On Saturday, January 03, 2015 12:14:54 via Digitalmars-d-learn wrote:
> On Saturday, 3 January 2015 at 00:12:35 UTC, Jonathan M Davis via Digitalmars-d-learn wrote:
> > In D, if a type is not marked as shared, then it is by
> > definition
> > thread-local, and the compiler is free to assume that it's
> > thread-local.
>
> I find this to be rather vague. If the compiler exploit this to the maximum wouldn't that lead to lots of bugs?

Only if you're declaring shared or __gshared variables and not protecting them properly or interacting with C or C++ code in a way that doesn't take their memory model into account. The vast majority of D code won't care one whit and won't have any problems, because very little of it needs to be shared, and thread communication most typically is done via message passing using std.concurrency, not by declaring shared variables. So, the risk of bugs related to the compiler taking advantage of its knowledge that a variable is thread-local if not mark as shared is pretty low in most cases.

Yes, there's definitely a risk of bugs if you're casting away shared or
use __gshared and screw it up (which is part of why we'd much rather have a
way for the language to safely strip away shared when it can guarantee that
a shared variable is properly protected), but for the most part, it's not a
problem at all and is _far_ less of a problem than what you get in languages
like C or C++ which default to shared rather than thread-local.

- Jonathan M Davis

On Saturday, 3 January 2015 at 12:17:52 UTC, ketmar via Digitalmars-d-learn wrote:
> why should it? thread locals are... well, local for each thread. you
> can't access local of different thread without resorting to low-level
> assembly and OS dependent tricks.

Of course you can, anything that is reachable through any chain of pointers/references is effectively "shared", not only the object you explicitly "share".

So when you cast away "shared" then call a function and that is safe in itself, you don't know what happens when someone modifies some function deep down in the call chain later on and access some private pointer chain and possibly retain a pointer to it.

The alternative is to put "shared" on all parameters in libraries or avoid using libraries...

On Saturday, 3 January 2015 at 12:34:10 UTC, Jonathan M Davis via Digitalmars-d-learn wrote:
> their memory model into account. The vast majority of D code won't care one
> whit and won't have any problems, because very little of it needs to be
> shared, and thread communication most typically is done via message passing
> using std.concurrency, not by declaring shared variables.

I don't agree with this. If you avoid the problem by message passing, then you should do like Go and make it a language feature. And accept that you loose out on efficiency and restrict the application domain.

If you make "shared" a language feature you also need to back it up with semantic analysis and prove that the concept is sound and useful (i.e. no need to break encapsulation, not making most libraries unusable without unsafe casting etc).

On Saturday, 3 January 2015 at 13:33:21 UTC, Ola Fosheim Grøstad wrote:
> On Saturday, 3 January 2015 at 12:17:52 UTC, ketmar via Digitalmars-d-learn wrote:
>> why should it? thread locals are... well, local for each thread. you
>> can't access local of different thread without resorting to low-level
>> assembly and OS dependent tricks.
>
> Of course you can, anything that is reachable through any chain of pointers/references is effectively "shared", not only the object you explicitly "share".
>
> So when you cast away "shared" then call a function and that is safe in itself, you don't know what happens when someone modifies some function deep down in the call chain later on and access some private pointer chain and possibly retain a pointer to it.
>
> The alternative is to put "shared" on all parameters in libraries or avoid using libraries...

I think you're talking cross-purposes. thread-local as in TLS v.s. thread-local as in not-shared.

On Saturday, 3 January 2015 at 13:53:09 UTC, John Colvin wrote:
> I think you're talking cross-purposes. thread-local as in TLS v.s. thread-local as in not-shared.

I am not talking TLS. TLS is related to object files, not types. You don't have shared vs non-shared. You have many different relevant situations:

- no other reader or writer
- no other writer
- not shared now, but later
- unknown
- always shared
++

On Saturday, January 03, 2015 13:39:51 via Digitalmars-d-learn wrote:
> On Saturday, 3 January 2015 at 12:34:10 UTC, Jonathan M Davis via Digitalmars-d-learn wrote:
> > their memory model into account. The vast majority of D code
> > won't care one
> > whit and won't have any problems, because very little of it
> > needs to be
> > shared, and thread communication most typically is done via
> > message passing
> > using std.concurrency, not by declaring shared variables.
>
> I don't agree with this. If you avoid the problem by message passing, then you should do like Go and make it a language feature. And accept that you loose out on efficiency and restrict the application domain.
>
> If you make "shared" a language feature you also need to back it up with semantic analysis and prove that the concept is sound and useful (i.e. no need to break encapsulation, not making most libraries unusable without unsafe casting etc).

It is far better in general to use message passing to separate threads, but by no means is it intended that it be the only way to do things in D. It is a systems language after all. We have synchronized blocks and mutexes. We have stuff like std.parallelism. We have fibers. There are quite a few ways to go about having threads communicate and share data. It's just that message passing is by far the cleanest and safest in most cases, so it's what most code should be doing. If that's not what fits a particular program, then there are plenty of other options available. However, that does tend to mean having to deal with shared more.

And shared is definitely of benefit regardless of whether you're forced to carefully cast it away some of the time to use it, just like C++'s const is useful even if it's not actually full-on physical const like D's const is. But we pretty much all agree that we want better guarantees than that. Ideally, you would never cast away shared, and it would be cast away for you by the compiler in sections of code where it can guarantee that it's safe to do so (that was part of the idea behind synchronized classes). But that's incredibly difficult to do, particularly in a useful way, so we don't currently have it. And yes, that sucks, and we definitely want to fix it, but I still think that it's far better than having everything be shared by default like you get in languages like C++ and Java.

- Jonathan M Davis

On Saturday, 3 January 2015 at 23:11:08 UTC, Jonathan M Davis via Digitalmars-d-learn wrote:
> Ideally, you would never cast away shared, and it would be cast away for you
> by the compiler in sections of code where it can guarantee that it's safe to
> do so (that was part of the idea behind synchronized classes). But that's
> incredibly difficult to do, particularly in a useful way, so we don't
> currently have it. And yes, that sucks, and we definitely want to fix it,
> but I still think that it's far better than having everything be shared by
> default like you get in languages like C++ and Java.

Efficient automatic synchronization is difficult, yes. You can try to tie groups of entities to a lock, but that will only work in some scenarios.

To me it sounds like having everything shared by default is the most conservative (safest) approach, and that it would make sense to put restrictions on parameters when you need more performance. If D's approach should make sense the compiler would allowed to elide atomics on members of an object the reference to the object is not marked as "shared". That can easily go horribly wrong.

I am also not overly happy with D making TLS default. That means new threads instantiate a lot of unused memory if the workload is heterogeneous (different threads do different type of work). TLS only make sense for things that all threads need.

On Monday, January 05, 2015 12:59:26 via Digitalmars-d-learn wrote:
> I am also not overly happy with D making TLS default. That means new threads instantiate a lot of unused memory if the workload is heterogeneous (different threads do different type of work). TLS only make sense for things that all threads need.

Well, if you don't like the choice of TLS by default, then you're going to be unhappy with shared and its related issues regardless. Personally, I think that having TLS be the default is a fantastic improvement over C++ and that it results in much cleaner and safer code, especially since the vast majority of code only lives on one thread anyway if you're dealing with threads cleanly. But it's definitely true that what we're up to is an experiment in how to handle TLS and shared storage, and by no means have we gotten it perfect.

- Jonathan M Davis