On 13.11.2012 10:20, Sönke Ludwig wrote:
> Am 13.11.2012 10:14, schrieb luka8088:
>> On Tuesday, 13 November 2012 at 09:11:15 UTC, luka8088 wrote:
>>> On 12.11.2012 3:30, Walter Bright wrote:
>>>> On 11/11/2012 10:46 AM, Alex Rønne Petersen wrote:
>>>>> It's starting to get outright embarrassing to talk to newcomers about D's
>>>>> concurrency support because the most fundamental part of it -- the
>>>>> shared type
>>>>> qualifier -- does not have well-defined semantics at all.
>>>>
>>>> I think a couple things are clear:
>>>>
>>>> 1. Slapping shared on a type is never going to make algorithms on that
>>>> type work in a concurrent context, regardless of what is done with
>>>> memory barriers. Memory barriers ensure sequential consistency, they do
>>>> nothing for race conditions that are sequentially consistent. Remember,
>>>> single core CPUs are all sequentially consistent, and still have major
>>>> concurrency problems. This also means that having templates accept
>>>> shared(T) as arguments and have them magically generate correct
>>>> concurrent code is a pipe dream.
>>>>
>>>> 2. The idea of shared adding memory barriers for access is not going to
>>>> ever work. Adding barriers has to be done by someone who knows what
>>>> they're doing for that particular use case, and the compiler inserting
>>>> them is not going to substitute.
>>>>
>>>>
>>>> However, and this is a big however, having shared as compiler-enforced
>>>> self-documentation is immensely useful. It flags where and when data is
>>>> being shared. So, your algorithm won't compile when you pass it a shared
>>>> type? That is because it is NEVER GOING TO WORK with a shared type. At
>>>> least you get a compile time indication of this, rather than random
>>>> runtime corruption.
>>>>
>>>> To make a shared type work in an algorithm, you have to:
>>>>
>>>> 1. ensure single threaded access by aquiring a mutex
>>>> 2. cast away shared
>>>> 3. operate on the data
>>>> 4. cast back to shared
>>>> 5. release the mutex
>>>>
>>>> Also, all op= need to be disabled for shared types.
>>>
>>>
>>> This clarifies a lot, but still a lot of people get confused with:
>>> http://dlang.org/faq.html#shared_memory_barriers
>>> is it a faq error ?
>>>
>>> and also with http://dlang.org/faq.html#shared_guarantees said, I come to think that the fact that
>>> the following code compiles is either lack of implementation, a compiler bug or a faq error ?
>>>
>>> //////////
>>>
>>> import core.thread;
>>>
>>> void main () {
>>>    shared int i;
>>>    (new Thread({ i++; })).start();
>>> }
>>
>> Um, sorry, the following code:
>>
>> //////////
>>
>> import core.thread;
>>
>> void main () {
>>    int i;
>>    (new Thread({ i++; })).start();
>> }
>>
>
> Only std.concurrency (using spawn() and send()) enforces that unshared data cannot be pass between
> threads. The core.thread module is just a low-level module that just represents the OS functionality.

In that case http://dlang.org/faq.html#shared_guarantees is wrong, it is not a correct guarantee. Or at least that should be noted there. If nothing else it is confusing...

On 11/12/12 20:08, Jacob Carlborg wrote:
> On 2012-11-12 17:57, Simen Kjaeraas wrote:
> 
>> Until someone writes a proper DIP on them, macros can write entire software packages, download Hitler, turn D into lisp, and bake bread. Can we please stop with the 'macros could do that' until there's any sort of consensus as to what macros *could* do?
> 
> Sure, I can try and stop doing that :)
> 

You know, AST macros could probably stop doing that.

Food for thought.

On Tuesday, 13 November 2012 at 10:06:12 UTC, luka8088 wrote:
> On 13.11.2012 10:20, Sönke Ludwig wrote:
>> Only std.concurrency (using spawn() and send()) enforces that unshared data cannot be pass between
>> threads. The core.thread module is just a low-level module that just represents the OS functionality.
>
> In that case http://dlang.org/faq.html#shared_guarantees is wrong, it is not a correct guarantee. Or at least that should be noted there. If nothing else it is confusing...

You are right, it could probably be added to avoid confusion. But then, non-@safe code is not guaranteed to maintain any type system invariants at all if you don't pay attention to what its requirements are, so memory sharing is not really special in that regard…

David

On 11/12/2012 02:48 AM, Michel Fortin wrote:
> On 2012-11-11 18:46:10 +0000, Alex Rønne Petersen <alex@lycus.org> said:
>
>> Something needs to be done about shared. I don't know what, but the
>> current situation is -- and I'm really not exaggerating here --
>> laughable. I think we either need to just make it perfectly clear that
>> shared is for documentation purposes and nothing else, or, figure out
>> an alternative system to shared, because I don't see shared actually
>> being useful for real world work no matter what we do with it.
>
> I feel like the concurrency aspect of D2 was rushed in the haste of
> having it ready for TDPL. Shared, deadlock-prone synchronized classes[1]
> as well as destructors running in any thread (thanks GC!) plus a couple
> of other irritants makes the whole concurrency scheme completely flawed
> if you ask me. D2 needs a near complete overhaul on the concurrency front.
>
> I'm currently working on a big code base in C++. While I do miss D when
> it comes to working with templates as well as for its compilation speed
> and a few other things, I can't say I miss D much when it comes to
> anything touching concurrency.
>
> [1]: http://michelf.ca/blog/2012/mutex-synchonization-in-d/
>

I am always irritated by shared-by-default static variables.

On 11/13/2012 1:11 AM, luka8088 wrote:
> This clarifies a lot, but still a lot of people get confused with:
> http://dlang.org/faq.html#shared_memory_barriers
> is it a faq error ?

Andrei is a proponent of having shared to memory barriers, I disagree with him. We haven't convinced each other yet, so this is a bit up in the air.


> and also with http://dlang.org/faq.html#shared_guarantees said, I come to think
> that the fact that the following code compiles is either lack of implementation,
> a compiler bug or a faq error ?
>
> //////////
>
> import core.thread;
>
> void main () {
>    shared int i;
>    (new Thread({ i++; })).start();
> }

I think it's a user bug.

On Tuesday, 13 November 2012 at 21:29:13 UTC, Walter Bright wrote:
> On 11/13/2012 1:11 AM, luka8088 wrote:
>> This clarifies a lot, but still a lot of people get confused with:
>> http://dlang.org/faq.html#shared_memory_barriers
>> is it a faq error ?
>
> Andrei is a proponent of having shared to memory barriers, I disagree with him. We haven't convinced each other yet, so this is a bit up in the air.

FWIW, I'm with you on this one. Memory barriers would not make shared more useful, as they do not solve the issue with concurrency (as you have explained earlier).

On 11/13/12 1:28 PM, Walter Bright wrote:
> On 11/13/2012 1:11 AM, luka8088 wrote:
>> This clarifies a lot, but still a lot of people get confused with:
>> http://dlang.org/faq.html#shared_memory_barriers
>> is it a faq error ?
>
> Andrei is a proponent of having shared to memory barriers, I disagree
> with him. We haven't convinced each other yet, so this is a bit up in
> the air.

Wait, then what would shared do? This is new to me as I've always assumed you and I have the same view on this.

Andrei

On Tuesday, 13 November 2012 at 21:56:21 UTC, Andrei Alexandrescu wrote:
> On 11/13/12 1:28 PM, Walter Bright wrote:
>> On 11/13/2012 1:11 AM, luka8088 wrote:
>>> This clarifies a lot, but still a lot of people get confused with:
>>> http://dlang.org/faq.html#shared_memory_barriers
>>> is it a faq error ?
>>
>> Andrei is a proponent of having shared to memory barriers, I disagree
>> with him. We haven't convinced each other yet, so this is a bit up in
>> the air.
>
> Wait, then what would shared do? This is new to me as I've always assumed you and I have the same view on this.
>
> Andrei

I'm speaking out of turn, but...

I'll flip that around: what would shared do if there were memory barriers?

Walter has said previously in this thread that shared is to be used to mark shared data, and disallow any potentially non-thread-safe operations. To use shared data, you need to manually lock it and then cast away the shared temporarily. This can be made more pleasant with library utilities.

On 11/13/2012 1:56 PM, Andrei Alexandrescu wrote:
> On 11/13/12 1:28 PM, Walter Bright wrote:
>> On 11/13/2012 1:11 AM, luka8088 wrote:
>>> This clarifies a lot, but still a lot of people get confused with:
>>> http://dlang.org/faq.html#shared_memory_barriers
>>> is it a faq error ?
>>
>> Andrei is a proponent of having shared to memory barriers, I disagree
>> with him. We haven't convinced each other yet, so this is a bit up in
>> the air.
>
> Wait, then what would shared do? This is new to me as I've always assumed you
> and I have the same view on this.

I'm just not convinced that having the compiler add memory barriers:

1. will result in correctly working code, when done by programmers who have only an incomplete understanding of memory barriers, which would be about 99.9% of us.

2. will result in efficient code

I also worry that it will lure programmers into a false sense of complacency about shared, that simply adding "shared" to a type will make their concurrent code work. Few seem to realize that adding memory barriers only makes code sequentially consistent, it does *not* eliminate race conditions. It just turns a multicore machine into (logically) a single core one, *not* a single threaded one.

But I do see enormous value in shared in that it logically (and rather forcefully) separates thread-local code from multi-thread code. For example, see the post here about adding a destructor to a shared struct, and having it fail to compile. The complaint was along the lines of shared being broken, whereas I viewed it along the lines of shared pointing out a logic problem in the code - what does destroying a struct accessible from multiple threads mean? I think it must be clear that destroying an object can only happen in one thread, i.e. the object must become thread local in order to be destroyed.

On 11/13/12 2:07 PM, Peter Alexander wrote:
> On Tuesday, 13 November 2012 at 21:56:21 UTC, Andrei Alexandrescu wrote:
>> On 11/13/12 1:28 PM, Walter Bright wrote:
>>> On 11/13/2012 1:11 AM, luka8088 wrote:
>>>> This clarifies a lot, but still a lot of people get confused with:
>>>> http://dlang.org/faq.html#shared_memory_barriers
>>>> is it a faq error ?
>>>
>>> Andrei is a proponent of having shared to memory barriers, I disagree
>>> with him. We haven't convinced each other yet, so this is a bit up in
>>> the air.
>>
>> Wait, then what would shared do? This is new to me as I've always
>> assumed you and I have the same view on this.
>>
>> Andrei
>
> I'm speaking out of turn, but...
>
> I'll flip that around: what would shared do if there were memory barriers?
>
> Walter has said previously in this thread that shared is to be used to
> mark shared data, and disallow any potentially non-thread-safe
> operations. To use shared data, you need to manually lock it and then
> cast away the shared temporarily. This can be made more pleasant with
> library utilities.

Oh ok, thanks. That does make sense. There's been quite a bit of discussion between Bartosz, Walter, and myself about allowing transparent loads and stores as opposed to defining intrinsics x.load and x.store(y). In C++11 both transparent and implicit are allowed, and an emergent idiom is "already use the explicit versions because they clarify flow and cost".


Andrei