After reading the recent "Lessons Learned" article [1], and reading a few comments on the thread, there was a mention of using __gshared over shared.

What exactly is the difference here?
Are they 2 keywords to do the same thing, or are there specific use cases to both?
Is there plans to 'converge' them at some point?

[1] https://www.reddit.com/r/programming/comments/3cg1r0/lessons_learned_writing_a_filesystem_in_d/

On Wednesday, 8 July 2015 at 09:20:58 UTC, wobbles wrote:
> After reading the recent "Lessons Learned" article [1], and reading a few comments on the thread, there was a mention of using __gshared over shared.
>
> What exactly is the difference here?
> Are they 2 keywords to do the same thing, or are there specific use cases to both?
> Is there plans to 'converge' them at some point?
>
> [1] https://www.reddit.com/r/programming/comments/3cg1r0/lessons_learned_writing_a_filesystem_in_d/

You can read it there : http://dlang.org/migrate-to-shared.html#gshared

Basically, __gshared is for interfacing with C (where everything is shared by default) without marking data as shared.

On Wednesday, 8 July 2015 at 09:37:37 UTC, Olivier Pisano wrote:
> On Wednesday, 8 July 2015 at 09:20:58 UTC, wobbles wrote:
>> After reading the recent "Lessons Learned" article [1], and reading a few comments on the thread, there was a mention of using __gshared over shared.
>>
>> What exactly is the difference here?
>> Are they 2 keywords to do the same thing, or are there specific use cases to both?
>> Is there plans to 'converge' them at some point?
>>
>> [1] https://www.reddit.com/r/programming/comments/3cg1r0/lessons_learned_writing_a_filesystem_in_d/
>
> You can read it there : http://dlang.org/migrate-to-shared.html#gshared
>
> Basically, __gshared is for interfacing with C (where everything is shared by default) without marking data as shared.

Ok, so we should prioritise using 'shared' over __gshared as much as possible. Good to know!

On Wed, 08 Jul 2015 09:43:38 +0000, wobbles wrote:

> Ok, so we should prioritise using 'shared' over __gshared as much as possible. Good to know!

only `shared` is PITA...

On Wednesday, 8 July 2015 at 09:54:01 UTC, ketmar wrote:
> On Wed, 08 Jul 2015 09:43:38 +0000, wobbles wrote:
>
>> Ok, so we should prioritise using 'shared' over __gshared as much as possible. Good to know!
>
> only `shared` is PITA...

The primary advantage of shared is that it allows most everything to be thread-local.

Though arguably, shared _should_ be a bit of a pain, since its usage should normally be very restricted. But we do need to revisit shared and figure out what we want/need to do with it. Synchronized classes as described in TDPL were never even implemented (though I contest that they really make shared usable in any kind of sane way; I really don't see how you can do anything other than really basic stuff with shared without requiring that the programmer deal with the locks and casting properly on their own). So, more work needs to be done there even if it's figuring out what shared _isn't_ going to be doing.

Really though, one of the bigger problems is dealing with allocation and deallocation of shared objects and passing objects across threads, since we keep wanting to be able to do stuff like have thread-specific allocators, but the way shared currently works doesn't actually allow for it. :|

Regardless, while I would very much like to see shared properly ironed out, I'm _very_ grateful that thread-local is the default in D. It's just so much saner.

- Jonathan M Davis

On Wednesday, 8 July 2015 at 10:10:58 UTC, Jonathan M Davis wrote:
> On Wednesday, 8 July 2015 at 09:54:01 UTC, ketmar wrote:
>> On Wed, 08 Jul 2015 09:43:38 +0000, wobbles wrote:
>>
>>> Ok, so we should prioritise using 'shared' over __gshared as much as possible. Good to know!
>>
>> only `shared` is PITA...
>
> The primary advantage of shared is that it allows most everything to be thread-local.
>
> Though arguably, shared _should_ be a bit of a pain, since its usage should normally be very restricted. But we do need to revisit shared and figure out what we want/need to do with it. Synchronized classes as described in TDPL were never even implemented (though I contest that they really make shared usable in any kind of sane way; I really don't see how you can do anything other than really basic stuff with shared without requiring that the programmer deal with the locks and casting properly on their own). So, more work needs to be done there even if it's figuring out what shared _isn't_ going to be doing.
>
> Really though, one of the bigger problems is dealing with allocation and deallocation of shared objects and passing objects across threads, since we keep wanting to be able to do stuff like have thread-specific allocators, but the way shared currently works doesn't actually allow for it. :|
>
> Regardless, while I would very much like to see shared properly ironed out, I'm _very_ grateful that thread-local is the default in D. It's just so much saner.
>
> - Jonathan M Davis

I still couldn't have found my answer though. I have three different use cases, one is missing in the language.

1. Thread-local object.
2. Shared, but implemented as not to be synchronised.
3. Shared, and implemented to be synchronised.

There is no simple way to design a class, so that you can implement it for points 2 and 3. The only way is to use _gshared with point one to solve it.

I use shared in many of my classes. Thus I experience different situations.

On Wednesday, 8 July 2015 at 09:54:01 UTC, ketmar wrote:
> On Wed, 08 Jul 2015 09:43:38 +0000, wobbles wrote:
>
>> Ok, so we should prioritise using 'shared' over __gshared as much as possible. Good to know!
>
> only `shared` is PITA...

I write a fair amount of threaded code myself and must say the experience with shared has also been horrible. To me, it feels like an attempt at fixing something in the way we write threaded code that someone gave up on, and now it's just noise in the language. I have given up on it too, and am now using only __gshared. It's fine, but it feels dirty to be using a __ prefixed keyword so often. :)

If I remember correctly, it was also a pain to use most of Phobos with 'shared' without casting left and right.

On Wednesday, 8 July 2015 at 10:49:05 UTC, tcak wrote:
> On Wednesday, 8 July 2015 at 10:10:58 UTC, Jonathan M Davis wrote:
>> On Wednesday, 8 July 2015 at 09:54:01 UTC, ketmar wrote:
>>> On Wed, 08 Jul 2015 09:43:38 +0000, wobbles wrote:
>>>
>>>> Ok, so we should prioritise using 'shared' over __gshared as much as possible. Good to know!
>>>
>>> only `shared` is PITA...
>>
>> The primary advantage of shared is that it allows most everything to be thread-local.
>>
>> Though arguably, shared _should_ be a bit of a pain, since its usage should normally be very restricted. But we do need to revisit shared and figure out what we want/need to do with it. Synchronized classes as described in TDPL were never even implemented (though I contest that they really make shared usable in any kind of sane way; I really don't see how you can do anything other than really basic stuff with shared without requiring that the programmer deal with the locks and casting properly on their own). So, more work needs to be done there even if it's figuring out what shared _isn't_ going to be doing.
>>
>> Really though, one of the bigger problems is dealing with allocation and deallocation of shared objects and passing objects across threads, since we keep wanting to be able to do stuff like have thread-specific allocators, but the way shared currently works doesn't actually allow for it. :|
>>
>> Regardless, while I would very much like to see shared properly ironed out, I'm _very_ grateful that thread-local is the default in D. It's just so much saner.
>>
>> - Jonathan M Davis
>
> I still couldn't have found my answer though. I have three different use cases, one is missing in the language.
>
> 1. Thread-local object.
> 2. Shared, but implemented as not to be synchronised.
> 3. Shared, and implemented to be synchronised.
>
> There is no simple way to design a class, so that you can implement it for points 2 and 3. The only way is to use _gshared with point one to solve it.
>
> I use shared in many of my classes. Thus I experience different situations.

By using __gshared, you're throwing away the compiler's help, and it's _much_ more likely that you're going to write code which causes the compiler to generate incorrect machine code, because it's assuming that an object is thread-local when it's not.

Generally what you have to do with shared is lock on a mutex, cast away shared on the object you want to operate on, do whatever you're going to do with it, and then release the lock after there are no more thread-local references to the shared object. And that's basically what you normally should be doing in C++ code except that you don't have to cast away shared, because C++ doesn't have it.

I know that there are a number of people who get frustrated with shared and using __gshared instead, but unless you fully understand what you're doing and how the language works, and you're _really_ careful, you're going to shoot yourself in the foot it subtle ways if you do that.

- Jonathan M Davis

On Wednesday, 8 July 2015 at 11:02:19 UTC, Márcio Martins wrote:
> If I remember correctly, it was also a pain to use most of Phobos with 'shared' without casting left and right.

In general, if you're doing much with a shared object other than passing it around, you're using shared incorrectly. Various operations can be done with core.atomic and shared objects, but a number of operations are outright forbidden by the language, and to use a non-thread local object correctly (be it __gshared or shared), you _have_ to protect it with lock except in cases where you know what you're doing and are _extremely_ careful with atomic operations. D is effectively trying to force you to not try and do operations on a shared object unless it's protected by a lock.

Per TDPL, we're supposed to have synchronized classes which then strip off the outer layer of shared internally when operating on their member variables so that the sharedness can be stripped safely without relying on the programmer getting the locks and casting right, but synchronized classes have never been implemented (just synchronized functions), so that doesn't work at the moment, and stripping off just the outer layer of shared often isn't enough anyway. So, what you end up having to do is something along the lines of

synchronized(mutexObj)
{
    auto unshared = cast(T)sharedT;

    // ...
    // do stuff on unshared, since it's considered thread-local, and normal
    // code will work on it.
    //...

    // make sure that there are no references to unshared before leaving the
    // synchronized block
}

That way, you can use the shared object with normal code as long as you've protected it properly with a mutex, and the language prevents you from accidentally operating no the shared object through the shared reference (which would then be when it's not protected by a lock). The problem is that unlike with synchronized classes, you have to explicitly cast away shared yourself and ensure that no thread-local references to that data escape the synchronized block, and if you screw it up, you could have subtle, entertaining bugs, whereas synchronized classes would be able to guarantee that the outer layer of shared was safely removed.

What we would ideally have would be a way to do the above code safely without having to do the cast explicitly and make sure that that was done correctly, but we haven't figured out how to do that yet. But this idiom ensures that you operate on shared objects only when they're protected by a lock. Actually operating on shared or __gshared objects without locking is just plain buggy unless you're talking about atomic operations and using core.atomic properly to deal with that (which isn't necessarily easy). So, if anything, the fact that folks find shared so annoying implies that they're trying to write code in a manner which is not thread-safe. That's not to say that shared doesn't need to be improved or that there aren't things that you can't do with it right now that you should be able to, but in general, the stuff that you can't do with a shared object is stuff that you shouldn't be doing with a shared object anyway.

- Jonathan M Davis

On Wednesday, 8 July 2015 at 12:21:22 UTC, Jonathan M Davis wrote:
> On Wednesday, 8 July 2015 at 11:02:19 UTC, Márcio Martins wrote:
>> [...]


Interesting, so the main pain of using shared is the requirement to cast away the shared whenever you want to work on the data in a synchronized block.

Is there any links do you know to the old conversations on what solutions are there for this?

My first thought is using the 'with' keyword.
shared int mySharedInt;
synchronised(mutexObj) with (mySharedInt){
    // any references to mySharedInt in this block are implicitly converted to non-shared
}