I don't know if we can answer this for sure at the moment given the ongoing discussion on shared, but looking at core.sync, it occurred to me that there's a major problem with the classes in there. None of the work with shared. And unless I'm missing something here, I don't see how many of them are even useful as anything other than shared. After all, what good is a mutex which is thread-local? But none of the methods on Mutex or its friends are shared.

So, the question is should the all have their methods shared? And if they should, is there any reason to have non-shared overloads for them? What good are they as anything other than shared? How is anyone using them right now?

- Jonathan M Davis

On 18-11-2012 05:46, Jonathan M Davis wrote:
> I don't know if we can answer this for sure at the moment given the ongoing
> discussion on shared, but looking at core.sync, it occurred to me that there's
> a major problem with the classes in there. None of the work with shared. And
> unless I'm missing something here, I don't see how many of them are even
> useful as anything other than shared. After all, what good is a mutex which is
> thread-local? But none of the methods on Mutex or its friends are shared.
>
> So, the question is should the all have their methods shared? And if they
> should, is there any reason to have non-shared overloads for them? What good
> are they as anything other than shared? How is anyone using them right now?
>
> - Jonathan M Davis
>

Not at this point in time. It would break a ridiculous amount of code if we did this, given the current extremely annoying nature of shared.

Most D code I have seen in the wild just shares mutexes, conditions, etc with __gshared or some other mechanism anyway, so I don't think there's anything to gain. Like, what would shared actually buy you here?

-- 
Alex Rønne Petersen
alex@lycus.org
http://lycus.org

On Sunday, November 18, 2012 05:51:00 Alex Rønne Petersen wrote:
> On 18-11-2012 05:46, Jonathan M Davis wrote:
> > I don't know if we can answer this for sure at the moment given the
> > ongoing
> > discussion on shared, but looking at core.sync, it occurred to me that
> > there's a major problem with the classes in there. None of the work with
> > shared. And unless I'm missing something here, I don't see how many of
> > them are even useful as anything other than shared. After all, what good
> > is a mutex which is thread-local? But none of the methods on Mutex or its
> > friends are shared.
> > 
> > So, the question is should the all have their methods shared? And if they should, is there any reason to have non-shared overloads for them? What good are they as anything other than shared? How is anyone using them right now?
> > 
> > - Jonathan M Davis
> 
> Not at this point in time. It would break a ridiculous amount of code if we did this, given the current extremely annoying nature of shared.
> 
> Most D code I have seen in the wild just shares mutexes, conditions, etc with __gshared or some other mechanism anyway, so I don't think there's anything to gain. Like, what would shared actually buy you here?

__gshared is a good reason for leaving non-shared overloads, but isn't code really supposed to be using shared and not __gshared unless it's specifically extern(C)? That being the case, I'd expect shared to be the correct thing to use with mutexes normally, and right now, that won't work without a ton of casting or adding shared overloads.

- Jonathan M Davis

On 18-11-2012 05:58, Jonathan M Davis wrote:
> On Sunday, November 18, 2012 05:51:00 Alex Rønne Petersen wrote:
>> On 18-11-2012 05:46, Jonathan M Davis wrote:
>>> I don't know if we can answer this for sure at the moment given the
>>> ongoing
>>> discussion on shared, but looking at core.sync, it occurred to me that
>>> there's a major problem with the classes in there. None of the work with
>>> shared. And unless I'm missing something here, I don't see how many of
>>> them are even useful as anything other than shared. After all, what good
>>> is a mutex which is thread-local? But none of the methods on Mutex or its
>>> friends are shared.
>>>
>>> So, the question is should the all have their methods shared? And if they
>>> should, is there any reason to have non-shared overloads for them? What
>>> good are they as anything other than shared? How is anyone using them
>>> right now?
>>>
>>> - Jonathan M Davis
>>
>> Not at this point in time. It would break a ridiculous amount of code if
>> we did this, given the current extremely annoying nature of shared.
>>
>> Most D code I have seen in the wild just shares mutexes, conditions, etc
>> with __gshared or some other mechanism anyway, so I don't think there's
>> anything to gain. Like, what would shared actually buy you here?
>
> __gshared is a good reason for leaving non-shared overloads, but isn't code
> really supposed to be using shared and not __gshared unless it's specifically
> extern(C)? That being the case, I'd expect shared to be the correct thing to
> use with mutexes normally, and right now, that won't work without a ton of
> casting or adding shared overloads.
>
> - Jonathan M Davis
>

I don't know what anything that has to do with shared is supposed or not supposed to do. It is not clear, even to the primary language designers, what shared actually is, so I really can't say anything to this...

All I can say is, let's wait with this until we know what shared is going to actually do. We don't want to do something we'll regret later.

-- 
Alex Rønne Petersen
alex@lycus.org
http://lycus.org

11/18/2012 8:46 AM, Jonathan M Davis пишет:
> I don't know if we can answer this for sure at the moment given the ongoing
> discussion on shared, but looking at core.sync, it occurred to me that there's
> a major problem with the classes in there. None of the work with shared. And
> unless I'm missing something here, I don't see how many of them are even
> useful as anything other than shared. After all, what good is a mutex which is
> thread-local? But none of the methods on Mutex or its friends are shared.
>

None of them are useful without shared.They should work with shared, but that'll break code outright. I think 2 overloads per method with TLS versions going through the deprecation path.

> So, the question is should the all have their methods shared? And if they
> should, is there any reason to have non-shared overloads for them? What good
> are they as anything other than shared? How is anyone using them right now?

I do suspect that most people use _gshared with them as it is the less painful way. Another way is to have them shared and cast them to TLS as needed. Since mutex, condition etc. are opaque types around OS primitives TLS doesn't make sense at all but currently allows to call their methods.

-- 
Dmitry Olshansky

11/18/2012 11:13 AM, Alex Rønne Petersen пишет:
> On 18-11-2012 05:58, Jonathan M Davis wrote:
>> On Sunday, November 18, 2012 05:51:00 Alex Rønne Petersen wrote:
>>> On 18-11-2012 05:46, Jonathan M Davis wrote:
>>>> I don't know if we can answer this for sure at the moment given the
>>>> ongoing
>>>> discussion on shared, but looking at core.sync, it occurred to me that
>>>> there's a major problem with the classes in there. None of the work
>>>> with
>>>> shared. And unless I'm missing something here, I don't see how many of
>>>> them are even useful as anything other than shared. After all, what
>>>> good
>>>> is a mutex which is thread-local? But none of the methods on Mutex
>>>> or its
>>>> friends are shared.
>>>>
>>>> So, the question is should the all have their methods shared? And if
>>>> they
>>>> should, is there any reason to have non-shared overloads for them? What
>>>> good are they as anything other than shared? How is anyone using them
>>>> right now?
>>>>
>>>> - Jonathan M Davis
>>>
>>> Not at this point in time. It would break a ridiculous amount of code if
>>> we did this, given the current extremely annoying nature of shared.
>>>
>>> Most D code I have seen in the wild just shares mutexes, conditions, etc
>>> with __gshared or some other mechanism anyway, so I don't think there's
>>> anything to gain. Like, what would shared actually buy you here?
>>
>> __gshared is a good reason for leaving non-shared overloads, but isn't
>> code
>> really supposed to be using shared and not __gshared unless it's
>> specifically
>> extern(C)? That being the case, I'd expect shared to be the correct
>> thing to
>> use with mutexes normally, and right now, that won't work without a
>> ton of
>> casting or adding shared overloads.
>>
>> - Jonathan M Davis
>>
>
> I don't know what anything that has to do with shared is supposed or not
> supposed to do. It is not clear, even to the primary language designers,
> what shared actually is, so I really can't say anything to this...
>

Let's please stop this mantra. It buy us nothing but FUD.

One thing is clear - shared is not going away over night. Yes, having auto-magic atomic ops for various builtins is discussed, the kind of memory model is discussed.

Everything else stays. It's the globally shared data and it has tight restrictions on what you can do with it. As such it is (just like immutable) is safely passable between threads.

> All I can say is, let's wait with this until we know what shared is
> going to actually do. We don't want to do something we'll regret later.

Obviously we know what shared is. It's just certain details need straightening up. And certainly druntime/std.concurrency should add more support for it where meaningful.

-- 
Dmitry Olshansky

On 18-11-2012 10:58, Dmitry Olshansky wrote:
> 11/18/2012 11:13 AM, Alex Rønne Petersen пишет:
>> On 18-11-2012 05:58, Jonathan M Davis wrote:
>>> On Sunday, November 18, 2012 05:51:00 Alex Rønne Petersen wrote:
>>>> On 18-11-2012 05:46, Jonathan M Davis wrote:
>>>>> I don't know if we can answer this for sure at the moment given the
>>>>> ongoing
>>>>> discussion on shared, but looking at core.sync, it occurred to me that
>>>>> there's a major problem with the classes in there. None of the work
>>>>> with
>>>>> shared. And unless I'm missing something here, I don't see how many of
>>>>> them are even useful as anything other than shared. After all, what
>>>>> good
>>>>> is a mutex which is thread-local? But none of the methods on Mutex
>>>>> or its
>>>>> friends are shared.
>>>>>
>>>>> So, the question is should the all have their methods shared? And if
>>>>> they
>>>>> should, is there any reason to have non-shared overloads for them?
>>>>> What
>>>>> good are they as anything other than shared? How is anyone using them
>>>>> right now?
>>>>>
>>>>> - Jonathan M Davis
>>>>
>>>> Not at this point in time. It would break a ridiculous amount of
>>>> code if
>>>> we did this, given the current extremely annoying nature of shared.
>>>>
>>>> Most D code I have seen in the wild just shares mutexes, conditions,
>>>> etc
>>>> with __gshared or some other mechanism anyway, so I don't think there's
>>>> anything to gain. Like, what would shared actually buy you here?
>>>
>>> __gshared is a good reason for leaving non-shared overloads, but isn't
>>> code
>>> really supposed to be using shared and not __gshared unless it's
>>> specifically
>>> extern(C)? That being the case, I'd expect shared to be the correct
>>> thing to
>>> use with mutexes normally, and right now, that won't work without a
>>> ton of
>>> casting or adding shared overloads.
>>>
>>> - Jonathan M Davis
>>>
>>
>> I don't know what anything that has to do with shared is supposed or not
>> supposed to do. It is not clear, even to the primary language designers,
>> what shared actually is, so I really can't say anything to this...
>>
>
> Let's please stop this mantra. It buy us nothing but FUD.
>
> One thing is clear - shared is not going away over night. Yes, having
> auto-magic atomic ops for various builtins is discussed, the kind of
> memory model is discussed.
>
> Everything else stays. It's the globally shared data and it has tight
> restrictions on what you can do with it. As such it is (just like
> immutable) is safely passable between threads.

There have been proposals to a completely different meaning of shared that has nothing to do with memory models or atomic ops. There is no FUD here. Shoehorning shared in its current state into core.sync is just plain unreasonable given that nothing has been set in stone about the meaning of shared at all yet.

>
>> All I can say is, let's wait with this until we know what shared is
>> going to actually do. We don't want to do something we'll regret later.
>
> Obviously we know what shared is. It's just certain details need
> straightening up. And certainly druntime/std.concurrency should add more
> support for it where meaningful.

We don't know at all what shared is. Have you been following the other thread? People have wildly different ideas of what it should do.



I don't see why we should be in a hurry to add shared to core.sync in particular. Practically all of the runtime and standard library can't work properly with shared anyway, so it's not going to change a whole lot. I think it's much more sensible to wait until shared's semantics are set in stone and *then* start adapting the runtime and standard library.

I can't stop anyone from just doing this, of course, but it seems like a premature change to me.

-- 
Alex Rønne Petersen
alex@lycus.org
http://lycus.org

11/18/2012 10:36 PM, Alex Rønne Petersen пишет:
[snip]
>> Everything else stays. It's the globally shared data and it has tight
>> restrictions on what you can do with it. As such it is (just like
>> immutable) is safely passable between threads.
>
> There have been proposals to a completely different meaning of shared
> that has nothing to do with memory models or atomic ops. There is no FUD
> here. Shoehorning shared in its current state into core.sync is just
> plain unreasonable given that nothing has been set in stone about the
> meaning of shared at all yet.
>

I'd argue that shared on these objects has nothing to do with memory models and atomics. It's about global visibility and access to API that is interlocked on OS-level anyway. It works with any sane definition that doesn't forget to add that 'shared' implies global visibility.

>>
>>> All I can say is, let's wait with this until we know what shared is
>>> going to actually do. We don't want to do something we'll regret later.
>>
>> Obviously we know what shared is. It's just certain details need
>> straightening up. And certainly druntime/std.concurrency should add more
>> support for it where meaningful.
>
> We don't know at all what shared is. Have you been following the other
> thread? People have wildly different ideas of what it should do.
>

I admit dismissing a lot of proposals from that thread on sight. The ones being about shared somehow 'just working' and getting you safe way to sharing and the one about auto-magically attaching mutexes to shared stuff.

Given now that the thread is mostly spent. If there many good different concepts I'd appreciate if you list these here.

> I don't see why we should be in a hurry to add shared to core.sync in
> particular.

> Practically all of the runtime and standard library can't
> work properly with shared anyway, so it's not going to change a whole
> lot.

In part because even things that are supposed to always be shared in fact are not working as shared. And that's because nobody have found the time to go on and do this work.

Add the fact that say class instances have monitor field when everything is TLS by default for me is indication of one thing only - during a move to TLS by default and shared a lot of things were not updated accordingly because of the lack of time and other issues.

> I think it's much more sensible to wait until shared's semantics
> are set in stone and *then* start adapting the runtime and standard
> library.
>
> I can't stop anyone from just doing this, of course, but it seems like a
> premature change to me.
>
What change in semantics of shared could stop mutex object from making sense only as a shared object? I mean the kind of change that can realistically can happen not some vague idea.

-- 
Dmitry Olshansky

On Sunday, November 18, 2012 23:28:03 Dmitry Olshansky wrote:
> What change in semantics of shared could stop mutex object from making sense only as a shared object? I mean the kind of change that can realistically can happen not some vague idea.

I thought that it was blindingly clear from TDPL and even the online spec what shared is for. It marks variables as being shared across threads rather than thread local. I don't see how that could possibly be changing. __gshared isn't even _mentioned_ in TDPL, and I think that it gets used as much as it does is entirely because of some of the current issues with shared's implementation in the compiler and the lack of support in druntime and Phobos.

There are definitely questions as to what exactly should happen when a variable is marked as shared, but they have to do with what protections you get and what conversions are possible. They still don't fundamentally change the fact that the point of shared is to share an instance across threads rather than have it be thread local like everything is by default.

So, I concur in that I don't see how the semantics of shared could possibly not be appropriate for mutexes. I started this thread primarily because I was shocked that Mutex, Condition, etc. weren't marked with shared already, and I didn't see how they could even be used without it (and apparently, the answer to that is that almost everyone cops out and uses __gshared).

- Jonathan M Davis

To separate an important notion I'm trying to express.

I assume that:

class X{
...
	shared foo(...);
	void bar(...);
...
}

shared X sharedStuff;
X localStuff;

Implies under any viable definition of shared that (including the current one and any pending clarifications):

a) foo can be called with sharedStuff in multiple threads at the same time. Thus it has to be thread safe (irregardless of possible @system/@trusted hackery inside).

b) localStuff on the other hand can called with  bar (and any non-shared methods) only in a single thread at any time. If localStuff is defined at global scope each thread has its own copy thus this requirement is fulfilled kind of automatically.

The major benefits of shared lie in cleaning up the opposite - unshared stuff. Having shared enables us to archieve great things we haven't looked at yet (maybe because of bugs/issues in the other departments). I'll name a few.

We can have lock-less and _safe_ file I/O  by default unlike e.g. C and pretty much everybody else. I should have said safe & _faster_  by default. This is why I believe D should stop relying on C's I/O as anything but fallback. We even had a partially complete design for new std.io that does it.

Same goes for memory allocations - lock-less thread-local allocators are possible _safely_ and by default (though it would have to go to the OS from time to time and likely to lock something here). It's not explored and remains as unused potential.

-- 
Dmitry Olshansky