View mode: basic / threaded / horizontal-split · Log in · Help
November 18, 2012
Should core.sync.mutex.Mutex, core.sync.condition.Condition, etc. have all their methods be shared?
I don't know if we can answer this for sure at the moment given the ongoing 
discussion on shared, but looking at core.sync, it occurred to me that there's 
a major problem with the classes in there. None of the work with shared. And 
unless I'm missing something here, I don't see how many of them are even 
useful as anything other than shared. After all, what good is a mutex which is 
thread-local? But none of the methods on Mutex or its friends are shared.

So, the question is should the all have their methods shared? And if they 
should, is there any reason to have non-shared overloads for them? What good 
are they as anything other than shared? How is anyone using them right now?

- Jonathan M Davis
November 18, 2012
Re: Should core.sync.mutex.Mutex, core.sync.condition.Condition, etc. have all their methods be shared?
On 18-11-2012 05:46, Jonathan M Davis wrote:
> I don't know if we can answer this for sure at the moment given the ongoing
> discussion on shared, but looking at core.sync, it occurred to me that there's
> a major problem with the classes in there. None of the work with shared. And
> unless I'm missing something here, I don't see how many of them are even
> useful as anything other than shared. After all, what good is a mutex which is
> thread-local? But none of the methods on Mutex or its friends are shared.
>
> So, the question is should the all have their methods shared? And if they
> should, is there any reason to have non-shared overloads for them? What good
> are they as anything other than shared? How is anyone using them right now?
>
> - Jonathan M Davis
>

Not at this point in time. It would break a ridiculous amount of code if 
we did this, given the current extremely annoying nature of shared.

Most D code I have seen in the wild just shares mutexes, conditions, etc 
with __gshared or some other mechanism anyway, so I don't think there's 
anything to gain. Like, what would shared actually buy you here?

-- 
Alex Rønne Petersen
alex@lycus.org
http://lycus.org
November 18, 2012
Re: Should core.sync.mutex.Mutex, core.sync.condition.Condition, etc. have all their methods be shared?
On Sunday, November 18, 2012 05:51:00 Alex Rønne Petersen wrote:
> On 18-11-2012 05:46, Jonathan M Davis wrote:
> > I don't know if we can answer this for sure at the moment given the
> > ongoing
> > discussion on shared, but looking at core.sync, it occurred to me that
> > there's a major problem with the classes in there. None of the work with
> > shared. And unless I'm missing something here, I don't see how many of
> > them are even useful as anything other than shared. After all, what good
> > is a mutex which is thread-local? But none of the methods on Mutex or its
> > friends are shared.
> > 
> > So, the question is should the all have their methods shared? And if they
> > should, is there any reason to have non-shared overloads for them? What
> > good are they as anything other than shared? How is anyone using them
> > right now?
> > 
> > - Jonathan M Davis
> 
> Not at this point in time. It would break a ridiculous amount of code if
> we did this, given the current extremely annoying nature of shared.
> 
> Most D code I have seen in the wild just shares mutexes, conditions, etc
> with __gshared or some other mechanism anyway, so I don't think there's
> anything to gain. Like, what would shared actually buy you here?

__gshared is a good reason for leaving non-shared overloads, but isn't code 
really supposed to be using shared and not __gshared unless it's specifically 
extern(C)? That being the case, I'd expect shared to be the correct thing to 
use with mutexes normally, and right now, that won't work without a ton of 
casting or adding shared overloads.

- Jonathan M Davis
November 18, 2012
Re: Should core.sync.mutex.Mutex, core.sync.condition.Condition, etc. have all their methods be shared?
On 18-11-2012 05:58, Jonathan M Davis wrote:
> On Sunday, November 18, 2012 05:51:00 Alex Rønne Petersen wrote:
>> On 18-11-2012 05:46, Jonathan M Davis wrote:
>>> I don't know if we can answer this for sure at the moment given the
>>> ongoing
>>> discussion on shared, but looking at core.sync, it occurred to me that
>>> there's a major problem with the classes in there. None of the work with
>>> shared. And unless I'm missing something here, I don't see how many of
>>> them are even useful as anything other than shared. After all, what good
>>> is a mutex which is thread-local? But none of the methods on Mutex or its
>>> friends are shared.
>>>
>>> So, the question is should the all have their methods shared? And if they
>>> should, is there any reason to have non-shared overloads for them? What
>>> good are they as anything other than shared? How is anyone using them
>>> right now?
>>>
>>> - Jonathan M Davis
>>
>> Not at this point in time. It would break a ridiculous amount of code if
>> we did this, given the current extremely annoying nature of shared.
>>
>> Most D code I have seen in the wild just shares mutexes, conditions, etc
>> with __gshared or some other mechanism anyway, so I don't think there's
>> anything to gain. Like, what would shared actually buy you here?
>
> __gshared is a good reason for leaving non-shared overloads, but isn't code
> really supposed to be using shared and not __gshared unless it's specifically
> extern(C)? That being the case, I'd expect shared to be the correct thing to
> use with mutexes normally, and right now, that won't work without a ton of
> casting or adding shared overloads.
>
> - Jonathan M Davis
>

I don't know what anything that has to do with shared is supposed or not 
supposed to do. It is not clear, even to the primary language designers, 
what shared actually is, so I really can't say anything to this...

All I can say is, let's wait with this until we know what shared is 
going to actually do. We don't want to do something we'll regret later.

-- 
Alex Rønne Petersen
alex@lycus.org
http://lycus.org
November 18, 2012
Re: Should core.sync.mutex.Mutex, core.sync.condition.Condition, etc. have all their methods be shared?
11/18/2012 8:46 AM, Jonathan M Davis пишет:
> I don't know if we can answer this for sure at the moment given the ongoing
> discussion on shared, but looking at core.sync, it occurred to me that there's
> a major problem with the classes in there. None of the work with shared. And
> unless I'm missing something here, I don't see how many of them are even
> useful as anything other than shared. After all, what good is a mutex which is
> thread-local? But none of the methods on Mutex or its friends are shared.
>

None of them are useful without shared.They should work with shared, but 
that'll break code outright. I think 2 overloads per method with TLS 
versions going through the deprecation path.

> So, the question is should the all have their methods shared? And if they
> should, is there any reason to have non-shared overloads for them? What good
> are they as anything other than shared? How is anyone using them right now?

I do suspect that most people use _gshared with them as it is the less 
painful way. Another way is to have them shared and cast them to TLS as 
needed. Since mutex, condition etc. are opaque types around OS 
primitives TLS doesn't make sense at all but currently allows to call 
their methods.

-- 
Dmitry Olshansky
November 18, 2012
Re: Should core.sync.mutex.Mutex, core.sync.condition.Condition, etc. have all their methods be shared?
11/18/2012 11:13 AM, Alex Rønne Petersen пишет:
> On 18-11-2012 05:58, Jonathan M Davis wrote:
>> On Sunday, November 18, 2012 05:51:00 Alex Rønne Petersen wrote:
>>> On 18-11-2012 05:46, Jonathan M Davis wrote:
>>>> I don't know if we can answer this for sure at the moment given the
>>>> ongoing
>>>> discussion on shared, but looking at core.sync, it occurred to me that
>>>> there's a major problem with the classes in there. None of the work
>>>> with
>>>> shared. And unless I'm missing something here, I don't see how many of
>>>> them are even useful as anything other than shared. After all, what
>>>> good
>>>> is a mutex which is thread-local? But none of the methods on Mutex
>>>> or its
>>>> friends are shared.
>>>>
>>>> So, the question is should the all have their methods shared? And if
>>>> they
>>>> should, is there any reason to have non-shared overloads for them? What
>>>> good are they as anything other than shared? How is anyone using them
>>>> right now?
>>>>
>>>> - Jonathan M Davis
>>>
>>> Not at this point in time. It would break a ridiculous amount of code if
>>> we did this, given the current extremely annoying nature of shared.
>>>
>>> Most D code I have seen in the wild just shares mutexes, conditions, etc
>>> with __gshared or some other mechanism anyway, so I don't think there's
>>> anything to gain. Like, what would shared actually buy you here?
>>
>> __gshared is a good reason for leaving non-shared overloads, but isn't
>> code
>> really supposed to be using shared and not __gshared unless it's
>> specifically
>> extern(C)? That being the case, I'd expect shared to be the correct
>> thing to
>> use with mutexes normally, and right now, that won't work without a
>> ton of
>> casting or adding shared overloads.
>>
>> - Jonathan M Davis
>>
>
> I don't know what anything that has to do with shared is supposed or not
> supposed to do. It is not clear, even to the primary language designers,
> what shared actually is, so I really can't say anything to this...
>

Let's please stop this mantra. It buy us nothing but FUD.

One thing is clear - shared is not going away over night. Yes, having 
auto-magic atomic ops for various builtins is discussed, the kind of 
memory model is discussed.

Everything else stays. It's the globally shared data and it has tight 
restrictions on what you can do with it. As such it is (just like 
immutable) is safely passable between threads.

> All I can say is, let's wait with this until we know what shared is
> going to actually do. We don't want to do something we'll regret later.

Obviously we know what shared is. It's just certain details need 
straightening up. And certainly druntime/std.concurrency should add more 
support for it where meaningful.

-- 
Dmitry Olshansky
November 18, 2012
Re: Should core.sync.mutex.Mutex, core.sync.condition.Condition, etc. have all their methods be shared?
On 18-11-2012 10:58, Dmitry Olshansky wrote:
> 11/18/2012 11:13 AM, Alex Rønne Petersen пишет:
>> On 18-11-2012 05:58, Jonathan M Davis wrote:
>>> On Sunday, November 18, 2012 05:51:00 Alex Rønne Petersen wrote:
>>>> On 18-11-2012 05:46, Jonathan M Davis wrote:
>>>>> I don't know if we can answer this for sure at the moment given the
>>>>> ongoing
>>>>> discussion on shared, but looking at core.sync, it occurred to me that
>>>>> there's a major problem with the classes in there. None of the work
>>>>> with
>>>>> shared. And unless I'm missing something here, I don't see how many of
>>>>> them are even useful as anything other than shared. After all, what
>>>>> good
>>>>> is a mutex which is thread-local? But none of the methods on Mutex
>>>>> or its
>>>>> friends are shared.
>>>>>
>>>>> So, the question is should the all have their methods shared? And if
>>>>> they
>>>>> should, is there any reason to have non-shared overloads for them?
>>>>> What
>>>>> good are they as anything other than shared? How is anyone using them
>>>>> right now?
>>>>>
>>>>> - Jonathan M Davis
>>>>
>>>> Not at this point in time. It would break a ridiculous amount of
>>>> code if
>>>> we did this, given the current extremely annoying nature of shared.
>>>>
>>>> Most D code I have seen in the wild just shares mutexes, conditions,
>>>> etc
>>>> with __gshared or some other mechanism anyway, so I don't think there's
>>>> anything to gain. Like, what would shared actually buy you here?
>>>
>>> __gshared is a good reason for leaving non-shared overloads, but isn't
>>> code
>>> really supposed to be using shared and not __gshared unless it's
>>> specifically
>>> extern(C)? That being the case, I'd expect shared to be the correct
>>> thing to
>>> use with mutexes normally, and right now, that won't work without a
>>> ton of
>>> casting or adding shared overloads.
>>>
>>> - Jonathan M Davis
>>>
>>
>> I don't know what anything that has to do with shared is supposed or not
>> supposed to do. It is not clear, even to the primary language designers,
>> what shared actually is, so I really can't say anything to this...
>>
>
> Let's please stop this mantra. It buy us nothing but FUD.
>
> One thing is clear - shared is not going away over night. Yes, having
> auto-magic atomic ops for various builtins is discussed, the kind of
> memory model is discussed.
>
> Everything else stays. It's the globally shared data and it has tight
> restrictions on what you can do with it. As such it is (just like
> immutable) is safely passable between threads.

There have been proposals to a completely different meaning of shared 
that has nothing to do with memory models or atomic ops. There is no FUD 
here. Shoehorning shared in its current state into core.sync is just 
plain unreasonable given that nothing has been set in stone about the 
meaning of shared at all yet.

>
>> All I can say is, let's wait with this until we know what shared is
>> going to actually do. We don't want to do something we'll regret later.
>
> Obviously we know what shared is. It's just certain details need
> straightening up. And certainly druntime/std.concurrency should add more
> support for it where meaningful.

We don't know at all what shared is. Have you been following the other 
thread? People have wildly different ideas of what it should do.



I don't see why we should be in a hurry to add shared to core.sync in 
particular. Practically all of the runtime and standard library can't 
work properly with shared anyway, so it's not going to change a whole 
lot. I think it's much more sensible to wait until shared's semantics 
are set in stone and *then* start adapting the runtime and standard library.

I can't stop anyone from just doing this, of course, but it seems like a 
premature change to me.

-- 
Alex Rønne Petersen
alex@lycus.org
http://lycus.org
November 18, 2012
Re: Should core.sync.mutex.Mutex, core.sync.condition.Condition, etc. have all their methods be shared?
11/18/2012 10:36 PM, Alex Rønne Petersen пишет:
[snip]
>> Everything else stays. It's the globally shared data and it has tight
>> restrictions on what you can do with it. As such it is (just like
>> immutable) is safely passable between threads.
>
> There have been proposals to a completely different meaning of shared
> that has nothing to do with memory models or atomic ops. There is no FUD
> here. Shoehorning shared in its current state into core.sync is just
> plain unreasonable given that nothing has been set in stone about the
> meaning of shared at all yet.
>

I'd argue that shared on these objects has nothing to do with memory 
models and atomics. It's about global visibility and access to API that 
is interlocked on OS-level anyway. It works with any sane definition 
that doesn't forget to add that 'shared' implies global visibility.

>>
>>> All I can say is, let's wait with this until we know what shared is
>>> going to actually do. We don't want to do something we'll regret later.
>>
>> Obviously we know what shared is. It's just certain details need
>> straightening up. And certainly druntime/std.concurrency should add more
>> support for it where meaningful.
>
> We don't know at all what shared is. Have you been following the other
> thread? People have wildly different ideas of what it should do.
>

I admit dismissing a lot of proposals from that thread on sight. The 
ones being about shared somehow 'just working' and getting you safe way 
to sharing and the one about auto-magically attaching mutexes to shared 
stuff.

Given now that the thread is mostly spent. If there many good different 
concepts I'd appreciate if you list these here.

> I don't see why we should be in a hurry to add shared to core.sync in
> particular.

> Practically all of the runtime and standard library can't
> work properly with shared anyway, so it's not going to change a whole
> lot.

In part because even things that are supposed to always be shared in 
fact are not working as shared. And that's because nobody have found the 
time to go on and do this work.

Add the fact that say class instances have monitor field when everything 
is TLS by default for me is indication of one thing only - during a move 
to TLS by default and shared a lot of things were not updated 
accordingly because of the lack of time and other issues.

> I think it's much more sensible to wait until shared's semantics
> are set in stone and *then* start adapting the runtime and standard
> library.
>
> I can't stop anyone from just doing this, of course, but it seems like a
> premature change to me.
>
What change in semantics of shared could stop mutex object from making 
sense only as a shared object? I mean the kind of change that can 
realistically can happen not some vague idea.

-- 
Dmitry Olshansky
November 18, 2012
Re: Should core.sync.mutex.Mutex, core.sync.condition.Condition, etc. have all their methods be shared?
On Sunday, November 18, 2012 23:28:03 Dmitry Olshansky wrote:
> What change in semantics of shared could stop mutex object from making
> sense only as a shared object? I mean the kind of change that can
> realistically can happen not some vague idea.

I thought that it was blindingly clear from TDPL and even the online spec what 
shared is for. It marks variables as being shared across threads rather than 
thread local. I don't see how that could possibly be changing. __gshared isn't 
even _mentioned_ in TDPL, and I think that it gets used as much as it does is 
entirely because of some of the current issues with shared's implementation in 
the compiler and the lack of support in druntime and Phobos.

There are definitely questions as to what exactly should happen when a variable 
is marked as shared, but they have to do with what protections you get and 
what conversions are possible. They still don't fundamentally change the fact 
that the point of shared is to share an instance across threads rather than 
have it be thread local like everything is by default.

So, I concur in that I don't see how the semantics of shared could possibly 
not be appropriate for mutexes. I started this thread primarily because I was 
shocked that Mutex, Condition, etc. weren't marked with shared already, and I 
didn't see how they could even be used without it (and apparently, the answer 
to that is that almost everyone cops out and uses __gshared).

- Jonathan M Davis
November 18, 2012
Re: Should core.sync.mutex.Mutex, core.sync.condition.Condition, etc. have all their methods be shared?
To separate an important notion I'm trying to express.

I assume that:

class X{
...
	shared foo(...);
	void bar(...);
...
}

shared X sharedStuff;
X localStuff;

Implies under any viable definition of shared that (including the 
current one and any pending clarifications):

a) foo can be called with sharedStuff in multiple threads at the same 
time. Thus it has to be thread safe (irregardless of possible 
@system/@trusted hackery inside).

b) localStuff on the other hand can called with  bar (and any non-shared 
methods) only in a single thread at any time. If localStuff is defined 
at global scope each thread has its own copy thus this requirement is 
fulfilled kind of automatically.

The major benefits of shared lie in cleaning up the opposite - unshared 
stuff. Having shared enables us to archieve great things we haven't 
looked at yet (maybe because of bugs/issues in the other departments). 
I'll name a few.

We can have lock-less and _safe_ file I/O  by default unlike e.g. C and 
pretty much everybody else. I should have said safe & _faster_  by 
default. This is why I believe D should stop relying on C's I/O as 
anything but fallback. We even had a partially complete design for new 
std.io that does it.

Same goes for memory allocations - lock-less thread-local allocators are 
possible _safely_ and by default (though it would have to go to the OS 
from time to time and likely to lock something here). It's not explored 
and remains as unused potential.

-- 
Dmitry Olshansky
« First   ‹ Prev
1 2
Top | Discussion index | About this forum | D home