On Friday, October 11, 2013 02:08:16 Sean Kelly wrote:
> On Thursday, 10 October 2013 at 23:33:27 UTC, Jonathan M Davis
> 
> wrote:
> > Yeah. The only times that something is going to accept shared
> > is when it was
> > specifically designed to work as shared (which most code
> > isn't), or if it's
> > templated and the template happens to work with shared. Regular
> > functions just
> > aren't going to work with shared without casting away shared,
> > because that
> > would usually mean either templating everything or duplicating
> > functions all over the place.
> 
> I think that's pretty reasonable though. Shared data needs to be treated differently, explicitly, or things go downhill fast.

I'm not disagreeing with how shared works. I'm disagreeing with the idea that it's not supposed to be normal to cast shared away when operating on shared objects. I expect that the most common idiom for dealing with shared is to protect it with a lock, cast it to thread-local, do whatever you're going to do with it, make sure that there are no thread-local references to it once you're done operating on it, and then release the lock. e.g.

synchronized
{
 auto tc = cast(T)mySharedT;
 tc.memberFunc();
 doStuff(tc);
 //no thread-local references to tc other than tc should
 //exist at this point.
}

That works perfectly fine and makes it so that shared objects are clearly delineated from thread-local ones by the type system, but it does require casting, and it requires that you make sure that the object is not misused while it's not being treated as shared.

The only real alternative to that is to create types which are designed to be operated on as shared, but I would expect that to be the rare case rather than the norm, because that requires creating new types just for sharing across threads rather than using the same types that you use in your thread-local code, and I don't expect programmers to want to do that in the average case.

However, from what I've seen, at the moment, the most typical reaction is to simply use __gshared, which is the least safe of the various options. So, people need to be better educated and/or we need figure out a different design for shared.

- Jonathan M Davis

On 11/10/13 02:30, Simen Kjaeraas wrote:
> TLDR: Do not use inout(T). Fix BigInt.

Yes, that's my feeling too.

Even if you do add inout (or "in", which also works in terms of allowing qualified BigInts) to std.math.abs, you immediately run back into problems ... with BigInt.

On 10/10/13 4:33 PM, Jonathan M Davis wrote:
> I just don't see how you could avoid casting when passing ownership of an
> object from one thread to another without having a way to pass an object
> across threads without having to make it shared or immutable to pass it.

By using restricted library types.

Andrei

On 10/10/13 5:36 PM, Jonathan M Davis wrote:
> On Thursday, October 10, 2013 10:55:49 Andrei Alexandrescu wrote:
>> On 10/10/13 12:33 AM, Jonathan M Davis wrote:
>>> I honestly don't think we can solve it a different way without completely
>>> redesigning shared. shared is specifically designed such that you have to
>>> either cast it way to do anything with it
>>
>> no
>>
>>> or write all of your code to
>>> explicitly work with shared, which is not something that generally makes
>>> sense to do unless you're creating a type whose only value is in being
>>> shared across threads.
>>
>> yes
>
> Really? Do you honestly expect the average use of shared to involve creating
> structs or classes which are designed specifically to be used as shared?

Yes. Data structures that can be shared are ALWAYS designed specifically for sharing, unless of course it's a trivial type like int. Sharing means careful interlocking and atomic operations and barriers and stuff. You can't EVER expect to obtain all of that magic by plastering "shared" on top of your type.


Andrei

On 11/10/13 02:44, Andrei Alexandrescu wrote:
> I'll look into this soon.

That would be fantastic, thank you very much.

Any chance you could ask Don Clugston to get in touch with me about these issues?  std.bigint is his, and I know he was in touch with David Simcha about std.rational.  I don't have his email address, as he (understandably) has a fake email address set up as his reply-to here.

On Thursday, October 10, 2013 18:21:52 Andrei Alexandrescu wrote:
> On 10/10/13 5:36 PM, Jonathan M Davis wrote:
> > On Thursday, October 10, 2013 10:55:49 Andrei Alexandrescu wrote:
> >> On 10/10/13 12:33 AM, Jonathan M Davis wrote:
> >>> or write all of your code to
> >>> explicitly work with shared, which is not something that generally makes
> >>> sense to do unless you're creating a type whose only value is in being
> >>> shared across threads.
> >> 
> >> yes
> > 
> > Really? Do you honestly expect the average use of shared to involve creating structs or classes which are designed specifically to be used as shared?
> Yes. Data structures that can be shared are ALWAYS designed specifically for sharing, unless of course it's a trivial type like int. Sharing means careful interlocking and atomic operations and barriers and stuff. You can't EVER expect to obtain all of that magic by plastering "shared" on top of your type.

It works just fine with the idiom that I described where you protect the usage of the object with a lock, cast it to thread-local to do stuff on it, and then release the lock (making sure that no thread-local references remain). Aside from the necessity of the cast, this is exactly what is typically done in every C++ code base that I've ever seen - and that's with complicated types and not just simple stuff like int. e.g.

synchronized
{
 auto tc = cast(T)mySharedT;
 tc.memberFunc();
 doStuff(tc);
 //no thread-local references to tc other than tc should
 //exist at this point.
}

I agree that designing types specifically to function as shared objects has its place (e.g. concurrent containers), but in my experience, that is very much the rare case, not the norm, and what most D programmers seem to describe when talking about shared is simply using __gshared with normal types, not even using shared, let alone using it with types specifically designed to function as shared. So, the most common approach at this point in D seems to be to avoid shared entirely.

- Jonathan M Davis

On 10/10/13 7:04 PM, Jonathan M Davis wrote:
> On Thursday, October 10, 2013 18:21:52 Andrei Alexandrescu wrote:
>> You can't EVER expect to obtain all of that magic by plastering "shared"
>> on top of your type.
>
> It works just fine with the idiom that I described where you protect the usage
> of the object with a lock, cast it to thread-local to do stuff on it, and then
> release the lock (making sure that no thread-local references remain).

TDPL describes how synchronized automatically peels off the "shared" off of direct members of the object. Unfortunately that feature is not yet implemented.


Andrei

On 2013-10-11 02:08:02 +0000, Andrei Alexandrescu <SeeWebsiteForEmail@erdani.org> said:

> On 10/10/13 7:04 PM, Jonathan M Davis wrote:
>> On Thursday, October 10, 2013 18:21:52 Andrei Alexandrescu wrote:
>>> You can't EVER expect to obtain all of that magic by plastering "shared"
>>> on top of your type.
>> 
>> It works just fine with the idiom that I described where you protect the usage
>> of the object with a lock, cast it to thread-local to do stuff on it, and then
>> release the lock (making sure that no thread-local references remain).
> 
> TDPL describes how synchronized automatically peels off the "shared" off of direct members of the object. Unfortunately that feature is not yet implemented.

That "direct member" limitation makes the feature pretty much worthless. It's rare you want to protect a single integer behind a mutex, generally you protect data structures like arrays or trees, and those always have indirections.

You could loosen it up a bit by allowing pure functions to use the member. But you must then make sure those functions won't escape a pointer to the protected structure through one of its argument, or the return value. That won't work with something like std.range.front on an array.

Anyway, that whole concept of synchronized class is a deadlock honeypot. It should be scrapped altogether.

-- 
Michel Fortin
michel.fortin@michelf.ca
http://michelf.ca

On 10/10/13 7:37 PM, Michel Fortin wrote:
> On 2013-10-11 02:08:02 +0000, Andrei Alexandrescu
> <SeeWebsiteForEmail@erdani.org> said:
>
>> On 10/10/13 7:04 PM, Jonathan M Davis wrote:
>>> On Thursday, October 10, 2013 18:21:52 Andrei Alexandrescu wrote:
>>>> You can't EVER expect to obtain all of that magic by plastering
>>>> "shared"
>>>> on top of your type.
>>>
>>> It works just fine with the idiom that I described where you protect
>>> the usage
>>> of the object with a lock, cast it to thread-local to do stuff on it,
>>> and then
>>> release the lock (making sure that no thread-local references remain).
>>
>> TDPL describes how synchronized automatically peels off the "shared"
>> off of direct members of the object. Unfortunately that feature is not
>> yet implemented.
>
> That "direct member" limitation makes the feature pretty much worthless.
> It's rare you want to protect a single integer behind a mutex, generally
> you protect data structures like arrays or trees, and those always have
> indirections.

It's not about a single integer, it's about multiple flat objects. True, many cases do involve indirections.

> You could loosen it up a bit by allowing pure functions to use the
> member. But you must then make sure those functions won't escape a
> pointer to the protected structure through one of its argument, or the
> return value. That won't work with something like std.range.front on an
> array.
>
> Anyway, that whole concept of synchronized class is a deadlock honeypot.
> It should be scrapped altogether.

People still use it.


Andrei

On Thursday, October 10, 2013 19:43:40 Andrei Alexandrescu wrote:
> On 10/10/13 7:37 PM, Michel Fortin wrote:
> > Anyway, that whole concept of synchronized class is a deadlock honeypot. It should be scrapped altogether.
> 
> People still use it.

To some extent in that some folks used synchronized functions, but synchronized classes haven't been implemented at all. What we have right now is basically just a copy of Java's synchronized function feature. Synchronized classes aren't drastically different, but whatever nuances come with the difference are completely unrealized at this point.

I think that synchronized classes have their uses, but I'd honestly still be inclined to use them sparingly. It's frequently too much to lock a whole object at once rather than the single member or group of members that actually need the lock.

- Jonathan M Davis