On Thursday, October 10, 2013 19:08:02 Andrei Alexandrescu wrote:
> On 10/10/13 7:04 PM, Jonathan M Davis wrote:
> > On Thursday, October 10, 2013 18:21:52 Andrei Alexandrescu wrote:
> >> You can't EVER expect to obtain all of that magic by plastering "shared" on top of your type.
> > 
> > It works just fine with the idiom that I described where you protect the usage of the object with a lock, cast it to thread-local to do stuff on it, and then release the lock (making sure that no thread-local references remain).
> TDPL describes how synchronized automatically peels off the "shared" off of direct members of the object. Unfortunately that feature is not yet implemented.

I think that it's generally overkill to create a whole class just to protect one shared object. Simply protecting it when you need to with mutex or synchronized block should be enough, and I don't see the cast as being all that big a deal. What TDPL describes is essentially the same except that the compiler does it automatically, but it forces you to create a class just for that. And as Michel points out, removing shared on a single level is frequently not all enough.

I think that synchronized classes have their place, but I also think that they're overkill for most situations. Why should I have to create a class just so that I can make a variable thread-local so that I can operate on it? I can do the same with a cast without the extra overhead and a lot more flexibly, since a cast doesn't have the single-level restriction that a synchronized class does.

- Jonathan M Davis

On 2013-10-11 03:05, Jonathan M Davis wrote:

> I'm not disagreeing with how shared works. I'm disagreeing with the idea that
> it's not supposed to be normal to cast shared away when operating on shared
> objects. I expect that the most common idiom for dealing with shared is to
> protect it with a lock, cast it to thread-local, do whatever you're going to
> do with it, make sure that there are no thread-local references to it once
> you're done operating on it, and then release the lock. e.g.
>
> synchronized
> {
>   auto tc = cast(T)mySharedT;
>   tc.memberFunc();
>   doStuff(tc);
>   //no thread-local references to tc other than tc should
>   //exist at this point.
> }

With Michel Fortin's proposal I think the above could work without a cast, if doStuff is pure function.

http://michelf.ca/blog/2012/mutex-synchonization-in-d/

-- 
/Jacob Carlborg

On Wednesday, 9 October 2013 at 09:01:12 UTC, Walter Bright wrote:
> On 10/9/2013 1:59 AM, JR wrote:
>> On Wednesday, 9 October 2013 at 02:22:35 UTC, Andrei Alexandrescu wrote:
>>> * Get Robert Schadek's precise GC in. Walter and I have become 101% convinced
>>> a precise GC is the one way to go about GC.
>>
>> An orthogonal question, but is Lucarella's CDGC (still) being ported? There's
>> nothing mutually exclusive between a precise and a concurrent gc, no?
>
> I thought that got stuck on the problem that the linux system libraries had some sort of threading problem with them.

This is not really what's stopping the porting, is a problem, but an independent one. My idea was to port the GC as it is in Tango, and then see how to overcome its limitations.

The problem is it's very hard for me to dedicate time to this porting effort. The decision to make the port happen is there, I just how to figure out how and when. I'll keep you posted when I have news.

On 2013-10-11 02:51, Jonathan M Davis wrote:

> At this point, I don't see how we can have thread-local pools unless casting
> to and from shared has hooks for managing that. Otherwise, it's far too likely
> that an object is going to be in the wrong pool, because it's being used as
> shared when it was constructed as thread-local or vice versa. And we may need
> some sort of hook with std.concurrency.send which understands that the object
> being sent is being transferred from one thread to another and would tell the
> GC to migrate the object from one pool to another (though to do that, it would
> probably have to not be typed as shared but rather as thread-local, which
> would jive better with what you're talking about doing with std.concurrency).
>
> Certainly, with how shared currently works, it's hard to see how we could get
> away with having thread-local GC pools as great as that would be. So, if we
> want that, something about how shared works is going to have to change.

A simple solution to the "hook" would be to pass a dummy type indicating the object should be transferred:

struct Transfer { }

send(tid, foo, Transfer());

"Transfer" would be defined in std.concurrency.

-- 
/Jacob Carlborg

Am Thu, 10 Oct 2013 22:04:16 -0400
schrieb "Jonathan M Davis" <jmdavisProg@gmx.com>:

> most D programmers seem to describe when talking about shared is simply using __gshared with normal types, not even using shared, let alone using it with types specifically designed to function as shared. So, the most common approach at this point in D seems to be to avoid shared entirely.

One important reason for this is that the types in core.sync still aren't shared.

--------
Mutex myMutex; //WRONG, myMutex is in TLS
shared Mutex myMutex; //WRONG, can't call .lock, new
__gshared Mutex myMutex; //Can't be used in @safe code...

//shared Mutex + casting to unshared when accessing: Can't be used in //@safe code

See also: http://forum.dlang.org/thread/mailman.2017.1353214033.5162.digitalmars-d@puremagic.com?page=2#post-mailman.2037.1353278884.5162.digitalmars-d:40puremagic.com

Sean Kelly:
"I tried this once and it cascaded to requiring modifications of
various definitions on core.sys.posix to add a "shared" qualifier, and
since I wasn't ready to do that I rolled back the changes.  I guess the
alternative would be to have a shared equivalent for every operation
that basically just casts away shared and then calls the non-shared
function, but that's such a terrible design I've been resisting it."

On Tuesday, 8 October 2013 at 22:37:28 UTC, Walter Bright wrote:
> On 10/8/2013 9:22 AM, Dicebot wrote:
>> It is simply "@nogc" which is lacking but absolutely
>> mandatory.
>
> Adding @nogc is fairly simple. The trouble, though, is (like purity) it is transitive. Every function an @nogc function calls will also have to be @nogc. This will entail a great deal of work updating phobos/druntime to add those annotations.

A very naive question but is there no way of analysing the subfunctions to check their purity or lack of GC use rather than having to annotate everything? D does need to be a little wary of becoming too heavily annotated.

11-Oct-2013 05:21, Andrei Alexandrescu пишет:
> On 10/10/13 5:36 PM, Jonathan M Davis wrote:
>> On Thursday, October 10, 2013 10:55:49 Andrei Alexandrescu wrote:
>>> On 10/10/13 12:33 AM, Jonathan M Davis wrote:
>>>> I honestly don't think we can solve it a different way without
>>>> completely
>>>> redesigning shared. shared is specifically designed such that you
>>>> have to
>>>> either cast it way to do anything with it
>>>
>>> no
>>>
>>>> or write all of your code to
>>>> explicitly work with shared, which is not something that generally
>>>> makes
>>>> sense to do unless you're creating a type whose only value is in being
>>>> shared across threads.
>>>
>>> yes
>>
>> Really? Do you honestly expect the average use of shared to involve
>> creating
>> structs or classes which are designed specifically to be used as shared?
>
> Yes. Data structures that can be shared are ALWAYS designed specifically
> for sharing, unless of course it's a trivial type like int.

This. And exactly the same for immutable. It's interesting how folks totally expect complex types (like containers) to meaningfully work with all 3 qualifiers.

> Sharing
> means careful interlocking and atomic operations and barriers and stuff.
> You can't EVER expect to obtain all of that magic by plastering "shared"
> on top of your type.
>

Yup.

>
> Andrei
>


-- 
Dmitry Olshansky

On 11/10/13 16:32, Dmitry Olshansky wrote:
> This. And exactly the same for immutable. It's interesting how folks totally
> expect complex types (like containers) to meaningfully work with all 3 qualifiers.

It's not so much that we expect it, as that we might expect that standard library types would _have the appropriate design work put in_ so that they would "just work" with these qualifiers.  (Admittedly shared is a bit of a special case right now that probably needs more work before support is rolled out.)

If you tell me that's an unreasonable expectation then fair enough, but it feels pretty bad if e.g. library-implemented number types (big integers or floats, rationals, complex numbers, ...) can't from a user perspective behave exactly like their built-in counterparts.

11-Oct-2013 18:46, Joseph Rushton Wakeling пишет:
> On 11/10/13 16:32, Dmitry Olshansky wrote:
>> This. And exactly the same for immutable. It's interesting how folks
>> totally
>> expect complex types (like containers) to meaningfully work with all 3
>> qualifiers.
>
> It's not so much that we expect it, as that we might expect that
> standard library types would _have the appropriate design work put in_
> so that they would "just work" with these qualifiers.  (Admittedly
> shared is a bit of a special case right now that probably needs more
> work before support is rolled out.)

It can't - it's like expecting 37 to modify an become 76.
Simply put built-ins are special. Imagine a ref-counted type - how would you copy it (and increment a count) with bit-wise immutability?
It simply doesn't make sense. (Yes, one can keep count elsewhere e.g. in a global hash-table).

More importantly there is little incentive to make immutable stuff ref-counted, especially COW-types. In this sense BigInt simply doesn't work with immutable by design, if need be one can make FixedBigInt that doesn't include COW, doesn't support read-modify-write and mixes well with BigInt.

Immutable works best as static data and/or as snapshot style data structures. It's tables, strings and some unique stuff that gets frozen and published at a certain point (usually at start up).

It makes a lot less sense at local scope aside from aesthetic beauty as there is plenty of invariants to check, and bitwise immutability is a minority of that.

I would even suggest to adopt a convention for a pair of freeze/thaw methods for any UDT that give you a deep copy of object this is made for mutation (thaw on immutable object) or immutable (freeze mutable).

>
> If you tell me that's an unreasonable expectation then fair enough, but
> it feels pretty bad if e.g. library-implemented number types (big
> integers or floats, rationals, complex numbers, ...) can't from a user
> perspective behave exactly like their built-in counterparts.

No magic paint would automatically expel reference count from the struct's body. With shared it's even more obvious.

In general user defined type has to be designed with one of 3 major use cases in mind: local, immutable, shared.


-- 
Dmitry Olshansky

On Friday, 11 October 2013 at 01:05:19 UTC, Jonathan M Davis wrote:
> On Friday, October 11, 2013 02:08:16 Sean Kelly wrote:
>> 
>> Shared data needs to be
>> treated differently, explicitly, or things go downhill fast.
>
> I'm not disagreeing with how shared works. I'm disagreeing with the idea that
> it's not supposed to be normal to cast shared away when operating on shared
> objects. I expect that the most common idiom for dealing with shared is to
> protect it with a lock, cast it to thread-local, do whatever you're going to
> do with it, make sure that there are no thread-local references to it once
> you're done operating on it, and then release the lock.

The thing with locks is that you need to use the same lock for all accesses to a set of mutated data or atomicity isn't guaranteed.  And if you're locking externally you don't know what might change inside a class during a method call, so you have to use the same lock for all operations on that object, regardless of what you're doing.  At that point you may as well just synchronize on the class itself and be done with it.  So sure, it saves you from having to define shared or synchronized methods, but I don't think this should be how we want to write concurrent code in D.