Le 2010-01-07 ? 10:00, Andrei Alexandrescu a ?crit :

> What real-world scenarios would require such a member?

I just realized that it might not be necessary, but it all this depends on the way shared propagates, so I'll need to clear that up first. Let's say I have this class:

	class A {
		int x;
		shared int y;
	}

	auto a1 = new A;
	auto a2 = new shared(A);

Here's what I'd expect from this (from what I explained earlier and from how const propagates):

	a1.x  is not shared  synchronization has no effect (a1 is not shared)
	a1.y  is shared  only accessible through atomic ops
	a2.x  is shared  synchronization is needed for access
	a2.y  is shared  only accessible through atomic ops

The issue is with a1.y. It being always only accessible through atomic ops could be bad if what I want to implement is a lock-free algorithm:

	class A {
		int x;
		shared int y;

		void increment() shared {
			synchronized { ++x; }
			++atomic(y);
		}
		void increment() {
			++x;
			++atomic(y);
		}
	}

It's good that the lock-free algorithm is forced to use atomic ops, but it's wasteful when the object itself is not shared. So the above could be changed like this:

	a1.x  is not shared  synchronization has no effect (a1 is not shared)
	a1.y  is not shared  no need for atomic ops, synchronization has no effect
	a2.x  is shared  synchronization is needed for access
	a2.y  is shared  only accessible through atomic ops

Now I've made a1.y not shared because its parent is not shared, even though this is counterintuitive since y is clearly marked as shared in the class. But now you can write your algorithm like that:

	class A {
		int x;
		shared int y;

		void incrementY() shared {
			synchronized { ++x; }
			++atomic(y);
		}
		void incrementY() {
			++x;
			++y;
		}
	}

If that's what we want, then we don't need another qualifier.


-- 
Michel Fortin
michel.fortin at michelf.com
http://michelf.com/

On Jan 7, 2010, at 9:27 AM, Michel Fortin wrote:
> 
> It's good that the lock-free algorithm is forced to use atomic ops, but it's wasteful when the object itself is not shared. So the above could be changed like this:
> 
> 	a1.x  is not shared  synchronization has no effect (a1 is not shared)
> 	a1.y  is not shared  no need for atomic ops, synchronization has no effect
> 	a2.x  is shared  synchronization is needed for access
> 	a2.y  is shared  only accessible through atomic ops

If this is the case then references would have to be exempt though.  A local field can have a reference to a shared class.  I'm not convinced that you'd often have lock-free algorithms that would also be used in a TLS scenario though.

Yes, shared is a type qualifier. For the most part it behaves like immutable. Note that you can define an immutable member of an otherwise mutable type.

Andrei

Steve Schveighoffer wrote:
> I'm not quite understanding everything being discussed here, because I came into this discussion quite late, but is shared going to remain a type modifier?
> 
> People are mentioning shared as a storage class on member variables, but I thought shared was going to be like const, where it transitively affects all members.  I thought a shared member function is one which the 'this' pointer is marked as shared, similar to a const function.  If that's the case, aren't all unmarked variables in a class shared?  Why the need to mark them?  And how do you have a 'partially shared' class where some members are marked shared and some are not?
> 
> I'm really confused :)
> 
> -Steve
> 
> 
> 
> 
> _______________________________________________
> dmd-concurrency mailing list
> dmd-concurrency at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/dmd-concurrency

It's probably needless complexity, but the only thingthat would need to change here is "head shared.". The tail would remain shared for shared and non-shared instances.  Then what you suggest would work for eerything.

Sent from my iPhone

On Jan 7, 2010, at 9:33 AM, Sean Kelly <sean at invisibleduck.org> wrote:

> On Jan 7, 2010, at 9:27 AM, Michel Fortin wrote:
>>
>> It's good that the lock-free algorithm is forced to use atomic ops, but it's wasteful when the object itself is not shared. So the above could be changed like this:
>>
>>    a1.x  is not shared  synchronization has no effect (a1 is not
>> shared)
>>    a1.y  is not shared  no need for atomic ops, synchronization has
>> no effect
>>    a2.x  is shared  synchronization is needed for access
>>    a2.y  is shared  only accessible through atomic ops
>
> If this is the case then references would have to be exempt though.
> A local field can have a reference to a shared class.  I'm not
> convinced that you'd often have lock-free algorithms that would also
> be used in a TLS scenario though.
> _______________________________________________
> dmd-concurrency mailing list
> dmd-concurrency at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/dmd-concurrency

----- Original Message ----

> From: Andrei Alexandrescu <andrei at erdani.com>
> Yes, shared is a type qualifier. For the most part it behaves like immutable.
> Note that you can define an immutable member of an otherwise mutable type.
> 

Right, but this to me makes sense -- I want to define something that is always immutable.  A piece of data becomes immutable the second you declare it is.

But when I mark a class member shared, it does not truly become shared by declaring it that way -- allocating an instance of such a class does not make its shared members available to other threads.  Compare that to a global, which is immediately shared.  I think this is going to have to be really explained well.  As a member storage class, shared really should be called "shareable".

Despite that, I think I get it.  The address to a shared member could be passed to other threads, even though the full object is not, is that correct?  I can see use cases for that.  From that point of view, it's not so important that you are declaring something is partially shared, but that it's partially thread-local.  That is, something that isn't marked shared can be used in a shared or unshared capacity, but something that is marked even partially shared you only ever want to use in a shared manner -- you'd never use such a class as a thread-local-only object.

OTOH, it's good to hear shared is going to remain a type modifier, because I need that in order to distinguish shared and non-shared array appends.

-Steve

Steve Schveighoffer wrote:
> ----- Original Message ----
> 
>> From: Andrei Alexandrescu <andrei at erdani.com> Yes, shared is a type
>> qualifier. For the most part it behaves like immutable. Note that
>> you can define an immutable member of an otherwise mutable type.
>> 
> 
> Right, but this to me makes sense -- I want to define something that is always immutable.  A piece of data becomes immutable the second you declare it is.
> 
> But when I mark a class member shared, it does not truly become shared by declaring it that way -- allocating an instance of such a class does not make its shared members available to other threads. Compare that to a global, which is immediately shared.  I think this is going to have to be really explained well.  As a member storage class, shared really should be called "shareable".
> 
> Despite that, I think I get it.  The address to a shared member could be passed to other threads, even though the full object is not, is that correct?

Of course, plus the more frequent case is:

class SList { ... }
struct A { int x; shared SList lst; }


Andrei

----- Original Message ----

> From: Andrei Alexandrescu <andrei at erdani.com>
> Steve Schveighoffer wrote:
> > Despite that, I think I get it.  The address to a shared member could be passed to other threads, even though the full object is not, is that correct?
> 
> Of course, plus the more frequent case is:
> 
> class SList { ... }
> struct A { int x; shared SList lst; }

This to me is a more normal usage.  The data that is shared is really the list, not the pointer to the list (but the reference is also shared because of the deficiencies of the type syntax).

To be more specific, the following case is odd to me because you are sharing data in the same memory segment as unshared data:

struct B
{
   int x;
   shared int y;
}

And in fact, declaring a variable of type B in a function puts shared data on the stack!

Hm... this brings up an interesting point.  If you want to say that a reference to a shared class is a thread-local, how do you do that?  For instance, in your struct A, you are most likely not going to share the actual reference lst (i.e. &a.lst), just what lst points to.  However, the compiler is going to assume you are sharing lst because that's what you declared.  So every access to lst is going to require special memory barriers (and technically wasted cycles).  Is there going to be a way around this?  With const it was not as important because const is a compile-time concept, but now the syntax deficiency is creeping into runtime and hurting performance.  I.e. are we going to get an equivalent "Rebindable" type constructor?

In fact, any shared class reference on the stack erroneously will create memory barriers around the stack variable itself -- you shouldn't be sharing any stack data.

-Steve

On Jan 7, 2010, at 12:34 PM, Steve Schveighoffer wrote:

> ----- Original Message ----
> 
>> From: Andrei Alexandrescu <andrei at erdani.com>
>> Steve Schveighoffer wrote:
>>> Despite that, I think I get it.  The address to a shared member could be passed to other threads, even though the full object is not, is that correct?
>> 
>> Of course, plus the more frequent case is:
>> 
>> class SList { ... }
>> struct A { int x; shared SList lst; }
> 
> This to me is a more normal usage.  The data that is shared is really the list, not the pointer to the list (but the reference is also shared because of the deficiencies of the type syntax).

Won't we be able to do:

    struct A { int x; shared(SList) lst; }

I'd think it would be pretty common to want a local reference to shared data.  Then you wouldn't have to pay for the atomic read required if the reference itself is shared.

> To be more specific, the following case is odd to me because you are sharing data in the same memory segment as unshared data:
> 
> struct B
> {
>   int x;
>   shared int y;
> }
> 
> And in fact, declaring a variable of type B in a function puts shared data on the stack!

Yeah, a fully shared field inside a non-shared class doesn't sound right.  The smoke test for me is that I think about whether it would work with per-thread GCs.  If one thread has a reference to "shared" data that actually lives in another thread's heap or stack, there's a dangling reference issue if the thread terminates (at least in theory--the heap data could easily be handed off to the shared GC instead, but let's pretend this is impossible).

> Hm... this brings up an interesting point.  If you want to say that a reference to a shared class is a thread-local, how do you do that?  For instance, in your struct A, you are most likely not going to share the actual reference lst (i.e. &a.lst), just what lst points to.  However, the compiler is going to assume you are sharing lst because that's what you declared.  So every access to lst is going to require special memory barriers (and technically wasted cycles).  Is there going to be a way around this?  With const it was not as important because const is a compile-time concept, but now the syntax deficiency is creeping into runtime and hurting performance.  I.e. are we going to get an equivalent "Rebindable" type constructor?

See my example of "shared (Something) p" above.  This is consistent with how const and such work anyway.

> In fact, any shared class reference on the stack erroneously will create memory barriers around the stack variable itself -- you shouldn't be sharing any stack data.

Right.

Le 2010-01-07 ? 15:34, Steve Schveighoffer a ?crit :

> In fact, any shared class reference on the stack erroneously will create memory barriers around the stack variable itself -- you shouldn't be sharing any stack data.

I'm not sure you should never be sharing stack data. Stack data can already be allocated on the heap with closures. If you're going to give that closure to another thread, the data it uses needs to be shared.


-- 
Michel Fortin
michel.fortin at michelf.com
http://michelf.com/

----- Original Message ----

> From: Sean Kelly <sean at invisibleduck.org>
> 
> On Jan 7, 2010, at 12:34 PM, Steve Schveighoffer wrote:
> 
> > ----- Original Message ----
> > 
> >> From: Andrei Alexandrescu Steve Schveighoffer wrote:
> >>> Despite that, I think I get it.  The address to a shared member could be passed to other threads, even though the full object is not, is that correct?
> >> 
> >> Of course, plus the more frequent case is:
> >> 
> >> class SList { ... }
> >> struct A { int x; shared SList lst; }
> > 
> > This to me is a more normal usage.  The data that is shared is really the
> list, not the pointer to the list (but the reference is also shared because of the deficiencies of the type syntax).
> 
> Won't we be able to do:
> 
>     struct A { int x; shared(SList) lst; }

currently, const(T) applies to *all* of T, even the reference itself.  There is no syntax to apply const only to what T references if T is a class.  There was a huge discussion about this in the early days of the current const system.

There is a type called Rebindable(T) in std.typecons which essentially is equivalent to only applying const or immutable to the referenced item.  Although last I remember there were some problems with it because it used opDot.

> I'd think it would be pretty common to want a local reference to shared data. Then you wouldn't have to pay for the atomic read required if the reference itself is shared.

I think more than const, we need something like this for shared.

> 
> > To be more specific, the following case is odd to me because you are sharing
> data in the same memory segment as unshared data:
> > 
> > struct B
> > {
> >   int x;
> >   shared int y;
> > }
> > 
> > And in fact, declaring a variable of type B in a function puts shared data on
> the stack!
> 
> Yeah, a fully shared field inside a non-shared class doesn't sound right.  The smoke test for me is that I think about whether it would work with per-thread GCs.  If one thread has a reference to "shared" data that actually lives in another thread's heap or stack, there's a dangling reference issue if the thread terminates (at least in theory--the heap data could easily be handed off to the shared GC instead, but let's pretend this is impossible).

Yeah, allowing this seems to imply that the global heap will have to be scanned for local heap collections:

class C
{
   shared(int)* ptr1;
   int *ptr2;
}

I assume that marking a single member shared makes C get allocated on the global heap.

If ptr1 points to a global-heap variable, and ptr2 points to a thread-local-heap variable, then you have to scan the global heap in order to collect the memory ptr2 is pointing to.

> > Hm... this brings up an interesting point.  If you want to say that a
> reference to a shared class is a thread-local, how do you do that?  For instance, in your struct A, you are most likely not going to share the actual reference lst (i.e. &a.lst), just what lst points to.  However, the compiler is going to assume you are sharing lst because that's what you declared.  So every access to lst is going to require special memory barriers (and technically wasted cycles).  Is there going to be a way around this?  With const it was not as important because const is a compile-time concept, but now the syntax deficiency is creeping into runtime and hurting performance.  I.e. are we going to get an equivalent "Rebindable" type constructor?
> 
> See my example of "shared (Something) p" above.  This is consistent with how const and such work anyway.

No it is not.  If you have const(T) x, you can never rebind x, no matter what type is inside the parens.

-Steve