I think there will be no more Thread class (sorry, Sean :o)). Just before starting this list we were discussing creating a small hierarchy that integrates threads, processes, and remote processes. The basic idea of that hierarchy is, again, that some users should be able to manipulate a concurrent entity without knowing whether it's a thread or a process.

Andrei

Graham St Jack wrote:
> 
>>>> I'm hoping this library-level trick will be enough to allow most of the classic synchronization mechanisms to be used in D 2.0.
>>>> 
>>>>> Re synchronized containers - I don't like the idea at all. That is going down the path of having many shared objects, which is notoriously difficult to get right, especially for non-experts. IMO, shared objects should be small in number, and serve as boundaries between threads, which otherwise play in their own separate sand-pits.
>>>>> 
>>>> A good pathological but functionally correct example is the Thread class in core.thread.  Thread instances should really be labeled as shared, but I know that when this happens the compiler will throw a fit.  The thing is, while some of the methods are already synchronized, there are quite a few which are not and don't need to be and neither do they need the variables they access to be made lock-free.  Even weirder is:
>>>> 
>>> Well I think Thread is a threading primitive so we shouldn't expect to be implementable without some oddities.
>>> 
>>
>> Sounds good.  Hopefully, it will be uniquely horrifying.
>> _______________________________________________
>> 
> One aspect of the Thread class that I find really annoying is that its name is of type char[] instead of string. Can we change this please? Also, it would be nice to have additional constructors that take a name.
> 
> I include thread names in my logs, so all my threads have names. This
> means I have code like this:
>     thread.name = "name".dup;
> in the constructors of all my active classes, which is nasty.
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> dmd-concurrency mailing list
> dmd-concurrency at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/dmd-concurrency

Le 2010-01-04 ? 22:49, Sean Kelly a ?crit :

>    class A {
>        void fnA() synchronized { fnB(); } // 1
>        private void fnB() synchronized {}
>    }
> 
>    void mutateA( A val ) {
>        val.fnB(); // 2
>    }
> 
> At 1, the compiler can easily tell that the lock is already held so it doesn't acquire the lock again.  At 2 the lock is (probably) not held, so it's acquired again.

To avoid that "lock is probably not held" situation, we could allow the standalone function to be declared like this:

	void mutateA(synchronized A val) {
	    val.fnB();
	}

Now 'val' can be locked at the call site, if necessary.

-- 
Michel Fortin
michel.fortin at michelf.com
http://michelf.com/

Sean's code has an error: mutateA takes an A (as opposed to a shared A) so no synchronization is assumed or necessary.

Again: non-shared objects _are_ thread-local objects.


Andrei

Michel Fortin wrote:
> Le 2010-01-04 ? 22:49, Sean Kelly a ?crit :
> 
>>    class A {
>>        void fnA() synchronized { fnB(); } // 1
>>        private void fnB() synchronized {}
>>    }
>>
>>    void mutateA( A val ) {
>>        val.fnB(); // 2
>>    }
>>
>> At 1, the compiler can easily tell that the lock is already held so it doesn't acquire the lock again.  At 2 the lock is (probably) not held, so it's acquired again.
> 
> To avoid that "lock is probably not held" situation, we could allow the standalone function to be declared like this:
> 
> 	void mutateA(synchronized A val) {
> 	    val.fnB();
> 	}
> 
> Now 'val' can be locked at the call site, if necessary.
> 

I think the thread class should remain in Druntime in some form, and that this new API should live in Phobos.  I'd expect the Druntime thread class to see little explicit use, but every once in a while there'll probably be someone that wants to do classic multithreading.

On Jan 4, 2010, at 8:49 PM, Andrei Alexandrescu wrote:

> I think there will be no more Thread class (sorry, Sean :o)). Just before starting this list we were discussing creating a small hierarchy that integrates threads, processes, and remote processes. The basic idea of that hierarchy is, again, that some users should be able to manipulate a concurrent entity without knowing whether it's a thread or a process.
> 
> Andrei
> 
> Graham St Jack wrote:
>>>>> I'm hoping this library-level trick will be enough to allow most of the classic synchronization mechanisms to be used in D 2.0.
>>>>> 
>>>>>> Re synchronized containers - I don't like the idea at all. That is going down the path of having many shared objects, which is notoriously difficult to get right, especially for non-experts. IMO, shared objects should be small in number, and serve as boundaries between threads, which otherwise play in their own separate sand-pits.
>>>>>> 
>>>>> A good pathological but functionally correct example is the Thread class in core.thread.  Thread instances should really be labeled as shared, but I know that when this happens the compiler will throw a fit.  The thing is, while some of the methods are already synchronized, there are quite a few which are not and don't need to be and neither do they need the variables they access to be made lock-free.  Even weirder is:
>>>>> 
>>>> Well I think Thread is a threading primitive so we shouldn't expect to be implementable without some oddities.
>>>> 
>>> 
>>> Sounds good.  Hopefully, it will be uniquely horrifying.
>>> _______________________________________________
>>> 
>> One aspect of the Thread class that I find really annoying is that its name is of type char[] instead of string. Can we change this please? Also, it would be nice to have additional constructors that take a name.
>> I include thread names in my logs, so all my threads have names. This means I have code like this:
>>    thread.name = "name".dup;
>> in the constructors of all my active classes, which is nasty.
>> ------------------------------------------------------------------------
>> _______________________________________________
>> dmd-concurrency mailing list
>> dmd-concurrency at puremagic.com
>> http://lists.puremagic.com/mailman/listinfo/dmd-concurrency
> _______________________________________________
> dmd-concurrency mailing list
> dmd-concurrency at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/dmd-concurrency

On Jan 4, 2010, at 8:47 PM, Graham St Jack wrote:

> One aspect of the Thread class that I find really annoying is that its name is of type char[] instead of string. Can we change this please? Also, it would be nice to have additional constructors that take a name.

Sure thing, though as I said in my other email, I hope people won't be using Thread explicitly very often.

Le 2010-01-04 ? 23:57, Andrei Alexandrescu a ?crit :

> Sean's code has an error: mutateA takes an A (as opposed to a shared A) so no synchronization is assumed or necessary.
> 
> Again: non-shared objects _are_ thread-local objects.
> 
> Andrei

Indeed, he probably meant the argument as shared. That doesn't really change my point however.

If we had synchronized function arguments, locking could be done more efficiently at the call site, where you know better what's already locked. That was your own reasoning for suggesting to Walter to acquire the lock at the call site for synchronized member functions, so I'm just generalizing it a bit more.



> Michel Fortin wrote:
>> Le 2010-01-04 ? 22:49, Sean Kelly a ?crit :
>>>   class A {
>>>       void fnA() synchronized { fnB(); } // 1
>>>       private void fnB() synchronized {}
>>>   }
>>> 
>>>   void mutateA( A val ) {
>>>       val.fnB(); // 2
>>>   }
>>> 
>>> At 1, the compiler can easily tell that the lock is already held so it doesn't acquire the lock again.  At 2 the lock is (probably) not held, so it's acquired again.
>> To avoid that "lock is probably not held" situation, we could allow the standalone function to be declared like this:
>> 	void mutateA(synchronized A val) {
>> 	    val.fnB();
>> 	}
>> Now 'val' can be locked at the call site, if necessary.

-- 
Michel Fortin
michel.fortin at michelf.com
http://michelf.com/

On Mon, Jan 4, 2010 at 6:58 PM, Andrei Alexandrescu <andrei at erdani.com>wrote:

>
> shared int x;
> ...
> ++x;
>
> The putative user notices that that doesn't work, so she's like, meh, I'll do this then:
>
> int y = x;
> ++y;
> x = y;
>
> And the user remains with this impression that the D compiler is a bit dumb. Of course that doesn't avoid the race condition though. If the user would have to call atomicIncrement(x) that would be clearly an improvement, but even this would be an improvement:
>
> int y = sharedRead(x);
> ++y;
> sharedWrite(y, x);
>
> When writing such code the user inevitably hits on the documentation for the two intrinsics, which clearly define their guarantees: only the sequence of sharedRead and sharedWrite is preserved. At that point, inspecting the code and understanding how it works is improved.
>
> ...
>
 Andrei
>

Just to clarify, this is still broken, right?  I mean that if two users are
calling a method that does this,
the value of x will only get incremented once if their calls to
sharedRead/sharedWrite are interleaved?

Kevin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.puremagic.com/pipermail/dmd-concurrency/attachments/20100105/066a6637/attachment.htm>

It is indeed broken, and the brokenness will be revealed by the following bits in the documentation:

"The language guarantees that the sequence of calls to sharedRead and sharedWrite is the same as syntactic order in each thread."

With this at hand, clearly there could be several possible interleavings of the calls from different thread, and the final value of x depends on the interleaving.


Andrei

Kevin Bealer wrote:
> On Mon, Jan 4, 2010 at 6:58 PM, Andrei Alexandrescu <andrei at erdani.com <mailto:andrei at erdani.com>> wrote:
> 
> 
>     shared int x;
>     ...
>     ++x;
> 
>     The putative user notices that that doesn't work, so she's like,
>     meh, I'll do this then:
> 
>     int y = x;
>     ++y;
>     x = y;
> 
>     And the user remains with this impression that the D compiler is a
>     bit dumb. Of course that doesn't avoid the race condition though. If
>     the user would have to call atomicIncrement(x) that would be clearly
>     an improvement, but even this would be an improvement:
> 
>     int y = sharedRead(x);
>     ++y;
>     sharedWrite(y, x);
> 
>     When writing such code the user inevitably hits on the documentation
>     for the two intrinsics, which clearly define their guarantees: only
>     the sequence of sharedRead and sharedWrite is preserved. At that
>     point, inspecting the code and understanding how it works is improved.
> 
>     ...
> 
>     Andrei
> 
> 
> Just to clarify, this is still broken, right?  I mean that if two users
> are calling a method that does this,
> the value of x will only get incremented once if their calls to
> sharedRead/sharedWrite are interleaved?
> 
> Kevin
> 
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> dmd-concurrency mailing list
> dmd-concurrency at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/dmd-concurrency

On Jan 5, 2010, at 10:14 AM, Kevin Bealer wrote:

> On Mon, Jan 4, 2010 at 6:58 PM, Andrei Alexandrescu <andrei at erdani.com> wrote:
> 
> shared int x;
> ...
> ++x;
> 
> The putative user notices that that doesn't work, so she's like, meh, I'll do this then:
> 
> int y = x;
> ++y;
> x = y;
> 
> And the user remains with this impression that the D compiler is a bit dumb. Of course that doesn't avoid the race condition though. If the user would have to call atomicIncrement(x) that would be clearly an improvement, but even this would be an improvement:
> 
> int y = sharedRead(x);
> ++y;
> sharedWrite(y, x);
> 
> When writing such code the user inevitably hits on the documentation for the two intrinsics, which clearly define their guarantees: only the sequence of sharedRead and sharedWrite is preserved. At that point, inspecting the code and understanding how it works is improved.
> 
> ...
> Andrei
> 
> Just to clarify, this is still broken, right?  I mean that if two users are calling a method that does this,
> the value of x will only get incremented once if their calls to sharedRead/sharedWrite are interleaved?

Yes.  You either need to call a special hardware instruction (LOCK INC on x86) or use a CAS loop:

void increment( ref shared int x )
{
    int t;
    do {
        t = atomicLoad( x );
    } while( !atomicStoreIf( x, t + 1, t ) );  // atomicStoreIf is a CAS
}

That's what I was guessing.  But then I thought maybe there might be some magic that either inserted a lock around the block of instructions from the first read to the last write, or else that detected these primatives and emitted a warning.  Not that I'm suggesting that approach, as a general solution is probably impractical / full of landmines.  And I guess getting lock free systems right is normally considered an "experts only" kind of thing, except maybe the one-pointer versions like linked list.

Kevin

On Tue, Jan 5, 2010 at 1:21 PM, Sean Kelly <sean at invisibleduck.org> wrote:

>  On Jan 5, 2010, at 10:14 AM, Kevin Bealer wrote:
>
> > On Mon, Jan 4, 2010 at 6:58 PM, Andrei Alexandrescu <andrei at erdani.com>
> wrote:
> >
> > shared int x;
> > ...
> > ++x;
> >
> > The putative user notices that that doesn't work, so she's like, meh,
> I'll do this then:
> >
> > int y = x;
> > ++y;
> > x = y;
> >
> > And the user remains with this impression that the D compiler is a bit
> dumb. Of course that doesn't avoid the race condition though. If the user would have to call atomicIncrement(x) that would be clearly an improvement, but even this would be an improvement:
> >
> > int y = sharedRead(x);
> > ++y;
> > sharedWrite(y, x);
> >
> > When writing such code the user inevitably hits on the documentation for
> the two intrinsics, which clearly define their guarantees: only the sequence of sharedRead and sharedWrite is preserved. At that point, inspecting the code and understanding how it works is improved.
> >
> > ...
> > Andrei
> >
> > Just to clarify, this is still broken, right?  I mean that if two users
> are calling a method that does this,
> > the value of x will only get incremented once if their calls to
> sharedRead/sharedWrite are interleaved?
>
> Yes.  You either need to call a special hardware instruction (LOCK INC on
> x86) or use a CAS loop:
>
> void increment( ref shared int x )
> {
>    int t;
>    do {
>        t = atomicLoad( x );
>    } while( !atomicStoreIf( x, t + 1, t ) );  // atomicStoreIf is a CAS
>  }
>
> _______________________________________________
> dmd-concurrency mailing list
> dmd-concurrency at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/dmd-concurrency
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.puremagic.com/pipermail/dmd-concurrency/attachments/20100105/1ececb2b/attachment.htm>