May 30, 2012
On 30-05-2012 18:42, Iain Buclaw wrote:
> On 30 May 2012 17:23, Alex Rønne Petersen<alex@lycus.org>  wrote:
>> On 30-05-2012 18:14, Andrei Alexandrescu wrote:
>>>
>>> On 5/30/12 9:03 AM, Regan Heath wrote:
>>>>
>>>> On Wed, 30 May 2012 16:46:54 +0100, Andrei Alexandrescu
>>>> <SeeWebsiteForEmail@erdani.org>  wrote:
>>>>
>>>>> On 5/30/12 2:34 AM, deadalnix wrote:
>>>>>>
>>>>>> Le 29/05/2012 23:33, Andrei Alexandrescu a écrit :
>>>>>>>
>>>>>>> On 5/29/12 1:37 AM, deadalnix wrote:
>>>>>>>>
>>>>>>>> I would say that breaking things here, with the right deprecation
>>>>>>>> process, is the way to go.
>>>>>>>
>>>>>>>
>>>>>>> So what should we use for mutex-based synchronization if we deprecate
>>>>>>> synchronized classes?
>>>>>>>
>>>>>>> Andrei
>>>>>>
>>>>>>
>>>>>> I think something similar to range design here is the way to go.
>>>>>>
>>>>>> It is easy to define something like
>>>>>>
>>>>>> template isLockable(T) {
>>>>>> enum isLockable = isShared!T&&  is(typeof(T.init.lock()))&&
>>>>>> is(typeof(T.init.release()));
>>>>>> }
>>>>>>
>>>>>> And allow locking only if(isLockable!Type) .
>>>>>>
>>>>>> Now we can create SyncObject or any structure we want. The point is
>>>>>> that
>>>>>> we lock explicit stuff.
>>>>>
>>>>>
>>>>> But in this design anyone can lock such an object, which was something
>>>>> you advocated against.
>>>>
>>>>
>>>> I think there is some confusion here as to what the "problem" is and is
>>>> not.
>>>>
>>>> The problem is /not/ that you can lock any object.
>>>> The problem is /not/ that we have synchronized(object) {}
>>>> The problem is /not/ that we have synchronized classes/methods.
>>>
>>>
>>> Several posts in this thread assert that such are problems.
>>>
>>>> The problem /is/ that synchronized classes/methods use a mutex which is
>>>> exposed publicly, and it the same mutex as used by synchronized(object)
>>>> {}. This exposure/re-use makes deadlocks more likely to happen, and
>>>> harder to spot.
>>>
>>>
>>> This is news to me. How do you publicly access the mutex of a
>>> synchronized class object?
>>
>>
>> Generally in two ways:
>>
>> 1) synchronized (obj)
>> 2) obj.__monitor
>>
>> (1) accesses it in order to actually do locking, but if obj is an
>> instantiation of a class that uses the this reference internally for
>> locking, you end up with potential deadlocks. Therefore, you can say that
>> obj's mutex is exposed. This isn't necessarily bad, because if obj does
>> *not* use its this reference for synchronized statements, nothing will blow
>> up. This is why the OP was about banning synchronized (this).
>>
>> (2) is an undocumented feature of the language that druntime (and maybe some
>> phobos code) makes use of to create/alter/delete monitors.
>>
>
> I'm pretty certain the use of .__monitor is stricted only to:
> object_.d - The frontend implementation which contains the underlying
> functions that are invoked when you synchronise(obj).
> monitor_.d - The backend implementation which contains the platform
> separated bits.
>
>
> Regards

I use it in my source base currently to implement weak references.

-- 
Alex Rønne Petersen
alex@lycus.org
http://lycus.org
May 30, 2012
On 05/30/12 17:50, Steven Schveighoffer wrote:
> So I propose it does this instead (entirely via lowering):
> 
> synchronized(x)
> {
> }
> 
> translates to:
> 
> auto l = x.__getMutex();
> l.lock();
> try
> {
> }
> finally
> {
>    l.unlock();
> }

I like this, it's slightly more complex, but can reduce the amount of boilerplate (by not requiring lock/unlock forwarders etc) and is more powerful than my proposal.

> So we have a few benefits here:
> 
> 1. if typeof(x) does not define __getMutex, the above is a compiler error.
> 2. We can control the visibility of the mutex, because we can attribute private or protected to the __getMutex function.
> 3. We can control what kinds of objects of typeof(x) use mutexes.  e.g. define __getMutex() shared only.
> 4. We have control over the type of the lock, it can be anything and does not need to fit into some runtime function's interface.

5. The lock doesn't have to be a real lock, but eg a lock manager. Hmm, lowering to
l.lock(x) and l.unlock(x) might be better; inlining will make it zero-cost for the
simple case, but this will make certain schemes much simplier (no need to encode
'x' inside the returned 'l').

> Second, I propose a somewhat gradual transition.  I think on first release, we *still* keep the allocated word in the class instance, and keep the druntime function which lazily allocates the mutex.  Therefore, changing an object that currently *properly* can use synchronized(this) can be made to work:
> 
> class Synchronizable
> {
>    xxx __getMutex() shared { return _d_getObjectMonitor(this);}
> }
> 
> We can deprecate that functionality, and the allocated space for the mutex later if desired.

Yes, UFCS could help too.

artur
May 30, 2012
On 5/30/12 9:23 AM, Alex Rønne Petersen wrote:
> On 30-05-2012 18:14, Andrei Alexandrescu wrote:
>> This is news to me. How do you publicly access the mutex of a
>> synchronized class object?
>
> Generally in two ways:
>
> 1) synchronized (obj)

This is not accessing the mutex for arbitrary operations.

> 2) obj.__monitor

All symbols starting with two underscores are reserved by the implementation.

> (1) accesses it in order to actually do locking, but if obj is an
> instantiation of a class that uses the this reference internally for
> locking, you end up with potential deadlocks.

Deadlocks are endemic to mutex-based programming and are not avoided inherently or preferentially by the alternative you discussed (exposing unrestricted mutex).

> Therefore, you can say
> that obj's mutex is exposed.

Nothing could stop one, but that claim is incorrect.


Andrei
May 30, 2012
On 5/30/12 9:43 AM, Regan Heath wrote:
> On Wed, 30 May 2012 17:00:43 +0100, Andrei Alexandrescu
> <SeeWebsiteForEmail@erdani.org> wrote:
>
>> On 5/30/12 5:32 AM, Regan Heath wrote:
>>> On Wed, 30 May 2012 10:21:00 +0100, deadalnix <deadalnix@gmail.com>
>>> wrote:
>>>> You don't want to synchronize on ANY object. You want to synchronize
>>>> on explicit mutexes.
>>>
>>> +1 .. this is the key point/issue.
>>
>> TDPL's design only allows for entire synchronized classes (not
>> separate synchronized and unsynchronized methods), which pair mutexes
>> with the data they protect. This is more restrictive than exposing
>> mutexes, but in a good way. We use such a library artifact in C++ at
>> Facebook all the time, to great success.
>
> Can you call pass them to a synchronized statement? i.e.
>
> TDPLStyleSynchClass a = new TDPLStyleSynchClass();
> synchronized(a) {
> }

Yes. Well I recommend acquiring the text! :o)

> ... because, if you can, then you're exposing the mutex.

No.

>> People shouldn't create designs that have synchronized classes
>> referring to one another naively. Designing with mutexes (explicit or
>> implicit) will always create the possibility of deadlock, so examples
>> how that could happen are easy to come across.
>
> True. But in my Example 1

Your example 1 should not compile.

>> TDPL improves on deadlocks by introducing synchronized statements with
>> more than one argument, see 13.15.
>
> Is there anywhere I can see this online? (for free :p)

http://goo.gl/ZhPM2

>> The implicit mutexes used by synchronized classes are recursive.
>
> :) why would you want anything else.

Efficiency.


Andrei

May 30, 2012
On 5/30/12 9:56 AM, deadalnix wrote:
> Le 30/05/2012 17:46, Andrei Alexandrescu a écrit :
>> On 5/30/12 2:34 AM, deadalnix wrote:
>>> Le 29/05/2012 23:33, Andrei Alexandrescu a écrit :
>>>> On 5/29/12 1:37 AM, deadalnix wrote:
>>>>> I would say that breaking things here, with the right deprecation
>>>>> process, is the way to go.
>>>>
>>>> So what should we use for mutex-based synchronization if we deprecate
>>>> synchronized classes?
>>>>
>>>> Andrei
>>>
>>> I think something similar to range design here is the way to go.
>>>
>>> It is easy to define something like
>>>
>>> template isLockable(T) {
>>> enum isLockable = isShared!T && is(typeof(T.init.lock())) &&
>>> is(typeof(T.init.release()));
>>> }
>>>
>>> And allow locking only if(isLockable!Type) .
>>>
>>> Now we can create SyncObject or any structure we want. The point is that
>>> we lock explicit stuff.
>>
>> But in this design anyone can lock such an object, which was something
>> you advocated against.
>>
>> Andrei
>
> I was advocating on being able to lock ANY object, which is out of control.

Must've been another poster. Anyhow, in TDPL's design only synchronized classes can be used with the synchronized statement.

> Such an object is known to be lockable, and most object will not be. It
> will avoid liquid lock, because most thing will not be lockable.

I'm celebrating day 2 of having no idea what a liquid lock is.

> Combined with encapsulation capabilities that D already have, exposing
> the lock to the entire worlds is easy. It will avoid unexpected lock
> from 3rd party code, which is a source of tedious deadlock.
>
> It also open door for locking on struct, that is easily embeded in a
> class as value (so constructed and destructed with the class).
>
> The fact that anyone or not can lock/unlock a mutex is a encapsulation
> problem and we already have solution in D for that.

... which can be used with synchronized classes.


Andrei
May 30, 2012
On Wed, 30 May 2012 01:10:41 +0200, Andrei Alexandrescu <SeeWebsiteForEmail@erdani.org> wrote:

> On 5/29/12 4:06 PM, Alex Rønne Petersen wrote:
>>  Synchronized blocks are good because they
>> operate on an implicit, hidden, global mutex. You can't screw up with that.
>
> I think there's quite some disconnect here. If there's any anti-pattern in this discussion, it's operating on an implicit, hidden, global mutex. Walter agreed to eliminate that from D, but never got around to it.
>

They're not really global, it's one per synchronized block.
https://github.com/D-Programming-Language/druntime/blob/master/src/rt/critical_.d
They're actually pretty limited/safe because you can't access/compose the underlying lock.

import std.stdio, core.thread;

void foo()
{
    synchronized
    {
        auto t = new Thread(&bar);
        t.start();
        t.join();
        writeln("foo");
    }
}

void bar()
{
    synchronized writeln("bar");
}

void main()
{
    foo();
}
May 30, 2012
On 5/30/12 10:20 AM, Martin Nowak wrote:
> On Wed, 30 May 2012 01:10:41 +0200, Andrei Alexandrescu
> <SeeWebsiteForEmail@erdani.org> wrote:
>
>> On 5/29/12 4:06 PM, Alex Rønne Petersen wrote:
>>> Synchronized blocks are good because they
>>> operate on an implicit, hidden, global mutex. You can't screw up with
>>> that.
>>
>> I think there's quite some disconnect here. If there's any
>> anti-pattern in this discussion, it's operating on an implicit,
>> hidden, global mutex. Walter agreed to eliminate that from D, but
>> never got around to it.
>>
>
> They're not really global, it's one per synchronized block.

That means global. They're terrible and should be eliminated from the language.


Andrei
May 30, 2012
On 30-05-2012 19:12, Andrei Alexandrescu wrote:
> On 5/30/12 9:23 AM, Alex Rønne Petersen wrote:
>> On 30-05-2012 18:14, Andrei Alexandrescu wrote:
>>> This is news to me. How do you publicly access the mutex of a
>>> synchronized class object?
>>
>> Generally in two ways:
>>
>> 1) synchronized (obj)
>
> This is not accessing the mutex for arbitrary operations.

No, it is indeed not. You didn't explicitly say you wanted to do "arbitrary operations", so I assumed that by accessing the mutex you meant doing the only two things that are sensible for a mutex: Locking and unlocking it. What else would you want to do? This discussion has always been about locking and unlocking publicly exposed mutexes. I don't think arguing about "arbitrary operations" is going to get the discussion anywhere exactly because of what you proceed to say ...

>
>> 2) obj.__monitor
>
> All symbols starting with two underscores are reserved by the
> implementation.

... since that effectively means there is no way, from an implementation-independent standpoint, to alter the memory of a mutex (or whatever) (see my answer below), thus making the only sensible operations on a mutex locking and unlocking.

But I may be misunderstanding you. If so, please clarify and/or correct me.

(Just so we're clear, I included (2) only for completeness. I know it's a dirty hack and I agree that it should never be in the language 'spec' or TDPL.)

>
>> (1) accesses it in order to actually do locking, but if obj is an
>> instantiation of a class that uses the this reference internally for
>> locking, you end up with potential deadlocks.
>
> Deadlocks are endemic to mutex-based programming and are not avoided
> inherently or preferentially by the alternative you discussed (exposing
> unrestricted mutex).
>
>> Therefore, you can say
>> that obj's mutex is exposed.
>
> Nothing could stop one, but that claim is incorrect.

It is exposed as far as using it for anything sensible goes. No, synchronized (obj) does not let you delete the monitor of obj or alter it or something along these lines, but I don't think that's worth bringing into this argument. This has always been about locking and unlocking a publicly exposed mutex, and about preventing common deadlock sources.

>
>
> Andrei

-- 
Alex Rønne Petersen
alex@lycus.org
http://lycus.org
May 30, 2012
On 30-05-2012 19:19, Andrei Alexandrescu wrote:
> On 5/30/12 9:56 AM, deadalnix wrote:
>> Le 30/05/2012 17:46, Andrei Alexandrescu a écrit :
>>> On 5/30/12 2:34 AM, deadalnix wrote:
>>>> Le 29/05/2012 23:33, Andrei Alexandrescu a écrit :
>>>>> On 5/29/12 1:37 AM, deadalnix wrote:
>>>>>> I would say that breaking things here, with the right deprecation
>>>>>> process, is the way to go.
>>>>>
>>>>> So what should we use for mutex-based synchronization if we deprecate
>>>>> synchronized classes?
>>>>>
>>>>> Andrei
>>>>
>>>> I think something similar to range design here is the way to go.
>>>>
>>>> It is easy to define something like
>>>>
>>>> template isLockable(T) {
>>>> enum isLockable = isShared!T && is(typeof(T.init.lock())) &&
>>>> is(typeof(T.init.release()));
>>>> }
>>>>
>>>> And allow locking only if(isLockable!Type) .
>>>>
>>>> Now we can create SyncObject or any structure we want. The point is
>>>> that
>>>> we lock explicit stuff.
>>>
>>> But in this design anyone can lock such an object, which was something
>>> you advocated against.
>>>
>>> Andrei
>>
>> I was advocating on being able to lock ANY object, which is out of
>> control.
>
> Must've been another poster. Anyhow, in TDPL's design only synchronized
> classes can be used with the synchronized statement.
>
>> Such an object is known to be lockable, and most object will not be. It
>> will avoid liquid lock, because most thing will not be lockable.
>
> I'm celebrating day 2 of having no idea what a liquid lock is.

I have to confess to this as well. I'm starting to suspect that he really means deadlock. Searching on Google for "liquid lock mutex" literally brings up this very NG thread. :)

>
>> Combined with encapsulation capabilities that D already have, exposing
>> the lock to the entire worlds is easy. It will avoid unexpected lock
>> from 3rd party code, which is a source of tedious deadlock.
>>
>> It also open door for locking on struct, that is easily embeded in a
>> class as value (so constructed and destructed with the class).
>>
>> The fact that anyone or not can lock/unlock a mutex is a encapsulation
>> problem and we already have solution in D for that.
>
> .... which can be used with synchronized classes.
>
>
> Andrei

-- 
Alex Rønne Petersen
alex@lycus.org
http://lycus.org
May 30, 2012
On 30-05-2012 19:22, Andrei Alexandrescu wrote:
> On 5/30/12 10:20 AM, Martin Nowak wrote:
>> On Wed, 30 May 2012 01:10:41 +0200, Andrei Alexandrescu
>> <SeeWebsiteForEmail@erdani.org> wrote:
>>
>>> On 5/29/12 4:06 PM, Alex Rønne Petersen wrote:
>>>> Synchronized blocks are good because they
>>>> operate on an implicit, hidden, global mutex. You can't screw up with
>>>> that.
>>>
>>> I think there's quite some disconnect here. If there's any
>>> anti-pattern in this discussion, it's operating on an implicit,
>>> hidden, global mutex. Walter agreed to eliminate that from D, but
>>> never got around to it.
>>>
>>
>> They're not really global, it's one per synchronized block.
>
> That means global. They're terrible and should be eliminated from the
> language.
>
>
> Andrei

So you do agree that explicit synchronization is better? Or only in this particular case?

-- 
Alex Rønne Petersen
alex@lycus.org
http://lycus.org