May 31, 2012
On 5/31/12 2:36 AM, Regan Heath wrote:
> On Wed, 30 May 2012 19:29:39 +0100, Andrei Alexandrescu
> <SeeWebsiteForEmail@erdani.org> wrote:
>> You can have deadlocks but with synchronized you can't leak locks or
>> doubly-unlock them. With free mutexes you have all of the above.
>
> I'm not suggesting using free mutexes. I'm suggesting keeping the mutex
> private inside the object.

Ergo, you are suggesting using free mutexes. Your second sentence destroys the first.


Andrei
May 31, 2012
On 5/31/12 2:45 AM, Regan Heath wrote:
> Ok, how about you tell us what term you want to use for the current
> state of affairs then, because arguing about this is pointless and I'm
> happy to use whatever term you deem appropriate (because I couldn't care
> less what the term is - as long as we all know what is meant, which I
> believe is the case here).

The idiom is called "scoped locking".

Andrei

May 31, 2012
On Thu, 31 May 2012 10:48:51 +0100, Andrei Alexandrescu <SeeWebsiteForEmail@erdani.org> wrote:

> On 5/31/12 2:36 AM, Regan Heath wrote:
>> On Wed, 30 May 2012 19:29:39 +0100, Andrei Alexandrescu
>> <SeeWebsiteForEmail@erdani.org> wrote:
>>> You can have deadlocks but with synchronized you can't leak locks or
>>> doubly-unlock them. With free mutexes you have all of the above.
>>
>> I'm not suggesting using free mutexes. I'm suggesting keeping the mutex
>> private inside the object.
>
> Ergo, you are suggesting using free mutexes. Your second sentence destroys the first.

Depends on your definition of "free".  You appear to have meant as an instance/pointer/object even one in a class, I initially read it as meaning as a separate object from the class you're locking.  In any case, you're right the compiler doesn't get synchronized() statements/classes/methods wrong and a programmer can.  The trade-off is the cause of this thread of discussion.

R

-- 
Using Opera's revolutionary email client: http://www.opera.com/mail/
May 31, 2012
Le 31/05/2012 11:55, Regan Heath a écrit :
> On Thu, 31 May 2012 10:48:51 +0100, Andrei Alexandrescu
> <SeeWebsiteForEmail@erdani.org> wrote:
>
>> On 5/31/12 2:36 AM, Regan Heath wrote:
>>> On Wed, 30 May 2012 19:29:39 +0100, Andrei Alexandrescu
>>> <SeeWebsiteForEmail@erdani.org> wrote:
>>>> You can have deadlocks but with synchronized you can't leak locks or
>>>> doubly-unlock them. With free mutexes you have all of the above.
>>>
>>> I'm not suggesting using free mutexes. I'm suggesting keeping the mutex
>>> private inside the object.
>>
>> Ergo, you are suggesting using free mutexes. Your second sentence
>> destroys the first.
>
> Depends on your definition of "free". You appear to have meant as an
> instance/pointer/object even one in a class, I initially read it as
> meaning as a separate object from the class you're locking. In any case,
> you're right the compiler doesn't get synchronized()
> statements/classes/methods wrong and a programmer can. The trade-off is
> the cause of this thread of discussion.
>
> R
>

I think you misunderstand each other. Here is what I nderstand :

Andrei is talking about free mutex in the sense of a mutex with lock and unlock function available. Such a mutex is error prone, because you can lock/unlock in an incorrect way.

synchronized is here a better alternative, because lock and unlock always goes by pair.

Regan, by free mutex, you mean a mutex that can be locked and unlocked by everybody. This is problematic too.

The ideal is to lock in a way that isn't a free mutex in both definition of the term.
May 31, 2012
On Thu, 31 May 2012 10:49:27 +0100, Andrei Alexandrescu <SeeWebsiteForEmail@erdani.org> wrote:

> On 5/31/12 2:45 AM, Regan Heath wrote:
>> Ok, how about you tell us what term you want to use for the current
>> state of affairs then, because arguing about this is pointless and I'm
>> happy to use whatever term you deem appropriate (because I couldn't care
>> less what the term is - as long as we all know what is meant, which I
>> believe is the case here).
>
> The idiom is called "scoped locking".

So.. the mutex is "scoped locking"?  Doesn't sound right to me.  I think the mutex is "available for locking and unlocking" <- need a word for that, which is not "exposed".  How about accessible, available, usable, or just plain lockable .. So, the problem here is that the mutex is lockable by external code via synchronized() and this means a certain type of deadlock is possible.

Moving on..

I think the change mentioned in TDPL to restrict synchronized to synchronized classes is a step in the right direction WRT wasted monitor space and people freely locking anything.  But, it is exactly the case which results in more possible deadlocks (the cause of this thread) AND I think it's actually far more likely people will want to use a synchronized statement on a class which is not itself synchronized, like for example an existing container class.

Given that, restricting synchronized statements to synchronized classes seems entirely wrong to me.  In fact, I would say you almost want to stop people using synchronized statements on synchronized classes because:
1. If a synchronized class is written correctly it should not be necessary in the general case(*)
2. It raises the chances of deadlocks (the cause of this thread).
3. It means that classes in general will be simpler to write (no need to worry about synchronization) and also more lightweight for use in non-threaded/non-shared cases.  (because we'd provide a template wrapper to make them synchronizable)

So, more and more I'm thinking it would be better to provide a library/runtime co-operative solution where we have an interface which is required for synchronized statements, and a wrapper template to implement that interface for any existing non-synchronized class.  Meaning, the choice is either to write a synchronized class (rare) or a non-synchronized class - knowing it can easily be synchronized if/when needed.

The "liquid lock" problem mentioned earlier is an interesting one that I have not personally experienced, perhaps because I don't lock anything but mutex primitives and I never to re-assign these.

(*) Locking on a larger scope can be achieved by providing a method taking a delegate (see earlier thread/replies) and/or exposing lock/unlock for those few situations where the delegate method cannot be used.  These are the few cases in which this cannot be avoided as the lock/unlock are separated by more than a single scope (so synchronized statements don't help in these cases either).

R

-- 
Using Opera's revolutionary email client: http://www.opera.com/mail/
May 31, 2012
On 5/31/12 3:27 AM, Regan Heath wrote:
> On Thu, 31 May 2012 10:49:27 +0100, Andrei Alexandrescu
> <SeeWebsiteForEmail@erdani.org> wrote:
>
>> On 5/31/12 2:45 AM, Regan Heath wrote:
>>> Ok, how about you tell us what term you want to use for the current
>>> state of affairs then, because arguing about this is pointless and I'm
>>> happy to use whatever term you deem appropriate (because I couldn't care
>>> less what the term is - as long as we all know what is meant, which I
>>> believe is the case here).
>>
>> The idiom is called "scoped locking".
>
> So.. the mutex is "scoped locking"?

No, the Java-style approach with the synchronized statement etc.

Andrei

May 31, 2012
On Thu, 31 May 2012 12:06:45 +0100, Andrei Alexandrescu <SeeWebsiteForEmail@erdani.org> wrote:

> On 5/31/12 3:27 AM, Regan Heath wrote:
>> On Thu, 31 May 2012 10:49:27 +0100, Andrei Alexandrescu
>> <SeeWebsiteForEmail@erdani.org> wrote:
>>
>>> On 5/31/12 2:45 AM, Regan Heath wrote:
>>>> Ok, how about you tell us what term you want to use for the current
>>>> state of affairs then, because arguing about this is pointless and I'm
>>>> happy to use whatever term you deem appropriate (because I couldn't care
>>>> less what the term is - as long as we all know what is meant, which I
>>>> believe is the case here).
>>>
>>> The idiom is called "scoped locking".
>>
>> So.. the mutex is "scoped locking"?
>
> No, the Java-style approach with the synchronized statement etc.

*sigh* I give up, this is pointless.

R

-- 
Using Opera's revolutionary email client: http://www.opera.com/mail/
May 31, 2012
On 5/31/12 3:27 AM, Regan Heath wrote:
> I think
> the mutex is "available for locking and unlocking" <- need a word for
> that, which is not "exposed". How about accessible, available, usable,
> or just plain lockable .. So, the problem here is that the mutex is
> lockable by external code via synchronized() and this means a certain
> type of deadlock is possible.

But this is a protection/visibility issue, which is orthogonal on the locking capability. It's as if you say "int is not good because anyone can overflow it." Okay! Make it private inside a CheckedInt class.

> Moving on..
>
> I think the change mentioned in TDPL to restrict synchronized to
> synchronized classes is a step in the right direction WRT wasted monitor
> space and people freely locking anything. But, it is exactly the case
> which results in more possible deadlocks (the cause of this thread) AND
> I think it's actually far more likely people will want to use a
> synchronized statement on a class which is not itself synchronized, like
> for example an existing container class.
>
> Given that, restricting synchronized statements to synchronized classes
> seems entirely wrong to me.

So where's the mutex that would be used to synchronize objects that are not synchronizable?

> In fact, I would say you almost want to stop
> people using synchronized statements on synchronized classes because:
> 1. If a synchronized class is written correctly it should not be
> necessary in the general case(*)
> 2. It raises the chances of deadlocks (the cause of this thread).
> 3. It means that classes in general will be simpler to write (no need to
> worry about synchronization) and also more lightweight for use in
> non-threaded/non-shared cases. (because we'd provide a template wrapper
> to make them synchronizable)

There are cases in which you want to do multiple operations under a single critical section, even though the API is otherwise well-designed. That may be for correctness, efficiency, or both. I don't see why we'd want to disallow that, it's a good idiom.

> So, more and more I'm thinking it would be better to provide a
> library/runtime co-operative solution where we have an interface which
> is required for synchronized statements, and a wrapper template to
> implement that interface for any existing non-synchronized class.
> Meaning, the choice is either to write a synchronized class (rare) or a
> non-synchronized class - knowing it can easily be synchronized if/when
> needed.

Looking forward for a fleshed out proposal. Make sure you motivate it properly.

> The "liquid lock" problem mentioned earlier is an interesting one that I
> have not personally experienced, perhaps because I don't lock anything
> but mutex primitives and I never to re-assign these.
>
> (*) Locking on a larger scope can be achieved by providing a method
> taking a delegate (see earlier thread/replies) and/or exposing
> lock/unlock for those few situations where the delegate method cannot be
> used. These are the few cases in which this cannot be avoided as the
> lock/unlock are separated by more than a single scope (so synchronized
> statements don't help in these cases either).

On first look, the inversion of control using delegates delegates has similar liabilities as straight scoped locking.


Andrei

May 31, 2012
On Wed, 30 May 2012 17:42:47 -0400, José Armando García Sancio <jsancio@gmail.com> wrote:

> On Wed, May 30, 2012 at 5:30 PM, Steven Schveighoffer
> <schveiguy@yahoo.com> wrote:
>> All deadlocks involving A and B mutexes.  Or if not, can you show how?  I
>> can't see it.
>>
>> -Steve
>
> This can cause a deadlock because locking order is not guaranteed, no?
>
> synchronized class A {
>   void delegate(B b) { b.call() }
>   void call() {}
> }
>
> synchronized class B {
>   void delegate(A a) { a.call() }
>   void call() {}
> }
>
> Thanks,
> -Jose
> PS. This may not compile not sure if it should be 'void
> delegate(shared B b)', etc.

A and B are not modifiable.  Given the implementation of A and B that I gave, how can you achieve deadlock?

The big problem I see with allowing anyone to synchronize on A and B is that one cannot achieve the property of "not possible to use me in a deadlock."

-Steve
May 31, 2012
On Wed, 30 May 2012 17:45:32 -0400, Andrei Alexandrescu <SeeWebsiteForEmail@erdani.org> wrote:

> On 5/30/12 1:31 PM, Steven Schveighoffer wrote:
>> On Wed, 30 May 2012 15:48:47 -0400, Andrei Alexandrescu
>> <SeeWebsiteForEmail@erdani.org> wrote:
>>> synchronized (object) {
>>> writeln("about to unlock the object");
>>> XXX
>>> writeln("unlocked the object");
>>> }
>>>
>>> Replace "XXX" with a construct that unlocks the object.
>>
>> This is not what we are talking about.
>
> Oh yes it is. "Expose the mutex" means "make the two mutex primitive operations lock() and unlock() freely usable against the mutex object".

Stop that!  You are blatantly arguing minutia when you should be trying to understand what we are saying!  This is like dealing with a stubborn child.

>> I guess the easiest way to say it is, the API used to "lock and then
>> subsequently unlock" the mutex. That is, even this:
>>
>> Object o = new Object;
>>
>> synchronized(o)
>> {
>> }
>>
>> is exposing the mutex in such a way that you can interfere with the
>> semantic meaning of the mutex, and cause preventable deadlocks.
>>
>> It would be preferrable for:
>>
>> synchronized(o)
>> {
>> }
>>
>> to be an error, unless typeof(o) allows it explicitly.
>
> Yah, it should be only allowed for synchronized classes.

Not good enough.  Whether I want all accesses to methods and members to be synchronized should be orthogonal to whether I want to allow arbitrary scoped locking and unlocking of my object.

>
>> I gave you a case where you can have a deadlock, even with simple fully
>> synchronized classes. You might say, "yeah, but all mutexes can have
>> deadlocks!".
>
> Not only I might say that, I'm actually gonna say it. Yeah, but all mutexes can have deadlocks!

Right, if used incorrectly.  In my example, you cannot have a deadlock unless the ability to scope-lock the object is exposed to the public.

>> My contention is that if the default is you do *not* expose the mutex to
>> external callers, there *cannot* be a deadlock.
>
> That's shuffling the fundamental issue from client code to library code. This can be done in either approach (synchronized vs. raw mutex), except it's clunkier with raw mutex.

Not talking about raw mutexes, my code is based on synchronized classes.  Period.

>> Here is the example again:
>>
>> synchronized class A
>> {
>> private int _foo;
>> @property int foo() { return _foo;}
>> }
>>
>> synchronized class B
>> {
>> private A _a;
>> @property A a() { return _a;}
>> void fun() {}
>> }
>>
>> void thread(B b)
>> {
>> synchronized(b.a)
>> {
>> b.fun();
>> }
>> }
>>
>> If synchronized(b.a) is valid, deadlock can occur. If it's invalid,
>> deadlock *cannot* occur.
>
> I don't get what this is supposed to prove.

If you cannot synchronize on b.a, as an *external* entity, then you cannot deadlock.  This is an important property.  The author of A and B can say "deadlock-proof".  How do you not see this as valuable?

Right now, we have a choice:

a. use synchronized -> prone to deadlocks that you cannot control or prevent.
b. roll your own locking -> possible to prevent deadlocks in all cases, but prone to error since you could accidentally miss an unlock or lock.

I want:

c. use synchronized with *no* public access -> possible to prevent deadlocks in all cases, safe to use without forgetting to unlock/lock.

-Steve