May 31, 2012
On 31-05-2012 16:49, Dmitry Olshansky wrote:
>>>
>>> But this is a protection/visibility issue, which is orthogonal on the
>>> locking capability. It's as if you say "int is not good because anyone
>>> can overflow it." Okay! Make it private inside a CheckedInt class.
>>
>> Sorry, that's a bad comparison. CheckedInt is to int, what CheckedMutex
>> is to mutex - but I'm not suggesting anything like a CheckedMutex. I'm
>> suggesting "mutex" but kept private inside the class /that it locks/.
>> Yes, it's a visibility issue, the issue is that the mutex used by
>> synchronized classes/methods is too visible/accessible and this opens it
>> up for deadlocks which are otherwise impossible.
>>
>
> Sure it's awful comparison.
>
>
>>>
>>> So where's the mutex that would be used to synchronize objects that
>>> are not synchronizable?
>>
>> In the wrapper class/struct/object which derives a synchronized
>> class/struct from the original. My D foo is not strong enough to just
>> come up with valid D code for the idiom on the fly, but essentially you
>> wrap the original object in a new object using a template which adds the
>> mutex member and the interface methods (lock, tryLock, and unlock)
>> required. No, this doesn't work with "final" classes.. but it shouldn't,
>> they're final after all. For them you need to add/manage the mutex
>> manually - the price you pay for "final".
>>
>
> OK let me land you a hand here. My proposal, that I think fits your
> ideas quite favorably.
>
> I'll enumerate certain assumptions beforehand since it's one of most
> confusing threads I ever followed.
>
> 1. synchronized class means: always allocate hidden monitor mutex, lock
> it on every public method of this class.

If this means that the monitor field in regular objects goes away, then I'm all for this, at least.

>
> 2. synchronized(x,y,z) is lowered to (I use Steven's idea):
> auto sorted = total_order(x,y,z);//conceptual, sorted is tuple of x,y,z
> sorted
> FOR EACH item IN sorted tuple add code in [[ ... ]]
> [[// conceptual
> item.__lock();
> scope(exit) item.__unlock();
> ]]
> In other words it works for every object that defines lock/unlock.
> Multiple object version works only if there is opCmp defined. (by
> address whatever, any total ordering should work)
>
>
> Per definition above synchronized classes AS IS can't be *synchronized*
> on. Instead their public methods are implicitly synchronized.
>
> The end result is:
>
> User can synchronize on various synchronization entities and even plug
> in custom synchronization primitives (say OS Y provides have this fancy
> lock type Z). It's explicit in as sense that object supports it via
> __lock__/unlock. Obviously one still can use __lock/__unlock explicitly
> just don't forget to wear big HERE BE DRAGONS banner.
>
> Synchronized class encapsulate mutex completely - nobody aside from
> designer of the class may (a)use it.
>
> Does it makes sense?
>

-- 
Alex Rønne Petersen
alex@lycus.org
http://lycus.org
May 31, 2012
On Thu, 31 May 2012 14:29:27 -0400, Andrei Alexandrescu <SeeWebsiteForEmail@erdani.org> wrote:

> On 5/31/12 7:01 AM, Regan Heath wrote:

>> Sorry, I have no spare time to spare. You're getting free ideas/thoughts
>> from me, feel free to ignore them.
>
> Thanks. Let me know if I understand correctly that your idea boils down to "I don't like synchronized, let's deprecate it and get back to core.sync.mutex and recommend the private thingamaroo." In that case, I disagree. I believe synchronized has good merits that are being ignored.

No, this is definitely *not* what we are saying.  The idea is that synchronized(x) is still present, but what objects you can call this on, and more importantly, *who* can do this is restricted.

Nobody is advocating abandoning synchronized in favor of manual locks.  In fact, I think we all want to *avoid* manual locks as much as possible.  It's all about controlling access.  If it comes down to "you must use a private, error-prone mutex member in order to prevent deadlocks," then I think we have room for improvement.

-Steve
May 31, 2012
On 31.05.2012 22:34, Alex Rønne Petersen wrote:
> On 31-05-2012 16:49, Dmitry Olshansky wrote:
>>>>
>>>> But this is a protection/visibility issue, which is orthogonal on the
>>>> locking capability. It's as if you say "int is not good because anyone
>>>> can overflow it." Okay! Make it private inside a CheckedInt class.
>>>
>>> Sorry, that's a bad comparison. CheckedInt is to int, what CheckedMutex
>>> is to mutex - but I'm not suggesting anything like a CheckedMutex. I'm
>>> suggesting "mutex" but kept private inside the class /that it locks/.
>>> Yes, it's a visibility issue, the issue is that the mutex used by
>>> synchronized classes/methods is too visible/accessible and this opens it
>>> up for deadlocks which are otherwise impossible.
>>>
>>
>> Sure it's awful comparison.
>>
>>
>>>>
>>>> So where's the mutex that would be used to synchronize objects that
>>>> are not synchronizable?
>>>
>>> In the wrapper class/struct/object which derives a synchronized
>>> class/struct from the original. My D foo is not strong enough to just
>>> come up with valid D code for the idiom on the fly, but essentially you
>>> wrap the original object in a new object using a template which adds the
>>> mutex member and the interface methods (lock, tryLock, and unlock)
>>> required. No, this doesn't work with "final" classes.. but it shouldn't,
>>> they're final after all. For them you need to add/manage the mutex
>>> manually - the price you pay for "final".
>>>
>>
>> OK let me land you a hand here. My proposal, that I think fits your
>> ideas quite favorably.
>>
>> I'll enumerate certain assumptions beforehand since it's one of most
>> confusing threads I ever followed.
>>
>> 1. synchronized class means: always allocate hidden monitor mutex, lock
>> it on every public method of this class.
>
> If this means that the monitor field in regular objects goes away, then
> I'm all for this, at least.
>

Yes, it removes monitor in non-synchronized object. Glad you we are in agreement here.

And all public methods lower to :

<signature>
{
this.hidden__monitor.lock();
scope(exit) this.hidden__monitor.unlock();
... //code
}


-- 
Dmitry Olshansky
May 31, 2012
On 5/31/12 11:35 AM, Steven Schveighoffer wrote:
> On Thu, 31 May 2012 14:29:27 -0400, Andrei Alexandrescu
> <SeeWebsiteForEmail@erdani.org> wrote:
>
>> On 5/31/12 7:01 AM, Regan Heath wrote:
>
>>> Sorry, I have no spare time to spare. You're getting free ideas/thoughts
>>> from me, feel free to ignore them.
>>
>> Thanks. Let me know if I understand correctly that your idea boils
>> down to "I don't like synchronized, let's deprecate it and get back to
>> core.sync.mutex and recommend the private thingamaroo." In that case,
>> I disagree. I believe synchronized has good merits that are being
>> ignored.
>
> No, this is definitely *not* what we are saying.

Then probably a more formal proposal is in order.

Andrei

May 31, 2012
On 31.05.2012 22:33, Andrei Alexandrescu wrote:
> On 5/31/12 7:49 AM, Dmitry Olshansky wrote:
>> 1. synchronized class means: always allocate hidden monitor mutex, lock
>> it on every public method of this class.
>
> Great.
>
>> 2. synchronized(x,y,z) is lowered to (I use Steven's idea):
>> auto sorted = total_order(x,y,z);//conceptual, sorted is tuple of x,y,z
>> sorted
>> FOR EACH item IN sorted tuple add code in [[ ... ]]
>> [[// conceptual
>> item.__lock();
>> scope(exit) item.__unlock();
>> ]]
>> In other words it works for every object that defines lock/unlock.
>> Multiple object version works only if there is opCmp defined. (by
>> address whatever, any total ordering should work)
>
> Great. What are regular calls to synchronized class methods lowered into?

Posted in another reply, basically same lock(); scope(exit)unlock() of hidden monitor.
>
>> Per definition above synchronized classes AS IS can't be *synchronized*
>> on. Instead their public methods are implicitly synchronized.
>>
>> The end result is:
>>
>> User can synchronize on various synchronization entities and even plug
>> in custom synchronization primitives (say OS Y provides have this fancy
>> lock type Z). It's explicit in as sense that object supports it via
>> __lock__/unlock. Obviously one still can use __lock/__unlock explicitly
>> just don't forget to wear big HERE BE DRAGONS banner.
>>
>> Synchronized class encapsulate mutex completely - nobody aside from
>> designer of the class may (a)use it.
>>
>> Does it makes sense?
>
> It does make sense, but I think we need a Lock struct type that makes
> sure code cannot screw up the number of lock and unlock calls. We
> shouldn't just expose bare lock() and unlock().
>

Dunno. In fact __lock/__unlock could be artificially hidden from user but not compiler with his magic rewrites.
But I believe there is value in having access to lock/unlock for those select cases where it's beneficial.


-- 
Dmitry Olshansky
May 31, 2012
On Thu, 31 May 2012 14:55:31 -0400, Dmitry Olshansky <dmitry.olsh@gmail.com> wrote:

>
> Dunno. In fact __lock/__unlock could be artificially hidden from user but not compiler with his magic rewrites.
> But I believe there is value in having access to lock/unlock for those select cases where it's beneficial.

__ctor and __dtor are accessible, but nobody abuses them.

I think if we use the double-underscore terminology, this would be fine.

-Steve
May 31, 2012
On May 31, 2012, at 11:33 AM, Andrei Alexandrescu wrote:
> 
> It does make sense, but I think we need a Lock struct type that makes sure code cannot screw up the number of lock and unlock calls. We shouldn't just expose bare lock() and unlock().

synchronized already works with classes that implement Object.Monitor, and we have scope guards.  Do we really need an RTTI Lock struct as well?
May 31, 2012
On 5/31/12 12:27 PM, Sean Kelly wrote:
> On May 31, 2012, at 11:33 AM, Andrei Alexandrescu wrote:
>>
>> It does make sense, but I think we need a Lock struct type that
>> makes sure code cannot screw up the number of lock and unlock
>> calls. We shouldn't just expose bare lock() and unlock().
>
> synchronized already works with classes that implement
> Object.Monitor, and we have scope guards.  Do we really need an RTTI
> Lock struct as well?

At this point it's unclear to me what different branches of this sprawling thread are proposing. I have extracted one thing that we should definitely look into - formalizing the lowering of the synchronized statement. Other than that I failed to derive much signal.

Andrei
May 31, 2012
On 31.05.2012 18:49, Dmitry Olshansky wrote:
>
> I'll enumerate certain assumptions beforehand since it's one of most
> confusing threads I ever followed.
>
> 1. synchronized class means: always allocate hidden monitor mutex, lock
> it on every public method of this class.
>
> 2. synchronized(x,y,z) is lowered to (I use Steven's idea):
> auto sorted = total_order(x,y,z);//conceptual, sorted is tuple of x,y,z
> sorted
> FOR EACH item IN sorted tuple add code in [[ ... ]]
> [[// conceptual
> item.__lock();
> scope(exit) item.__unlock();
> ]]
> In other words it works for every object that defines lock/unlock.
> Multiple object version works only if there is opCmp defined. (by
> address whatever, any total ordering should work)
>
>
> Per definition above synchronized classes AS IS can't be *synchronized*
> on. Instead their public methods are implicitly synchronized.
>
> The end result is:
>
> User can synchronize on various synchronization entities and even plug
> in custom synchronization primitives (say OS Y provides have this fancy
> lock type Z). It's explicit in as sense that object supports it via
> __lock__/unlock. Obviously one still can use __lock/__unlock explicitly
> just don't forget to wear big HERE BE DRAGONS banner.
>
> Synchronized class encapsulate mutex completely - nobody aside from
> designer of the class may (a)use it.

It could even go one step forward. Though it require a great deal of though, here is an initial idea.

So with the above we have  synchronized class with monitor that is eagerly initialized (it's the propose of sync-ed class after all to use this mutex safely) and ordinary class that don't have it.

Now if we let shared classes to have monitor field lazily initialized something interesting shows up. For the moment let's skip painful detail of cast(shared).

It's then possible to introduce a restricted version of ownership system. Since ever member of shared class is shared or synchronized(?) it should be OK to access member of any depth in it by following the chain of monitor locks:
(i.e. I'm assuming composition implies "owns" relationship,
this ownership relation comes only for shared types)

shared class A
{
	shared B b;
	...
}

shared class B
{
	shared C c;
	...
}


then:
{
	...
	A a = ....;
	// locks A's monitor, then B's monitor, unlocks A's
	// then C's monitor, unlocks B's
	// then calls do_smth, then unlocks C's
	//all of the above lock/unlock sequence has to be exception proof
	a.b.c.do_smth();

}


Grunted assuming compositon == ownership is wrong in general, but could serve as a sensible default. Then weak non-owning references would be provided via dirty casting in special wrappers like
{
	NotOwning!(shared T) x;//shared but doesn't own it, thus if
//calling any resource inside of x, we should keep x.monitor locked //through out the operation. (unlike in the above case)
}	
...

Obviously it's iteration 1 of idea, it needs to deal with possible atomic modifications, cast(shared), etc.


-- 
Dmitry Olshansky
May 31, 2012
On 01.06.2012 0:39, Dmitry Olshansky wrote:
> On 31.05.2012 18:49, Dmitry Olshansky wrote:
>>
>> I'll enumerate certain assumptions beforehand since it's one of most
>> confusing threads I ever followed.
>>
>> 1. synchronized class means: always allocate hidden monitor mutex, lock
>> it on every public method of this class.
>>
>> 2. synchronized(x,y,z) is lowered to (I use Steven's idea):
>> auto sorted = total_order(x,y,z);//conceptual, sorted is tuple of x,y,z
>> sorted
>> FOR EACH item IN sorted tuple add code in [[ ... ]]
>> [[// conceptual
>> item.__lock();
>> scope(exit) item.__unlock();
>> ]]
>> In other words it works for every object that defines lock/unlock.
>> Multiple object version works only if there is opCmp defined. (by
>> address whatever, any total ordering should work)
>>
>>
>> Per definition above synchronized classes AS IS can't be *synchronized*
>> on. Instead their public methods are implicitly synchronized.
>>
>> The end result is:
>>
>> User can synchronize on various synchronization entities and even plug
>> in custom synchronization primitives (say OS Y provides have this fancy
>> lock type Z). It's explicit in as sense that object supports it via
>> __lock__/unlock. Obviously one still can use __lock/__unlock explicitly
>> just don't forget to wear big HERE BE DRAGONS banner.
>>
>> Synchronized class encapsulate mutex completely - nobody aside from
>> designer of the class may (a)use it.
>
> It could even go one step forward. Though it require a great deal of
> though, here is an initial idea.
>
> So with the above we have synchronized class with monitor that is
> eagerly initialized (it's the propose of sync-ed class after all to use
> this mutex safely) and ordinary class that don't have it.
>
> Now if we let shared classes to have monitor field lazily initialized
> something interesting shows up. For the moment let's skip painful detail
> of cast(shared).
>
> It's then possible to introduce a restricted version of ownership
> system. Since ever member of shared class is shared or synchronized(?)
> it should be OK to access member of any depth in it by following the
> chain of monitor locks:
> (i.e. I'm assuming composition implies "owns" relationship,
> this ownership relation comes only for shared types)
>
> shared class A
> {
> shared B b;
> ...
> }
>
> shared class B
> {
> shared C c;
> ...
> }
>
>
> then:
> {
> ...
> A a = ....;
> // locks A's monitor, then B's monitor, unlocks A's
> // then C's monitor, unlocks B's
> // then calls do_smth, then unlocks C's
> //all of the above lock/unlock sequence has to be exception proof
> a.b.c.do_smth();
>
> }
>
>
> Grunted assuming compositon == ownership is wrong in general, but could

s/Grunted/Granted

> serve as a sensible default. Then weak non-owning references would be
> provided via dirty casting in special wrappers like
> {
> NotOwning!(shared T) x;//shared but doesn't own it, thus if
> //calling any resource inside of x, we should keep x.monitor locked
> //through out the operation. (unlike in the above case)
> }
> ...
>
> Obviously it's iteration 1 of idea, it needs to deal with possible
> atomic modifications, cast(shared), etc.
>
>


-- 
Dmitry Olshansky