May 29, 2012
On Wednesday, May 30, 2012 00:01:49 Alex Rønne Petersen wrote:
> Besides, it seems to me that D can't quite make up its mind. We have TLS by default, and we encourage message-passing (through a library mechanism), and then we have the synchronized statement and attribute. It just seems so incredibly inconsistent. synchronized encourages doing the wrong thing (locks and synchronization).

It allows multiple approaches at multiple levels. TLS and message passing is the preferred way, and shared combined with synchronized or mutexes to protect it is available when you need it. synchronized classes are more straightforward approach to locking when you need to protect an entire object (and less error-prone in some respects, because you then have the guarantee that it happens for all functions and don't run the risk of not locking in some functions; it also works with inheritance that way, unlike mutexes). sychronized blocks on the other hand are a quick and easy way to protect specific sections of code without having to bother creating mutexes for it. And mutexes are available when you need full control. Which approach you go with depends on what you're doing and what your needs are.

Now, I could definitely see an argument that using shared is more low level and that it would be just simpler to only have mutexes and no synchronized, but we ended up with a more tiered approach, and that's not all bad. Each approach has its advantages and disadvantages, and D gives you all of them, so you can pick and choose what works best for you.

- Jonathan M Davis
May 29, 2012
On 5/29/12 2:56 PM, Alex Rønne Petersen wrote:
> On 29-05-2012 23:38, Andrei Alexandrescu wrote:
>> On 5/29/12 5:26 AM, Alex Rønne Petersen wrote:
>>> Generalized object monitors is the worst idea in programming language
>>> and virtual machine design, ever.
>>
>> I think that's an exaggeration. Care to elaborate a bit?
>>
>> Andrei
>>
>
> 1) You waste an entire word of memory in *every single object you ever
> create*. And for the majority of objects, this word will always be
> zero/null. Not to mention that when you do allocate a monitor, you also
> get the actual memory of that monitor in addition.

I agree. It would be better if synchronizable objects would be part of a sub-hierarchy such that not everyone pays for the word.

> 2) Anyone can lock on any object meaning it's near impossible to see
> where a deadlock might come from.

What would be the alternative? Deadlocks are a natural consequence of the fact that indeed any thread can wait on a resource.

> 3) Encapsulation is completely broken as a result of (2).

Again, what would be the alternative?

> 4) There's a whole bunch of runtime and GC plumbing (which is even
> broken in druntime right now) to support object monitors. To fix this
> entire mess, we'd need weak references.

That's more like an argument the implementation is imperfect, not that the idea is the worst ever.

> 5) This whole object monitor model goes against our "thread-local by
> default" model. Synchronization is evil and should be explicit with
> core.sync.mutex.

But doing things with core.sync.mutex is a definite step backwards as it is prey to all of the issues above. For example, "anyone can lock any mutex". How is that progress?


Andrei


May 29, 2012
On 5/29/12 2:57 PM, Alex Rønne Petersen wrote:
> On 29-05-2012 23:33, Andrei Alexandrescu wrote:
>> On 5/29/12 1:37 AM, deadalnix wrote:
>>> I would say that breaking things here, with the right deprecation
>>> process, is the way to go.
>>
>> So what should we use for mutex-based synchronization if we deprecate
>> synchronized classes?
>>
>> Andrei
>
> core.sync.mutex

That's worse.

Andrei

May 29, 2012
On 5/29/12 2:59 PM, Alex Rønne Petersen wrote:
> On 29-05-2012 23:31, Andrei Alexandrescu wrote:
>> On 5/29/12 1:32 AM, deadalnix wrote:
>>> I already did some comment about this.
>>>
>>> Making any object synchronization is a very bad design decision. It is
>>> deadlock prone, it is liquid lock prone, it cause an overhead on any
>>> object, and even worse, it is useless most of the time as D promote
>>> thread locality (which is very good).
>>
>> Actually I think such a characterization is superficial and biased to
>> the extent it becomes wrong.
>
> Really ? I think he's spot on.

I'd be glad to think the same. Good arguments might be helpful. So far I've seen exaggerated statements and posturing. Such are entertaining but are of limited effectiveness in furthering a point.

>> Locks are deadlock-prone by design, whether used with objects or with
>> classic procedural programs. In fact they are better confined to objects
>> because the association between the protected data and the
>> synchronization object is clear.
>
> There is no doubt that synchronization of any kind is hard and
> error-prone no matter how you do it. But do you really think that the
> situation is made better by allowing locking on *anything*, meaning that
> locks are effectively resources shared publicly?
>
> Besides, what's wrong with core.sync.mutex?

Your very argument that anyone can lock on it. To that, I'll add that core.sync.mutex is not properly associated with the resource it protects, so coding with bare mutexes is in fact inferior to coding with synchronized objects.


Andrei
May 29, 2012
On 5/29/12 3:01 PM, Alex Rønne Petersen wrote:
> On 29-05-2012 23:54, Andrei Alexandrescu wrote:
>> On 5/29/12 2:49 PM, Alex Rønne Petersen wrote:
>>> On 29-05-2012 23:32, Andrei Alexandrescu wrote:
>>>> On 5/29/12 1:35 AM, deadalnix wrote:
>>>>> Le 29/05/2012 01:38, Alex Rønne Petersen a écrit :
>>>>>> I should probably add that Java learned it long ago, and yet we
>>>>>> adopted
>>>>>> it anyway... blergh.
>>>>>>
>>>>>
>>>>> That is what I was about to say. No point of doing D if it is to
>>>>> repeat
>>>>> previously done errors.
>>>>
>>>> So what is the lesson Java learned, and how does it address
>>>> multithreaded programming in wake of that lesson?
>>>>
>>>> Andrei
>>>
>>> It learned that allowing locking on arbitrary objects makes controlling
>>> locking (and thus reducing the chance for deadlocks) impossible.
>>
>> And how does Java address multithreading in general, and those issues in
>> particular, today?
>>
>> Andrei
>>
>
> It doesn't, and neither does C#. Java still encourages using
> synchronized, and C# still encourages using lock, but many prominent
> figures in those programming language communities have written blog
> posts on why these language constructs are evil and should be avoided.

Some citations (beyond the known fallacies of Java 1.0) would be great. Thanks.

> Besides, it seems to me that D can't quite make up its mind. We have TLS
> by default, and we encourage message-passing (through a library
> mechanism), and then we have the synchronized statement and attribute.
> It just seems so incredibly inconsistent. synchronized encourages doing
> the wrong thing (locks and synchronization).

Each paradigm has its place. Lock-based programming is definitely here to stay, and when the paradigm is needed, doing it with synchronized objects is better than most alternatives.


Andrei
May 29, 2012
On 30-05-2012 00:50, Andrei Alexandrescu wrote:
> On 5/29/12 2:59 PM, Alex Rønne Petersen wrote:
>> On 29-05-2012 23:31, Andrei Alexandrescu wrote:
>>> On 5/29/12 1:32 AM, deadalnix wrote:
>>>> I already did some comment about this.
>>>>
>>>> Making any object synchronization is a very bad design decision. It is
>>>> deadlock prone, it is liquid lock prone, it cause an overhead on any
>>>> object, and even worse, it is useless most of the time as D promote
>>>> thread locality (which is very good).
>>>
>>> Actually I think such a characterization is superficial and biased to
>>> the extent it becomes wrong.
>>
>> Really ? I think he's spot on.
>
> I'd be glad to think the same. Good arguments might be helpful. So far
> I've seen exaggerated statements and posturing. Such are entertaining
> but are of limited effectiveness in furthering a point.
>
>>> Locks are deadlock-prone by design, whether used with objects or with
>>> classic procedural programs. In fact they are better confined to objects
>>> because the association between the protected data and the
>>> synchronization object is clear.
>>
>> There is no doubt that synchronization of any kind is hard and
>> error-prone no matter how you do it. But do you really think that the
>> situation is made better by allowing locking on *anything*, meaning that
>> locks are effectively resources shared publicly?
>>
>> Besides, what's wrong with core.sync.mutex?
>
> Your very argument that anyone can lock on it. To that, I'll add that
> core.sync.mutex is not properly associated with the resource it
> protects, so coding with bare mutexes is in fact inferior to coding with
> synchronized objects.
>
>
> Andrei

I'm not sure I follow.

A mutex can be stored privately. Any object can be locked on, meaning no lock is effectively protected or private or... anything. It encourages shared locks, which is seriously the worst deadlock inducing anti-pattern to have ever manifested in multithreaded programming.

-- 
Alex Rønne Petersen
alex@lycus.org
http://lycus.org
May 29, 2012
On 30-05-2012 00:45, Andrei Alexandrescu wrote:
> On 5/29/12 2:57 PM, Alex Rønne Petersen wrote:
>> On 29-05-2012 23:33, Andrei Alexandrescu wrote:
>>> On 5/29/12 1:37 AM, deadalnix wrote:
>>>> I would say that breaking things here, with the right deprecation
>>>> process, is the way to go.
>>>
>>> So what should we use for mutex-based synchronization if we deprecate
>>> synchronized classes?
>>>
>>> Andrei
>>
>> core.sync.mutex
>
> That's worse.
>
> Andrei
>

I don't agree. Also, it is faster than object monitors. This was proven by David Simcha recently where he sped up GC allocations by some 40%-ish factor just by changing to a final class derived from core.sync.mutex. And no, that's not a bug in the monitor implementation. It's a limitation of the design.

-- 
Alex Rønne Petersen
alex@lycus.org
http://lycus.org
May 29, 2012
On 5/29/12 3:06 PM, Dmitry Olshansky wrote:
> On 30.05.2012 1:41, Andrei Alexandrescu wrote:
>>> If anything I'd rather re-implement the whole v-table infrastructure via
>>> mixins, templates & friends.
>>
>> Could you please elaborate how that would help multithreading?
>>
>
> It's unrelated to "ease multithreading part strictly speaking.
> My observation is that it leads to at least removal of 1 word per
> object. Not a small thing if you happen to use GC, that "sells his
> memory" in 16 byte chunks, chances are you'd waste 2 instead of one.

So there is concern about the word per object wasted by the possible mutex. I understand.

> Again strictly speaking I'm of an opinion that having mutex together
> with object guarded by it is at least better then 2 separate entities
> bound together by virtue of code comments :)

Absolutely. I hope you agree that this essentially means you're advocating a Java-style approach in which the mutex is implicitly present...

> In any case if mutex is desired, object could have had some other base
> type say SyncObject.

... albeit not in all objects, only a subhierarchy thereof. I posted the same thing. Nice :o).

> Or use "synchronized class" to that end, what it does now by the way
> - locks on each method?

TDPL's design of synchronized still hasn't been implemented. The design indeed prescribes that all public access to the resource is synchronized.

> More about the actual point is that I've come to believe that there is
> satisfactory way to implement whatever scheme of polymorphism* we want
> within the language on top of structs without 'class' and 'interface'
> keywords, special "flawed" pointer type (i.e. tail-const anyone?), and
> last but not least without new/delete/finalizes (new/delete are still
> overridable, hint-hint) madness.
>
> Ideally I think it should be possible to lower the whole
> interface/object/class infrastructure to code that uses structs with
> direct function pointer tables, etc. Multiple alias this is the key,
> sadly so, otherwise subtyping to multiple interfaces seem not likely.
> Then some future compiler may even chose to not provide OOP as built-in
> but lower to this manual implementation on top of struct(!).

Well all of these are nice thoughts but at some point we must acknowledge we're operating within the confines of an already-defined language.

> *I like the one in Smalltalk or Obj-C. Also I think exposing type-tag as
> ordinal (index inside one global master v-table) instead of pointless
> _hidden_ v-table pointer could be interesting in certain designs.
> Another idea is to try tackling multi-methods via some form of
> compressed 2-stage v-table. (my recent work on generalized tries in D
> sparked some ideas)

Any post that starts with taking an issue against the waste of one word and ends advocating Smalltalk and Obj-C is... ho-hum.


Andrei
May 29, 2012
On 30-05-2012 00:45, Andrei Alexandrescu wrote:
> On 5/29/12 2:56 PM, Alex Rønne Petersen wrote:
>> On 29-05-2012 23:38, Andrei Alexandrescu wrote:
>>> On 5/29/12 5:26 AM, Alex Rønne Petersen wrote:
>>>> Generalized object monitors is the worst idea in programming language
>>>> and virtual machine design, ever.
>>>
>>> I think that's an exaggeration. Care to elaborate a bit?
>>>
>>> Andrei
>>>
>>
>> 1) You waste an entire word of memory in *every single object you ever
>> create*. And for the majority of objects, this word will always be
>> zero/null. Not to mention that when you do allocate a monitor, you also
>> get the actual memory of that monitor in addition.
>
> I agree. It would be better if synchronizable objects would be part of a
> sub-hierarchy such that not everyone pays for the word.
>
>> 2) Anyone can lock on any object meaning it's near impossible to see
>> where a deadlock might come from.
>
> What would be the alternative? Deadlocks are a natural consequence of
> the fact that indeed any thread can wait on a resource.

But mutexes allow proper encapsulation by hiding the mutex resource. As I've proven in the very OP of this thread, both druntime and phobos suffer from the anti-pattern that is locking on a public resource when you shouldn't. The language encourages doing it with this synchronized business.

>
>> 3) Encapsulation is completely broken as a result of (2).
>
> Again, what would be the alternative?
>
>> 4) There's a whole bunch of runtime and GC plumbing (which is even
>> broken in druntime right now) to support object monitors. To fix this
>> entire mess, we'd need weak references.
>
> That's more like an argument the implementation is imperfect, not that
> the idea is the worst ever.

Of course. But it's enough reason for me to have banned synchronized from my source base entirely. This point is a practical matter more than it's a design matter.

>
>> 5) This whole object monitor model goes against our "thread-local by
>> default" model. Synchronization is evil and should be explicit with
>> core.sync.mutex.
>
> But doing things with core.sync.mutex is a definite step backwards as it
> is prey to all of the issues above. For example, "anyone can lock any
> mutex". How is that progress?

You encapsulate the mutex and thereby avoid the anti-pattern I pointed out in the OP, which has evidently found its way into both druntime and phobos.

>
>
> Andrei
>
>

-- 
Alex Rønne Petersen
alex@lycus.org
http://lycus.org
May 29, 2012
On 5/29/12 3:56 PM, Alex Rønne Petersen wrote:
> A mutex can be stored privately.

It can, but that doesn't mean it will.

> Any object can be locked on, meaning no
> lock is effectively protected or private or... anything.

To paraphrase you, "An object can be stored privately".

> It encourages
> shared locks, which is seriously the worst deadlock inducing
> anti-pattern to have ever manifested in multithreaded programming.

I'll ignore the hyperbole and continued posturing. Please understand it does absolutely nothing in carrying your point. A technical question I have is - what are shared locks, and what are superior alternatives to them?


Andrei