May 30, 2012
On 2012-05-30 00:06, Dmitry Olshansky wrote:

> *I like the one in Smalltalk or Obj-C. Also I think exposing type-tag as
> ordinal (index inside one global master v-table) instead of pointless
> _hidden_ v-table pointer could be interesting in certain designs.
> Another idea is to try tackling multi-methods via some form of
> compressed 2-stage v-table. (my recent work on generalized tries in D
> sparked some ideas)

I like that classes are first class citizens in Objective-C and Ruby. Class methods instead of static methods. Easily extendable from outside the class. But I still like to have the "class" and "interface" keywords.

Would this solve the problem with const:

https://github.com/D-Programming-Language/dmd/pull/3

-- 
/Jacob Carlborg
May 30, 2012
Le 29/05/2012 23:33, Andrei Alexandrescu a écrit :
> On 5/29/12 1:37 AM, deadalnix wrote:
>> I would say that breaking things here, with the right deprecation
>> process, is the way to go.
>
> So what should we use for mutex-based synchronization if we deprecate
> synchronized classes?
>
> Andrei

I think something similar to range design here is the way to go.

It is easy to define something like

template isLockable(T) {
	enum isLockable = isShared!T && is(typeof(T.init.lock())) && is(typeof(T.init.release()));
}

And allow locking only if(isLockable!Type) .

Now we can create SyncObject or any structure we want. The point is that we lock explicit stuff.
May 30, 2012
Le 30/05/2012 08:23, Sean Kelly a écrit :
> On May 29, 2012, at 4:07 PM, Andrei Alexandrescu<SeeWebsiteForEmail@erdani.org>  wrote:
>
>> On 5/29/12 3:58 PM, Alex Rønne Petersen wrote:
>>> On 30-05-2012 00:45, Andrei Alexandrescu wrote:
>>>> On 5/29/12 2:57 PM, Alex Rønne Petersen wrote:
>>>>> On 29-05-2012 23:33, Andrei Alexandrescu wrote:
>>>>>> On 5/29/12 1:37 AM, deadalnix wrote:
>>>>>>> I would say that breaking things here, with the right deprecation
>>>>>>> process, is the way to go.
>>>>>>
>>>>>> So what should we use for mutex-based synchronization if we deprecate
>>>>>> synchronized classes?
>>>>>>
>>>>>> Andrei
>>>>>
>>>>> core.sync.mutex
>>>>
>>>> That's worse.
>>>>
>>>> Andrei
>>>>
>>>
>>> I don't agree.
>>
>> One simple thing to understand is that core.sync.mutex does everything synchronized objects do, in a much less structured way. So it's tenuous to build an argument that synchronized classes do something wrong but bare, unstructured mutexes do something good.
>>
>>> Also, it is faster than object monitors. This was proven
>>> by David Simcha recently where he sped up GC allocations by some 40%-ish
>>> factor just by changing to a final class derived from core.sync.mutex.
>>> And no, that's not a bug in the monitor implementation. It's a
>>> limitation of the design.
>>
>> We'd need to take a look at that. I recall at a point Bartosz was working on cheap locks using the mutex word as a spin lock in certain circumstances. Anyhow, it's something that would be interesting to look at.
>
> I bet this is because monitors are lazily initialized, so the cost of acquiring a lock is more than just locking the underlying mutex. The implementation for built-in monitors really isnt great. I've been meaning to do something about that.

And it use double check locking the wrong way, so can eventually explode sometime.
May 30, 2012
Le 30/05/2012 00:50, Andrei Alexandrescu a écrit :
> On 5/29/12 2:59 PM, Alex Rønne Petersen wrote:
>> On 29-05-2012 23:31, Andrei Alexandrescu wrote:
>>> On 5/29/12 1:32 AM, deadalnix wrote:
>>>> I already did some comment about this.
>>>>
>>>> Making any object synchronization is a very bad design decision. It is
>>>> deadlock prone, it is liquid lock prone, it cause an overhead on any
>>>> object, and even worse, it is useless most of the time as D promote
>>>> thread locality (which is very good).
>>>
>>> Actually I think such a characterization is superficial and biased to
>>> the extent it becomes wrong.
>>
>> Really ? I think he's spot on.
>
> I'd be glad to think the same. Good arguments might be helpful. So far
> I've seen exaggerated statements and posturing. Such are entertaining
> but are of limited effectiveness in furthering a point.
>

I have provided link in other posts to document the point. Not to mention that my personal experience back up this too.

I don't like what you are doing here, because you don't provide anything and simply discard what don't fit the current design. For instance, you didn't even considered the liquid lock problem.
May 30, 2012
On 30.05.2012 13:28, Jacob Carlborg wrote:
> On 2012-05-30 00:06, Dmitry Olshansky wrote:
>
>> *I like the one in Smalltalk or Obj-C. Also I think exposing type-tag as
>> ordinal (index inside one global master v-table) instead of pointless
>> _hidden_ v-table pointer could be interesting in certain designs.
>> Another idea is to try tackling multi-methods via some form of
>> compressed 2-stage v-table. (my recent work on generalized tries in D
>> sparked some ideas)
>
> I like that classes are first class citizens in Objective-C and Ruby.
> Class methods instead of static methods. Easily extendable from outside
> the class. But I still like to have the "class" and "interface" keywords.
>

Yup. Even Java has Class as object. If D copied most of OOP from Java it seems strange that it was replaced by halfhearted factory method in object.di
kind of saying "you have RTTI but it's crippled just in case if it is of no use".

Still I'd think it's reasonable course to enact slow but steady evolution of existing OOP design, changing synchronized/monitors seems as good start as any.

> Would this solve the problem with const:
>
> https://github.com/D-Programming-Language/dmd/pull/3
>

I know, and hoped it would happen sooner. But at least it's possible.

-- 
Dmitry Olshansky
May 30, 2012
On 30.05.2012 2:27, Dmitry Olshansky wrote:
>
> I just wish it was different. To set things straight I still believe
> that OOP part of language is not what I want it to be, and thus I just
> don't use it. Like in the old saying: there is always a choice.
>

Minor correction, when I first come to try D I used classes, I confess :). The toy project of 2D graphics engine needed resource manager, thus I used Flyweight pattern with lazy-loading/unloding behind the scenes. The concrete resource were struct on C heap but handles were  small objects were derived from Resource fro obvious reasons.

Brings us back to extra word per object problem, flyweight is exactly kind of pattern that is hit by collateral damage of this decision!

OT: Actually algorithms to lazily cache resources are very similar to virtual memory management, same strategies, same action just replace "swap-out page" with "unload resource".

TL;DR all of this worked out quite well then I lost interest in the project but certainly not D itself.

> Things that stopped me from using it are:
> - hidden v-table design like C++ and Java, doesn't help things much.
> Encapsulation is good, but if it has no advantages other then hiding
> things it's bad.
> - GCed by default, no provision for custom allocation until very recently
> - monitor field, and who knows what else (v-table obviously, maybe
> something else?)
> - no tail-const, no solutions aside from rebindable and casts. Pointers
> to struct look so much better in this regard ;)
> - special slow built-in protocols for equality (it's robust, but I don't
> need it)
> - opEquals signature madness (probably fixed?)
> - final vs virtual, no inlining of known ahead-of-time virtual call
> - nowdays enforced purity, @safe, nothrow whatever. Why should I follow
> these restrictions - dubious.
>
> Observe that all of the above have very few advantages brought to me by
> compiler:
> - polymorphism
> - inheritance
> - contract inheritance
>
> Of the above I only ever needed the first 2. Call me dinosaur.
>

Throwing in this one:
	-useless RTTi that still occupies space.
(the useless part might have been fixed recently? Some customizable field added to Typeinfo - dunno)

-- 
Dmitry Olshansky
May 30, 2012
On 30-05-2012 11:15, deadalnix wrote:
> Le 30/05/2012 01:25, Alex Rønne Petersen a écrit :
>> On 30-05-2012 01:22, Andrei Alexandrescu wrote:
>>> On 5/29/12 4:17 PM, Alex Rønne Petersen wrote:
>>>> On 30-05-2012 01:10, Andrei Alexandrescu wrote:
>>>>> On 5/29/12 4:06 PM, Alex Rønne Petersen wrote:
>>>>>> Synchronized blocks are good because they
>>>>>> operate on an implicit, hidden, global mutex. You can't screw up with
>>>>>> that.
>>>>>
>>>>> I think there's quite some disconnect here. If there's any
>>>>> anti-pattern
>>>>> in this discussion, it's operating on an implicit, hidden, global
>>>>> mutex.
>>>>> Walter agreed to eliminate that from D, but never got around to it.
>>>>>
>>>>> Andrei
>>>>
>>>> I'd love to hear why you think this design is problematic as opposed to
>>>> one that lets users accidentally expose synchronization issues to
>>>> consumers of their API surface, which is what many people end up doing
>>>> since synchronized (this) or even synchronized (this.classinfo) are
>>>> allowed at all.
>>>>
>>>> (I've seen countless cases of those two horrible abuses of synchronized
>>>> especially from people asking questions on e.g. IRC.)
>>>
>>> I think the most egregious example is synchronization by global lock
>>> done in Python and other languages. It has very nice semantics (stop the
>>> world, do something, resume the world), but scales poorly enough to be
>>> universally considered a failure. Quite honest I'm shocked that you're
>>> advocating it.
>>>
>>> Andrei
>>>
>>>
>>
>> I'm not advocating it. I don't use it myself. I'm just saying that I
>> find it to be less of an issue than synchronizing on arbitrary objects,
>> because the latter is error-prone.
>>
>> And yes, Python is a textbook example of synchronization gone completely
>> wrong. But that doesn't mean that you don't sometimes have to
>> synchronize on some global resource, and in those cases, it can be
>> useful syntactic sugar (but certainly not essential).
>>
>
> I have to agree with Andrei here. The global mutex isn't the solution
> either. The mutex should be explicit, and under control (not this).

I've been arguing for being explicit all along. :)

What I've been saying in this particular part of the thread is just that synchronized blocks that don't operate on a specific monitor object are less error-prone. Nothing else.

-- 
Alex Rønne Petersen
alex@lycus.org
http://lycus.org
May 30, 2012
On 30-05-2012 08:23, Sean Kelly wrote:
> On May 29, 2012, at 4:07 PM, Andrei Alexandrescu<SeeWebsiteForEmail@erdani.org>  wrote:
>
>> On 5/29/12 3:58 PM, Alex Rønne Petersen wrote:
>>> On 30-05-2012 00:45, Andrei Alexandrescu wrote:
>>>> On 5/29/12 2:57 PM, Alex Rønne Petersen wrote:
>>>>> On 29-05-2012 23:33, Andrei Alexandrescu wrote:
>>>>>> On 5/29/12 1:37 AM, deadalnix wrote:
>>>>>>> I would say that breaking things here, with the right deprecation
>>>>>>> process, is the way to go.
>>>>>>
>>>>>> So what should we use for mutex-based synchronization if we deprecate
>>>>>> synchronized classes?
>>>>>>
>>>>>> Andrei
>>>>>
>>>>> core.sync.mutex
>>>>
>>>> That's worse.
>>>>
>>>> Andrei
>>>>
>>>
>>> I don't agree.
>>
>> One simple thing to understand is that core.sync.mutex does everything synchronized objects do, in a much less structured way. So it's tenuous to build an argument that synchronized classes do something wrong but bare, unstructured mutexes do something good.
>>
>>> Also, it is faster than object monitors. This was proven
>>> by David Simcha recently where he sped up GC allocations by some 40%-ish
>>> factor just by changing to a final class derived from core.sync.mutex.
>>> And no, that's not a bug in the monitor implementation. It's a
>>> limitation of the design.
>>
>> We'd need to take a look at that. I recall at a point Bartosz was working on cheap locks using the mutex word as a spin lock in certain circumstances. Anyhow, it's something that would be interesting to look at.
>
> I bet this is because monitors are lazily initialized, so the cost of acquiring a lock is more than just locking the underlying mutex. The implementation for built-in monitors really isnt great. I've been meaning to do something about that.

We also need to fix the monitor memory leak that sometimes manifests itself...

-- 
Alex Rønne Petersen
alex@lycus.org
http://lycus.org
May 30, 2012
On Wed, 30 May 2012 10:21:00 +0100, deadalnix <deadalnix@gmail.com> wrote:

> Le 30/05/2012 00:53, Andrei Alexandrescu a écrit :
>> On 5/29/12 3:01 PM, Alex Rønne Petersen wrote:
>>> On 29-05-2012 23:54, Andrei Alexandrescu wrote:
>>>> On 5/29/12 2:49 PM, Alex Rønne Petersen wrote:
>>>>> On 29-05-2012 23:32, Andrei Alexandrescu wrote:
>>>>>> On 5/29/12 1:35 AM, deadalnix wrote:
>>>>>>> Le 29/05/2012 01:38, Alex Rønne Petersen a écrit :
>>>>>>>> I should probably add that Java learned it long ago, and yet we
>>>>>>>> adopted
>>>>>>>> it anyway... blergh.
>>>>>>>>
>>>>>>>
>>>>>>> That is what I was about to say. No point of doing D if it is to
>>>>>>> repeat
>>>>>>> previously done errors.
>>>>>>
>>>>>> So what is the lesson Java learned, and how does it address
>>>>>> multithreaded programming in wake of that lesson?
>>>>>>
>>>>>> Andrei
>>>>>
>>>>> It learned that allowing locking on arbitrary objects makes controlling
>>>>> locking (and thus reducing the chance for deadlocks) impossible.
>>>>
>>>> And how does Java address multithreading in general, and those issues in
>>>> particular, today?
>>>>
>>>> Andrei
>>>>
>>>
>>> It doesn't, and neither does C#. Java still encourages using
>>> synchronized, and C# still encourages using lock, but many prominent
>>> figures in those programming language communities have written blog
>>> posts on why these language constructs are evil and should be avoided.
>>
>> Some citations (beyond the known fallacies of Java 1.0) would be great.
>> Thanks.
>>
>>> Besides, it seems to me that D can't quite make up its mind. We have TLS
>>> by default, and we encourage message-passing (through a library
>>> mechanism), and then we have the synchronized statement and attribute.
>>> It just seems so incredibly inconsistent. synchronized encourages doing
>>> the wrong thing (locks and synchronization).
>>
>> Each paradigm has its place. Lock-based programming is definitely here
>> to stay, and when the paradigm is needed, doing it with synchronized
>> objects is better than most alternatives.
>>
>>
>> Andrei
>
> No !
>
> You don't want to synchronize on ANY object. You want to synchronize on explicit mutexes.

+1 .. this is the key point/issue.  If all objects can be locked, and if synchronized classes/methods use the /same/ mutex you can easily/accidentally have hard to find deadlocks.  The deadlocks in Q are usually (at least) two threads locking (at least) two objects in the opposite order and with synchronized classes/methods the fact that a lock is being taken is not always obvious from the code (at the call site) so the problem code can look fairly innocuous.

**[Example 1]**

class C
{
  synchronized void ccc() { }
}

class D
{
  synchronized void ddd() { }
}

C c = new C();
D d = new D();

[thread1]
..lock(c)  // locks c
{          // deadlock window
  d.ddd(); // locks d
}

[thread2]
..lock(d)  // locks d
{          // deadlock window
  c.ccc(); // locks c
}

**[Example 2]**

class A
{
  B b;
  ..
  synchronized void foo()  // locks a
  {                        // deadlock window
    b.bar();               // locks b
  }
}

class B
{
  A a;
  ..
  synchronized void bar()  // locks b
  {                        // deadlock window
    a.foo();               // locks a
  }
}

A a = new A();
B b = new B();

a.b = b;
b.a = a;

[thread1]
a.foo(); // locks a then b

[thread2]
b.bar(); // locks b then a

So, the root cause of the problem is that synchronized classes/methods use a publicly visible mutex (the object/this pointer) to perform their locking.  The solution is to lock on a private mutex, which no-one else can lock unexpectedly as described in the link below:

> See that link for instance : http://msdn.microsoft.com/en-us/library/ms173179.aspx

Now, ideally we want to make it hard for people to screw this up in this manner and there are several ways to do it.

1. Prevent locking on any/every object.  People would have to create a separate mutex object for locking.  This is a little tedious, but it does make people think about the scope of the locking (where to put the mutex, who can see/use it, etc).  On the flipside it can make people re-use a mutex for multiple conceptual critical section areas of code, which is less than ideal, but at least it's not 'broken'.

2. Allow locking on any/every object but use a different mutex for synchronized class/methods.  So 'synchronized' on a method call locks a private mutex object, not the object itself.  And synchronized(object) locks the public mutex object.  The downside here is that now every object potentially has 2 mutex objects/handles internally - a public and a private one.

.. there are probably a number of other ways to do it.  I favour #1, especially as thread-local is the default storage type now.  100% of objects in a single threaded application don't need a built-in mutex, most objects (80%??) in a threaded application will be thread-local and don't need one.  Only those few shared objects, which are actually manually locked with synchronized or have synchronized class/methods actually need a mutex.  It seems that removing this will save memory and avoid tricky deadlock bugs and the cost is having to manually create a mutex object (1/2 lines of code) to lock instead.. it seems like a good idea to me.

R

-- 
Using Opera's revolutionary email client: http://www.opera.com/mail/
May 30, 2012
Le 30/05/2012 14:32, Regan Heath a écrit :
> On Wed, 30 May 2012 10:21:00 +0100, deadalnix <deadalnix@gmail.com> wrote:
>
>> Le 30/05/2012 00:53, Andrei Alexandrescu a écrit :
>>> On 5/29/12 3:01 PM, Alex Rønne Petersen wrote:
>>>> On 29-05-2012 23:54, Andrei Alexandrescu wrote:
>>>>> On 5/29/12 2:49 PM, Alex Rønne Petersen wrote:
>>>>>> On 29-05-2012 23:32, Andrei Alexandrescu wrote:
>>>>>>> On 5/29/12 1:35 AM, deadalnix wrote:
>>>>>>>> Le 29/05/2012 01:38, Alex Rønne Petersen a écrit :
>>>>>>>>> I should probably add that Java learned it long ago, and yet we
>>>>>>>>> adopted
>>>>>>>>> it anyway... blergh.
>>>>>>>>>
>>>>>>>>
>>>>>>>> That is what I was about to say. No point of doing D if it is to
>>>>>>>> repeat
>>>>>>>> previously done errors.
>>>>>>>
>>>>>>> So what is the lesson Java learned, and how does it address
>>>>>>> multithreaded programming in wake of that lesson?
>>>>>>>
>>>>>>> Andrei
>>>>>>
>>>>>> It learned that allowing locking on arbitrary objects makes
>>>>>> controlling
>>>>>> locking (and thus reducing the chance for deadlocks) impossible.
>>>>>
>>>>> And how does Java address multithreading in general, and those
>>>>> issues in
>>>>> particular, today?
>>>>>
>>>>> Andrei
>>>>>
>>>>
>>>> It doesn't, and neither does C#. Java still encourages using
>>>> synchronized, and C# still encourages using lock, but many prominent
>>>> figures in those programming language communities have written blog
>>>> posts on why these language constructs are evil and should be avoided.
>>>
>>> Some citations (beyond the known fallacies of Java 1.0) would be great.
>>> Thanks.
>>>
>>>> Besides, it seems to me that D can't quite make up its mind. We have
>>>> TLS
>>>> by default, and we encourage message-passing (through a library
>>>> mechanism), and then we have the synchronized statement and attribute.
>>>> It just seems so incredibly inconsistent. synchronized encourages doing
>>>> the wrong thing (locks and synchronization).
>>>
>>> Each paradigm has its place. Lock-based programming is definitely here
>>> to stay, and when the paradigm is needed, doing it with synchronized
>>> objects is better than most alternatives.
>>>
>>>
>>> Andrei
>>
>> No !
>>
>> You don't want to synchronize on ANY object. You want to synchronize
>> on explicit mutexes.
>
> +1 .. this is the key point/issue. If all objects can be locked, and if
> synchronized classes/methods use the /same/ mutex you can
> easily/accidentally have hard to find deadlocks. The deadlocks in Q are
> usually (at least) two threads locking (at least) two objects in the
> opposite order and with synchronized classes/methods the fact that a
> lock is being taken is not always obvious from the code (at the call
> site) so the problem code can look fairly innocuous.
>
> **[Example 1]**
>
> class C
> {
> synchronized void ccc() { }
> }
>
> class D
> {
> synchronized void ddd() { }
> }
>
> C c = new C();
> D d = new D();
>
> [thread1]
> ..lock(c) // locks c
> { // deadlock window
> d.ddd(); // locks d
> }
>
> [thread2]
> ..lock(d) // locks d
> { // deadlock window
> c.ccc(); // locks c
> }
>
> **[Example 2]**
>
> class A
> {
> B b;
> ..
> synchronized void foo() // locks a
> { // deadlock window
> b.bar(); // locks b
> }
> }
>
> class B
> {
> A a;
> ..
> synchronized void bar() // locks b
> { // deadlock window
> a.foo(); // locks a
> }
> }
>
> A a = new A();
> B b = new B();
>
> a.b = b;
> b.a = a;
>
> [thread1]
> a.foo(); // locks a then b
>
> [thread2]
> b.bar(); // locks b then a
>
> So, the root cause of the problem is that synchronized classes/methods
> use a publicly visible mutex (the object/this pointer) to perform their
> locking. The solution is to lock on a private mutex, which no-one else
> can lock unexpectedly as described in the link below:
>
>> See that link for instance :
>> http://msdn.microsoft.com/en-us/library/ms173179.aspx
>
> Now, ideally we want to make it hard for people to screw this up in this
> manner and there are several ways to do it.
>
> 1. Prevent locking on any/every object. People would have to create a
> separate mutex object for locking. This is a little tedious, but it does
> make people think about the scope of the locking (where to put the
> mutex, who can see/use it, etc). On the flipside it can make people
> re-use a mutex for multiple conceptual critical section areas of code,
> which is less than ideal, but at least it's not 'broken'.
>

It is suboptimal, but correct.

> 2. Allow locking on any/every object but use a different mutex for
> synchronized class/methods. So 'synchronized' on a method call locks a
> private mutex object, not the object itself. And synchronized(object)
> locks the public mutex object. The downside here is that now every
> object potentially has 2 mutex objects/handles internally - a public and
> a private one.
>

It doesn't address most of the drawback cited. Notably the fact that every object have a monitor field, but most of them will not use it.