Would it be useful if 'shared' in D did something like 'volatile' in C++ (as in, Andrei's article on volatile-correctness)?
http://www.drdobbs.com/cpp/volatile-the-multithreaded-programmers-b/184403766

11/15/2012 8:33 AM, Michel Fortin пишет:

> If you want to declare the mutex separately, you could do it by
> specifying a variable instead of a type in the variable declaration:
>
>      Mutex m;
>      synchronized(m) int i;
>
>      synchronized(i)
>      {
>          // implicit: m.lock();
>          // implicit: scope (exit) m.unlock();
>          i++;
>      }

While the rest of proposal was more or less fine. I don't get why we need escape control of mutex at all - in any case it just opens a possibility to shout yourself in the foot.

I'd say:
"Need direct access to mutex? - Go on with the manual way it's still right there (and scope(exit) for that matter)".

Another problem is that somebody clever can escape reference to unlocked 'i' inside of synchronized to somewhere else.

But anyway we can make it in the library right about now.

synchronized T ---> Synchronized!T
synchronized(i){ ... } --->

i.access((x){
//will lock & cast away shared T inside of it
	...
});

I fail to see what it doesn't solve (aside of syntactic sugar).

The key point is that Synchronized!T is otherwise an opaque type.
We could pack a few other simple primitives like 'load', 'store' etc. All of them will go through lock-unlock.

Even escaping a reference can be solved by passing inside of 'access'
a proxy of T. It could even asserts that the lock is in indeed locked.

Same goes about Atomic!T. Though the set of primitives is quite limited depending on T.
(I thought that built-in shared(T) is already atomic though so no need to reinvent this wheel)

It's time we finally agree that 'shared' qualifier is an assembly language of multi-threading based on sharing. It just needs some safe patterns in the library.

That and clarifying explicitly what guarantees (aside from being well.. being shared) it provides w.r.t. memory model.

Until reaching this thread I was under impression that shared means:
- globally visible
- atomic operations for stuff that fits in one word
- sequentially consistent guarantee
- any other forms of access are disallowed except via casts

-- 
Dmitry Olshansky

On Nov 15, 2012, at 3:16 AM, Regan Heath <regan@netmail.co.nz> wrote:
> 
> I suggested something similar as did Sönke: http://forum.dlang.org/thread/k7orpj$1tt5$1@digitalmars.com?page=2#post-op.wnnuiio554xghj:40puck.auriga.bhead.co.uk
> 
> According to deadalnix the compiler magic I suggested to add the mutex isn't possible: http://forum.dlang.org/thread/k7orpj$1tt5$1@digitalmars.com?page=3#post-k7qsb5:242gqk:241:40digitalmars.com
> 
> Most of our ideas can be implemented with a wrapper template containing the sync object (mutex, etc).

If I understand you correctly, you don't need anything that explicitly contains the sync object.  A global table of mutexes used according to the address of the value to be mutated should work.


> So... my feeling is that the best solution for "shared", ignoring the memory barrier aspect which I would relegate to a different feature and solve a different way, is..
> 
> 1. Remove the existing mutex from object.
> 2. Require that all objects passed to synchronized() {} statements implement a synchable(*) interface
> 3. Design a Shared(*) wrapper template/struct that contains a mutex and implements synchable(*)
> 4. Design a Shared(*) base class which contains a mutex and implements synchable(*)

It would be nice to eliminate the mutex that's optionally built into classes now.  The possibility of having to allocate a new mutex on whatever random function call happens to be the first one with "synchronized" is kinda not great.

On Nov 14, 2012, at 6:28 PM, Andrei Alexandrescu <SeeWebsiteForEmail@erdani.org> wrote:

> On 11/14/12 4:50 PM, Sean Kelly wrote:
>> On Nov 14, 2012, at 2:25 PM, Andrei Alexandrescu<SeeWebsiteForEmail@erdani.org>  wrote:
>> 
>>> On 11/14/12 1:09 PM, Walter Bright wrote:
>>>> Yes. And also, I agree that having something typed as "shared" must prevent the compiler from reordering them. But that's separate from inserting memory barriers.
>>> 
>>> It's the same issue at hand: ordering properly and inserting barriers are two ways to ensure one single goal, sequential consistency. Same thing.
>> 
>> Sequential consistency is great and all, but it doesn't render concurrent code correct.  At worst, it provides a false sense of security that somehow it does accomplish this, and people end up actually using it as such.
> 
> Yah, but the baseline here is acquire-release which has subtle differences that are all the more maddening.

Really?  Acquire-release always seemed to have equivalent safety to me.  Typically, the user doesn't even have to understand that optimization can occur upwards across the trailing boundary of the block, etc, to produce correct code.  Though I do agree that the industry is moving towards sequential consistency, so there may be no point in trying for something weaker.

On Nov 11, 2012, at 6:30 PM, Walter Bright <newshound2@digitalmars.com> wrote:
> 
> To make a shared type work in an algorithm, you have to:
> 
> 1. ensure single threaded access by aquiring a mutex
> 2. cast away shared
> 3. operate on the data
> 4. cast back to shared
> 5. release the mutex


So what happens if you pass a reference to the now non-shared object to a function that caches a local reference to it?  Half the point of the attribute is to protect us from accidents like this.

On Nov 15, 2012, at 4:54 AM, deadalnix <deadalnix@gmail.com> wrote:

> Le 14/11/2012 21:01, Sean Kelly a écrit :
>> On Nov 14, 2012, at 6:32 AM, Andrei Alexandrescu<SeeWebsiteForEmail@erdani.org>  wrote:
>>> 
>>> This is a simplification of what should be going on. The core.atomic.{atomicLoad, atomicStore} functions must be intrinsics so the compiler generate sequentially consistent code with them (i.e. not perform certain reorderings). Then there are loads and stores with weaker consistency semantics (acquire, release, acquire/release, and consume).
>> 
>> No.  These functions all contain volatile ask blocks.  If the compiler respected the "volatile" it would be enough.
> 
> It is sufficient for monocore and mostly correct for x86. But isn't enough.
> 
> volatile isn't for concurency, but memory mapping.

Traditionally, the term "volatile" is for memory mapping.  The description of "volatile" for D1, though, would have worked for concurrency.  Or is there some example you can provide where this isn't true?

On Nov 15, 2012, at 5:10 AM, deadalnix <deadalnix@gmail.com> wrote:

> Le 14/11/2012 23:21, Andrei Alexandrescu a écrit :
>> On 11/14/12 12:00 PM, Sean Kelly wrote:
>>> On Nov 14, 2012, at 6:16 AM, Andrei Alexandrescu<SeeWebsiteForEmail@erdani.org> wrote:
>>> 
>>>> On 11/14/12 1:20 AM, Walter Bright wrote:
>>>>> On 11/13/2012 11:37 PM, Jacob Carlborg wrote:
>>>>>> If the compiler should/does not add memory barriers, then is there a
>>>>>> reason for
>>>>>> having it built into the language? Can a library solution be enough?
>>>>> 
>>>>> Memory barriers can certainly be added using library functions.
>>>> 
>>>> The compiler must understand the semantics of barriers such as e.g. it doesn't hoist code above an acquire barrier or below a release barrier.
>>> 
>>> That was the point of the now deprecated "volatile" statement. I still don't entirely understand why it was deprecated.
>> 
>> Because it's better to associate volatility with data than with code.
> 
> Happy to see I'm not alone on that one.
> 
> Plus, volatile and sequential consistency are 2 different beast. Volatile means no register promotion and no load/store reordering. It is required, but not sufficient for concurrency.

It's sufficient for concurrency when coupled with library code that does the hardware-level synchronization.  In short, a program has two separate machines doing similar optimizations on it: the compiler and the CPU.  In D we can use ASM to control CPU optimizations, and in D1 we had "volatile" to control compiler optimizations.  "volatile" was the minimum required for handling the compiler portion and was easy to get wrong, but it used only one keyword and I suspect was relatively easy to implement on the compiler side as well.

On Nov 15, 2012, at 5:16 AM, deadalnix <deadalnix@gmail.com> wrote:
> 
> What is the point of ensuring that the compiler does not reorder load/stores if the CPU is allowed to do so ?

Because we can write ASM to tell the CPU not to.  We don't have any such ability for the compiler right now.

On Nov 15, 2012, at 7:17 AM, Andrei Alexandrescu <SeeWebsiteForEmail@erdani.org> wrote:

> On 11/15/12 1:08 AM, Manu wrote:
>> 
>> Side note: I still think a convenient and fairly practical solution is
>> to make 'shared' things 'lockable'; where you can lock()/unlock() them,
>> and assignment to/from shared things is valid (no casting), but a
>> runtime assert insists that the entity is locked whenever it is
>> accessed.
> 
> This (IIUC) is conflating mutex-based synchronization with memory models and atomic operations. I suggest we postpone anything related to that for the sake of staying focused.

By extension, I'd suggest postponing anything related to classes as well.

On 2012-11-15 11:52, Manu wrote:

> Interesting concept. Nice idea, could certainly be useful, but it
> doesn't address the problem as directly as my suggestion.
> There are still many problem situations, for instance, any time a
> template is involved. The template doesn't know to do that internally,
> but under my proposal, you lock it prior to the workload, and then the
> template works as expected. Templates won't just break and fail whenever
> shared is involved, because assignments would be legal. They'll just
> assert that the thing is locked at the time, which is the programmers
> responsibility to ensure.

I don't understand how a template would cause problems.

-- 
/Jacob Carlborg