> What guarantees is shared supposed to provide?
>
> Shared means that multiple threads can access the data. The guarantee is that if it is not shared, and not immutable, that only the current thread can see it.

What if I have a thread that contains some shared data? Should the thread be created as shared, be cast to shared after construction or not be shared and fine grained shared applied to the respective shared data fields?

> What does shared have to do with synchronization?
>
> Only shared data can be synchronized. It makes no sense to synchronize thread local data.

Define synchronized. With atomic ops on word size items this is clear, but what does it mean for aggregates? The language shows us no connection between synchronization and the shared data. What is one unit of data that is to be synchronized?

> What does shared have to do with memory barriers?
>
> Reading/writing shared data emits memory barriers to ensure
> sequential consistency (not implemented).

That's exactly the problem. It assumes the unit is a word size item. If I have a Mutex to protect my unit of shared data, I don't need "volatile" handling of shared data.

    private shared class SomeThread : Thread
    {
    private:

        Condition m_condition;
        bool m_shutdown = false;
        ...
    }

m_shutdown will be shared and it is shared data, but it is synchronized by the Mutex contained in that condition. Automatic memory barriers and such would only slow down execution.

> What are the semantics of casting FROM unshared TO shared?
>
> Make sure there are no other unshared references to that same data.
>
> What are the semantics of casting FROM shared TO
> unshared?
>
> Make sure there are no other shared references to that same data.

That's just wrong to ask. `SomeThread` is a worker thread and
data is passed to it regularly through a shared reference that
is certainly never going away until the thread dies.
Yet I must be able to "unshare" it's list of work items to
process them.

Now let's say I have an "empty" property. Shared or unshared?

	override @property bool empty() const
	{
		return m_list.empty;
	}

It is only called internally by the thread itself after entering a certain critical section. I _know_ that m_list wont be accessible by other threads while .empty is running.

But this seeming 1:1 relationship between entering "the" critical section and stripping shared is of course non-existent. Aggregates may contain Mutexes protecting different fields or even stacking on top of each other.

So the text should read:

> What are the semantics of casting FROM shared TO unshared?
>
> Make sure that during the period the data is unshared, no other thread can modify those parts of it that you will be accessing. If you don't use synchronization objects with built-in memory-barriers like a Mutex, it is your responsibility to properly synchronize data access through e.g. atomicLoad/Store.

That at least in general sanctifies casting away shared for the purpose of calling a method under protection of a user defined critical section.

-- 
Marco

On Monday, 20 October 2014 at 13:29:47 UTC, Marco Leise wrote:
>> What guarantees is shared supposed to provide?
>>
>> Shared means that multiple threads can access the data. The
>> guarantee is that if it is not shared, and not immutable, that
>> only the current thread can see it.
>
> What if I have a thread that contains some shared data? Should
> the thread be created as shared, be cast to shared after
> construction or not be shared and fine grained shared applied
> to the respective shared data fields?

Since Thread is by its very nature a shared thing, Thread should probably be defined as shared.  But more generally it depends on the use case.


>> What does shared have to do with synchronization?
>>
>> Only shared data can be synchronized. It makes no sense to
>> synchronize thread local data.
>
> Define synchronized. With atomic ops on word size items this
> is clear, but what does it mean for aggregates? The language
> shows us no connection between synchronization and the shared
> data. What is one unit of data that is to be synchronized?

I think there's some conflation of two separate uses of "synchronized" here.  I think the above is actually talking about synchronized methods (ie. involving a mutex).


>> What does shared have to do with memory barriers?
>>
>> Reading/writing shared data emits memory barriers to ensure
>> sequential consistency (not implemented).
>
> That's exactly the problem. It assumes the unit is a word size
> item.

I'd say the real problem is more that it assumes, or at least suggests, that sequential consistency of shared variables will result in a correct program.  It won't, for any non-trivial uses of shared variables.  Lock-free programming is really, really hard, even for experts.  Using shared variables in this way shouldn't be easy semantically because it provides a false sense of security, resulting in programs that are silently broken in weird ways under some conditions but not others.


> If I have a Mutex to protect my unit of shared data, I
> don't need "volatile" handling of shared data.
>
>     private shared class SomeThread : Thread
>     {
>     private:
>
>         Condition m_condition;
>         bool m_shutdown = false;
>         ...
>     }

Yep.  This is one of my biggest issues with shared as it applies to user-defined types.  I even raised it in the now defunct concurrency mailing list before the design was finalized.  Sadly, there's no good way to sort this out, because:

shared class A {
    int m_count = 0;
    void increment() shared {
        m_count.atomicOp!"+="(1);
    }

    int getCount() synchronized {
        return m_count;
    }
}

If we make accesses of shared variables non-atomic inside synchronized methods, there may be conflicts with their use in shared methods.  Also:

shared class A {
    void doSomething() synchronized {
        doSomethingElse();
    }

    private void doSomethingElse() synchronized {

    }
}

doSomethingElse must be synchronized even if I as a programmer know it doesn't have to be because the compiler insists it must be.  And I know that private methods are visible within the module, but the same rule applies.  In essence, we can't avoid recursive mutexes for implementing synchronized, and we're stuck with a lot of recursive locks and unlocks no matter what, as soon as we slap a "shared" label on something.


> m_shutdown will be shared and it is shared data, but it is
> synchronized by the Mutex contained in that condition.
> Automatic memory barriers and such would only slow down
> execution.

Yes.  Though there's no overhead for having a Mutex synchronize one more operation.  A Mutex is basically just a shared variable indicating locked state.  When you leave a Mutex a shared variable is written to to indicate that the Mutex is unlocked, and the memory model in that language/platform/cpu guarantees that all operations logically occurring before that shared write actually do complete before the shared write, at least to anyone who acquires that same mutex before looking at the protected data (ie. there's a reader-writer contract).


>> What are the semantics of casting FROM unshared TO shared?
>>
>> Make sure there are no other unshared references to that same
>> data.
>>
>> What are the semantics of casting FROM shared TO
>> unshared?
>>
>> Make sure there are no other shared references to that same
>> data.
>
> That's just wrong to ask. `SomeThread` is a worker thread and
> data is passed to it regularly through a shared reference that
> is certainly never going away until the thread dies.
> Yet I must be able to "unshare" it's list of work items to
> process them.

Sure, but at that point they are no longer referenced by the shared Thread, correct?  The rule is simply that you can't be trying to read or write data using both shared and unshared operations, because of that reader-writer contract I mentioned above.


> Now let's say I have an "empty" property. Shared or unshared?
>
> 	override @property bool empty() const
> 	{
> 		return m_list.empty;
> 	}
>
> It is only called internally by the thread itself after
> entering a certain critical section. I _know_ that m_list wont
> be accessible by other threads while .empty is running.

So one thing about shared that Walter confirmed at some point is that atomic ops won't be imposed on operations within a shared method.  But I suspect that someone is likely to read the preceding sentence and say "woah! We never said that!  And if we did, that's wrong!"  In short, I agree with you that shared, as described, kind of sucks here because you're stuck with a ton of inefficiency that you, as an intelligent programmer, know is unnecessary.


> But this seeming 1:1 relationship between entering "the"
> critical section and stripping shared is of course
> non-existent. Aggregates may contain Mutexes protecting
> different fields or even stacking on top of each other.
>
> So the text should read:
>
>> What are the semantics of casting FROM shared TO unshared?
>>
>> Make sure that during the period the data is unshared, no
>> other thread can modify those parts of it that you will be
>> accessing. If you don't use synchronization objects with
>> built-in memory-barriers like a Mutex, it is your
>> responsibility to properly synchronize data access through
>> e.g. atomicLoad/Store.
>
> That at least in general sanctifies casting away shared for
> the purpose of calling a method under protection of a user
> defined critical section.

It's more complicated than that, because you don't know how long a given operation needs to propagate to another CPU.  Simply performing a shared write is meaningless if something else is performing an unshared read because the optimization happens at both points--the write side and the read side.

In essence, the CPU performs the same optimizations as a compiler.  Depending on the architecture, reads may be rearranged to occur before other reads or writes, and writes may be rearranged to occur before other reads and writes.  On most architectures the CPU makes some intelligent guesses about what operations are safe to rearrange (look into "dependent loads"), though on some few others like the DEC Alpha (a CPU invented by crazy people), they do not and if you don't explicitly tell them what needs to happen, it won't.

Basically what's needed is some way to have the compiler optimize according to the same rules as the CPU (the goal of "shared").  Or in lieu of that, to have some "don't optimize this" instruction to tell the compiler to keep it's dirty hands off your carefully constructed code so the only thing you need to worry about is what the CPU is trying to do.  This is what "volatile" was meant for in D1 and I really liked it, but I think I was the only one.

There's a paper on release consistency that I think is fantastic.  I'll link it later if I can find it on the interweb.  CPUs seem to be converging on even more strict memory ordering than release consistency, but the release consistency model is really fantastic as it's basically equivalent to how mutexes work and so it's a model everyone already understands.

Am Mon, 20 Oct 2014 16:18:51 +0000
schrieb "Sean Kelly" <sean@invisibleduck.org>:

> On Monday, 20 October 2014 at 13:29:47 UTC, Marco Leise wrote:
> >
> > What if I have a thread that contains some shared data? Should the thread be created as shared, be cast to shared after construction or not be shared and fine grained shared applied to the respective shared data fields?
> 
> Since Thread is by its very nature a shared thing, Thread should probably be defined as shared.  But more generally it depends on the use case.

In a single-threaded application in particular, there is an
unshared thread :p
But to the point: Doesn't defining it as shared means that it
can not have _any_ unshared methods? Ok, fair enough. So even
if a method is only working on technically unshared parts of
the thread's data, it has to cast everything to unshared
itself. This makes sense since `this`, the Thread itself is
still shared.

> […]
> > If I have a Mutex to protect my unit of shared data, I don't need "volatile" handling of shared data.
> >
> >     private shared class SomeThread : Thread
> >     {
> >     private:
> >
> >         Condition m_condition;
> >         bool m_shutdown = false;
> >         ...
> >     }
> 
> Yep.  This is one of my biggest issues with shared as it applies to user-defined types.  I even raised it in the now defunct concurrency mailing list before the design was finalized.  Sadly, there's no good way to sort this out, because:
> 
> shared class A {
>      int m_count = 0;
>      void increment() shared {
>          m_count.atomicOp!"+="(1);
>      }
> 
>      int getCount() synchronized {
>          return m_count;
>      }
> }
> 
> If we make accesses of shared variables non-atomic inside synchronized methods, there may be conflicts with their use in shared methods.  Also:

Well, when you talk about "shared and unshared operations" further down, I took it as the set of operations ensuring thread-safety over a particular set of shared data. That code above is just a broken set of such operations. I.e. in this case the programmer must decide between mutex synchronization and atomic read-modify-write. That's not too much to ask.

> shared class A {
>      void doSomething() synchronized {
>          doSomethingElse();
>      }
> 
>      private void doSomethingElse() synchronized {
> 
>      }
> }
> 
> doSomethingElse must be synchronized even if I as a programmer know it doesn't have to be because the compiler insists it must be.  And I know that private methods are visible within the module, but the same rule applies.  In essence, we can't avoid recursive mutexes for implementing synchronized, and we're stuck with a lot of recursive locks and unlocks no matter what, as soon as we slap a "shared" label on something.

Imagine you have a shared root object that contains a deeply
nested private data structure that is technically unshared.
Then it becomes not only one more method of the root object
that needs to be `synchronized` but it cascades all the way
down its private fields as well. One ends up requiring data
structures designed for single-threaded execution to
grow synchronized methods over night even though they aren't
_really_ used concurrently my multiple threads.

> >> What are the semantics of casting FROM shared TO
> >> unshared?
> >>
> >> Make sure there are no other shared references to that same data.
> >
> > That's just wrong to ask. `SomeThread` is a worker thread and
> > data is passed to it regularly through a shared reference that
> > is certainly never going away until the thread dies.
> > Yet I must be able to "unshare" it's list of work items to
> > process them.
> 
> Sure, but at that point they are no longer referenced by the shared Thread, correct?

The work items? They stay referenced by the shared Thread
until it is done with them. In this particular implementation
an item is moved from the list to a separate field that
denotes the current item and then the Mutex is released.
This current item is technically unshared now, because only
this thread can really see it, but as far as the language is
concerned there is a shared reference to it because shared
applies transitively.
The same goes for the list of items while it is under the
Mutex protection.

> The rule is simply that you can't be trying to read or write data using both shared and unshared operations, because of that reader-writer contract I mentioned above.

Something along that line yes. The exact formulation may need
to be ironed out, but what the FAQ says right now doesn't
work. When you say (un)shared operations that maps to any means
of ensuring thread-safe operations on a set of data.
It can range from putting synchronized in front of your class
to the exact order of executing a series of loads and stored
in a lock-free algorithm.
Anything between a single shared basic data type and full
blown synchronized class is too complicated for the compiler
to see through. So a simple definition of `shared` like the
FAQ attempts wont fly. Most methods are "somewhat shared":

private void workOnAnItem() shared
{
	// m_current is technically never shared,
	// but we cannot describe this with `shared`.
	// Hence I manually unshare where appropriate.

	synchronized (m_condition.unshared.mutex)
	{
		m_current = m_list.unshared.front;
		m_list.unshared.removeFront();
	}
	m_current.unshared.doSomthing();
}

> > […]
> >
> > So the text should read:
> >
> >> What are the semantics of casting FROM shared TO unshared?
> >>
> >> Make sure that during the period the data is unshared, no other thread can modify those parts of it that you will be accessing. If you don't use synchronization objects with built-in memory-barriers like a Mutex, it is your responsibility to properly synchronize data access through e.g. atomicLoad/Store.
> >
> > That at least in general sanctifies casting away shared for the purpose of calling a method under protection of a user defined critical section.
> 
> It's more complicated than that, because you don't know how long a given operation needs to propagate to another CPU.  Simply performing a shared write is meaningless if something else is performing an unshared read because the optimization happens at both points--the write side and the read side.

You are right, my point was that the original formulation is so strict that can only come from the point of view of using shared in message passing. It doesn't spend a thought on how a shared(Thread) is supposed to be both referable and able to unshare internal lists during processing.

> In essence, the CPU performs the same optimizations as a compiler. […]

Yeah, I know, except for the DEC Alpha part.

> Basically what's needed is some way to have the compiler optimize according to the same rules as the CPU (the goal of "shared"). Or in lieu of that, to have some "don't optimize this" instruction to tell the compiler to keep it's dirty hands off your carefully constructed code so the only thing you need to worry about is what the CPU is trying to do.  This is what "volatile" was meant for in D1 and I really liked it, but I think I was the only one.

Count me in. Anecdotally I once tried to see if I can write a
minimal typed malloc() that is faster than temalloc. It went
way over budget from a single CAS instruction, where temalloc
mostly works on thread local pools.
These synchronizations stall the CPU _that_ much, that I don't
see how someone writing lock-free algorithms with `shared`
will accept implicit full barriers placed by the language.
This is a dead-end to me.
Mostly what I use is load-acquire and store-release, but
sometimes raw atomic read access is sufficient as well.

So ideally I would like to see:

volatile -> compiler doesn't reorder stuff

and on top of that:

atomicLoad/Store -> CPU doesn't reorder stuff in the pipeline
                    in the way I described by MemoryOrder.xxx

A shared variable need not be volatile, but a volatile variable is implicitly shared.

> There's a paper on release consistency that I think is fantastic.
>   I'll link it later if I can find it on the interweb.  CPUs seem
> to be converging on even more strict memory ordering than release
> consistency, but the release consistency model is really
> fantastic as it's basically equivalent to how mutexes work and so
> it's a model everyone already understands.

-- 
Marco

On Tuesday, 21 October 2014 at 13:10:57 UTC, Marco Leise wrote:
> Am Mon, 20 Oct 2014 16:18:51 +0000
> schrieb "Sean Kelly" <sean@invisibleduck.org>:
>
> But to the point: Doesn't defining it as shared means that it
> can not have _any_ unshared methods? Ok, fair enough. So even
> if a method is only working on technically unshared parts of
> the thread's data, it has to cast everything to unshared
> itself. This makes sense since `this`, the Thread itself is
> still shared.

Good point about a shared class not having any unshared methods.  I guess that almost entirely eliminates the cases where I might define a class as shared.  For example, the MessageBox class in std.concurrency has one or two ostensibly shared methods and the rest are unshared.  And it's expected for there to be both shared and unshared references to the object held simultaneously.  This is by design, and the implementation would either be horribly slow or straight-up broken if done another way.

Also, of the shared methods that exist, there are synchronized blocks but they occur at a fine grain within the shared methods rather than the entire method being shared.  I think that labeling entire methods as synchronized is an inherently flawed concept, as it contradicts the way mutexes are supposed to be used (which is to hold the lock for as short a time as possible).  I hate to say it, but if I were to apply shared/synchronized labels to class methods it would simply be to service user requests rather than because I think it would actually make the code better or safer.


>> shared class A {
>>      int m_count = 0;
>>      void increment() shared {
>>          m_count.atomicOp!"+="(1);
>>      }
>> 
>>      int getCount() synchronized {
>>          return m_count;
>>      }
>> }
>> 
>> If we make accesses of shared variables non-atomic inside synchronized methods, there may be conflicts with their use in shared methods.  Also:
>
> Well, when you talk about "shared and unshared operations"
> further down, I took it as the set of operations ensuring
> thread-safety over a particular set of shared data. That code
> above is just a broken set of such operations. I.e. in this
> case the programmer must decide between mutex synchronization
> and atomic read-modify-write. That's not too much to ask.

I agree.  I was being pedantic for the sake of informing anyone who wasn't aware.  There are times where I have some fields be lock-free and others protected by a mutex though.  See Thread.isRunning, for example.  There are times where a write delay is acceptable and the possibility of tearing is irrelevant.  But I think this falls pretty squarely into the "expert" category--I don't care if the language makes it easy.


>> shared class A {
>>      void doSomething() synchronized {
>>          doSomethingElse();
>>      }
>> 
>>      private void doSomethingElse() synchronized {
>> 
>>      }
>> }
>> 
>> doSomethingElse must be synchronized even if I as a programmer know it doesn't have to be because the compiler insists it must be.  And I know that private methods are visible within the module, but the same rule applies.  In essence, we can't avoid recursive mutexes for implementing synchronized, and we're stuck with a lot of recursive locks and unlocks no matter what, as soon as we slap a "shared" label on something.
>
> Imagine you have a shared root object that contains a deeply
> nested private data structure that is technically unshared.
> Then it becomes not only one more method of the root object
> that needs to be `synchronized` but it cascades all the way
> down its private fields as well. One ends up requiring data
> structures designed for single-threaded execution to
> grow synchronized methods over night even though they aren't
> _really_ used concurrently my multiple threads.

I need to give it some more thought, but I think the way this should work is for shared to not be transitive, but for the compiler to require that non-local variables accessed within a shared method must either be declared as shared or the access must occur within a synchronized block.  This does trust the programmer a bit more than the current design, but in exchange it encourages a programming model that actually makes sense.  It doesn't account for the case where I'm calling pthread_mutex_lock on an unshared variable though.  Still not sure about that one.


>> Sure, but at that point they are no longer referenced by the shared Thread, correct?
>
> The work items? They stay referenced by the shared Thread
> until it is done with them. In this particular implementation
> an item is moved from the list to a separate field that
> denotes the current item and then the Mutex is released.
> This current item is technically unshared now, because only
> this thread can really see it, but as far as the language is
> concerned there is a shared reference to it because shared
> applies transitively.

Oh I see what you're getting at.  This sort of thing is why Thread can be initialized with an unshared delegate.  Since join() is an implicit synchronization point, it's completely normal to launch a thread that modifies local data, then call join and expect the local data to be in a coherent state.  Work queues are much the same.


> The same goes for the list of items while it is under the
> Mutex protection.
>
>> The rule is simply that you can't be trying to read or write data using both shared and unshared operations, because of that reader-writer contract I mentioned above.
>
> Something along that line yes. The exact formulation may need
> to be ironed out, but what the FAQ says right now doesn't
> work.

Yes.  See my suggestion about shared being non-transitive above.  I think that's at least in the right ballpark.


> Anything between a single shared basic data type and full
> blown synchronized class is too complicated for the compiler
> to see through. So a simple definition of `shared` like the
> FAQ attempts wont fly. Most methods are "somewhat shared":
>
> private void workOnAnItem() shared
> {
> 	// m_current is technically never shared,
> 	// but we cannot describe this with `shared`.
> 	// Hence I manually unshare where appropriate.
>
> 	synchronized (m_condition.unshared.mutex)
> 	{
> 		m_current = m_list.unshared.front;
> 		m_list.unshared.removeFront();
> 	}
> 	m_current.unshared.doSomthing();
> }

As they should be.  This is the correct way to use mutexes.


> You are right, my point was that the original formulation is
> so strict that can only come from the point of view of using
> shared in message passing. It doesn't spend a thought on how a
> shared(Thread) is supposed to be both referable and able to
> unshare internal lists during processing.

And since message passing is encapsulated in an API, we really don't need the type system to do anything special.  We can just make correct use an artifact of the API itself.


> Count me in. Anecdotally I once tried to see if I can write a
> minimal typed malloc() that is faster than temalloc. It went
> way over budget from a single CAS instruction, where temalloc
> mostly works on thread local pools.
> These synchronizations stall the CPU _that_ much, that I don't
> see how someone writing lock-free algorithms with `shared`
> will accept implicit full barriers placed by the language.
> This is a dead-end to me.

Last time I checked boost::shared_ptr (which admittedly was years ago) they'd dropped atomic operations in favor of spins on non-atomics.  LOCK has gotten a lot more efficient--I think it's on the order of ~75 cycles these days.  And the FENCE operations are supposed to work for normal programming now, though it's hard to find out whether this is true of all CPUs or only those from Intel.  But yes, truly atomic ops are terribly expensive.


> Mostly what I use is load-acquire and store-release, but
> sometimes raw atomic read access is sufficient as well.
>
> So ideally I would like to see:
>
> volatile -> compiler doesn't reorder stuff

Me too.  For example, GCC can optimize around inline assembler.  I used to have the inline asm code in core.atomic labeled as volatile for this reason, but was forced to remove it because it's deprecated in D2.


> and on top of that:
>
> atomicLoad/Store -> CPU doesn't reorder stuff in the pipeline
>                     in the way I described by MemoryOrder.xxx
>
> A shared variable need not be volatile, but a volatile
> variable is implicitly shared.

I'm not quite following you here.  Above, I thought you meant the volatile statement.  Are you saying we would have both shared and volatile as attributes for variables?

On Tuesday, 21 October 2014 at 16:05:58 UTC, Sean Kelly wrote:
> Also, of the shared methods that exist, there are synchronized blocks but they occur at a fine grain within the shared methods rather than the entire method being shared.  I think that labeling entire methods as synchronized is an inherently flawed concept, as it contradicts the way mutexes are supposed to be used (which is to hold the lock for as short a time as possible).

`shared` applies to the implicit `this` parameter, not to the method. It's really no different from normal parameters in this respect, which can either be shared, or not. There's no way to make them temporarily shared either (apart from casting).

I think `shared` by itself is fine, as long as it is only take to mean "this method can cope with the parameters being shared". It's `synchronized` and casting that cause the trouble, because they are too coarse, and synchronized doesn't specify which parts of an object it protects. This makes more detailed compiler checks impossible.

On Tuesday, 21 October 2014 at 16:36:30 UTC, Marc Schütz wrote:
>
> I think `shared` by itself is fine, as long as it is only take to mean "this method can cope with the parameters being shared". It's `synchronized` and casting that cause the trouble, because they are too coarse, and synchronized doesn't specify which parts of an object it protects. This makes more detailed compiler checks impossible.

I think "synchronized" should work basically the same way.  For
example:

class C {
     int u;
     shared int s;
     shared Mutex m;

     void doSomething() shared { // can be called from a shared
reference
         u = 5; // error, u is not labeled as shared and operation
not synchronized
         s = 5; // ok, though maybe should require s.atomicStore(5)

         synchronized(m) {
             u = 5; // ok
             doSomething2(); // okay because labeled synchronized
         }
     }

     void doSomething2() synchronized { // can be called within a
synchronized block
         u = 5; // ok because synchronized
         s = 5; // ok, though maybe should require s.atomicStore(5)
     }
}


I'd like to avoid having a synchronized label on a method
implicitly lock anything because object monitors are terrible and
should not exist, and because even if the compiler tries to
optimize away recursive locking it won't always happen and the
result will be too expensive for anyone to actually want to use
it.

Am Tue, 21 Oct 2014 16:05:57 +0000
schrieb "Sean Kelly" <sean@invisibleduck.org>:

> Good point about a shared class not having any unshared methods. I guess that almost entirely eliminates the cases where I might define a class as shared. For example, the MessageBox class in std.concurrency has one or two ostensibly shared methods and the rest are unshared.  And it's expected for there to be both shared and unshared references to the object held simultaneously.  This is by design, and the implementation would either be horribly slow or straight-up broken if done another way.
>
> Also, of the shared methods that exist, there are synchronized
> blocks but they occur at a fine grain within the shared methods
> rather than the entire method being shared.  I think that
> labeling entire methods as synchronized is an inherently flawed
> concept, as it contradicts the way mutexes are supposed to be
> used (which is to hold the lock for as short a time as possible).
>   I hate to say it, but if I were to apply shared/synchronized
> labels to class methods it would simply be to service user
> requests rather than because I think it would actually make the
> code better or safer.

I have nothing to add.

> > […] I.e. in this
> > case the programmer must decide between mutex synchronization
> > and atomic read-modify-write. That's not too much to ask.
> 
> I agree.  I was being pedantic for the sake of informing anyone who wasn't aware.  There are times where I have some fields be lock-free and others protected by a mutex though.  See Thread.isRunning, for example.

Yep, and the reason I carefully formulated "read-modify-write", hehe.

> There are times where a write
> delay is acceptable and the possibility of tearing is irrelevant.
>   But I think this falls pretty squarely into the "expert"
> category--I don't care if the language makes it easy.

> > Imagine you have a shared root object that contains a deeply
> > nested private data structure that is technically unshared.
> > Then it becomes not only one more method of the root object
> > that needs to be `synchronized` but it cascades all the way
> > down its private fields as well. One ends up requiring data
> > structures designed for single-threaded execution to
> > grow synchronized methods over night even though they aren't
> > _really_ used concurrently my multiple threads.
> 
> I need to give it some more thought, but I think the way this should work is for shared to not be transitive, but for the compiler to require that non-local variables accessed within a shared method must either be declared as shared or the access must occur within a synchronized block.  This does trust the programmer a bit more than the current design, but in exchange it encourages a programming model that actually makes sense.  It doesn't account for the case where I'm calling pthread_mutex_lock on an unshared variable though.  Still not sure about that one.

Do you think it would be bad if a pthread_mutex_t* was declared as shared or only usable when shared ?

> > The work items? They stay referenced by the shared Thread until it is done with them. In this particular implementation an item is moved from the list to a separate field that denotes the current item and then the Mutex is released. This current item is technically unshared now, because only this thread can really see it, but as far as the language is concerned there is a shared reference to it because shared applies transitively.
> 
> Oh I see what you're getting at.  This sort of thing is why Thread can be initialized with an unshared delegate.  Since join() is an implicit synchronization point, it's completely normal to launch a thread that modifies local data, then call join and expect the local data to be in a coherent state.  Work queues are much the same.

I have to think about that.

[…]

> > Mostly what I use is load-acquire and store-release, but sometimes raw atomic read access is sufficient as well.
> >
> > So ideally I would like to see:
> >
> > volatile -> compiler doesn't reorder stuff
> 
> Me too.  For example, GCC can optimize around inline assembler. I used to have the inline asm code in core.atomic labeled as volatile for this reason, but was forced to remove it because it's deprecated in D2.
> 
> 
> > and on top of that:
> >
> > atomicLoad/Store -> CPU doesn't reorder stuff in the pipeline
> >                     in the way I described by MemoryOrder.xxx
> >
> > A shared variable need not be volatile, but a volatile variable is implicitly shared.
> 
> I'm not quite following you here.  Above, I thought you meant the volatile statement.  Are you saying we would have both shared and volatile as attributes for variables?

I haven't been around in the D1 times. There was a volatile
statement? Anyways what I don't want is that the compiler
emits memory barriers everywhere shared variables are accessed.
When I use mutex synchronization I don't need it and when I use
atomics, I want control over barriers.
I thought that could end up in two attributes for variables,
but it need not be the case.

-- 
Marco

On Tuesday, 21 October 2014 at 19:32:17 UTC, Marco Leise wrote:
>
> Do you think it would be bad if a pthread_mutex_t* was
> declared as shared or only usable when shared ?

No.  But the issue in general concerns me.  Like say my class
contains a C style FILE*.  If shared is transitive, then
fprintf() must be modified to, what, take shared(FILE*)?  Maybe
conditionally, since it won't always be shared?  I don't know
that the shared equivalent of "const" is appropriate here either,
since you'd pretty much stick it at the top of every module in
core.stdc and core.sys.


> I haven't been around in the D1 times. There was a volatile
> statement?

http://digitalmars.com/d/1.0/statement.html#VolatileStatement


> Anyways what I don't want is that the compiler
> emits memory barriers everywhere shared variables are accessed.
> When I use mutex synchronization I don't need it and when I use
> atomics, I want control over barriers.

Oh okay.  Yes, I'd be fine of all built-in operations were simply
prohibited on shared variables, so you were forced to use
core.atomic for everything.

Am Tue, 21 Oct 2014 19:43:45 +0000
schrieb "Sean Kelly" <sean@invisibleduck.org>:

> On Tuesday, 21 October 2014 at 19:32:17 UTC, Marco Leise wrote:
> >
> > Do you think it would be bad if a pthread_mutex_t* was declared as shared or only usable when shared ?
> 
> No.  But the issue in general concerns me.  Like say my class contains a C style FILE*.  If shared is transitive, then fprintf() must be modified to, what, take shared(FILE*)?  Maybe conditionally, since it won't always be shared?  I don't know that the shared equivalent of "const" is appropriate here either, since you'd pretty much stick it at the top of every module in core.stdc and core.sys.

No no, a FILE* is obviously immutable.

-- 
Marco