Hey, I've got a few questions if anybody's got a minute.

I'm trying to wrap my head around the threading situation in D. So far, things seem to be working as expected, but I want to verify my solutions.

1) Are the following two snippets exactly equivalent(not just in observable behaviour)?
a)

Mutex mut;
mut.lock();
scope(exit) mut.unlock();

b)
Mutex mut;
synchronized(mut) { }

Will 'synchronized' call 'lock' on the Mutex, or do something else(possibly related to the interface Object.Monitor)?

2) Phobos has 'Condition' which takes a Mutex in the constructor. The documentation doesn't exactly specify this, but should I assume it works the same as std::condition_variable in C++?

For example, is this correct?

Mutex mut;
Condition cond = new Condition(mut);

// mut must be locked before calling Condition.wait
synchronized(mut)  // depends on answer to (1)
{
    // wait() unlocks the mutex and enters wait state
    // wait() must re-acquire the mutex before returning when cond is signalled
    cond.wait();
}

3) Why do I have to pass a "Mutex" to "Condition"? Why can't I just pass an "Object"?

4) Will D's Condition ever experience spurious wakeups?

5) Why doesn't D's Condition.wait take a predicate? I assume this is because the answer to (4) is no.

6) Does 'shared' actually have any effect on non-global variables beside the syntactic regulations?

I know that all global variables are TLS unless explicitly marked as 'shared', but someone once told me something about 'shared' affecting member variables in that accessing them from a separate thread would return T.init instead of the actual value... or something like that. This seems to be wrong(thankfully).

For example, I have created this simple Worker class which seems to work fine without a 'shared' keyword in sight(thankfully). I'm wondering though, if there would be any unexpected consequences of doing things this way.

http://dpaste.com/2ZG2QZV




Thanks!
    Bit

Pretty please? :)

Sorry I don't know the answers but these questions are interesting so BUMP ;)

On Friday, 25 September 2015 at 15:19:27 UTC, bitwise wrote:
>
> 1) Are the following two snippets exactly equivalent(not just in observable behaviour)?
> a)
>
> Mutex mut;
> mut.lock();
> scope(exit) mut.unlock();
>
> b)
> Mutex mut;
> synchronized(mut) { }
>
> Will 'synchronized' call 'lock' on the Mutex, or do something else(possibly related to the interface Object.Monitor)?


Don't know.
Is this Object monitor a mutex or something else?



> 6) Does 'shared' actually have any effect on non-global variables beside the syntactic regulations?

Don't think so.

I hadn't answered as I do not have answers to the questions you ask. My reason: people should not be doing their codes using these low-level shared memory techniques. Data parallel things should be using the std.parallelism module. Dataflow-style things should be using spawn and channels – akin to the way you do things in Go.

So to give you an answer I would go back a stage, forget threads, mutexes, synchronized, etc. and ask what do you want you workers to do? If they are to do something and return a result then spawn and channel is exactly the right abstraction to use. Think "farmer–worker", the farmer spawns the workers and then collects their results. No shared memory anywyere – at least not mutable.

On Fri, 2015-09-25 at 15:19 +0000, bitwise via Digitalmars-d-learn wrote:
> Hey, I've got a few questions if anybody's got a minute.
> 
> I'm trying to wrap my head around the threading situation in D. So far, things seem to be working as expected, but I want to verify my solutions.
> 
> 1) Are the following two snippets exactly equivalent(not just in
> observable behaviour)?
> a)
> 
> Mutex mut;
> mut.lock();
> scope(exit) mut.unlock();
> 
> b)
> Mutex mut;
> synchronized(mut) { }
> 
> Will 'synchronized' call 'lock' on the Mutex, or do something else(possibly related to the interface Object.Monitor)?
> 
> 2) Phobos has 'Condition' which takes a Mutex in the constructor. The documentation doesn't exactly specify this, but should I assume it works the same as std::condition_variable in C++?
> 
> For example, is this correct?
> 
> Mutex mut;
> Condition cond = new Condition(mut);
> 
> // mut must be locked before calling Condition.wait
> synchronized(mut)  // depends on answer to (1)
> {
>      // wait() unlocks the mutex and enters wait state
>      // wait() must re-acquire the mutex before returning when
> cond is signalled
>      cond.wait();
> }
> 
> 3) Why do I have to pass a "Mutex" to "Condition"? Why can't I just pass an "Object"?
> 
> 4) Will D's Condition ever experience spurious wakeups?
> 
> 5) Why doesn't D's Condition.wait take a predicate? I assume this is because the answer to (4) is no.
> 
> 6) Does 'shared' actually have any effect on non-global variables beside the syntactic regulations?
> 
> I know that all global variables are TLS unless explicitly marked as 'shared', but someone once told me something about 'shared' affecting member variables in that accessing them from a separate thread would return T.init instead of the actual value... or something like that. This seems to be wrong(thankfully).
> 
> For example, I have created this simple Worker class which seems to work fine without a 'shared' keyword in sight(thankfully). I'm wondering though, if there would be any unexpected consequences of doing things this way.
> 
> http://dpaste.com/2ZG2QZV
> 
> 
> 
> 
> Thanks!
>      Bit
-- 
Russel. ============================================================================= Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder@ekiga.net 41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel@winder.org.uk London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder

On Monday, 28 September 2015 at 11:47:38 UTC, Russel Winder wrote:
> I hadn't answered as I do not have answers to the questions you ask. My reason: people should not be doing their codes using these low-level shared memory techniques. Data parallel things should be using the std.parallelism module. Dataflow-style things should be using spawn and channels – akin to the way you do things in Go.
>
> So to give you an answer I would go back a stage, forget threads, mutexes, synchronized, etc. and ask what do you want you workers to do? If they are to do something and return a result then spawn and channel is exactly the right abstraction to use. Think "farmer–worker", the farmer spawns the workers and then collects their results. No shared memory anywyere – at least not mutable.

https://www.youtube.com/watch?v=S7pGs7JU7eM

    Bit

On Tue, 2015-09-29 at 03:05 +0000, bitwise via Digitalmars-d-learn wrote:
> On Monday, 28 September 2015 at 11:47:38 UTC, Russel Winder wrote:
> > I hadn't answered as I do not have answers to the questions you ask. My reason: people should not be doing their codes using these low-level shared memory techniques. Data parallel things should be using the std.parallelism module. Dataflow-style things should be using spawn and channels – akin to the way you do things in Go.
> > 
> > So to give you an answer I would go back a stage, forget threads, mutexes, synchronized, etc. and ask what do you want you workers to do? If they are to do something and return a result then spawn and channel is exactly the right abstraction to use. Think "farmer–worker", the farmer spawns the workers and then collects their results. No shared memory anywyere – at least not mutable.
> 
> https://www.youtube.com/watch?v=S7pGs7JU7eM
> 
>      Bit

What's the tl;dr as text, I very, very rarely watch videos.

-- 
Russel. ============================================================================= Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder@ekiga.net 41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel@winder.org.uk London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder

On 9/25/15 11:19 AM, bitwise wrote:
> Hey, I've got a few questions if anybody's got a minute.
>
> I'm trying to wrap my head around the threading situation in D. So far,
> things seem to be working as expected, but I want to verify my solutions.
>
> 1) Are the following two snippets exactly equivalent(not just in
> observable behaviour)?
> a)
>
> Mutex mut;
> mut.lock();
> scope(exit) mut.unlock();
>
> b)
> Mutex mut;
> synchronized(mut) { }
>
> Will 'synchronized' call 'lock' on the Mutex, or do something
> else(possibly related to the interface Object.Monitor)?

Yes. A mutex object has it's internal lock as its monitor.

> 2) Phobos has 'Condition' which takes a Mutex in the constructor. The
> documentation doesn't exactly specify this, but should I assume it works
> the same as std::condition_variable in C++?

I am not sure about std::condition_variable. core.sync.condition works like a standard condition (https://en.wikipedia.org/wiki/Monitor_%28synchronization%29)

> For example, is this correct?
>
> Mutex mut;
> Condition cond = new Condition(mut);
>
> // mut must be locked before calling Condition.wait
> synchronized(mut)  // depends on answer to (1)
> {
>      // wait() unlocks the mutex and enters wait state
>      // wait() must re-acquire the mutex before returning when cond is
> signalled
>      cond.wait();
> }

Yes, I believe it is.

> 3) Why do I have to pass a "Mutex" to "Condition"? Why can't I just pass
> an "Object"?

An object that implements the Monitor interface may not actually be a mutex. For example, a pthread_cond_t requires a pthread_mutex_t to operate properly. If you passed it anything that can act like a lock, it won't work. So the Condition needs to know that it has an actual Mutex, not just any lock-like object.

I think I advocated in the past to Sean that Condition should provide a default ctor that just constructs a mutex, but it doesn't look like that was done.

>
> 4) Will D's Condition ever experience spurious wakeups?

What do you mean by "spurious"? If you notify a condition, anything that is waiting on it can be woken up. Since the condition itself is user defined, there is no way for the actual Condition to verify you will only be woken up when it is satisfied.

In terms of whether a condition could be woken when notify *isn't* called, I suppose it's possible (perhaps interrupted by a signal?). But I don't know why it would matter -- per above you should already be checking the condition while within the lock.

I think there are cases with multiple threads where you can potentially wake up the thread waiting on a condition AFTER the condition was already reset by another.

> 5) Why doesn't D's Condition.wait take a predicate? I assume this is
> because the answer to (4) is no.

The actual "condition" that you are waiting on is up to you to check/define.

> 6) Does 'shared' actually have any effect on non-global variables beside
> the syntactic regulations?

I believe shared doesn't alter code generation at all. It only prevents certain things and affects the type.

> I know that all global variables are TLS unless explicitly marked as
> 'shared', but someone once told me something about 'shared' affecting
> member variables in that accessing them from a separate thread would
> return T.init instead of the actual value... or something like that.
> This seems to be wrong(thankfully).

No, this isn't true.

> For example, I have created this simple Worker class which seems to work
> fine without a 'shared' keyword in sight(thankfully). I'm wondering
> though, if there would be any unexpected consequences of doing things
> this way.
>
> http://dpaste.com/2ZG2QZV

Some errors:

1. When calling notifyAll, you should ALWAYS have the mutex locked.
2. Since the mutex is protecting _run, it should only be checked/modified with the lock held.
3. After you have been woken up, you should check that the condition is satisfied.
4. Technically, you shouldn't access member variables that are GC allocated from a dtor. I know it's a struct, but structs can be GC allocated as well.

I would replace your if(tasks.empty) with while(tasks.empty && _run) to fix issue 3.

-Steve

Am Tue, 29 Sep 2015 15:10:58 -0400
schrieb Steven Schveighoffer <schveiguy@yahoo.com>:

> 
> > 3) Why do I have to pass a "Mutex" to "Condition"? Why can't I just pass an "Object"?
> 
> An object that implements the Monitor interface may not actually be a mutex. For example, a pthread_cond_t requires a pthread_mutex_t to operate properly. If you passed it anything that can act like a lock, it won't work. So the Condition needs to know that it has an actual Mutex, not just any lock-like object.
> 
> I think I advocated in the past to Sean that Condition should provide a default ctor that just constructs a mutex, but it doesn't look like that was done.
> 

But you'll need access to the Mutex in user code as well. And often you use multiple Conditions with one Mutex so a Condition doesn't really own the Mutex.

> >
> > 4) Will D's Condition ever experience spurious wakeups?
> 
> What do you mean by "spurious"? If you notify a condition, anything that is waiting on it can be woken up. Since the condition itself is user defined, there is no way for the actual Condition to verify you will only be woken up when it is satisfied.
> 
> In terms of whether a condition could be woken when notify *isn't* called, I suppose it's possible (perhaps interrupted by a signal?). But I don't know why it would matter -- per above you should already be checking the condition while within the lock.

Spurious wakeup is a common term when talking about posix conditions
and it does indeed mean a wait() call can return without ever calling
notify():
https://en.wikipedia.org/wiki/Spurious_wakeup
http://stackoverflow.com/questions/8594591/why-does-pthread-cond-wait-have-spurious-wakeups

And yes, this does happen for core.sync.condition as well. As a result you'll always have to check in a loop:

synchronized(mutex)
{
    while(some_flag_or_expression)
    {
        cond.wait();
    }
}

-----------------
synchronized(mutex)
{
    some_flag_or_expression = true;
    cond.notify();
}

> 
> I think there are cases with multiple threads where you can potentially wake up the thread waiting on a condition AFTER the condition was already reset by another.
> 
> > 5) Why doesn't D's Condition.wait take a predicate? I assume this is
> > because the answer to (4) is no.
> 
> The actual "condition" that you are waiting on is up to you to check/define.
> 

He probably means that you could pass an expression to wait and wait would do the looping / check internally. That's probably a nicer API but not implemented.

> > 6) Does 'shared' actually have any effect on non-global variables beside the syntactic regulations?
> 
> I believe shared doesn't alter code generation at all. It only prevents certain things and affects the type.
> 

It shouldn't. I think in GDC it does generate different code, but that's an implementation detail that needs to be fixed.

On 9/29/15 4:38 PM, Johannes Pfau wrote:
> Am Tue, 29 Sep 2015 15:10:58 -0400
> schrieb Steven Schveighoffer <schveiguy@yahoo.com>:
>
>>
>>> 3) Why do I have to pass a "Mutex" to "Condition"? Why can't I just
>>> pass an "Object"?
>>
>> An object that implements the Monitor interface may not actually be a
>> mutex. For example, a pthread_cond_t requires a pthread_mutex_t to
>> operate properly. If you passed it anything that can act like a lock,
>> it won't work. So the Condition needs to know that it has an actual
>> Mutex, not just any lock-like object.
>>
>> I think I advocated in the past to Sean that Condition should provide
>> a default ctor that just constructs a mutex, but it doesn't look like
>> that was done.
>>
>
> But you'll need access to the Mutex in user code as well.

synchronized(condition.mutex)

> And often you
> use multiple Conditions with one Mutex so a Condition doesn't really
> own the Mutex.

It's just a different option. Often times, you have a condition variable, and a mutex variable.

It's not super-important, you can always do:

new Condition(new Mutex);

>
>>>
>>> 4) Will D's Condition ever experience spurious wakeups?
>>
>> What do you mean by "spurious"? If you notify a condition, anything
>> that is waiting on it can be woken up. Since the condition itself is
>> user defined, there is no way for the actual Condition to verify you
>> will only be woken up when it is satisfied.
>>
>> In terms of whether a condition could be woken when notify *isn't*
>> called, I suppose it's possible (perhaps interrupted by a signal?).
>> But I don't know why it would matter -- per above you should already
>> be checking the condition while within the lock.
>
> Spurious wakeup is a common term when talking about posix conditions
> and it does indeed mean a wait() call can return without ever calling
> notify():
> https://en.wikipedia.org/wiki/Spurious_wakeup
> http://stackoverflow.com/questions/8594591/why-does-pthread-cond-wait-have-spurious-wakeups

OK thanks.

>>> 5) Why doesn't D's Condition.wait take a predicate? I assume this is
>>> because the answer to (4) is no.
>>
>> The actual "condition" that you are waiting on is up to you to
>> check/define.
>>
>
> He probably means that you could pass an expression to wait and wait
> would do the looping / check internally. That's probably a nicer API
> but not implemented.

yeah, that could probably be done. One thing to note is that these classes are from ages ago (probably close to 10 years). New API suggestions may be allowed.

I just wanted to stress that there isn't some sort of built-in condition predicate (like a boolean).

-Steve

On Tuesday, September 29, 2015 22:38:42 Johannes Pfau via Digitalmars-d-learn wrote:
> Am Tue, 29 Sep 2015 15:10:58 -0400
> schrieb Steven Schveighoffer <schveiguy@yahoo.com>:
>
> >
> > > 3) Why do I have to pass a "Mutex" to "Condition"? Why can't I just pass an "Object"?
> >
> > An object that implements the Monitor interface may not actually be a mutex. For example, a pthread_cond_t requires a pthread_mutex_t to operate properly. If you passed it anything that can act like a lock, it won't work. So the Condition needs to know that it has an actual Mutex, not just any lock-like object.
> >
> > I think I advocated in the past to Sean that Condition should provide a default ctor that just constructs a mutex, but it doesn't look like that was done.
> >
>
> But you'll need access to the Mutex in user code as well. And often you use multiple Conditions with one Mutex so a Condition doesn't really own the Mutex.
>
> > >
> > > 4) Will D's Condition ever experience spurious wakeups?
> >
> > What do you mean by "spurious"? If you notify a condition, anything that is waiting on it can be woken up. Since the condition itself is user defined, there is no way for the actual Condition to verify you will only be woken up when it is satisfied.
> >
> > In terms of whether a condition could be woken when notify *isn't* called, I suppose it's possible (perhaps interrupted by a signal?). But I don't know why it would matter -- per above you should already be checking the condition while within the lock.
>
> Spurious wakeup is a common term when talking about posix conditions
> and it does indeed mean a wait() call can return without ever calling
> notify():
> https://en.wikipedia.org/wiki/Spurious_wakeup
> http://stackoverflow.com/questions/8594591/why-does-pthread-cond-wait-have-spurious-wakeups
>
> And yes, this does happen for core.sync.condition as well. As a result you'll always have to check in a loop:
>
> synchronized(mutex)
> {
>     while(some_flag_or_expression)
>     {
>         cond.wait();
>     }
> }
>
> -----------------
> synchronized(mutex)
> {
>     some_flag_or_expression = true;
>     cond.notify();
> }

What I took from the answers to that SO question was that in general, it really doesn't matter whether a condition variable has spurious wakeups. You're going to have to check that the associated bool is true when you wake up anyway. Maybe without spurious wakeups, it wouldn't be required if only one thread was waiting for the signal, but you'd almost certainly still need an associated bool in case it becomes true prior to waiting. In addition, if you want to avoid locking up your program, it's ferquently the case that you want a timed wait so that you can check whether the program is trying to exit (or at least that the thread in question is being terminated), and you'd need a separate bool in that case as well so that you can check whether the condition has actually been signaled. So, ultimately, while spurious wakeups do seem wrong from a correctness perspective, when you look at what a condition variable needs to do, it usually doesn't matter that spurious wakeups exist, and a correctly used condition variable will just handle spurious wakeups as a side effect of how it's used.

- Jonathan M Davis