14-May-2013 21:02, Steven Schveighoffer пишет:
> On Tue, 14 May 2013 04:58:27 -0400, Dmitry Olshansky
> <dmitry.olsh@gmail.com> wrote:
>
>> 14-May-2013 08:33, Heinz пишет:
>>>> BTW, given recent discussion on memory barriers, I think my previous
>>>> statement that the mutex does not need to be locked to call notify is
>>>> probably incorrect.
>>>
>>
>> Have to lock it otherwise you have a race condition on a condition
>> variable (wow!).
>
> No, the issue would be reordering (is that possible in this case?).  The
> actual signaling of the condition would not require the lock, but you
> still need to lock to send the message (e.g. set the boolean) or you
> will definitely have issues.

Never had to notify without sending some message (otherwise how you'd affect the thread on the other side of fence?). I'd have to dig into how D's condition variables are done to say for sure. But I thought that notify is not re-entrant that is it needs to be locked (as there is a queue of threads to manage).

I just had fresh experience in Java where you basically can't notify threads waiting on the monitor without grabbing the lock, same with call to wait. It's a clean-cut run-time error (when it can detect it).

>
> But since you have to lock anyway, signaling while holding the lock, or
> while being outside the lock isn't really a difference.
>

On the level of gut feeling there must be something about it as you don't see:

synchronized(m){
	// ... send message
}
notify();

anytime of day. And hosting work out of mutex seems natural isn't it?
Not in condition to analyze just yet but one clue is that it seems like you may be notifying a thread that would wake up only to block on mutex somebody else holds.

> Maybe I'm wrong...

Maybe I am.

>>> Haven't had any issues calling notify outside a synchronized statement,
>>> even from multiple threads. At least this works under Win32 with my
>>> producer thread all wrapped inside synchronized(). Only a single
>>> consumer thread is inside synchronized() but then i have 2 more threads
>>> making naked calls to notify(). Not a single crash.
>>
>> Doesn't prove anything, it could happen that you just miss a
>> notification, for instance. Another common case is that it so happens
>> that wait will (with luck) always happen before any of notify and
>> notifications come spaced out in time.
>
> I don't see how you could miss a notification, can you explain further?

Not particularly well put. In general I was implying that program not crushing doesn't mean anything useful. It might work either due to happenstance. Depending on the implementation of notify, if it's not re-entrant then technically anything fishy can happen.

-- 
Dmitry Olshansky

On May 14, 2013, at 12:02 PM, Dmitry Olshansky <dmitry.olsh@gmail.com> wrote:

> 14-May-2013 21:02, Steven Schveighoffer пишет:
>> 
>> But since you have to lock anyway, signaling while holding the lock, or while being outside the lock isn't really a difference.
>> 
> 
> On the level of gut feeling there must be something about it as you don't see:
> 
> synchronized(m){
> 	// ... send message
> }
> notify();
> 
> anytime of day. And hosting work out of mutex seems natural isn't it?


If you move the notify out of the mutex then you can end up with multiple threads competing for the same value.  Say the producer puts a value in the queue, leaves the mutex, and notifies a waiting thread.  Then consumer A enters the mutex, sees that there's something in the container and takes it, then consumer B receives the notification and wakes up to discover that the container is empty.  So long as your wait loops are done properly the only bad result will be pointless wakeups, but worst case you could have a crash or exception if you're removing data from the container without checking if it's empty.

On Tue, 14 May 2013 15:19:29 -0400, Sean Kelly <sean@invisibleduck.org> wrote:

> If you move the notify out of the mutex then you can end up with multiple threads competing for the same value.  Say the producer puts a value in the queue, leaves the mutex, and notifies a waiting thread.  Then consumer A enters the mutex, sees that there's something in the container and takes it, then consumer B receives the notification and wakes up to discover that the container is empty.  So long as your wait loops are done properly the only bad result will be pointless wakeups, but worst case you could have a crash or exception if you're removing data from the container without checking if it's empty.

Any thread that does not check to ensure his data is ready while waiting for a condition, I would argue is incorrectly implemented -- cond.notify is not directly tied to the data, and there's always cond.notifyAll which could wake up any number of threads.

I also don't think there's a requirement that threads waiting on a condition have priority over threads simply trying to lock.

But I see little reason to avoid putting the cond.notify inside the lock.  You HAVE to lock to insert the data anyway.

And Dmitry seems to have some Java experience that indicates it's at least required there.  I'd say go with notifying only while locked.

I learn something every day :)

-Steve

To Steven:

> 1. D mutex locks are re-entrant.  That is, you can do:
>
> synchronized(mutex)
> {
>    synchronized(mutex)
>    {
>       ...
>    }
> }
>
> and all is fine.

Hmmm, i guess this is what the docs meant by "mutexes are recursive", i didn't get it at first but now i do.
To test this i wrapped SetLoop() in synchronized(cond.mutex) and called it from consumer wich was already all wrapped by the same synchronized(cond.mutex), i thought at first it was going to be a deadlock but i guess this is what you mean because it works like a charm.

Thanks for your code example, i'll update my code with your suggestions. But there's one thing i don't understand in your example: If the producer is inside a while() generating multiple messages once in a while then when the lock is released in the consumer, the producer will replace "my_variable" a couple of times with the latest message until consumer gets to wait() again. That's why i locked the whole consumer, to sync all messages. Am i wrong here?

To Sean:

> Let's back up a minute.  Can you be more specific about what you're trying to do?  I think you shouldn't need to use the "loop" var at all, but I'm not entirely sure.  Also, loop and my_variable will be thread-local given how they're declared.

Fair enough, i uploaded a bunch of code hoping that experienced programmers like you to understand right away what i'm doing.

I have this "main loop" wich is named here as my consumer function. I didn't want this main loop to be cycling all the time if there was nothing to do. So i needed an event object (condition/mutex) that locked the loop until any of the 2 program events (or both) occur:

1) A custom program message was sent by producer to main loop to notify for example mouse move, key press, timer tick, etc.
2) The user explicitly want to do something else (execute custom code or perform an action not covered by program messages) or did something that now needs to be processed inside the main loop. In this case the user calls SetLoop(true) to notify the main loop and then do what the user want.

That's why i have and need both types of messages: bool loop; and Object my_variable;

My code seems to work fine but i did not know if it was the best approach. Now i know after your help that i can and must protect both messages with the condition's mutex and have all in sync. Also i learned that wait() and notify() must be inside a lock.

To ALL:

Sorry for my incompetence. I'm not new to D but i am to D2 and multithreading. I got stuck in D1, specially in the DMD1.030 era (those were the most stable and beautiful times) but now i decided it was time to evolve. So far, learning D2 and multithreading hasn't been easy, but with your help guys i've accomplished many things.

Thank you again for your support.

On Wednesday, 15 May 2013 at 02:25:02 UTC, Heinz wrote:
> ....
> I have this "main loop" wich is named here as my consumer function. I didn't want this main loop to be cycling all the time if there was nothing to do. So i needed an event object (condition/mutex) that locked the loop until any of the 2 program events (or both) occur:
>
> 1) A custom program message was sent by producer to main loop to notify for example mouse move, key press, timer tick, etc.
> 2) The user explicitly want to do something else (execute custom code or perform an action not covered by program messages) or did something that now needs to be processed inside the main loop. In this case the user calls SetLoop(true) to notify the main loop and then do what the user want.
>
> That's why i have and need both types of messages: bool loop; and Object my_variable;
> ....

It sounds like you need to:
    1) use a Message Queue.
    2) Copy the message while you work on it with the consumer, so that you can exit the mutex.

If you don't use a message queue, then the thread that feeds the consumer will have to wait on the consumer to finish processing the last message, just to post a new message. Thus removing the advantage of processing the message in a different thread.

If you use a message queue, make it so that you exit the mutex lock while you process the message, and then reaquire the mutex to use the message queue. This is so that you can keep adding messages to the queue without having to wait until the last message is consumed.

There are many ways to implement a queue for thread message passing, some are more advanced and require less locking. But you can start with a fixed array.

--jm

On Wednesday, 15 May 2013 at 15:35:05 UTC, Juan Manuel Cabo wrote:
>
> It sounds like you need to:
>     1) use a Message Queue.
>     2) Copy the message while you work on it with the consumer, so that you can exit the mutex.

At which point I'll suggest considering std.concurrency instead of rolling everything yourself.