First, the answer may be as simple as "use core.thread.thread_joinAll". Is that the proper way of waiting for all threads?

Second, my question may not be a valid example as starting a thread without communicating with it may be under the umbrella of parallelization. Maybe in concurrency, threads communicate with each other so that the following situation should not occur in practice.

Third is my question: :) If I spawn a single thread in main, the single thread seems to run to completion.

import std.stdio;
import std.concurrency;
import core.thread;

void foo()
{
    foreach (i; 0 .. 5) {
        Thread.sleep(dur!"msecs"(500));
        writeln(i, " foo");
    }
}

void main()
{
    spawn(&foo);
    writeln("main done");
}

I get all of foo's output after "main done":

main done
0 foo
1 foo
2 foo
3 foo
4 foo

If I introduce an intermediate thread that spawns the foo thread, now foo sometimes terminates early:

import std.stdio;
import std.concurrency;
import core.thread;

void foo()
{
    foreach (i; 0 .. 5) {
        Thread.sleep(dur!"msecs"(500));
        writeln(i, " foo");
    }
}

void intermediate()
{
    spawn(&foo);
    writeln("intermediate done");
}

void main()
{
    spawn(&intermediate);
    writeln("main done");
}

The output is inconsistent. Sometimes there is nothing from foo:

main done
intermediate done

Sometimes it runs fully:

main done
intermediate done
0 foo
1 foo
2 foo
3 foo
4 foo

Is the inconsistency a bug or a natural consequence of something? :) (Perhaps even the first example that seems to run correctly just has a higher probability of showing this behavior.)

I am aware of thread_joinAll(). Is that the recommended way of waiting for all threads?

Thank you,
Ali

On Mon, 06 Jun 2011 14:09:25 -0400, Ali Çehreli <acehreli@yahoo.com> wrote:

> First, the answer may be as simple as "use core.thread.thread_joinAll". Is that the proper way of waiting for all threads?

main (the C main, not D main) does this already:

https://github.com/D-Programming-Language/druntime/blob/master/src/rt/dmain2.d#L512

But note that "daemonized" threads will not be included:

http://www.digitalmars.com/d/2.0/phobos/core_thread.html#isDaemon

-Steve

On 06/06/2011 12:07 PM, Steven Schveighoffer wrote:
> On Mon, 06 Jun 2011 14:09:25 -0400, Ali Çehreli <acehreli@yahoo.com> wrote:
>
>> First, the answer may be as simple as "use
>> core.thread.thread_joinAll". Is that the proper way of waiting for all
>> threads?
>
> main (the C main, not D main) does this already:
>
> https://github.com/D-Programming-Language/druntime/blob/master/src/rt/dmain2.d#L512
>
>
> But note that "daemonized" threads will not be included:
>
> http://www.digitalmars.com/d/2.0/phobos/core_thread.html#isDaemon
>
> -Steve

Thank you but it doesn't explain the inconsistent behavior. It seems like thread_joinAll() has a different idea at different times. Now I also print the result of isDaemon():

import std.stdio;
import std.concurrency;
import core.thread;

void tell_daemon_state(string name)
{
    writeln(name, " isDaemon: ", Thread.getThis.isDaemon);
}

void foo()
{
    tell_daemon_state("foo");

    foreach (i; 0 .. 5) {
        Thread.sleep(dur!"msecs"(500));
        writeln(i, " foo");
    }
}

void intermediate()
{
    tell_daemon_state("intermediate");

    spawn(&foo);
    writeln("intermediate done");
}

void main()
{
    tell_daemon_state("main");

    spawn(&intermediate);
    writeln("main done");
}

I see that only the main thread is a daemon:

main isDaemon: true
main done
intermediate isDaemon: false
intermediate done
foo isDaemon: false
0 foo
1 foo
2 foo
3 foo
4 foo

That makes sense.

There is a race condition: Just because I added the printing of the isDaemon state, now foo() runs to completion seemingly everytime I start the program. When I remove the printing AND run the program under 'time', I get inconsistent behavior.

The following are two consecutive runs:

$ time ./deneme
main done
intermediate done
0 foo                    <--- foo()'s output is present
1 foo
2 foo
3 foo
4 foo

real	0m2.504s
user	0m0.000s
sys	0m0.000s

$ time ./deneme
main done
intermediate done        <--- foo()'s output is missing

real	0m0.003s
user	0m0.000s
sys	0m0.000s

As if thread_joinAll() misses the fact that there is still the non-daemon foo() thread.

Note that it's not failing to flush stdout either. The program runs shorter in the case where foo()'s output is missing.

Thank you,
Ali

On 2011-06-06 14:37, Ali Çehreli wrote:
> On 06/06/2011 12:07 PM, Steven Schveighoffer wrote:
> > On Mon, 06 Jun 2011 14:09:25 -0400, Ali Çehreli <acehreli@yahoo.com>
wrote:
> >> First, the answer may be as simple as "use core.thread.thread_joinAll". Is that the proper way of waiting for all threads?
> > 
> > main (the C main, not D main) does this already:
> > 
> > https://github.com/D-Programming-Language/druntime/blob/master/src/rt/dma in2.d#L512
> > 
> > 
> > But note that "daemonized" threads will not be included:
> > 
> > http://www.digitalmars.com/d/2.0/phobos/core_thread.html#isDaemon
> > 
> > -Steve
> 
> Thank you but it doesn't explain the inconsistent behavior. It seems
> like thread_joinAll() has a different idea at different times. Now I
> also print the result of isDaemon():
> 
> import std.stdio;
> import std.concurrency;
> import core.thread;
> 
> void tell_daemon_state(string name)
> {
> writeln(name, " isDaemon: ", Thread.getThis.isDaemon);
> }
> 
> void foo()
> {
> tell_daemon_state("foo");
> 
> foreach (i; 0 .. 5) {
> Thread.sleep(dur!"msecs"(500));
> writeln(i, " foo");
> }
> }
> 
> void intermediate()
> {
> tell_daemon_state("intermediate");
> 
> spawn(&foo);
> writeln("intermediate done");
> }
> 
> void main()
> {
> tell_daemon_state("main");
> 
> spawn(&intermediate);
> writeln("main done");
> }
> 
> I see that only the main thread is a daemon:
> 
> main isDaemon: true
> main done
> intermediate isDaemon: false
> intermediate done
> foo isDaemon: false
> 0 foo
> 1 foo
> 2 foo
> 3 foo
> 4 foo
> 
> That makes sense.
> 
> There is a race condition: Just because I added the printing of the isDaemon state, now foo() runs to completion seemingly everytime I start the program. When I remove the printing AND run the program under 'time', I get inconsistent behavior.
> 
> The following are two consecutive runs:
> 
> $ time ./deneme
> main done
> intermediate done
> 0 foo <--- foo()'s output is present
> 1 foo
> 2 foo
> 3 foo
> 4 foo
> 
> real 0m2.504s
> user 0m0.000s
> sys 0m0.000s
> 
> $ time ./deneme
> main done
> intermediate done <--- foo()'s output is missing
> 
> real 0m0.003s
> user 0m0.000s
> sys 0m0.000s
> 
> As if thread_joinAll() misses the fact that there is still the
> non-daemon foo() thread.
> 
> Note that it's not failing to flush stdout either. The program runs shorter in the case where foo()'s output is missing.

Unless the code has changed (and Sean was working on it a couple of months back, so I'm not sure what the current state is), on Linux, none of the spawned threads ever get joined, and they're all joinable - which causes problems. As I understand it, they should all be detached (as in the pthread concept of detached, not detached from the GC like core.Thread talks about) rather than joinable.

At this point, I don't trust spawn at all (on Linux at least). I've had too many problems with it. But I don't know what the current state is, because Sean was at least working on improving the situation. It's possible that the joinable issues and whatnot were worked out, but I don't know and kind of doubt it.

Regardless, spawned threads aren't intended to be joined by you. They should run until they're done doing whatever they're doing and then exit. And the program should wait for them all to exit, even if main finishes. If that's not happening, then there are bugs that need to be fixed. You shouldn't ever have to worry about joining spawned threads.

- Jonathan M Davis

On 06/06/2011 03:52 PM, Jonathan M Davis wrote:

> At this point, I don't trust spawn at all (on Linux at least). I've had too
> many problems with it.

Thank you.

I've spawned threads from within threads and now received ThreadException and segmentation faults as well:

  http://d.puremagic.com/issues/show_bug.cgi?id=6116

Ali