http://d.puremagic.com/issues/show_bug.cgi?id=4307

           Summary: spawn()'ed thread doesn't terminate
           Product: D
           Version: D2
          Platform: x86
        OS/Version: Windows
            Status: NEW
          Severity: normal
          Priority: P2
         Component: DMD
        AssignedTo: nobody@puremagic.com
        ReportedBy: torhu@yahoo.com


--- Comment #0 from torhu@yahoo.com 2010-06-13 18:50:11 PDT ---
Using DMD 2.047.

This example hangs after printing '9'.  From reading Andrei's book, my understanding is that the spawned thread should terminate automatically when its owner thread terminates.  But that doesn't happen here.

---
import std.concurrency;
import std.stdio;

void f()
{
    for (;;)
    {
        int i = receiveOnly!int();
        writeln(i);
    }
}

void main()
{
    Tid tid = spawn(&f);
    foreach (int i; 0..10)
    {
        send(tid, i);
    }
}
---

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

The main thread is somewhat special in D--it doesn't actually terminate until all other threads have terminated.  And because the "OwnerTerminated" message is sent in a static dtor, the ordering is wrong for having spawned threads receive this message.  I'll leave it up to Andrei to decide whether this is correct behavior or if the OwnerTerminated message should be sent when D main() exits (I'm leaning towards this latter behavior myself).  If a change is necessary I'll have to add a stack for onMainExit callbacks to be executed in druntime.

Never mind, I'll fix it.  This will require adding an atexit() sort of routine to druntime, which I believe was requested in another ticket anyway.

http://d.puremagic.com/issues/show_bug.cgi?id=4307


Jonathan M Davis <jmdavisProg@gmx.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |jmdavisProg@gmx.com


--- Comment #1 from Jonathan M Davis <jmdavisProg@gmx.com> 2010-11-01 01:13:14 PDT ---
I don't know if this is the same problem or not, but the most threads that I seem to be able to create in a program is 505 or 506.

import std.concurrency;
import std.stdio;

void main()
{
    int currThreads = 0;
    enum maxThreads = 6;

    size_t totalThreads = 0;
    auto recProc = (Tid tid)
                   {
                       writeln(++totalThreads);
                   };

    for(size_t i = 0; i < 1_000; ++i)
    {
        if(currThreads < maxThreads)
            receiveTimeout(1, recProc);
        else
            receive(recProc);

        spawn(&threadFunc, thisTid);
    }

    while(currThreads > 0)
        receive(recProc);
}

void threadFunc(Tid parentTid)
{
    send(parentTid, thisTid);
}


prints out to either 505 or 506 and then throws an exception:

core.thread.ThreadException: Error creating thread
----------------
./d(void core.thread.Thread.start()) [0x80902b0]
./d(_D3std11concurrency34__T6_spawnTS3std11concurrency3TidZ6_spawnFbPFS3std11concurrency3TidZvS3std11concurrency3TidZS3std11concurrency3Tid+0x7f)
[0x808c3c7]
./d(_D3std11concurrency33__T5spawnTS3std11concurrency3TidZ5spawnFPFS3std11concurrency3TidZvS3std11concurrency3TidZS3std11concurrency3Tid+0x10)
[0x808c344]
./d(_Dmain+0x6b) [0x8087d0f]
./d(extern (C) int rt.dmain2.main(int, char**)) [0x8091a56]
./d(extern (C) int rt.dmain2.main(int, char**)) [0x80919b0]
./d(extern (C) int rt.dmain2.main(int, char**)) [0x8091a9a]
./d(extern (C) int rt.dmain2.main(int, char**)) [0x80919b0]
./d(main+0x96) [0x8091956]
/usr/lib32/libc.so.6(__libc_start_main+0xe6) [0xf756ec76]
./d() [0x8087bf1]


My best guess is that the threads aren't really terminating, so the OS is running out of threads.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=4307



--- Comment #2 from Jonathan M Davis <jmdavisProg@gmx.com> 2010-11-01 02:12:09 PDT ---
If I put a print statement at the end of exec() in std.concurrency._spawn(), it appears to print out just fine every time that a spawned thread is supposed to be terminating, so whatever the problem is, I think that it pretty much has to be in core.Thread (or maybe dmd itself, depending on what's causing the issue) rather than std.concurrency.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=4307



--- Comment #3 from Jonathan M Davis <jmdavisProg@gmx.com> 2010-11-01 02:39:58 PDT ---
I think what's happening is that pthread_join() never gets called on a thread
started with spawn(), and IIRC, if pthread_join() never gets called on a
thread, it never actually terminates. The only place that I see pthread_join()
getting called is in Thread.join(), and that never gets called for threads
started with spawn(). What we really want here, I think, is for threads which
succesfully terminate to just join to their parent thread themselves without
the parent thread having to call join() on them, but I'm not sure that you can
really do that with pthreads. Assuming that the parent thread has to join() on
all threads created with pthread_create(), we're going to need to find a way to
get the parent thread to call join() on its spawned threads. About all I can
think of is to have a thread whose entire job is to create threads and and make
sure that they join. But it's been too long since I had to deal with pthreads
for me to remember all of the details.

In any case, I believe that the problem stems from the fact that the spawned threads are never actually joined.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=4307



--- Comment #4 from Jonathan M Davis <jmdavisProg@gmx.com> 2010-11-01 03:17:58 PDT ---
Actually, what I think needs to happen is for there to be a way to start threads as detached rather than joinable and have spawn() start detached threads rather than joinable threads.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=4307



--- Comment #5 from Jonathan M Davis <jmdavisProg@gmx.com> 2010-11-02 20:33:50 PDT ---
Actually, the program I posted earlier was buggy (that's what I get for simplifying code and not studying the simplified version enough, I guess). Here's the corrected version:

import std.concurrency;
import std.stdio;

void main()
{
    int currThreads = 0;
    enum maxThreads = 6;

    size_t totalThreads = 0;
    auto recProc = (Tid tid)
                   {
                       writeln(++totalThreads);
                       --currThreads;
                   };

    for(size_t i = 0; i < 1_000; ++i)
    {
        if(currThreads < maxThreads)
            receiveTimeout(1, recProc);
        else
            receive(recProc);

        spawn(&threadFunc, thisTid);
        ++currThreads;
    }

    while(currThreads > 0)
        receive(recProc);
}

void threadFunc(Tid parentTid)
{
    send(parentTid, thisTid);
}

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=4307



--- Comment #6 from Jonathan M Davis <jmdavisProg@gmx.com> 2010-11-02 21:06:34 PDT ---
Created an attachment (id=797)
Patch for core.thread for linux with changes based off of dmd 2.050.

Here's a patch for core.thread which appears to solve the problem on Linux. Essentially, I made it so that you can give start() a bool to tell it to start the thread as detached (as in pthread_detach(), not detached from the D runtime), which I chose to call joinable (to distinguish from being detached from the D runtime). So, spawn() can call thread.start(false), and then the thread should terminate properly without needing to be joined. The changes are essentially Posix only, so if this problem is also on Windows, then other changes may be required for Windows and/or MacOS X, but I know essentially nothing about threads on either of those systems. So, I don't know if this patch is the best overall solution, but it does appear to fix the problem on Linux, so even if it's not enough of a fix, it should at least help solve the problem.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=4307



--- Comment #7 from Jonathan M Davis <jmdavisProg@gmx.com> 2010-11-02 21:07:34 PDT ---
Created an attachment (id=798)
Patch for std.concurrency for linux with changes based off of dmd 2.050.

The patch for std.concurrency to go with the patch for core.thread.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------