Hi all,

I am a newbie with D but experienced C++ developer.. this is my first post, so I MUST say that D ROCKS! Really impressive language. Walter, you are definitely the Lord of Compilers. Back to the subject, I am writing a massively multithreaded piece of code, and after some time I get a "failed to start" error. I digged in the forums and found someone else with same problem, but no answer. So was time to dig into Thread sources...

...and I see a static destructor cleaning a single handle (the main thread, I suppose), but no CloseHandle on any other handle created via _beginthreadex (I am talking about Windows, I should have specified before, sorry).

I believe there should be an explicit Thread destructor able to free the handle via CloseHandle; also, because thread handles under Windows are a limited resource, probably a RIAA scheme (or explicit "delete" call) should be used, instead of waiting for the GC to pass by..

So, I added a CloseHandle(mythread.hdl) call and now my handles count (in the Task Manager) is much more under control...

ciao
Bab

I suspect you have found a bug, in which case it should be reported in the bug tracker...  Now, as I've been away for 6 months and forgotten, can someone tell us both how that's done.

Regan

Regan Heath schrieb:
> I suspect you have found a bug, in which case it should be reported in the bug tracker...  Now, as I've been away for 6 months and forgotten, can someone tell us both how that's done.
> 
> Regan

look here:
http://d.puremagic.com/issues/index.cgi


david

On Fri, 11 May 2007 18:47:04 -0400, Regan Heath <regan@netmail.co.nz> wrote:

> I suspect you have found a bug, in which case it should be reported in the bug tracker...  Now, as I've been away for 6 months and forgotten, can someone tell us both how that's done.
>
> Regan

It has been reported and confirmed several times:
   http://d.puremagic.com/issues/show_bug.cgi?id=318

Babele Dunnit wrote:

> I am a newbie with D but experienced C++ developer.. this is my first post, so I MUST say that D ROCKS! Really impressive language. Walter, you are definitely the Lord of Compilers. Back to the subject, I am writing a massively multithreaded piece of code, and after some time I get a "failed to start" error. I digged in the forums and found someone else with same problem, but no answer. So was time to dig into Thread sources...
> 
> ...and I see a static destructor cleaning a single handle (the main thread, I suppose), but no CloseHandle on any other handle created via _beginthreadex (I am talking about Windows, I should have specified before, sorry). 
> 
> I believe there should be an explicit Thread destructor able to free the handle via CloseHandle; also, because thread handles under Windows are a limited resource, probably a RIAA scheme (or explicit "delete" call) should be used, instead of waiting for the GC to pass by..
> 
> So, I added a CloseHandle(mythread.hdl) call and now my handles count (in the Task Manager) is much more under control...

I'm not too experienced with threading, so if you could post a patch that would be welcome.

Walter Bright wrote:
> Babele Dunnit wrote:
> 
>> I am a newbie with D but experienced C++ developer.. this is my first post, so I MUST say that D ROCKS! Really impressive language. Walter, you are definitely the Lord of Compilers. Back to the subject, I am writing a massively multithreaded piece of code, and after some time I get a "failed to start" error. I digged in the forums and found someone else with same problem, but no answer. So was time to dig into Thread sources...
>>
>> ...and I see a static destructor cleaning a single handle (the main
>> thread, I suppose), but no CloseHandle on any other handle created via
>> _beginthreadex (I am talking about Windows, I should have specified
>> before, sorry).
>> I believe there should be an explicit Thread destructor able to free
>> the handle via CloseHandle; also, because thread handles under Windows
>> are a limited resource, probably a RIAA scheme (or explicit "delete"
>> call) should be used, instead of waiting for the GC to pass by..
>>
>> So, I added a CloseHandle(mythread.hdl) call and now my handles count
>> (in the Task Manager) is much more under control...
> 
> I'm not too experienced with threading, so if you could post a patch that would be welcome.

I also experience the same issue in linux. It happens after I create/delete over 8000 threads. I dunno why 8000 is the magic number, but it is for my machine. I'll try and look into it as well on linux. Just wanted to let you know that the problem also occurs in linux

kenny Wrote:
> I also experience the same issue in linux. It happens after I create/delete over 8000 threads. I dunno why 8000 is the magic number, but it is for my machine. I'll try and look into it as well on linux. Just wanted to let you know that the problem also occurs in linux

Each operating system has a maximum number of handles, it differs for operating system, each version of each operating system, and can be defined for each process upon execution as well.  Or so I recall from my travels.

Regan Heath

Walter Bright Wrote:
> Babele Dunnit wrote:
> 
> > I am a newbie with D but experienced C++ developer.. this is my first post, so I MUST say that D ROCKS! Really impressive language. Walter, you are definitely the Lord of Compilers. Back to the subject, I am writing a massively multithreaded piece of code, and after some time I get a "failed to start" error. I digged in the forums and found someone else with same problem, but no answer. So was time to dig into Thread sources...
> > 
> > ...and I see a static destructor cleaning a single handle (the main thread, I suppose), but no CloseHandle on any other handle created via _beginthreadex (I am talking about Windows, I should have specified before, sorry).
> > 
> > I believe there should be an explicit Thread destructor able to free the handle via CloseHandle; also, because thread handles under Windows are a limited resource, probably a RIAA scheme (or explicit "delete" call) should be used, instead of waiting for the GC to pass by..
> > 
> > So, I added a CloseHandle(mythread.hdl) call and now my handles count (in the Task Manager) is much more under control...
> 
> I'm not too experienced with threading, so if you could post a patch that would be welcome.

I've no idea how to do a patch as such but I believe all you need is a call to CloseHandle in:

extern (Windows) static uint threadstart(void *p)

as this wrapper is used for all threads and the stuff at the end of the wrapper removes the thread from allThreads and does cleanup.

eg

	debug (thread) printf("Ending thread %d\n", t.idx);
                version (Win32) {
                       CloseHandle(t.hdl);
                       t.hdl = cast(thread_hdl)0;
                }
	t.state = TS.TERMINATED;
	allThreads[t.idx] = null;

I cannot recall (nor do I have old code I can refer to, sadly) whether there is a similar call required on Linux etc, I suspect not given that pthread_create returns a thread id, as opposed to a handle per-se.

I have vague recollections of calling kill, but that may be to forcibly terminate a thread as opposed to cleaning up a handle.

Regan Heath

Regan Heath wrote:
> kenny Wrote:
>> I also experience the same issue in linux. It happens after I create/delete over 8000 threads. I dunno why 8000 is the magic number, but it is for my machine. I'll try and look into it as well on linux. Just wanted to let you know that the problem also occurs in linux
> 
> Each operating system has a maximum number of handles, it differs for operating system, each version of each operating system, and can be defined for each process upon execution as well.  Or so I recall from my travels.
> 
> Regan Heath

wow, now it's only giving me 382 threads... here is test code. Later today, I will dip into phobos and try and find the problem.

kenny wrote:

> Regan Heath wrote:
>> kenny Wrote:
>>> I also experience the same issue in linux. It happens after I create/delete over 8000 threads. I dunno why 8000 is the magic number, but it is for my machine. I'll try and look into it as well on linux. Just wanted to let you know that the problem also occurs in linux
>> 
>> Each operating system has a maximum number of handles, it differs for operating system, each version of each operating system, and can be defined for each process upon execution as well.  Or so I recall from my travels.
>> 
>> Regan Heath
> 
> wow, now it's only giving me 382 threads... here is test code. Later today, I will dip into phobos and try and find the problem.

For what it's worth, the Tango version of Thread has several fixes and changes compared to the Phobos version, and also provides other features like thread local storage and fibers/co-routines.

-- 
Lars Ivar Igesund
blog at http://larsivi.net
DSource, #d.tango & #D: larsivi
Dancing the Tango