View mode: basic / threaded / horizontal-split · Log in · Help
September 24, 2003
Re: GC & multiple thread causes program hang
I have written a small C test app on XP and suspending the primary thread has no
effect on the process, execution continues. this makes me think that the primary
is in a critical, and common section when the pause is executed.

My synchronization implementation in thread.d is entirely independent of the gc
synchronization, thread uses threadLock, gc uses gclock, ir seem that the gc,
nenory allocator and thread suspension/resume might need to share a global
lock(s)

I'll let you know how I get on with Linux tonight.

Justin

In article <bkt5ue$25tp$1@digitaldaemon.com>, dickl says...
>
>I'm running XP.
>
>I don't think the synchronized() call is needed in resumeAll..
>The synchronized call in pauseAll should keep the main thread from doing stuff
>while a child thread is re-arranging memory. Its probably an very uncommon for
>this to happen.
>
>I plan on taking a more serious look at the gc tomorrow..
>--------------
>In article <bkt1ne$1vvi$1@digitaldaemon.com>, jhenzie@mac.com says...
>>
>>I have found the same results, see previous.
>>
>>Are you running on Win or Lin?
>>
>>
>>
>>
>>In article <bkt0sq$1umd$1@digitaldaemon.com>, dickl says...
>>>
>>>I've added synchronized() to Thread.pauseAll. I have also found that you can not
>>>pause allThreads[0] or the app will freeze. I believe this is because
>>>allThreads[0] is the main thread and pausing it will pause all the children.
>>>
>>>A better solution would be to put checks in the gc routines so that any memory
>>>alloc/dealloc waits for collectfull() to finish. There is no reason to pause
>>>threads which are not alloc/dealloc-ing memory. I don't understand the gc enough
>>>yet to suggest a change.
>>>
>>>
>>>
>>>static void pauseAll()
>>>{
>>>if (nthreads > 1)
>>>{
>>>synchronized (threadLock)
>>>{
>>>Thread tthis = getThis();
>>>for (int i = 1; i < allThreadsDim; i++)
>>>{   
>>>Thread t;
>>>t = allThreads[i];
>>>if (t && t !== tthis && t.state == TS.RUNNING)
>>>t.pause();
>>>}
>>>}
>>>}
>>>}
>>>=============================
>>>In article <bkrhoh$2vhg$1@digitaldaemon.com>, Walter says...
>>>>
>>>>I think you're on the right track here. If you want to make the
>>>>modifications to thread.d, that'd be a great contribution to D!
>>>>
>>>><jhenzie@mac.com> wrote in message news:bkrbr7$2mhf$1@digitaldaemon.com...
>>>>> Walter
>>>>>
>>>>> Let me just say that I am new to D but I can not tell you how much I
>>>>appeciate
>>>>> your efforts.  It has
>>>>> given me a sense of excitement that I have not felt since Objective-C.
>>>>> [Objective-c retain] by the
>>>>> way.
>>>>>
>>>>> In any case my concerns with the Thread class are as follows.
>>>>>
>>>>> 1.  It seems to me that resumeAll and pauseAll ought to be considered
>>>>atomic
>>>>> operations.  In my
>>>>> opinion, a call to pauseAll should leave the calling thread as the only
>>>>running
>>>>> thread in the
>>>>> process and it should be serial. That is calls to pauseAll should be
>>>>serialized
>>>>> such that if thread1
>>>>> calls pauseAll before thread2, thread1 will be the controlling thread when
>>>>all
>>>>> is said and done.  A
>>>>> resumeAll will then allow thread2 to take control.
>>>>>
>>>>> To my mind this type of call should be extremely rare and only used by
>>>>people
>>>>> with GURU Status,
>>>>> after all suspending other threads ignores the possibility that they hold
>>>>a lock
>>>>> on a specific
>>>>> resource that the calling thread may soon need.  This is the reason
>>>>suspend and
>>>>> resume have been
>>>>> deprecated in Java.
>>>>>
>>>>> NB:  I realize this may be a somewhat naive view and am willing to be
>>>>educated.
>>>>> <smile/>
>>>>>
>>>>> The most basic approach would be (using resumeAll)
>>>>>
>>>>> static void resumeAll()
>>>>> {
>>>>> if (nthreads > 1)
>>>>> {
>>>>> Thread tthis = getThis();
>>>>>
>>>>> synchronized (threadlock) {
>>>>> for (int i = 0; i < allThreadsDim; i++)
>>>>> {   Thread t;
>>>>>
>>>>> t = allThreads[i];
>>>>> if (t && t !== tthis && t.state == TS.RUNNING)
>>>>> t.resume();
>>>>> }
>>>>> }
>>>>> }
>>>>> }
>>>>>
>>>>> of course this would require a recursive lock on Thread::resume
>>>>Thread::pause.
>>>>>
>>>>> 2.  Given the sychronized(object) stnrax ir would seem that every object
>>>>can act
>>>>> as mutex in D. If
>>>>> this is the case perhaps the addition of wait, notify and notifyall, a la
>>>>java,
>>>>> on object would allow
>>>>> for a more flexible and complete approach to multithreading.  Of course
>>>>this
>>>>> would require the
>>>>> Thread::wait method to be renamed Thread::join <smile/>.  No flames
>>>>please.
>>>>>
>>>>> I will consider it further and hopefully give as much feedback as I can
>>>>and is
>>>>> wanted.
>>>>>
>>>>> I will keep looking for the cause of the deadlock and let you know what I
>>>>find.
>>>>>
>>>>> What are people using as a D Debugger, is there such a beast?
>>>>>
>>>>> Justin
>>>>>
>>>>>
>>>>>
>>>>> In article <bkr2gh$29sc$1@digitaldaemon.com>, Walter says...
>>>>> >
>>>>> >
>>>>> ><jhenzie@mac.com> wrote in message
>>>>news:bkqu8s$24dq$1@digitaldaemon.com...
>>>>> >> It seems that the first call to the garbage collectors causes the
>>>>> >deadlock.  The
>>>>> >> interesting thing is
>>>>> >> that the Main thread is not deadlocked, amend your example and add the
>>>>> >following
>>>>> >> after
>>>>> >> Sleep(10000);
>>>>> >>
>>>>> >> Thread.resumeAll()
>>>>> >>
>>>>> >> The application exits as you would expect which tends to point to
>>>>another
>>>>> >> problem. that  of
>>>>> >> threads that are sleeping being ignored as part of a
>>>>Thread.suspendAll()
>>>>> >>
>>>>> >> I am new to D so it may take me some time to figure this out but you
>>>>are
>>>>> >not
>>>>> >> alone.
>>>>> >>
>>>>> >> Walter if you are reading, is there any reason why sychronization
>>>>> >semantics are
>>>>> >> not present in
>>>>> >> Thread.suspendAll.  I fairly sure you are handling the locking lower
>>>>down
>>>>> >but it
>>>>> >> would appear
>>>>> >> logica;l to implement the lock on the Thread class.  But again I'm new
>>>>> >here so
>>>>> >> feel free to disabuse
>>>>> >> of that notion.
>>>>> >>
>>>>> >> More soon.
>>>>> >>
>>>>> >> Justin
>>>>> >
>>>>> >Can you suggest any fixes to thread.d?
>>>>> >
>>>>> >
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>
>
September 25, 2003
Re: GC & multiple thread causes program hang
Ok so Linux hangs as well but that should not be that much of a suprise since a
process is active in unix and so suspending it means that the process will not
be allocated an cpu time until woken up so it can't schedule threads, of course
this depends on the threading model.  More annoyingly SIGINT is blocked so you
can't ctrl-c tha application, must change that.

I am going to look at the windows code again today because the only reason a
thread would cease to run if its parent thread were paused woul dbe if it were a
fibre.

Let me know if you make any progress.


Justin 





>In article <bkt5ue$25tp$1@digitaldaemon.com>, dickl says...
>>
>>I'm running XP.
>>
>>I don't think the synchronized() call is needed in resumeAll..
>>The synchronized call in pauseAll should keep the main thread from doing stuff
>>while a child thread is re-arranging memory. Its probably an very uncommon for
>>this to happen.
>>
>>I plan on taking a more serious look at the gc tomorrow..
>>--------------
>>In article <bkt1ne$1vvi$1@digitaldaemon.com>, jhenzie@mac.com says...
>>>
>>>I have found the same results, see previous.
>>>
>>>Are you running on Win or Lin?
>>>
>>>
>>>
>>>
>>>In article <bkt0sq$1umd$1@digitaldaemon.com>, dickl says...
>>>>
>>>>I've added synchronized() to Thread.pauseAll. I have also found that you can not
>>>>pause allThreads[0] or the app will freeze. I believe this is because
>>>>allThreads[0] is the main thread and pausing it will pause all the children.
>>>>
>>>>A better solution would be to put checks in the gc routines so that any memory
>>>>alloc/dealloc waits for collectfull() to finish. There is no reason to pause
>>>>threads which are not alloc/dealloc-ing memory. I don't understand the gc enough
>>>>yet to suggest a change.
>>>>
>>>>
>>>>
>>>>static void pauseAll()
>>>>{
>>>>if (nthreads > 1)
>>>>{
>>>>synchronized (threadLock)
>>>>{
>>>>Thread tthis = getThis();
>>>>for (int i = 1; i < allThreadsDim; i++)
>>>>{   
>>>>Thread t;
>>>>t = allThreads[i];
>>>>if (t && t !== tthis && t.state == TS.RUNNING)
>>>>t.pause();
>>>>}
>>>>}
>>>>}
>>>>}
>>>>=============================
>>>>In article <bkrhoh$2vhg$1@digitaldaemon.com>, Walter says...
>>>>>
>>>>>I think you're on the right track here. If you want to make the
>>>>>modifications to thread.d, that'd be a great contribution to D!
>>>>>
>>>>><jhenzie@mac.com> wrote in message news:bkrbr7$2mhf$1@digitaldaemon.com...
>>>>>> Walter
>>>>>>
>>>>>> Let me just say that I am new to D but I can not tell you how much I
>>>>>appeciate
>>>>>> your efforts.  It has
>>>>>> given me a sense of excitement that I have not felt since Objective-C.
>>>>>> [Objective-c retain] by the
>>>>>> way.
>>>>>>
>>>>>> In any case my concerns with the Thread class are as follows.
>>>>>>
>>>>>> 1.  It seems to me that resumeAll and pauseAll ought to be considered
>>>>>atomic
>>>>>> operations.  In my
>>>>>> opinion, a call to pauseAll should leave the calling thread as the only
>>>>>running
>>>>>> thread in the
>>>>>> process and it should be serial. That is calls to pauseAll should be
>>>>>serialized
>>>>>> such that if thread1
>>>>>> calls pauseAll before thread2, thread1 will be the controlling thread when
>>>>>all
>>>>>> is said and done.  A
>>>>>> resumeAll will then allow thread2 to take control.
>>>>>>
>>>>>> To my mind this type of call should be extremely rare and only used by
>>>>>people
>>>>>> with GURU Status,
>>>>>> after all suspending other threads ignores the possibility that they hold
>>>>>a lock
>>>>>> on a specific
>>>>>> resource that the calling thread may soon need.  This is the reason
>>>>>suspend and
>>>>>> resume have been
>>>>>> deprecated in Java.
>>>>>>
>>>>>> NB:  I realize this may be a somewhat naive view and am willing to be
>>>>>educated.
>>>>>> <smile/>
>>>>>>
>>>>>> The most basic approach would be (using resumeAll)
>>>>>>
>>>>>> static void resumeAll()
>>>>>> {
>>>>>> if (nthreads > 1)
>>>>>> {
>>>>>> Thread tthis = getThis();
>>>>>>
>>>>>> synchronized (threadlock) {
>>>>>> for (int i = 0; i < allThreadsDim; i++)
>>>>>> {   Thread t;
>>>>>>
>>>>>> t = allThreads[i];
>>>>>> if (t && t !== tthis && t.state == TS.RUNNING)
>>>>>> t.resume();
>>>>>> }
>>>>>> }
>>>>>> }
>>>>>> }
>>>>>>
>>>>>> of course this would require a recursive lock on Thread::resume
>>>>>Thread::pause.
>>>>>>
>>>>>> 2.  Given the sychronized(object) stnrax ir would seem that every object
>>>>>can act
>>>>>> as mutex in D. If
>>>>>> this is the case perhaps the addition of wait, notify and notifyall, a la
>>>>>java,
>>>>>> on object would allow
>>>>>> for a more flexible and complete approach to multithreading.  Of course
>>>>>this
>>>>>> would require the
>>>>>> Thread::wait method to be renamed Thread::join <smile/>.  No flames
>>>>>please.
>>>>>>
>>>>>> I will consider it further and hopefully give as much feedback as I can
>>>>>and is
>>>>>> wanted.
>>>>>>
>>>>>> I will keep looking for the cause of the deadlock and let you know what I
>>>>>find.
>>>>>>
>>>>>> What are people using as a D Debugger, is there such a beast?
>>>>>>
>>>>>> Justin
>>>>>>
>>>>>>
>>>>>>
>>>>>> In article <bkr2gh$29sc$1@digitaldaemon.com>, Walter says...
>>>>>> >
>>>>>> >
>>>>>> ><jhenzie@mac.com> wrote in message
>>>>>news:bkqu8s$24dq$1@digitaldaemon.com...
>>>>>> >> It seems that the first call to the garbage collectors causes the
>>>>>> >deadlock.  The
>>>>>> >> interesting thing is
>>>>>> >> that the Main thread is not deadlocked, amend your example and add the
>>>>>> >following
>>>>>> >> after
>>>>>> >> Sleep(10000);
>>>>>> >>
>>>>>> >> Thread.resumeAll()
>>>>>> >>
>>>>>> >> The application exits as you would expect which tends to point to
>>>>>another
>>>>>> >> problem. that  of
>>>>>> >> threads that are sleeping being ignored as part of a
>>>>>Thread.suspendAll()
>>>>>> >>
>>>>>> >> I am new to D so it may take me some time to figure this out but you
>>>>>are
>>>>>> >not
>>>>>> >> alone.
>>>>>> >>
>>>>>> >> Walter if you are reading, is there any reason why sychronization
>>>>>> >semantics are
>>>>>> >> not present in
>>>>>> >> Thread.suspendAll.  I fairly sure you are handling the locking lower
>>>>>down
>>>>>> >but it
>>>>>> >> would appear
>>>>>> >> logica;l to implement the lock on the Thread class.  But again I'm new
>>>>>> >here so
>>>>>> >> feel free to disabuse
>>>>>> >> of that notion.
>>>>>> >>
>>>>>> >> More soon.
>>>>>> >>
>>>>>> >> Justin
>>>>>> >
>>>>>> >Can you suggest any fixes to thread.d?
>>>>>> >
>>>>>> >
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>
>
September 25, 2003
Re: GC & multiple thread causes program hang
I've looked a the GC code in more detail. It appears anything to do anything
with dynamic memory,  has a call to synchronized() in it. So, it theory, any
thread making a call will wait for any GC in progress. So, I don't see a
need for the calls to thread.pauseAll and thread.resumeAll.

I commented them out in gcx.fullCollect. So far I have not seen any
problems. I've
run my app for a for a few hours with no problems. I also then added another
thread which
calls fullCollect every 1 Sec and have my worker thread calling  fullCollect
every loop (about  every 50mS). Didn't see a problem there either.

For all the tests above, I put that for-loop in pauseAll back the way it
was. I'm not using thread.pauseAll in my app. More checking on the Thread
code is needed to see if pauseAll works ok when used by itself (outside of
the GC code).

The gc code is somewhat disruptive to my worker thread since it is doing
real time signal processing, so I am looking at removing all dynamic memory
stuff from it.


<jhenzie@mac.com> wrote in message news:bkt7p3$28e7$1@digitaldaemon.com...
> I have written a small C test app on XP and suspending the primary thread
has no
> effect on the process, execution continues. this makes me think that the
primary
> is in a critical, and common section when the pause is executed.
>
> My synchronization implementation in thread.d is entirely independent of
the gc
> synchronization, thread uses threadLock, gc uses gclock, ir seem that the
gc,
> nenory allocator and thread suspension/resume might need to share a global
> lock(s)
>
> I'll let you know how I get on with Linux tonight.
>
> Justin
>
> In article <bkt5ue$25tp$1@digitaldaemon.com>, dickl says...
> >
> >I'm running XP.
> >
> >I don't think the synchronized() call is needed in resumeAll..
> >The synchronized call in pauseAll should keep the main thread from doing
stuff
> >while a child thread is re-arranging memory. Its probably an very
uncommon for
> >this to happen.
> >
> >I plan on taking a more serious look at the gc tomorrow..
> >--------------
> >In article <bkt1ne$1vvi$1@digitaldaemon.com>, jhenzie@mac.com says...
> >>
> >>I have found the same results, see previous.
> >>
> >>Are you running on Win or Lin?
> >>
> >>
> >>
> >>
> >>In article <bkt0sq$1umd$1@digitaldaemon.com>, dickl says...
> >>>
> >>>I've added synchronized() to Thread.pauseAll. I have also found that
you can not
> >>>pause allThreads[0] or the app will freeze. I believe this is because
> >>>allThreads[0] is the main thread and pausing it will pause all the
children.
> >>>
> >>>A better solution would be to put checks in the gc routines so that any
memory
> >>>alloc/dealloc waits for collectfull() to finish. There is no reason to
pause
> >>>threads which are not alloc/dealloc-ing memory. I don't understand the
gc enough
> >>>yet to suggest a change.
> >>>
> >>>
> >>>
> >>>static void pauseAll()
> >>>{
> >>>if (nthreads > 1)
> >>>{
> >>>synchronized (threadLock)
> >>>{
> >>>Thread tthis = getThis();
> >>>for (int i = 1; i < allThreadsDim; i++)
> >>>{
> >>>Thread t;
> >>>t = allThreads[i];
> >>>if (t && t !== tthis && t.state == TS.RUNNING)
> >>>t.pause();
> >>>}
> >>>}
> >>>}
> >>>}
> >>>=============================
> >>>In article <bkrhoh$2vhg$1@digitaldaemon.com>, Walter says...
> >>>>
> >>>>I think you're on the right track here. If you want to make the
> >>>>modifications to thread.d, that'd be a great contribution to D!
> >>>>
> >>>><jhenzie@mac.com> wrote in message
news:bkrbr7$2mhf$1@digitaldaemon.com...
> >>>>> Walter
> >>>>>
> >>>>> Let me just say that I am new to D but I can not tell you how much I
> >>>>appeciate
> >>>>> your efforts.  It has
> >>>>> given me a sense of excitement that I have not felt since
Objective-C.
> >>>>> [Objective-c retain] by the
> >>>>> way.
> >>>>>
> >>>>> In any case my concerns with the Thread class are as follows.
> >>>>>
> >>>>> 1.  It seems to me that resumeAll and pauseAll ought to be
considered
> >>>>atomic
> >>>>> operations.  In my
> >>>>> opinion, a call to pauseAll should leave the calling thread as the
only
> >>>>running
> >>>>> thread in the
> >>>>> process and it should be serial. That is calls to pauseAll should be
> >>>>serialized
> >>>>> such that if thread1
> >>>>> calls pauseAll before thread2, thread1 will be the controlling
thread when
> >>>>all
> >>>>> is said and done.  A
> >>>>> resumeAll will then allow thread2 to take control.
> >>>>>
> >>>>> To my mind this type of call should be extremely rare and only used
by
> >>>>people
> >>>>> with GURU Status,
> >>>>> after all suspending other threads ignores the possibility that they
hold
> >>>>a lock
> >>>>> on a specific
> >>>>> resource that the calling thread may soon need.  This is the reason
> >>>>suspend and
> >>>>> resume have been
> >>>>> deprecated in Java.
> >>>>>
> >>>>> NB:  I realize this may be a somewhat naive view and am willing to
be
> >>>>educated.
> >>>>> <smile/>
> >>>>>
> >>>>> The most basic approach would be (using resumeAll)
> >>>>>
> >>>>> static void resumeAll()
> >>>>> {
> >>>>> if (nthreads > 1)
> >>>>> {
> >>>>> Thread tthis = getThis();
> >>>>>
> >>>>> synchronized (threadlock) {
> >>>>> for (int i = 0; i < allThreadsDim; i++)
> >>>>> {   Thread t;
> >>>>>
> >>>>> t = allThreads[i];
> >>>>> if (t && t !== tthis && t.state == TS.RUNNING)
> >>>>> t.resume();
> >>>>> }
> >>>>> }
> >>>>> }
> >>>>> }
> >>>>>
> >>>>> of course this would require a recursive lock on Thread::resume
> >>>>Thread::pause.
> >>>>>
> >>>>> 2.  Given the sychronized(object) stnrax ir would seem that every
object
> >>>>can act
> >>>>> as mutex in D. If
> >>>>> this is the case perhaps the addition of wait, notify and notifyall,
a la
> >>>>java,
> >>>>> on object would allow
> >>>>> for a more flexible and complete approach to multithreading.  Of
course
> >>>>this
> >>>>> would require the
> >>>>> Thread::wait method to be renamed Thread::join <smile/>.  No flames
> >>>>please.
> >>>>>
> >>>>> I will consider it further and hopefully give as much feedback as I
can
> >>>>and is
> >>>>> wanted.
> >>>>>
> >>>>> I will keep looking for the cause of the deadlock and let you know
what I
> >>>>find.
> >>>>>
> >>>>> What are people using as a D Debugger, is there such a beast?
> >>>>>
> >>>>> Justin
> >>>>>
> >>>>>
> >>>>>
> >>>>> In article <bkr2gh$29sc$1@digitaldaemon.com>, Walter says...
> >>>>> >
> >>>>> >
> >>>>> ><jhenzie@mac.com> wrote in message
> >>>>news:bkqu8s$24dq$1@digitaldaemon.com...
> >>>>> >> It seems that the first call to the garbage collectors causes the
> >>>>> >deadlock.  The
> >>>>> >> interesting thing is
> >>>>> >> that the Main thread is not deadlocked, amend your example and
add the
> >>>>> >following
> >>>>> >> after
> >>>>> >> Sleep(10000);
> >>>>> >>
> >>>>> >> Thread.resumeAll()
> >>>>> >>
> >>>>> >> The application exits as you would expect which tends to point to
> >>>>another
> >>>>> >> problem. that  of
> >>>>> >> threads that are sleeping being ignored as part of a
> >>>>Thread.suspendAll()
> >>>>> >>
> >>>>> >> I am new to D so it may take me some time to figure this out but
you
> >>>>are
> >>>>> >not
> >>>>> >> alone.
> >>>>> >>
> >>>>> >> Walter if you are reading, is there any reason why sychronization
> >>>>> >semantics are
> >>>>> >> not present in
> >>>>> >> Thread.suspendAll.  I fairly sure you are handling the locking
lower
> >>>>down
> >>>>> >but it
> >>>>> >> would appear
> >>>>> >> logica;l to implement the lock on the Thread class.  But again
I'm new
> >>>>> >here so
> >>>>> >> feel free to disabuse
> >>>>> >> of that notion.
> >>>>> >>
> >>>>> >> More soon.
> >>>>> >>
> >>>>> >> Justin
> >>>>> >
> >>>>> >Can you suggest any fixes to thread.d?
> >>>>> >
> >>>>> >
> >>>>>
> >>>>>
> >>>>
> >>>>
> >>>
> >>>
> >>
> >>
> >
> >
>
>
September 25, 2003
Re: GC & multiple thread causes program hang
Totally agree the synchonized semantics are unnecessary in Thread.

The problem is clearly to do with suspending the primary thread.  On unix one
might expect the app to hang given that what D uses as allThreads[0] is actually
the process, but this does not explain the windows hang since windows processes
are not active.

It is very strange.  

Justin


In article <bkvl68$1s3f$1@digitaldaemon.com>, dickl says...
>
>I've looked a the GC code in more detail. It appears anything to do anything
>with dynamic memory,  has a call to synchronized() in it. So, it theory, any
>thread making a call will wait for any GC in progress. So, I don't see a
>need for the calls to thread.pauseAll and thread.resumeAll.
>
>I commented them out in gcx.fullCollect. So far I have not seen any
>problems. I've
>run my app for a for a few hours with no problems. I also then added another
>thread which
>calls fullCollect every 1 Sec and have my worker thread calling  fullCollect
>every loop (about  every 50mS). Didn't see a problem there either.
>
>For all the tests above, I put that for-loop in pauseAll back the way it
>was. I'm not using thread.pauseAll in my app. More checking on the Thread
>code is needed to see if pauseAll works ok when used by itself (outside of
>the GC code).
>
>The gc code is somewhat disruptive to my worker thread since it is doing
>real time signal processing, so I am looking at removing all dynamic memory
>stuff from it.
>
>
><jhenzie@mac.com> wrote in message news:bkt7p3$28e7$1@digitaldaemon.com...
>> I have written a small C test app on XP and suspending the primary thread
>has no
>> effect on the process, execution continues. this makes me think that the
>primary
>> is in a critical, and common section when the pause is executed.
>>
>> My synchronization implementation in thread.d is entirely independent of
>the gc
>> synchronization, thread uses threadLock, gc uses gclock, ir seem that the
>gc,
>> nenory allocator and thread suspension/resume might need to share a global
>> lock(s)
>>
>> I'll let you know how I get on with Linux tonight.
>>
>> Justin
>>
>> In article <bkt5ue$25tp$1@digitaldaemon.com>, dickl says...
>> >
>> >I'm running XP.
>> >
>> >I don't think the synchronized() call is needed in resumeAll..
>> >The synchronized call in pauseAll should keep the main thread from doing
>stuff
>> >while a child thread is re-arranging memory. Its probably an very
>uncommon for
>> >this to happen.
>> >
>> >I plan on taking a more serious look at the gc tomorrow..
>> >--------------
>> >In article <bkt1ne$1vvi$1@digitaldaemon.com>, jhenzie@mac.com says...
>> >>
>> >>I have found the same results, see previous.
>> >>
>> >>Are you running on Win or Lin?
>> >>
>> >>
>> >>
>> >>
>> >>In article <bkt0sq$1umd$1@digitaldaemon.com>, dickl says...
>> >>>
>> >>>I've added synchronized() to Thread.pauseAll. I have also found that
>you can not
>> >>>pause allThreads[0] or the app will freeze. I believe this is because
>> >>>allThreads[0] is the main thread and pausing it will pause all the
>children.
>> >>>
>> >>>A better solution would be to put checks in the gc routines so that any
>memory
>> >>>alloc/dealloc waits for collectfull() to finish. There is no reason to
>pause
>> >>>threads which are not alloc/dealloc-ing memory. I don't understand the
>gc enough
>> >>>yet to suggest a change.
>> >>>
>> >>>
>> >>>
>> >>>static void pauseAll()
>> >>>{
>> >>>if (nthreads > 1)
>> >>>{
>> >>>synchronized (threadLock)
>> >>>{
>> >>>Thread tthis = getThis();
>> >>>for (int i = 1; i < allThreadsDim; i++)
>> >>>{
>> >>>Thread t;
>> >>>t = allThreads[i];
>> >>>if (t && t !== tthis && t.state == TS.RUNNING)
>> >>>t.pause();
>> >>>}
>> >>>}
>> >>>}
>> >>>}
>> >>>=============================
>> >>>In article <bkrhoh$2vhg$1@digitaldaemon.com>, Walter says...
>> >>>>
>> >>>>I think you're on the right track here. If you want to make the
>> >>>>modifications to thread.d, that'd be a great contribution to D!
>> >>>>
>> >>>><jhenzie@mac.com> wrote in message
>news:bkrbr7$2mhf$1@digitaldaemon.com...
>> >>>>> Walter
>> >>>>>
>> >>>>> Let me just say that I am new to D but I can not tell you how much I
>> >>>>appeciate
>> >>>>> your efforts.  It has
>> >>>>> given me a sense of excitement that I have not felt since
>Objective-C.
>> >>>>> [Objective-c retain] by the
>> >>>>> way.
>> >>>>>
>> >>>>> In any case my concerns with the Thread class are as follows.
>> >>>>>
>> >>>>> 1.  It seems to me that resumeAll and pauseAll ought to be
>considered
>> >>>>atomic
>> >>>>> operations.  In my
>> >>>>> opinion, a call to pauseAll should leave the calling thread as the
>only
>> >>>>running
>> >>>>> thread in the
>> >>>>> process and it should be serial. That is calls to pauseAll should be
>> >>>>serialized
>> >>>>> such that if thread1
>> >>>>> calls pauseAll before thread2, thread1 will be the controlling
>thread when
>> >>>>all
>> >>>>> is said and done.  A
>> >>>>> resumeAll will then allow thread2 to take control.
>> >>>>>
>> >>>>> To my mind this type of call should be extremely rare and only used
>by
>> >>>>people
>> >>>>> with GURU Status,
>> >>>>> after all suspending other threads ignores the possibility that they
>hold
>> >>>>a lock
>> >>>>> on a specific
>> >>>>> resource that the calling thread may soon need.  This is the reason
>> >>>>suspend and
>> >>>>> resume have been
>> >>>>> deprecated in Java.
>> >>>>>
>> >>>>> NB:  I realize this may be a somewhat naive view and am willing to
>be
>> >>>>educated.
>> >>>>> <smile/>
>> >>>>>
>> >>>>> The most basic approach would be (using resumeAll)
>> >>>>>
>> >>>>> static void resumeAll()
>> >>>>> {
>> >>>>> if (nthreads > 1)
>> >>>>> {
>> >>>>> Thread tthis = getThis();
>> >>>>>
>> >>>>> synchronized (threadlock) {
>> >>>>> for (int i = 0; i < allThreadsDim; i++)
>> >>>>> {   Thread t;
>> >>>>>
>> >>>>> t = allThreads[i];
>> >>>>> if (t && t !== tthis && t.state == TS.RUNNING)
>> >>>>> t.resume();
>> >>>>> }
>> >>>>> }
>> >>>>> }
>> >>>>> }
>> >>>>>
>> >>>>> of course this would require a recursive lock on Thread::resume
>> >>>>Thread::pause.
>> >>>>>
>> >>>>> 2.  Given the sychronized(object) stnrax ir would seem that every
>object
>> >>>>can act
>> >>>>> as mutex in D. If
>> >>>>> this is the case perhaps the addition of wait, notify and notifyall,
>a la
>> >>>>java,
>> >>>>> on object would allow
>> >>>>> for a more flexible and complete approach to multithreading.  Of
>course
>> >>>>this
>> >>>>> would require the
>> >>>>> Thread::wait method to be renamed Thread::join <smile/>.  No flames
>> >>>>please.
>> >>>>>
>> >>>>> I will consider it further and hopefully give as much feedback as I
>can
>> >>>>and is
>> >>>>> wanted.
>> >>>>>
>> >>>>> I will keep looking for the cause of the deadlock and let you know
>what I
>> >>>>find.
>> >>>>>
>> >>>>> What are people using as a D Debugger, is there such a beast?
>> >>>>>
>> >>>>> Justin
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> In article <bkr2gh$29sc$1@digitaldaemon.com>, Walter says...
>> >>>>> >
>> >>>>> >
>> >>>>> ><jhenzie@mac.com> wrote in message
>> >>>>news:bkqu8s$24dq$1@digitaldaemon.com...
>> >>>>> >> It seems that the first call to the garbage collectors causes the
>> >>>>> >deadlock.  The
>> >>>>> >> interesting thing is
>> >>>>> >> that the Main thread is not deadlocked, amend your example and
>add the
>> >>>>> >following
>> >>>>> >> after
>> >>>>> >> Sleep(10000);
>> >>>>> >>
>> >>>>> >> Thread.resumeAll()
>> >>>>> >>
>> >>>>> >> The application exits as you would expect which tends to point to
>> >>>>another
>> >>>>> >> problem. that  of
>> >>>>> >> threads that are sleeping being ignored as part of a
>> >>>>Thread.suspendAll()
>> >>>>> >>
>> >>>>> >> I am new to D so it may take me some time to figure this out but
>you
>> >>>>are
>> >>>>> >not
>> >>>>> >> alone.
>> >>>>> >>
>> >>>>> >> Walter if you are reading, is there any reason why sychronization
>> >>>>> >semantics are
>> >>>>> >> not present in
>> >>>>> >> Thread.suspendAll.  I fairly sure you are handling the locking
>lower
>> >>>>down
>> >>>>> >but it
>> >>>>> >> would appear
>> >>>>> >> logica;l to implement the lock on the Thread class.  But again
>I'm new
>> >>>>> >here so
>> >>>>> >> feel free to disabuse
>> >>>>> >> of that notion.
>> >>>>> >>
>> >>>>> >> More soon.
>> >>>>> >>
>> >>>>> >> Justin
>> >>>>> >
>> >>>>> >Can you suggest any fixes to thread.d?
>> >>>>> >
>> >>>>> >
>> >>>>>
>> >>>>>
>> >>>>
>> >>>>
>> >>>
>> >>>
>> >>
>> >>
>> >
>> >
>>
>>
>
>
September 26, 2003
Re: GC & multiple thread causes program hang
The Thread class looks like it needs some work. There isn't a way to kill a
thread and remove if from the list of threads, if the thread completes, you
can't re-start it later on.

Walter, whats the best way to address these issues with the Thread class ?

1) suspending allthread[0] suspends the application in pauseAll()
2) doing a delete on a thread causes problems (no destructor to remove the
thread from the list)
3) letting a thread complete and then restarting it causes problems.(no way
to mark a thread as terminated and allow it  to restarted)
4) the call to pauseAll() and resumeAll() does not seem to be necessary in
gcx.fullcollect()


<jhenzie@mac.com> wrote in message news:bkvq9i$2325$1@digitaldaemon.com...
> Totally agree the synchonized semantics are unnecessary in Thread.
>
> The problem is clearly to do with suspending the primary thread.  On unix
one
> might expect the app to hang given that what D uses as allThreads[0] is
actually
> the process, but this does not explain the windows hang since windows
processes
> are not active.
>
> It is very strange.
>
> Justin
>
>
> In article <bkvl68$1s3f$1@digitaldaemon.com>, dickl says...
> >
> >I've looked a the GC code in more detail. It appears anything to do
anything
> >with dynamic memory,  has a call to synchronized() in it. So, it theory,
any
> >thread making a call will wait for any GC in progress. So, I don't see a
> >need for the calls to thread.pauseAll and thread.resumeAll.
> >
> >I commented them out in gcx.fullCollect. So far I have not seen any
> >problems. I've
> >run my app for a for a few hours with no problems. I also then added
another
> >thread which
> >calls fullCollect every 1 Sec and have my worker thread calling
fullCollect
> >every loop (about  every 50mS). Didn't see a problem there either.
> >
> >For all the tests above, I put that for-loop in pauseAll back the way it
> >was. I'm not using thread.pauseAll in my app. More checking on the Thread
> >code is needed to see if pauseAll works ok when used by itself (outside
of
> >the GC code).
> >
> >The gc code is somewhat disruptive to my worker thread since it is doing
> >real time signal processing, so I am looking at removing all dynamic
memory
> >stuff from it.
> >
> >
> ><jhenzie@mac.com> wrote in message
news:bkt7p3$28e7$1@digitaldaemon.com...
> >> I have written a small C test app on XP and suspending the primary
thread
> >has no
> >> effect on the process, execution continues. this makes me think that
the
> >primary
> >> is in a critical, and common section when the pause is executed.
> >>
> >> My synchronization implementation in thread.d is entirely independent
of
> >the gc
> >> synchronization, thread uses threadLock, gc uses gclock, ir seem that
the
> >gc,
> >> nenory allocator and thread suspension/resume might need to share a
global
> >> lock(s)
> >>
> >> I'll let you know how I get on with Linux tonight.
> >>
> >> Justin
> >>
> >> In article <bkt5ue$25tp$1@digitaldaemon.com>, dickl says...
> >> >
> >> >I'm running XP.
> >> >
> >> >I don't think the synchronized() call is needed in resumeAll..
> >> >The synchronized call in pauseAll should keep the main thread from
doing
> >stuff
> >> >while a child thread is re-arranging memory. Its probably an very
> >uncommon for
> >> >this to happen.
> >> >
> >> >I plan on taking a more serious look at the gc tomorrow..
> >> >--------------
> >> >In article <bkt1ne$1vvi$1@digitaldaemon.com>, jhenzie@mac.com says...
> >> >>
> >> >>I have found the same results, see previous.
> >> >>
> >> >>Are you running on Win or Lin?
> >> >>
> >> >>
> >> >>
> >> >>
> >> >>In article <bkt0sq$1umd$1@digitaldaemon.com>, dickl says...
> >> >>>
> >> >>>I've added synchronized() to Thread.pauseAll. I have also found that
> >you can not
> >> >>>pause allThreads[0] or the app will freeze. I believe this is
because
> >> >>>allThreads[0] is the main thread and pausing it will pause all the
> >children.
> >> >>>
> >> >>>A better solution would be to put checks in the gc routines so that
any
> >memory
> >> >>>alloc/dealloc waits for collectfull() to finish. There is no reason
to
> >pause
> >> >>>threads which are not alloc/dealloc-ing memory. I don't understand
the
> >gc enough
> >> >>>yet to suggest a change.
> >> >>>
> >> >>>
> >> >>>
> >> >>>static void pauseAll()
> >> >>>{
> >> >>>if (nthreads > 1)
> >> >>>{
> >> >>>synchronized (threadLock)
> >> >>>{
> >> >>>Thread tthis = getThis();
> >> >>>for (int i = 1; i < allThreadsDim; i++)
> >> >>>{
> >> >>>Thread t;
> >> >>>t = allThreads[i];
> >> >>>if (t && t !== tthis && t.state == TS.RUNNING)
> >> >>>t.pause();
> >> >>>}
> >> >>>}
> >> >>>}
> >> >>>}
> >> >>>=============================
> >> >>>In article <bkrhoh$2vhg$1@digitaldaemon.com>, Walter says...
> >> >>>>
> >> >>>>I think you're on the right track here. If you want to make the
> >> >>>>modifications to thread.d, that'd be a great contribution to D!
> >> >>>>
> >> >>>><jhenzie@mac.com> wrote in message
> >news:bkrbr7$2mhf$1@digitaldaemon.com...
> >> >>>>> Walter
> >> >>>>>
> >> >>>>> Let me just say that I am new to D but I can not tell you how
much I
> >> >>>>appeciate
> >> >>>>> your efforts.  It has
> >> >>>>> given me a sense of excitement that I have not felt since
> >Objective-C.
> >> >>>>> [Objective-c retain] by the
> >> >>>>> way.
> >> >>>>>
> >> >>>>> In any case my concerns with the Thread class are as follows.
> >> >>>>>
> >> >>>>> 1.  It seems to me that resumeAll and pauseAll ought to be
> >considered
> >> >>>>atomic
> >> >>>>> operations.  In my
> >> >>>>> opinion, a call to pauseAll should leave the calling thread as
the
> >only
> >> >>>>running
> >> >>>>> thread in the
> >> >>>>> process and it should be serial. That is calls to pauseAll should
be
> >> >>>>serialized
> >> >>>>> such that if thread1
> >> >>>>> calls pauseAll before thread2, thread1 will be the controlling
> >thread when
> >> >>>>all
> >> >>>>> is said and done.  A
> >> >>>>> resumeAll will then allow thread2 to take control.
> >> >>>>>
> >> >>>>> To my mind this type of call should be extremely rare and only
used
> >by
> >> >>>>people
> >> >>>>> with GURU Status,
> >> >>>>> after all suspending other threads ignores the possibility that
they
> >hold
> >> >>>>a lock
> >> >>>>> on a specific
> >> >>>>> resource that the calling thread may soon need.  This is the
reason
> >> >>>>suspend and
> >> >>>>> resume have been
> >> >>>>> deprecated in Java.
> >> >>>>>
> >> >>>>> NB:  I realize this may be a somewhat naive view and am willing
to
> >be
> >> >>>>educated.
> >> >>>>> <smile/>
> >> >>>>>
> >> >>>>> The most basic approach would be (using resumeAll)
> >> >>>>>
> >> >>>>> static void resumeAll()
> >> >>>>> {
> >> >>>>> if (nthreads > 1)
> >> >>>>> {
> >> >>>>> Thread tthis = getThis();
> >> >>>>>
> >> >>>>> synchronized (threadlock) {
> >> >>>>> for (int i = 0; i < allThreadsDim; i++)
> >> >>>>> {   Thread t;
> >> >>>>>
> >> >>>>> t = allThreads[i];
> >> >>>>> if (t && t !== tthis && t.state == TS.RUNNING)
> >> >>>>> t.resume();
> >> >>>>> }
> >> >>>>> }
> >> >>>>> }
> >> >>>>> }
> >> >>>>>
> >> >>>>> of course this would require a recursive lock on Thread::resume
> >> >>>>Thread::pause.
> >> >>>>>
> >> >>>>> 2.  Given the sychronized(object) stnrax ir would seem that every
> >object
> >> >>>>can act
> >> >>>>> as mutex in D. If
> >> >>>>> this is the case perhaps the addition of wait, notify and
notifyall,
> >a la
> >> >>>>java,
> >> >>>>> on object would allow
> >> >>>>> for a more flexible and complete approach to multithreading.  Of
> >course
> >> >>>>this
> >> >>>>> would require the
> >> >>>>> Thread::wait method to be renamed Thread::join <smile/>.  No
flames
> >> >>>>please.
> >> >>>>>
> >> >>>>> I will consider it further and hopefully give as much feedback as
I
> >can
> >> >>>>and is
> >> >>>>> wanted.
> >> >>>>>
> >> >>>>> I will keep looking for the cause of the deadlock and let you
know
> >what I
> >> >>>>find.
> >> >>>>>
> >> >>>>> What are people using as a D Debugger, is there such a beast?
> >> >>>>>
> >> >>>>> Justin
> >> >>>>>
> >> >>>>>
> >> >>>>>
> >> >>>>> In article <bkr2gh$29sc$1@digitaldaemon.com>, Walter says...
> >> >>>>> >
> >> >>>>> >
> >> >>>>> ><jhenzie@mac.com> wrote in message
> >> >>>>news:bkqu8s$24dq$1@digitaldaemon.com...
> >> >>>>> >> It seems that the first call to the garbage collectors causes
the
> >> >>>>> >deadlock.  The
> >> >>>>> >> interesting thing is
> >> >>>>> >> that the Main thread is not deadlocked, amend your example and
> >add the
> >> >>>>> >following
> >> >>>>> >> after
> >> >>>>> >> Sleep(10000);
> >> >>>>> >>
> >> >>>>> >> Thread.resumeAll()
> >> >>>>> >>
> >> >>>>> >> The application exits as you would expect which tends to point
to
> >> >>>>another
> >> >>>>> >> problem. that  of
> >> >>>>> >> threads that are sleeping being ignored as part of a
> >> >>>>Thread.suspendAll()
> >> >>>>> >>
> >> >>>>> >> I am new to D so it may take me some time to figure this out
but
> >you
> >> >>>>are
> >> >>>>> >not
> >> >>>>> >> alone.
> >> >>>>> >>
> >> >>>>> >> Walter if you are reading, is there any reason why
sychronization
> >> >>>>> >semantics are
> >> >>>>> >> not present in
> >> >>>>> >> Thread.suspendAll.  I fairly sure you are handling the locking
> >lower
> >> >>>>down
> >> >>>>> >but it
> >> >>>>> >> would appear
> >> >>>>> >> logica;l to implement the lock on the Thread class.  But again
> >I'm new
> >> >>>>> >here so
> >> >>>>> >> feel free to disabuse
> >> >>>>> >> of that notion.
> >> >>>>> >>
> >> >>>>> >> More soon.
> >> >>>>> >>
> >> >>>>> >> Justin
> >> >>>>> >
> >> >>>>> >Can you suggest any fixes to thread.d?
> >> >>>>> >
> >> >>>>> >
> >> >>>>>
> >> >>>>>
> >> >>>>
> >> >>>>
> >> >>>
> >> >>>
> >> >>
> >> >>
> >> >
> >> >
> >>
> >>
> >
> >
>
>
September 26, 2003
Re: GC & multiple thread causes program hang FIXED
At least on windows.  I was so excited by the find that I have not refactored to
make it clean but here is the problem.

On windows GetCurrentThread returns -2  This is a pseudo handle that always
represents the current thread so which ever thread is calling SuspendThread with

allThreads[0].hdl is effectively pausing itself.

The following additions will resolve the problem, walter is there an official
submission form?

***windows.d***

extern (Windows)
{
export BOOL DuplicateHandle (HANDLE sourceProcess, HANDLE sourceThread,
HANDLE targetProcessHandle, HANDLE *targetHandle, DWORD access, 
BOOL inheritHandle, DWORD options);
}

***thread.d***

static this()
{
threadLock = new Object();

Thread t = new Thread();

t.state = TS.RUNNING;
t.id = GetCurrentThreadId();

thread_hdl currentThread = GetCurrentThread();

HANDLE process = (HANDLE)-1;        

DWORD access = (DWORD)0x00000002;

DuplicateHandle(process, currentThread, process, 
&(t.hdl), (DWORD)0, TRUE, access);

t.stackBottom = os_query_stackBottom();
synchronized (threadLock)
{
assert(!allThreads[0]);
allThreads[0] = t;
allThreadsDim = 1;
t.idx = 0;
}
}
September 26, 2003
Re: GC & multiple thread causes program hang FIXED
At least on windows.  I was so excited by the find that I have not refactored to
make it clean but here is the problem.

On windows GetCurrentThread returns -2  This is a pseudo handle that always
represents the current thread so which ever thread is calling SuspendThread with

allThreads[0].hdl is effectively pausing itself.

The following additions will resolve the problem, walter is there an official
submission form?

***windows.d***

extern (Windows)
{
export BOOL DuplicateHandle (HANDLE sourceProcess, HANDLE sourceThread,
HANDLE targetProcessHandle, HANDLE *targetHandle, DWORD access, 
BOOL inheritHandle, DWORD options);
}

***thread.d***

static this()
{
threadLock = new Object();

Thread t = new Thread();

t.state = TS.RUNNING;
t.id = GetCurrentThreadId();

thread_hdl currentThread = GetCurrentThread();

HANDLE process = (HANDLE)-1;        

DWORD access = (DWORD)0x00000002;

DuplicateHandle(process, currentThread, process, 
&(t.hdl), (DWORD)0, TRUE, access);

t.stackBottom = os_query_stackBottom();
synchronized (threadLock)
{
assert(!allThreads[0]);
allThreads[0] = t;
allThreadsDim = 1;
t.idx = 0;
}
}
October 02, 2003
Re: GC & multiple thread causes program hang
"dickl" <dickl_member@pathlink.com> wrote in message
news:bkt0sq$1umd$1@digitaldaemon.com...
> There is no reason to pause
> threads which are not alloc/dealloc-ing memory.

Yes, there is, because they may hold (on their stacks) a reference to
allocated memory.
Next ›   Last »
1 2
Top | Discussion index | About this forum | D home