View mode: basic / threaded / horizontal-split · Log in · Help
December 05, 2008
Hard-to-reproduce GC bug
I'm having issues where it appears that the D2 druntime GC ignores roots
located in thread-local storage when determining what objects are alive.  This
leads to spurious deletion of objects in certain cases.  This is clearly a GC
issue, since disabling the GC solves it, but I can't figure out how to
reproduce it in a canonical way.  Has anyone else run into anything like this?
December 05, 2008
Re: Hard-to-reproduce GC bug
dsimcha wrote:
> I'm having issues where it appears that the D2 druntime GC ignores roots
> located in thread-local storage when determining what objects are alive.  This
> leads to spurious deletion of objects in certain cases.  This is clearly a GC
> issue, since disabling the GC solves it, but I can't figure out how to
> reproduce it in a canonical way.  Has anyone else run into anything like this?

Weird.  The actual storage for TLS in druntime is an array of void* 
within the Thread class.  I can't imagine that it wouldn't be scanned by 
the GC.  Do you have a reproducible test case?


Sean
December 05, 2008
Re: Hard-to-reproduce GC bug
there is no simple way to scan TSL from another thread, TSL was designed to be thread-local after all :)
December 05, 2008
Re: Hard-to-reproduce GC bug
One thing I forgot to mention in the orig. post:  This happens in single-threaded
apps.  I discovered this when writing a library struct, and haven't even tried to
actually use multithreading yet.
December 05, 2008
Re: Hard-to-reproduce GC bug
== Quote from Sean Kelly (sean@invisibleduck.org)'s article
> Weird.  The actual storage for TLS in druntime is an array of void*
> within the Thread class.  I can't imagine that it wouldn't be scanned by
> the GC.  Do you have a reproducible test case?
> Sean

I just now managed to play around with this some more and come up with a small
test case, as opposed to a much larger real-world case, that reproduces this.  I
still haven't the slightest clue *why* my latest test case reproduces the bug and
some others that I had tried didn't, but I've filed a bug report.  See:

http://d.puremagic.com/issues/show_bug.cgi?id=2491
December 05, 2008
Re: Hard-to-reproduce GC bug
"dsimcha" wrote
> == Quote from Sean Kelly (sean@invisibleduck.org)'s article
>> Weird.  The actual storage for TLS in druntime is an array of void*
>> within the Thread class.  I can't imagine that it wouldn't be scanned by
>> the GC.  Do you have a reproducible test case?
>> Sean
>
> I just now managed to play around with this some more and come up with a 
> small
> test case, as opposed to a much larger real-world case, that reproduces 
> this.  I
> still haven't the slightest clue *why* my latest test case reproduces the 
> bug and
> some others that I had tried didn't, but I've filed a bug report.  See:
>
> http://d.puremagic.com/issues/show_bug.cgi?id=2491

I don't know if that __thread keyword is fleshed out yet.  There is no 
documentation on it in the spec.  The only place it is referenced is in the 
changelog, and there it says: "This is for testing purposes only to check 
out the machinery in the back end."

I'd say most likely that the GC doesn't see anything declared as __thread, 
so when you use that pointer as the only reference to GC allocated data, it 
doesn't see that it's still in use, and will collect.

-Steve
December 05, 2008
Re: Hard-to-reproduce GC bug
== Quote from Steven Schveighoffer (schveiguy@yahoo.com)'s article
> "dsimcha" wrote
> > == Quote from Sean Kelly (sean@invisibleduck.org)'s article
> >> Weird.  The actual storage for TLS in druntime is an array of void*
> >> within the Thread class.  I can't imagine that it wouldn't be scanned by
> >> the GC.  Do you have a reproducible test case?
> >> Sean
> >
> > I just now managed to play around with this some more and come up with a
> > small
> > test case, as opposed to a much larger real-world case, that reproduces
> > this.  I
> > still haven't the slightest clue *why* my latest test case reproduces the
> > bug and
> > some others that I had tried didn't, but I've filed a bug report.  See:
> >
> > http://d.puremagic.com/issues/show_bug.cgi?id=2491
> I don't know if that __thread keyword is fleshed out yet.  There is no
> documentation on it in the spec.  The only place it is referenced is in the
> changelog, and there it says: "This is for testing purposes only to check
> out the machinery in the back end."
> I'd say most likely that the GC doesn't see anything declared as __thread,
> so when you use that pointer as the only reference to GC allocated data, it
> doesn't see that it's still in use, and will collect.
> -Steve

Ok, well now that I'm aware of it, I'll just use the Tango/druntime TLS
implementation to do what I want to do.
December 05, 2008
Re: Hard-to-reproduce GC bug
Steven Schveighoffer wrote:
> I'd say most likely that the GC doesn't see anything declared as __thread, 
> so when you use that pointer as the only reference to GC allocated data, it 
> doesn't see that it's still in use, and will collect.

Looks like I need to do some research to see how the gc can discover the 
extent of tls data.
December 05, 2008
Re: Hard-to-reproduce GC bug
== Quote from dsimcha (dsimcha@yahoo.com)'s article
> == Quote from Sean Kelly (sean@invisibleduck.org)'s article
> > Weird.  The actual storage for TLS in druntime is an array of void*
> > within the Thread class.  I can't imagine that it wouldn't be scanned by
> > the GC.  Do you have a reproducible test case?
> > Sean
> I just now managed to play around with this some more and come up with a small
> test case, as opposed to a much larger real-world case, that reproduces this.  I
> still haven't the slightest clue *why* my latest test case reproduces the bug and
> some others that I had tried didn't, but I've filed a bug report.  See:
> http://d.puremagic.com/issues/show_bug.cgi?id=2491

Oh!  You're using the built-in thread-local storage.  I don't think that's fully
implemented yet (Walter, please correct me if I'm wrong).  You might want to
use the thread-local storage feature in the Thread class for now.  Depending
on your memory / performance requirements:

import core.thread;

void main()
{
   auto t = new ThreadLocal!(int);
   t.val = 5;
   writefln( t.val );
}

-or-

import core.thread;

void main()
{
   auto key = Thread.createLocal();
   Thread.setLocal( key, cast(void*) 5 );
   writefln( cast(int) Thread.getLocal( key ) );
}

the second approach is closer to how C/C++ TLS works and saves
the allocation of a wrapper struct, but is clearly more complicated
in exchange.  your best bet is probably to simply use ThreadLocal
for now, since it will be easier to change later when built-in TLS
works properly.


Sean
December 05, 2008
Re: Hard-to-reproduce GC bug
== Quote from Sean Kelly (sean@invisibleduck.org)'s article
> Oh!  You're using the built-in thread-local storage.  I don't think that's fully
> implemented yet (Walter, please correct me if I'm wrong).  You might want to
> use the thread-local storage feature in the Thread class for now.  Depending
> on your memory / performance requirements:
> import core.thread;
> void main()
> {
>     auto t = new ThreadLocal!(int);
>     t.val = 5;
>     writefln( t.val );
> }
> -or-
> import core.thread;
> void main()
> {
>     auto key = Thread.createLocal();
>     Thread.setLocal( key, cast(void*) 5 );
>     writefln( cast(int) Thread.getLocal( key ) );
> }
> the second approach is closer to how C/C++ TLS works and saves
> the allocation of a wrapper struct, but is clearly more complicated
> in exchange.  your best bet is probably to simply use ThreadLocal
> for now, since it will be easier to change later when built-in TLS
> works properly.
> Sean

Thanks, though I'm way ahead of you in that I already did this.  Works great,
except it's a little bit slow.

I'm actually working on an implementation of the SuperStack proposed by Andrei
about a month ago, which was why I needed good TLS.  It seems like with the
current implementation (using the faster explicit key solution instead of the
slower class-based solution), about 1/3 of my time is being spent on retrieving
TLS.  I got this number by caching the stuff from TLS on the stack of the calling
function and passing it in as a parameter.  This may become a semi-hidden feature
for wringing out that last bit of performance from SuperStack.  Is TLS inherently
slow, or is the druntime implementation relatively quick and dirty and likely to
improve in the future?
« First   ‹ Prev
1 2 3
Top | Discussion index | About this forum | D home