I'm having issues where it appears that the D2 druntime GC ignores roots located in thread-local storage when determining what objects are alive.  This leads to spurious deletion of objects in certain cases.  This is clearly a GC issue, since disabling the GC solves it, but I can't figure out how to reproduce it in a canonical way.  Has anyone else run into anything like this?

dsimcha wrote:
> I'm having issues where it appears that the D2 druntime GC ignores roots
> located in thread-local storage when determining what objects are alive.  This
> leads to spurious deletion of objects in certain cases.  This is clearly a GC
> issue, since disabling the GC solves it, but I can't figure out how to
> reproduce it in a canonical way.  Has anyone else run into anything like this?

Weird.  The actual storage for TLS in druntime is an array of void* within the Thread class.  I can't imagine that it wouldn't be scanned by the GC.  Do you have a reproducible test case?


Sean

there is no simple way to scan TSL from another thread, TSL was designed to be thread-local after all :)

One thing I forgot to mention in the orig. post:  This happens in single-threaded apps.  I discovered this when writing a library struct, and haven't even tried to actually use multithreading yet.

== Quote from Sean Kelly (sean@invisibleduck.org)'s article
> Weird.  The actual storage for TLS in druntime is an array of void*
> within the Thread class.  I can't imagine that it wouldn't be scanned by
> the GC.  Do you have a reproducible test case?
> Sean

I just now managed to play around with this some more and come up with a small test case, as opposed to a much larger real-world case, that reproduces this.  I still haven't the slightest clue *why* my latest test case reproduces the bug and some others that I had tried didn't, but I've filed a bug report.  See:

http://d.puremagic.com/issues/show_bug.cgi?id=2491

"dsimcha" wrote
> == Quote from Sean Kelly (sean@invisibleduck.org)'s article
>> Weird.  The actual storage for TLS in druntime is an array of void*
>> within the Thread class.  I can't imagine that it wouldn't be scanned by
>> the GC.  Do you have a reproducible test case?
>> Sean
>
> I just now managed to play around with this some more and come up with a
> small
> test case, as opposed to a much larger real-world case, that reproduces
> this.  I
> still haven't the slightest clue *why* my latest test case reproduces the
> bug and
> some others that I had tried didn't, but I've filed a bug report.  See:
>
> http://d.puremagic.com/issues/show_bug.cgi?id=2491

I don't know if that __thread keyword is fleshed out yet.  There is no documentation on it in the spec.  The only place it is referenced is in the changelog, and there it says: "This is for testing purposes only to check out the machinery in the back end."

I'd say most likely that the GC doesn't see anything declared as __thread, so when you use that pointer as the only reference to GC allocated data, it doesn't see that it's still in use, and will collect.

-Steve

== Quote from Steven Schveighoffer (schveiguy@yahoo.com)'s article
> "dsimcha" wrote
> > == Quote from Sean Kelly (sean@invisibleduck.org)'s article
> >> Weird.  The actual storage for TLS in druntime is an array of void*
> >> within the Thread class.  I can't imagine that it wouldn't be scanned by
> >> the GC.  Do you have a reproducible test case?
> >> Sean
> >
> > I just now managed to play around with this some more and come up with a
> > small
> > test case, as opposed to a much larger real-world case, that reproduces
> > this.  I
> > still haven't the slightest clue *why* my latest test case reproduces the
> > bug and
> > some others that I had tried didn't, but I've filed a bug report.  See:
> >
> > http://d.puremagic.com/issues/show_bug.cgi?id=2491
> I don't know if that __thread keyword is fleshed out yet.  There is no
> documentation on it in the spec.  The only place it is referenced is in the
> changelog, and there it says: "This is for testing purposes only to check
> out the machinery in the back end."
> I'd say most likely that the GC doesn't see anything declared as __thread,
> so when you use that pointer as the only reference to GC allocated data, it
> doesn't see that it's still in use, and will collect.
> -Steve

Ok, well now that I'm aware of it, I'll just use the Tango/druntime TLS implementation to do what I want to do.

Steven Schveighoffer wrote:
> I'd say most likely that the GC doesn't see anything declared as __thread, so when you use that pointer as the only reference to GC allocated data, it doesn't see that it's still in use, and will collect.

Looks like I need to do some research to see how the gc can discover the extent of tls data.

== Quote from dsimcha (dsimcha@yahoo.com)'s article
> == Quote from Sean Kelly (sean@invisibleduck.org)'s article
> > Weird.  The actual storage for TLS in druntime is an array of void*
> > within the Thread class.  I can't imagine that it wouldn't be scanned by
> > the GC.  Do you have a reproducible test case?
> > Sean
> I just now managed to play around with this some more and come up with a small test case, as opposed to a much larger real-world case, that reproduces this.  I still haven't the slightest clue *why* my latest test case reproduces the bug and some others that I had tried didn't, but I've filed a bug report.  See: http://d.puremagic.com/issues/show_bug.cgi?id=2491

Oh!  You're using the built-in thread-local storage.  I don't think that's fully
implemented yet (Walter, please correct me if I'm wrong).  You might want to
use the thread-local storage feature in the Thread class for now.  Depending
on your memory / performance requirements:

import core.thread;

void main()
{
    auto t = new ThreadLocal!(int);
    t.val = 5;
    writefln( t.val );
}

-or-

import core.thread;

void main()
{
    auto key = Thread.createLocal();
    Thread.setLocal( key, cast(void*) 5 );
    writefln( cast(int) Thread.getLocal( key ) );
}

the second approach is closer to how C/C++ TLS works and saves
the allocation of a wrapper struct, but is clearly more complicated
in exchange.  your best bet is probably to simply use ThreadLocal
for now, since it will be easier to change later when built-in TLS
works properly.


Sean

== Quote from Sean Kelly (sean@invisibleduck.org)'s article
> Oh!  You're using the built-in thread-local storage.  I don't think that's fully
> implemented yet (Walter, please correct me if I'm wrong).  You might want to
> use the thread-local storage feature in the Thread class for now.  Depending
> on your memory / performance requirements:
> import core.thread;
> void main()
> {
>     auto t = new ThreadLocal!(int);
>     t.val = 5;
>     writefln( t.val );
> }
> -or-
> import core.thread;
> void main()
> {
>     auto key = Thread.createLocal();
>     Thread.setLocal( key, cast(void*) 5 );
>     writefln( cast(int) Thread.getLocal( key ) );
> }
> the second approach is closer to how C/C++ TLS works and saves
> the allocation of a wrapper struct, but is clearly more complicated
> in exchange.  your best bet is probably to simply use ThreadLocal
> for now, since it will be easier to change later when built-in TLS
> works properly.
> Sean

Thanks, though I'm way ahead of you in that I already did this.  Works great, except it's a little bit slow.

I'm actually working on an implementation of the SuperStack proposed by Andrei about a month ago, which was why I needed good TLS.  It seems like with the current implementation (using the faster explicit key solution instead of the slower class-based solution), about 1/3 of my time is being spent on retrieving TLS.  I got this number by caching the stuff from TLS on the stack of the calling function and passing it in as a parameter.  This may become a semi-hidden feature for wringing out that last bit of performance from SuperStack.  Is TLS inherently slow, or is the druntime implementation relatively quick and dirty and likely to improve in the future?