March 23, 2012
On Mon, 19 Mar 2012 09:15:08 +0100, Johannes Pfau <nospam@example.com> wrote:

> And how do we get the TLS initialization data? If we placed it into an
> array, like DMD does on OSX, we could use dlsym for dlopened libraries,
> but what about initially loaded libraries?

That doesn't work because the symbols would collide.
If you made them local symbols OTOH you can't access
them through dlsym.
Use dl_iterate_phdr to get the initial image.
March 23, 2012
Am Fri, 23 Mar 2012 06:06:39 +0100
schrieb "Martin Nowak" <dawg@dawgfoto.de>:

> On Mon, 19 Mar 2012 09:15:08 +0100, Johannes Pfau <nospam@example.com> wrote:
> 
> > And how do we get the TLS initialization data? If we placed it into an array, like DMD does on OSX, we could use dlsym for dlopened libraries, but what about initially loaded libraries?
> 
> That doesn't work because the symbols would collide.
> If you made them local symbols OTOH you can't access
> them through dlsym.
> Use dl_iterate_phdr to get the initial image.

I just saw your latest work on DSO yesterday (I was looking for a
status update for shared libraries as Android does not officially
support native applications. The supported way is to build a shared
library and load it into a JAVA app. And indeed, native applications
have some wierd corner cases (https://github.com/jpf91/GDC/issues/4).
Android is such a crappy platform for native apps...)
You're doing some awesome work there. I'm not sure what issues gdc had
with the original TLS support, but I guess that your new code will
probably solve those.

I guess the OSX emulated tls code will also be adapted to support multiple modules? I guess we can just wait until your changes are merged into DMD and then think about emulated TLS again.
March 23, 2012
Am Fri, 23 Mar 2012 05:48:46 +0100
schrieb "Martin Nowak" <dawg@dawgfoto.de>:

> On Mon, 19 Mar 2012 16:57:29 +0100, Johannes Pfau <nospam@example.com> wrote:
> 
> > The only way to access all loaded library is dl_iterate_phdr. But I'm not sure if it provides all necessary information.
> 
> Yes it does.
> 
> The drawback is that it eagerly allocates the TLS block. https://github.com/dawgfoto/druntime/blob/SharedRuntime/src/rt/dso.d#L408 https://github.com/dawgfoto/druntime/blob/SharedRuntime/src/rt/dso.d#L459

As written in some comment in your code, we can avoid eager allocation
using some architecture dependent 'hacks'. I think we'd have to
* get the thread pointer (architecture specific)
* find the dtv (there's only variant 1 / variant 2?)
* access the correct dtv entry (C library dependent?)
* check if the dtv entry is initialized (C library dependent?)

For FreeBSD step 3/4 means checking if dtv[index + 1] == 0. It's probably the same for most other C libraries. The tricky part is that we have to check first if the dtv is big enough for the current index. For FreeBSD, this is easy again, dtv[1] contains the size of the dtv. But that's probably nonstandard.

All this is not hard to do, but quite system specific. Normal Desktop OS probably don't need this optimization. For systems which benifit from it, adding it shouldn't be too difficult.

In case you're interested, the FreeBSD linker source is here (BSD
licensed, of course):
http://www.freebsd.org/cgi/cvsweb.cgi/src/libexec/rtld-elf/rtld.c?rev=1.196;content-type=text%2Fplain
search for tls_get_addr_slow, __tls_get_addr, dtv, tls
March 23, 2012
> I guess the OSX emulated tls code will also be adapted to support
> multiple modules? I guess we can just wait until your changes are
> merged into DMD and then think about emulated TLS again.

We're already merging since 3 month or so.
March 23, 2012
Just another point about TLS.

extern(C) /*__thread*/ int foo;

At some point you want to be able to access C++ TLS variables
so emulation should not replace native TLS support.
March 23, 2012
On Fri, 23 Mar 2012 11:02:44 +0100, Johannes Pfau <nospam@example.com> wrote:

> For FreeBSD, this is easy again, dtv[1] contains the size of the dtv.
> But that's probably nonstandard.

Yeah, seems to be non-standard.
There might also be issues with outdated dtv's.
March 23, 2012
Am Fri, 23 Mar 2012 13:05:55 +0100
schrieb "Martin Nowak" <dawg@dawgfoto.de>:

> On Fri, 23 Mar 2012 11:02:44 +0100, Johannes Pfau <nospam@example.com> wrote:
> 
> > For FreeBSD, this is easy again, dtv[1] contains the size of the dtv. But that's probably nonstandard.
> 
> Yeah, seems to be non-standard.

> There might also be issues with outdated dtv's.

Which means we'd have to check the generation counter. And if the counter mismatches, we'd need the C runtime to update it. I'm not sure if we can access tls_dtv_generation on FreeBSD, but updating the counter is easy: every call to __tls_get_addr works, so we could just use module id 1 offset 0. AFAIK the TLS memory for the application module is always allocated anyway.

I'll probably try this at some point, but I have to set up a FreeBSD VM first.
March 25, 2012
On 2012-03-23 06:03, Martin Nowak wrote:
> On Mon, 19 Mar 2012 10:40:25 +0100, Jacob Carlborg <doob@me.com> wrote:
>
>> As I understand it, in the native ELF implementation, assembly is used
>> to access the current module id, this is for FreeBSD:
>> http://people.freebsd.org/~marcel/tls.html
>> This is how ___tls_get_addr is implemented on FreeBSD ELF i386:
>> https://bitbucket.org/freebsd/freebsd-head/src/4e8f50fe2f05/libexec/rtld-elf/i386/reloc.c#cl-355
>>
>
> Not quite.
> Access to the static image is done through %fs relative addressing
> which is super-fast and requires no runtime linking.
> The general dynamic addressing needs one tls_index
> struct in the GOT for every variable and a call to
> _tls_get_addr(tls_index*).
> The module index and the offset are filled by the runtime linker.

Ok, I see.

-- 
/Jacob Carlborg
March 25, 2012
On 2012-03-23 12:55, Martin Nowak wrote:
>
> Just another point about TLS.
>
> extern(C) /*__thread*/ int foo;
>
> At some point you want to be able to access C++ TLS variables
> so emulation should not replace native TLS support.

So C++ TLS is not using the same implementation as the C extension __thread?

-- 
/Jacob Carlborg
March 25, 2012
On Sun, 25 Mar 2012 16:29:25 +0200, Jacob Carlborg <doob@me.com> wrote:

> On 2012-03-23 12:55, Martin Nowak wrote:
>>
>> Just another point about TLS.
>>
>> extern(C) /*__thread*/ int foo;
>>
>> At some point you want to be able to access C++ TLS variables
>> so emulation should not replace native TLS support.
>
> So C++ TLS is not using the same implementation as the C extension __thread?
>

Sorry,
that might have been misleading.
The point I was trying to make is that D's TLS support shouldn't
deviate from the native platform TLS if one is available. I've just
tried it out, and indeed I can access C TLS variables from D.