On 2 dec 2011, at 20:45, Sean Kelly wrote:
> I don't know if it matters, but __thread should be natively supported in Lion (by llvm-gcc, not DMD).  OSX is doing away with GCC so it wouldn't surprise me if no effort had been made to add __thread support to that.


I downloaded the source code for dyld, the version that ships with 10.7.2. There seems to be support for TLS, they call it TLV (Thread Local Variable). But it seems impossible to find any documentation about this.

The function used to get a variable  is called "tlv_get_addr".

-- 
/Jacob Carlborg

On 2 dec 2011, at 21:54, Sean Kelly wrote:

> On Dec 2, 2011, at 12:51 PM, Brad Roberts wrote:
> 
>> On Fri, 2 Dec 2011, Walter Bright wrote:
>> 
>>> On 12/2/2011 12:19 PM, Alex wrote:
>>>> Just FYI, LLVM 3.0 has officially deprecated llvm-gcc in favor of Clang and DragonEgg.
>>> 
>>> I expect that soon we'll switch to llvm for dmd on the Mac. Just not this upcoming release.
>>> 
>>> It's good to use multiple compilers to compile dmd, such tends to flush out latent source code bugs.
>> 
>> I don't remember what the problems were, but there were definitly problems with using llvm on lion when Sean first upgraded the box the tester runs on so we switched it back to gcc.  At some point, someone will have to take the plunge and figure out what the issue(s) was(were).
> 
> I ram into one or two segfaults with the generated dmd built by clang.  If I remember correctly, one was an alignment issue calling a struct's default actor (the one generated by the compiler, not the user).  This is why I suggested moving to clang as a desired future step rather than something to be done immediately.  I don't think it's worth it if we have to work around compiler bugs.


Clang is supposed to be a drop-in replacement for GCC. But of course they may not succeed in being compatible with GCC in every single way.

-- 
/Jacob Carlborg

On 2 dec 2011, at 21:27, Walter Bright wrote:

> 
> 
> On 12/1/2011 11:48 AM, Jacob Carlborg wrote:
>> On 30 nov 2011, at 22:46, Walter Bright wrote:
>> 
>>> I'm waiting for gcc on the Mac to support TLS, which it must do soon. When it does, I'll change dmd to match it.
>> 
>> That was kind of a disappointing answer.
>> 
> 
> What that means is there's a ton of work to do on dmd, and implementing TLS three times is a waste of effort. dmd will eventually have to match what the default C compiler does on the Mac.

I can understand that. I was naively thinking that we could implement it the same way it's implemented for ELF and assuming Apple would implemented it like that as well. Then we would only need to change the compiler now and not later. I was wrong about that, see below.

> I'm also not terribly concerned if TLS access is slow, only that it is correct. This is because a program that depends on fast TLS access is badly designed, for several reasons.

Fair enough. I was thinking it would be easier to implement support for dynamic libraries if TLS was implemented "properly". If that also would mean that accessing thread local variables would be faster, than all the better.

> And lastly, if you wish to work on the OSX TLS, I do appreciate the efforts. I think it can only help. Please look at how clang is doing it, if it is doing it.


Sean mentioned that TLS is supported natively in Mac OS X Lion. I've downloaded the source code for dyld, the release that ships with Lion and it seems to be correct. There's a function called "tlv_get_addr" that seems to be doing the same as "__tls_get_addr" does on Linux. It looks a bit different from the ELF implementation, different function names and similar, but it seems to be doing the same thing.

Now to the question: how should we implement this? Do we have two implementations, a new implementation targeting 10.7 and the existing one for all previous versions of Mac OS X? Or do we want to just have one implementation, somehow.

-- 
/Jacob Carlborg

On 12/3/2011 2:37 PM, Jacob Carlborg wrote:
>
> Now to the question: how should we implement this? Do we have two implementations, a new implementation targeting 10.7 and the existing one for all previous versions of Mac OS X? Or do we want to just have one implementation, somehow.
>

It either has to have a runtime switch if the 10.7 scheme is not backwards compatible, or we have to stick with the old scheme until the older OSX's disappear.

On 4 dec 2011, at 01:24, Walter Bright wrote:

> 
> 
> On 12/3/2011 2:37 PM, Jacob Carlborg wrote:
>> 
>> Now to the question: how should we implement this? Do we have two implementations, a new implementation targeting 10.7 and the existing one for all previous versions of Mac OS X? Or do we want to just have one implementation, somehow.
>> 
> 
> It either has to have a runtime switch if the 10.7 scheme is not backwards compatible, or we have to stick with the old scheme until the older OSX's disappear.


I'm pretty sure it's not backwards compatible. The tlv_get_addr exists in dyld for Lion but not for Snow Leopard. I would hate to see that we need to wait with this until Snow Leopard disappears. I see three different alternatives:

1. Runtime switch (as you suggests above)

2. The user need to replace dyld with the version from Lion, I have no idea if that will work. That's probably not a good idea to do anyway

3. We implement the runtime part ourself. Now that it is implemented in Lion we can make it forwards compatible (as long as Apple doesn't change the implementation in later versions of OSX)

I think number three is the best alternative, but also the most difficult to implement.

-- 
/Jacob Carlborg

On Dec 3, 2011, at 2:37 PM, Jacob Carlborg wrote:
> 
> Sean mentioned that TLS is supported natively in Mac OS X Lion. I've downloaded the source code for dyld, the release that ships with Lion and it seems to be correct. There's a function called "tlv_get_addr" that seems to be doing the same as "__tls_get_addr" does on Linux. It looks a bit different from the ELF implementation, different function names and similar, but it seems to be doing the same thing.

The implementation is different from ELF.  I don't know how this was implemented at a code level but I do know a bit about where stuff lives and how to get at it.  The bigger issue to me is whether we compile different versions of TLS for 10.7 and 10.6 or if we want to stick with what we've got until 10.8 is released.

On Dec 4, 2011, at 3:09 AM, Jacob Carlborg wrote:
> 
> On 4 dec 2011, at 01:24, Walter Bright wrote:
>> 
>> 
>> It either has to have a runtime switch if the 10.7 scheme is not backwards compatible, or we have to stick with the old scheme until the older OSX's disappear.
> 
> 
> I'm pretty sure it's not backwards compatible. The tlv_get_addr exists in dyld for Lion but not for Snow Leopard. I would hate to see that we need to wait with this until Snow Leopard disappears. I see three different alternatives:
> 
> 1. Runtime switch (as you suggests above)
> 
> 2. The user need to replace dyld with the version from Lion, I have no idea if that will work. That's probably not a good idea to do anyway

I don't think it will.  The object file format changed to support this as well.


> 3. We implement the runtime part ourself. Now that it is implemented in Lion we can make it forwards compatible (as long as Apple doesn't change the implementation in later versions of OSX)

Sure, but this still won't work in 10.6 because of the object file and dyld issue.  Or am I misunderstanding?

On 12/7/2011 2:49 PM, Sean Kelly wrote:
> On Dec 3, 2011, at 2:37 PM, Jacob Carlborg wrote:
>> Sean mentioned that TLS is supported natively in Mac OS X Lion. I've downloaded the source code for dyld, the release that ships with Lion and it seems to be correct. There's a function called "tlv_get_addr" that seems to be doing the same as "__tls_get_addr" does on Linux. It looks a bit different from the ELF implementation, different function names and similar, but it seems to be doing the same thing.
> The implementation is different from ELF.  I don't know how this was implemented at a code level but I do know a bit about where stuff lives and how to get at it.  The bigger issue to me is whether we compile different versions of TLS for 10.7 and 10.6 or if we want to stick with what we've got until 10.8 is released.
>

Sounds like we should stick with what we have now for now.

On 7 dec 2011, at 23:49, Sean Kelly wrote:

> On Dec 3, 2011, at 2:37 PM, Jacob Carlborg wrote:
>> 
>> Sean mentioned that TLS is supported natively in Mac OS X Lion. I've downloaded the source code for dyld, the release that ships with Lion and it seems to be correct. There's a function called "tlv_get_addr" that seems to be doing the same as "__tls_get_addr" does on Linux. It looks a bit different from the ELF implementation, different function names and similar, but it seems to be doing the same thing.
> 
> The implementation is different from ELF.  I don't know how this was implemented at a code level but I do know a bit about where stuff lives and how to get at it.  The bigger issue to me is whether we compile different versions of TLS for 10.7 and 10.6 or if we want to stick with what we've got until 10.8 is released.


I guess that means to drop the support for 10.6 when 10.6 is released. It starts to get a bit annoying to only be able to support one version back.

-- 
/Jacob Carlborg

On 7 dec 2011, at 23:51, Sean Kelly wrote:

> On Dec 4, 2011, at 3:09 AM, Jacob Carlborg wrote:
>> 
>> On 4 dec 2011, at 01:24, Walter Bright wrote:
>>> 
>>> 
>>> It either has to have a runtime switch if the 10.7 scheme is not backwards compatible, or we have to stick with the old scheme until the older OSX's disappear.
>> 
>> 
>> I'm pretty sure it's not backwards compatible. The tlv_get_addr exists in dyld for Lion but not for Snow Leopard. I would hate to see that we need to wait with this until Snow Leopard disappears. I see three different alternatives:
>> 
>> 1. Runtime switch (as you suggests above)
>> 
>> 2. The user need to replace dyld with the version from Lion, I have no idea if that will work. That's probably not a good idea to do anyway
> 
> I don't think it will.  The object file format changed to support this as well.
> 
> 
>> 3. We implement the runtime part ourself. Now that it is implemented in Lion we can make it forwards compatible (as long as Apple doesn't change the implementation in later versions of OSX)
> 
> Sure, but this still won't work in 10.6 because of the object file and dyld issue.  Or am I misunderstanding?


I was thinking that we implement what dylb does but in druntime. That assumes it's possible to do everything dyld does in druntime. In druntime we would need to check if the application is running on 10.7 or later. If it is, druntime won't have to do anything since dyld have already handled everything.

The whole point is to change the compiler to generate code that is compatible with 10.7 and implement all the missing pieces ourself for 10.6.

I don't think there's any problem with the object file. W'e're already using non-standard segments/sections as it is now. The runtime implementation just need to know how to find the segments, since we implement it ourself it won't be any problem.

-- 
/Jacob Carlborg