Bug 4854. "Regression(2.047, Mac only) writefln Segmentation fault if no globals"  Confirmed by 3 people.

Thread local storage isn't getting set up correctly. Perhaps this
happens when TLS is used in a library, but not in user code.
This behaviour was introduced in 2.047, though it may have been a
latent bug which was only triggered in that release.
This may be a druntime bug, though it is more likely to be a compiler bug.

Although this is OSX-specific, this bug could still be a PR disaster. Perhaps an OSX user can track down which svn commit triggered the regression.

I can't reproduce using 2.050 and OSX 10.6.  Is anyone still on 10.5 that can try this with 2.050?

On Nov 8, 2010, at 3:46 AM, Don Clugston wrote:

> Bug 4854. "Regression(2.047, Mac only) writefln Segmentation fault if no globals"  Confirmed by 3 people.
> 
> Thread local storage isn't getting set up correctly. Perhaps this
> happens when TLS is used in a library, but not in user code.
> This behaviour was introduced in 2.047, though it may have been a
> latent bug which was only triggered in that release.
> This may be a druntime bug, though it is more likely to be a compiler bug.
> 
> Although this is OSX-specific, this bug could still be a PR disaster.
> Perhaps an OSX user can track down which svn commit triggered the regression.
> _______________________________________________
> phobos mailing list
> phobos at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/phobos

I can reproduce the error using 2.050 on Mac OS X 10.5.8.

On 8 nov 2010, at 20:02, Sean Kelly wrote:

> I can't reproduce using 2.050 and OSX 10.6.  Is anyone still on 10.5 that can try this with 2.050?
> 
> On Nov 8, 2010, at 3:46 AM, Don Clugston wrote:
> 
>> Bug 4854. "Regression(2.047, Mac only) writefln Segmentation fault if no globals"  Confirmed by 3 people.
>> 
>> Thread local storage isn't getting set up correctly. Perhaps this
>> happens when TLS is used in a library, but not in user code.
>> This behaviour was introduced in 2.047, though it may have been a
>> latent bug which was only triggered in that release.
>> This may be a druntime bug, though it is more likely to be a compiler bug.
>> 
>> Although this is OSX-specific, this bug could still be a PR disaster.
>> Perhaps an OSX user can track down which svn commit triggered the regression.
>> _______________________________________________
>> phobos mailing list
>> phobos at puremagic.com
>> http://lists.puremagic.com/mailman/listinfo/phobos
> 
> _______________________________________________
> phobos mailing list
> phobos at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/phobos

-- 
/Jacob Carlborg

The library code isn't different between OS versions so my guess is that this is somehow related to object file generation within the compiler.  backend/machobj.c didn't change between May 10th and Jun 11th however, so I'm not sure what to suggest.  If it helps, here are the druntime and DMD timelines for DMD 2.047:

http://dsource.org/projects/druntime/timeline?from=6%2F11%2F10&daysback=32&wiki=on&changeset=on&milestone=on&ticket=on&update=Update
http://dsource.org/projects/dmd/timeline?from=6%2F11%2F10&daysback=32&wiki=on&changeset=on&milestone=on&ticket=on&update=Update

I don't see anything in the druntime changelog that should affect TLS.  The DMD changelog will take longer to go through though.

On Nov 8, 2010, at 12:33 PM, Jacob Carlborg wrote:

> I can reproduce the error using 2.050 on Mac OS X 10.5.8.
> 
> On 8 nov 2010, at 20:02, Sean Kelly wrote:
> 
>> I can't reproduce using 2.050 and OSX 10.6.  Is anyone still on 10.5 that can try this with 2.050?
>> 
>> On Nov 8, 2010, at 3:46 AM, Don Clugston wrote:
>> 
>>> Bug 4854. "Regression(2.047, Mac only) writefln Segmentation fault if no globals"  Confirmed by 3 people.
>>> 
>>> Thread local storage isn't getting set up correctly. Perhaps this
>>> happens when TLS is used in a library, but not in user code.
>>> This behaviour was introduced in 2.047, though it may have been a
>>> latent bug which was only triggered in that release.
>>> This may be a druntime bug, though it is more likely to be a compiler bug.
>>> 
>>> Although this is OSX-specific, this bug could still be a PR disaster.
>>> Perhaps an OSX user can track down which svn commit triggered the regression.
>>> _______________________________________________
>>> phobos mailing list
>>> phobos at puremagic.com
>>> http://lists.puremagic.com/mailman/listinfo/phobos
>> 
>> _______________________________________________
>> phobos mailing list
>> phobos at puremagic.com
>> http://lists.puremagic.com/mailman/listinfo/phobos
> 
> -- 
> /Jacob Carlborg
> 
> _______________________________________________
> phobos mailing list
> phobos at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/phobos

See my reply: http://d.puremagic.com/issues/show_bug.cgi?id=4854

Sean Kelly wrote:
> The library code isn't different between OS versions so my guess is that this is somehow related to object file generation within the compiler.  backend/machobj.c didn't change between May 10th and Jun 11th however, so I'm not sure what to suggest.  If it helps, here are the druntime and DMD timelines for DMD 2.047:
>
> http://dsource.org/projects/druntime/timeline?from=6%2F11%2F10&daysback=32&wiki=on&changeset=on&milestone=on&ticket=on&update=Update
> http://dsource.org/projects/dmd/timeline?from=6%2F11%2F10&daysback=32&wiki=on&changeset=on&milestone=on&ticket=on&update=Update
>
> I don't see anything in the druntime changelog that should affect TLS.  The DMD changelog will take longer to go through though.
>
> On Nov 8, 2010, at 12:33 PM, Jacob Carlborg wrote:
>
> 
>> I can reproduce the error using 2.050 on Mac OS X 10.5.8.
>>
>> On 8 nov 2010, at 20:02, Sean Kelly wrote:
>>
>> 
>>> I can't reproduce using 2.050 and OSX 10.6.  Is anyone still on 10.5 that can try this with 2.050?
>>>
>>> On Nov 8, 2010, at 3:46 AM, Don Clugston wrote:
>>>
>>> 
>>>> Bug 4854. "Regression(2.047, Mac only) writefln Segmentation fault if no globals"  Confirmed by 3 people.
>>>>
>>>> Thread local storage isn't getting set up correctly. Perhaps this
>>>> happens when TLS is used in a library, but not in user code.
>>>> This behaviour was introduced in 2.047, though it may have been a
>>>> latent bug which was only triggered in that release.
>>>> This may be a druntime bug, though it is more likely to be a compiler bug.
>>>>
>>>> Although this is OSX-specific, this bug could still be a PR disaster.
>>>> Perhaps an OSX user can track down which svn commit triggered the regression.
>>>> _______________________________________________
>>>> phobos mailing list
>>>> phobos at puremagic.com
>>>> http://lists.puremagic.com/mailman/listinfo/phobos
>>>> 
>>> _______________________________________________
>>> phobos mailing list
>>> phobos at puremagic.com
>>> http://lists.puremagic.com/mailman/listinfo/phobos
>>> 
>> -- 
>> /Jacob Carlborg
>>
>> _______________________________________________
>> phobos mailing list
>> phobos at puremagic.com
>> http://lists.puremagic.com/mailman/listinfo/phobos
>> 
>
> _______________________________________________
> phobos mailing list
> phobos at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/phobos
>
>
> 

Le 2010-11-08 ? 17:12, Walter Bright a ?crit :

> See my reply: http://d.puremagic.com/issues/show_bug.cgi?id=4854

As a side note, I discovered this week that the Objective-C runtime also depends on knowing the size of a section to initialize itself properly. This is for a section that contains all the protocol definitions aggregated together; upon loading an image the list is traversed to 'fixup' the protocols...

The interesting point is that it does so without this begin,content,end section trickery, so it doesn't matter if sections get reordered during linking, which would be much more robust.

From what I found out by digging in the source code, this involves calling 'getsectdatafromheader' (or 'getsectdatafromheader_64') which returns the size as part of its last argument (see header "mach-o/getsect.h"). I'm not exactly sure how this all works, but there's definitely a way to make it work because it does work for the Objective-C runtime. Perhaps I should investigate a little more and send what I find on the list for druntime.

libobjc:

Source Browser: <http://www.opensource.apple.com/source/objc4/objc4-437.1/>

Source Download link: <http://www.opensource.apple.com/tarballs/objc4/objc4-437.1.tar.gz>

-- 
Michel Fortin
michel.fortin at michelf.com
http://michelf.com/

Michel Fortin wrote:
> Le 2010-11-08 ? 17:12, Walter Bright a ?crit :
>
> 
>> See my reply: http://d.puremagic.com/issues/show_bug.cgi?id=4854
>> 
>
> As a side note, I discovered this week that the Objective-C runtime also depends on knowing the size of a section to initialize itself properly. This is for a section that contains all the protocol definitions aggregated together; upon loading an image the list is traversed to 'fixup' the protocols...
>
> The interesting point is that it does so without this begin,content,end section trickery, so it doesn't matter if sections get reordered during linking, which would be much more robust.
>
> >From what I found out by digging in the source code, this involves calling 'getsectdatafromheader' (or 'getsectdatafromheader_64') which returns the size as part of its last argument (see header "mach-o/getsect.h"). I'm not exactly sure how this all works, but there's definitely a way to make it work because it does work for the Objective-C runtime. Perhaps I should investigate a little more and send what I find on the list for druntime.
>
> libobjc:
>
> Source Browser: <http://www.opensource.apple.com/source/objc4/objc4-437.1/>
>
> Source Download link: <http://www.opensource.apple.com/tarballs/objc4/objc4-437.1.tar.gz>
>
> 

If such works, then I'm all for using it to replace the current method.

It should work using getsectdatafromheader, that's what I changed when I added support for dynamic libraries to Tango. Have a look at the Tango code:

http://dsource.org/projects/tango/browser/trunk/tango/core/rt/compiler/dmd/object_.d#L1270 and http://dsource.org/projects/tango/browser/trunk/tango/core/rt/compiler/dmd/darwin/Image.d#L103 .

The file in the last link is completely made by me so you won't have to be afraid for looking at it. Or if you don't want to look at Tango source files at all you can look at this druntime change: http://www.dsource.org/projects/druntime/changeset/372 , to be more specific: http://www.dsource.org/projects/druntime/browser/trunk/src/rt/image.d?rev=372#L105

On 9 nov 2010, at 00:26, Michel Fortin wrote:

> Le 2010-11-08 ? 17:12, Walter Bright a ?crit :
> 
>> See my reply: http://d.puremagic.com/issues/show_bug.cgi?id=4854
> 
> As a side note, I discovered this week that the Objective-C runtime also depends on knowing the size of a section to initialize itself properly. This is for a section that contains all the protocol definitions aggregated together; upon loading an image the list is traversed to 'fixup' the protocols...
> 
> The interesting point is that it does so without this begin,content,end section trickery, so it doesn't matter if sections get reordered during linking, which would be much more robust.
> 
> From what I found out by digging in the source code, this involves calling 'getsectdatafromheader' (or 'getsectdatafromheader_64') which returns the size as part of its last argument (see header "mach-o/getsect.h"). I'm not exactly sure how this all works, but there's definitely a way to make it work because it does work for the Objective-C runtime. Perhaps I should investigate a little more and send what I find on the list for druntime.
> 
> libobjc:
> 
> Source Browser: <http://www.opensource.apple.com/source/objc4/objc4-437.1/>
> 
> Source Download link: <http://www.opensource.apple.com/tarballs/objc4/objc4-437.1.tar.gz>
> 
> -- 
> Michel Fortin
> michel.fortin at michelf.com
> http://michelf.com/
> 
> 
> 
> _______________________________________________
> phobos mailing list
> phobos at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/phobos

-- 
/Jacob Carlborg

See my replay to Michel Fortin.

Just out of curiosity why does DMD use this approach with begin, content, end to determine a section on Mac OS X? Looking at the source code for object_.d in Tango for LDC, the same implementation is used for all Posix systems and that is the same approach which is used in druntime for Linux, Solaris and FreeBSD.

On 9 nov 2010, at 00:51, Walter Bright wrote:

> 
> 
> Michel Fortin wrote:
>> Le 2010-11-08 ? 17:12, Walter Bright a ?crit :
>> 
>> 
>>> See my reply: http://d.puremagic.com/issues/show_bug.cgi?id=4854
>>> 
>> 
>> As a side note, I discovered this week that the Objective-C runtime also depends on knowing the size of a section to initialize itself properly. This is for a section that contains all the protocol definitions aggregated together; upon loading an image the list is traversed to 'fixup' the protocols...
>> 
>> The interesting point is that it does so without this begin,content,end section trickery, so it doesn't matter if sections get reordered during linking, which would be much more robust.
>> 
>> >From what I found out by digging in the source code, this involves calling 'getsectdatafromheader' (or 'getsectdatafromheader_64') which returns the size as part of its last argument (see header "mach-o/getsect.h"). I'm not exactly sure how this all works, but there's definitely a way to make it work because it does work for the Objective-C runtime. Perhaps I should investigate a little more and send what I find on the list for druntime.
>> 
>> libobjc:
>> 
>> Source Browser: <http://www.opensource.apple.com/source/objc4/objc4-437.1/>
>> 
>> Source Download link: <http://www.opensource.apple.com/tarballs/objc4/objc4-437.1.tar.gz>
>> 
>> 
> 
> If such works, then I'm all for using it to replace the current method.
> _______________________________________________
> phobos mailing list
> phobos at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/phobos

-- 
/Jacob Carlborg

Le 2010-11-09 ? 3:33, Jacob Carlborg a ?crit :

> See my replay to Michel Fortin.
> 
> Just out of curiosity why does DMD use this approach with begin, content, end to determine a section on Mac OS X? Looking at the source code for object_.d in Tango for LDC, the same implementation is used for all Posix systems and that is the same approach which is used in druntime for Linux, Solaris and FreeBSD.

I believed the reason was that Walter could not find at the time a way to get the size of a section, so he developed this little hack.

Now, that's surprising because if you look at rt/memory_osx.c in druntime you'll find it does exactly that: it calls getsectbynamefromheader to get a section's address and size to add ranges to the GC, and _dyld_register_func_for_add_image/_dyld_register_func_for_remove_image to register callbacks for when an image is loaded/unloaded. That file's authors are Walter Bright and Sean Kelly. I don't see a reason the same technique can't be used to handle modules infos, tls, and the like.

Looking at druntime makes me wonder why the initialization code is scattered everywhere... surely it'd be easier to figure out these kinds of problems if initialization was all happening in one place.

-- 
Michel Fortin
michel.fortin at michelf.com
http://michelf.com/