Context: On OSX, a C program can load a D shared library but once unloaded the next dlopen will crash, jumping into a callback that doesn't exist anymore.

I've filed it here: https://issues.dlang.org/show_bug.cgi?id=15060


It looks like this was known and discussed several times already:
http://forum.dlang.org/post/vixoqmidlbizawbxmsao@forum.dlang.org (2015)
https://github.com/D-Programming-Language/druntime/pull/228 (2012)


Any idea to work-around this problem would be awesome.

I'm not looking for something correct, future-proof, or pretty. Any shit that still stick to the wall will do. Anything!

The only case I need to support is: C host, D shared library, with runtime statically linked.

Please help!

On Wednesday, 16 September 2015 at 22:29:46 UTC, ponce wrote:
> Context: On OSX, a C program can load a D shared library but once unloaded the next dlopen will crash, jumping into a callback that doesn't exist anymore.
>
> I've filed it here: https://issues.dlang.org/show_bug.cgi?id=15060
>
>
> It looks like this was known and discussed several times already:
> http://forum.dlang.org/post/vixoqmidlbizawbxmsao@forum.dlang.org (2015)
> https://github.com/D-Programming-Language/druntime/pull/228 (2012)
>
>
> Any idea to work-around this problem would be awesome.
>
> I'm not looking for something correct, future-proof, or pretty. Any shit that still stick to the wall will do. Anything!
>
> The only case I need to support is: C host, D shared library, with runtime statically linked.
>
> Please help!

I was trying to solve this one myself, but the modifications to DMD's backend that are needed are out of reach for me right now.

If you're willing to build your own druntime, you may be able to get by.

If I understand correctly, you want to repeatedly load/unload the same shared library, correct? I ask because druntime for osx only supports loading a single image at a time:

https://github.com/D-Programming-Language/druntime/blob/1e25749cd01ad08dc08319a3853fbe86356c3e62/src/rt/sections_osx.d#L26

Anyways, when main() of a D program runs, it calls rt_init() and rt_term(). If you don't have a D entry point in your program, you have to retrieve these from your shared lib(which has druntime statically linked) using dlsym() and call them yourself.

https://github.com/D-Programming-Language/druntime/blob/478b6c5354470bc70e688c45821eea71b766e70d/src/rt/dmain2.d#L158

Now, initSections() and finiSections() are responsible for setting up the image. If you look at initSections(), the function "_dyld_register_func_for_add_image" is the one that causes the crash, because there is no way to remove the callback, which will reside in your shared lib.

https://github.com/D-Programming-Language/druntime/blob/1e25749cd01ad08dc08319a3853fbe86356c3e62/src/rt/sections_osx.d#L76

So what happens is, when you call _dyld_register_func_for_add_image, dyld will call your callback for every shared-library/image(including the main application's image) that is currently loaded. However, you can skip the callback and just call "sections_osx_onAddImage" yourself.

You would have to add something like this to sections_osx.d, and call it instead of adding the callback:

void callOnAddImage()
{
    // dladdr() should give you information about the
    // shared lib in which the symbol you pass resides.
    // Passing the address of this function should work.
    Dl_info info;
    int ret = dladdr(cast(void*)&callOnAddImage, &info);
    assert(ret);

    // "dli_fbase" is actually a pointer to
    // the mach_header for the shared library.
    // once you have the mach_header, you can
    // also retrieve the image slide, and finally
    // call sections_osx_onAddImage().
    mach_header* header = cast(mach_header*)info.dli_fbase;
    intptr_t slide = _dyld_get_image_slide(header);
    sections_osx_onAddImage(header, slide);
}

Now, if you look at finiSections(), it seems to be incomplete. There is nothing like sections_osx_onRemoveImage, so you'll have to complete it to make sure the library is unloaded correctly:

https://github.com/D-Programming-Language/druntime/blob/1e25749cd01ad08dc08319a3853fbe86356c3e62/src/rt/sections_osx.d#L83

You'll may have to make other mods here and there to get this working correctly, but this is the bulk of it.

     Bit

On Wednesday, 16 September 2015 at 23:24:29 UTC, bitwise wrote:
>
> I was trying to solve this one myself, but the modifications to DMD's backend that are needed are out of reach for me right now.
>
> If you're willing to build your own druntime, you may be able to get by.

I'd prefer a solution that works with existing compilers, but maybe building a custom LDC is possible if I figure it out.

> If I understand correctly, you want to repeatedly load/unload the same shared library, correct? I ask because druntime for osx only supports loading a single image at a time:
>
> https://github.com/D-Programming-Language/druntime/blob/1e25749cd01ad08dc08319a3853fbe86356c3e62/src/rt/sections_osx.d#L26
>

In practive I've found that the D shared libraries I produce can be dlopen/dlclose any number of times, simultaneous too. Using both LDC and DMD, don't know why it works.
The thing that doesn't work is the C host program dlopen'ing the shared library, dlclose it, then dlopen another shared library written in C.


> Anyways, when main() of a D program runs, it calls rt_init() and rt_term(). If you don't have a D entry point in your program, you have to retrieve these from your shared lib(which has druntime statically linked) using dlsym() and call them yourself.

I don't control the host program. My shared libs do have an entrypoint, from which I call Runtime.initialize().

I can also use LDC global constructor/destructor to call Runtime.initialize / Runtime.terminate, but it doesn't work any better because of the callback.



>
> https://github.com/D-Programming-Language/druntime/blob/478b6c5354470bc70e688c45821eea71b766e70d/src/rt/dmain2.d#L158
>
> Now, initSections() and finiSections() are responsible for setting up the image. If you look at initSections(), the function "_dyld_register_func_for_add_image" is the one that causes the crash, because there is no way to remove the callback, which will reside in your shared lib.
>
> https://github.com/D-Programming-Language/druntime/blob/1e25749cd01ad08dc08319a3853fbe86356c3e62/src/rt/sections_osx.d#L76
>
> So what happens is, when you call _dyld_register_func_for_add_image, dyld will call your callback for every shared-library/image(including the main application's image) that is currently loaded. However, you can skip the callback and just call "sections_osx_onAddImage" yourself.
>
> You would have to add something like this to sections_osx.d, and call it instead of adding the callback:
>
> void callOnAddImage()
> {
>     // dladdr() should give you information about the
>     // shared lib in which the symbol you pass resides.
>     // Passing the address of this function should work.
>     Dl_info info;
>     int ret = dladdr(cast(void*)&callOnAddImage, &info);
>     assert(ret);
>
>     // "dli_fbase" is actually a pointer to
>     // the mach_header for the shared library.
>     // once you have the mach_header, you can
>     // also retrieve the image slide, and finally
>     // call sections_osx_onAddImage().
>     mach_header* header = cast(mach_header*)info.dli_fbase;
>     intptr_t slide = _dyld_get_image_slide(header);
>     sections_osx_onAddImage(header, slide);
> }
>
> Now, if you look at finiSections(), it seems to be incomplete. There is nothing like sections_osx_onRemoveImage, so you'll have to complete it to make sure the library is unloaded correctly:
>
> https://github.com/D-Programming-Language/druntime/blob/1e25749cd01ad08dc08319a3853fbe86356c3e62/src/rt/sections_osx.d#L83
>
> You'll may have to make other mods here and there to get this working correctly, but this is the bulk of it.
>
>      Bit

Thanks for your answer. This is really helpful, though I don't understand the first thing about what images, headers and sections are in this context.

On Thursday, 17 September 2015 at 06:40:56 UTC, ponce wrote:
> On Wednesday, 16 September 2015 at 23:24:29 UTC, bitwise wrote:
>>
>> I was trying to solve this one myself, but the modifications to DMD's backend that are needed are out of reach for me right now.
>>
>> If you're willing to build your own druntime, you may be able to get by.
>
> I'd prefer a solution that works with existing compilers, but maybe building a custom LDC is possible if I figure it out.

Unfortunately, this is a complex problem and you can't just flip a few switches and make it work.

> In practive I've found that the D shared libraries I produce can be dlopen/dlclose any number of times, simultaneous too. Using both LDC and DMD, don't know why it works.
> The thing that doesn't work is the C host program dlopen'ing the shared library, dlclose it, then dlopen another shared library written in C.

Calling dlopen won't initialize the runtime in the shared library. Although your program doesn't crash, you may find that it does once you start calling into the shared lib... or it may leak memory because the GC was never initialized. You can call Runtime.initialize() from druntime yourself, but as you know, there is the callback issue.

I think LDC may be farther along in terms of shared library support. You may want to ask David Nadlinger about this. I think the problem with LDC was that it only supported statically loaded shared libs for osx. I'm not sure if this is still the case, but it looks like it:

https://github.com/ldc-developers/druntime/blob/002d0aed65cd24dfcbbc782d9aed2f9112f27b4e/src/rt/sections_ldc.d#L372

>> Anyways, when main() of a D program runs, it calls rt_init() and rt_term(). If you don't have a D entry point in your program, you have to retrieve these from your shared lib(which has druntime statically linked) using dlsym() and call them yourself.
>
> I don't control the host program. My shared libs do have an entrypoint, from which I call Runtime.initialize().
>
> I can also use LDC global constructor/destructor to call Runtime.initialize / Runtime.terminate, but it doesn't work any better because of the callback.

I don't think there is any way around this beside building a custom runtime and implementing the function I suggested.

> Thanks for your answer. This is really helpful, though I don't understand the first thing about what images, headers and sections are in this context.

As a last resort, you may be able to use OSX Distributed Objects, depending on what your shared library does. This is basically a wrapper around linux pipes, but with a simple RPC-like interface. Depending on how much talk there is between the host program and your shared lib, this may or may not be a good solution.

https://developer.apple.com/library/mac/documentation/Cocoa/Conceptual/DistrObjects/DistrObjects.html

Basically, you would make an Objective-C shared library with a C interface, into which the host program could call. This shared library would then spawn a second process as needed, which would load the D shared library. You would then use distributed objects to call from the Objective-C shared library to the D shared library. When you were done with the D shared library, you would simply terminate the second process.

   Bit

On 9/16/2015 11:40 PM, ponce wrote:
> On Wednesday, 16 September 2015 at 23:24:29 UTC, bitwise wrote:
>>
>> I was trying to solve this one myself, but the modifications to DMD's backend
>> that are needed are out of reach for me right now.
>>
>> If you're willing to build your own druntime, you may be able to get by.
>
> I'd prefer a solution that works with existing compilers, but maybe building a
> custom LDC is possible if I figure it out.

I seriously doubt this issue has anything to do with the compiler's code generator back end. It's more likely the problem is in druntime.

On 2015-09-17 00:29, ponce wrote:
> Context: On OSX, a C program can load a D shared library but once
> unloaded the next dlopen will crash, jumping into a callback that
> doesn't exist anymore.
>
> I've filed it here: https://issues.dlang.org/show_bug.cgi?id=15060
>
>
> It looks like this was known and discussed several times already:
> http://forum.dlang.org/post/vixoqmidlbizawbxmsao@forum.dlang.org (2015)
> https://github.com/D-Programming-Language/druntime/pull/228 (2012)
>
>
> Any idea to work-around this problem would be awesome.
>
> I'm not looking for something correct, future-proof, or pretty. Any shit
> that still stick to the wall will do. Anything!
>
> The only case I need to support is: C host, D shared library, with
> runtime statically linked.
>
> Please help!

Easiest would be to not unload the library. If that doesn't work, replace "_dyld_register_func_for_add_image" [1] with "dyld_register_image_state_change_handler" [2].

[1] https://github.com/D-Programming-Language/druntime/blob/master/src/rt/sections_osx.d#L76

[2] http://www.opensource.apple.com/source/dyld/dyld-353.2.3/include/mach-o/dyld_priv.h

-- 
/Jacob Carlborg

On Thursday, 17 September 2015 at 10:12:57 UTC, Walter Bright wrote:
> On 9/16/2015 11:40 PM, ponce wrote:
>> On Wednesday, 16 September 2015 at 23:24:29 UTC, bitwise wrote:
>>>
>>> I was trying to solve this one myself, but the modifications to DMD's backend
>>> that are needed are out of reach for me right now.
>>>
>>> If you're willing to build your own druntime, you may be able to get by.
>>
>> I'd prefer a solution that works with existing compilers, but maybe building a
>> custom LDC is possible if I figure it out.
>
> I seriously doubt this issue has anything to do with the compiler's code generator back end. It's more likely the problem is in druntime.

Based on previous posts you've made, it seems you already know that Martin's solution involves having DMD's back end add linux style ctors/dtors to the COMDAT section of every compiled D file, so I'm not sure what you mean by this.

If a callback provided to _dyld_register_func_for_add_image does not call sections_osx_onAddImage, then it must be called somehow.

   Bit

On Thursday, 17 September 2015 at 15:12:43 UTC, Jacob Carlborg wrote:
> On 2015-09-17 00:29, ponce wrote:
>> [...]
>
> Easiest would be to not unload the library. If that doesn't work, replace "_dyld_register_func_for_add_image" [1] with "dyld_register_image_state_change_handler" [2].
>
> [1] https://github.com/D-Programming-Language/druntime/blob/master/src/rt/sections_osx.d#L76
>
> [2] http://www.opensource.apple.com/source/dyld/dyld-353.2.3/include/mach-o/dyld_priv.h

dyld_register_image_state_change_handler does not provide a way to unregister the callback either, so I don't see how this helps.

On Thursday, 17 September 2015 at 15:12:43 UTC, Jacob Carlborg wrote:
>
> Easiest would be to not unload the library.

I don't control what the host program does.


> If that doesn't work, replace "_dyld_register_func_for_add_image" [1] with "dyld_register_image_state_change_handler" [2].
>
> [1] https://github.com/D-Programming-Language/druntime/blob/master/src/rt/sections_osx.d#L76
>
> [2] http://www.opensource.apple.com/source/dyld/dyld-353.2.3/include/mach-o/dyld_priv.h

Looks good.

On Thursday, 17 September 2015 at 16:17:11 UTC, bitwise wrote:
> On Thursday, 17 September 2015 at 10:12:57 UTC, Walter Bright wrote:
>> On 9/16/2015 11:40 PM, ponce wrote:
>>> On Wednesday, 16 September 2015 at 23:24:29 UTC, bitwise wrote:
>>>>
>>>> I was trying to solve this one myself, but the modifications to DMD's backend
>>>> that are needed are out of reach for me right now.
>>>>
>>>> If you're willing to build your own druntime, you may be able to get by.
>>>
>>> I'd prefer a solution that works with existing compilers, but maybe building a
>>> custom LDC is possible if I figure it out.
>>
>> I seriously doubt this issue has anything to do with the compiler's code generator back end. It's more likely the problem is in druntime.
>
> Based on previous posts you've made, it seems you already know that Martin's solution involves having DMD's back end add linux style ctors/dtors to the COMDAT section of every compiled D file, so I'm not sure what you mean by this.
>
> If a callback provided to _dyld_register_func_for_add_image does not call sections_osx_onAddImage, then it must be called somehow.
>
>    Bit

One solution which could work is to disallow static linking of druntime on OSX completely....meaning, either don't even distribute a static druntime for OSX, or make shared druntime the default. This way, druntime would only ever be initialized once per process. Knowing this, linux ctors/dtors could be added to druntime to initialize it, eliminating the need to call rt_init/rt_term before and after main(). Also, if druntime were loaded into a C-hosted binary, druntime would automatically be initialized by the ctors/dtors. Also, for the ctors/dtors, they could be added to the druntime build, and wouldn't having to be compiler generated.

I'm not sure about the difference in performance cost over static vs shared druntime, but it seems that this is the way things are done on OSX. If you look in /usr/lib/ on a mac, practically everything is a shared lib.

    Bit