Thread overview
Shared library support: Where we are at
Jun 07, 2022
Rikki Cattermole
Jun 07, 2022
Adam Ruppe
Jun 07, 2022
jmh530
Jun 12, 2022
rikki cattermole
Jun 12, 2022
Adam D Ruppe
Jun 12, 2022
rikki cattermole
Jun 13, 2022
rikki cattermole
Nov 25, 2022
rikki cattermole
June 07, 2022

Hello!

Currently, I'm doing a push towards getting shared library support in D into a much more usable state.

My approach has been this:

Build -betterC shared library, build full D executable that depends upon said shared library. Regardless of how you feel about -betterC, the reason it is in use here is to try and prevent issues surrounding druntime during my initial push.

I have not tested on Posix or used GDC. Right now I am only interested in Windows support as it has the most platform-specific issues and I am entirely ignoring Optlink (as far as I'm concerned that is WONTFIX).

Both ldc and dmd can work in this arrangement. However in both cases you need to manually generate ModuleInfo stubs. Like this:

export extern(C) void _D6dynlib3app12__ModuleInfoZ() {
    asm {
        naked;
        di 0;
        di 0x1000;
        db 0;
    }
}

Both LDC and DMD do not have a way to configure ModuleInfo generation. GDC does have a way to turn it on and off. LDC bug report [0].

If you remove -betterC, that stub will no longer work in dmd due to duplicate symbols (that wasn't exported). LDC will compile okay.

In my codebase I have found that --fvisibility=public is a very useful crutch given export is a linker directive that is pretending to be a visibility modifier (however let's ignore that since that is a DIP).

As part of my codebase I'm working towards getting dub working. I'm currently waiting on a PR that is blocked by a vibe.d regression with openssl[1]. This should mean build artifacts from dependencies will be copied to the target directory of a project and DLLs won't be linked against but instead will use their import libraries.

I've cherry-picked a few issues surrounding this [2], [3], [4].

However here are the things that if fixed, should lead to this working:

  • In both LDC and DMD, implement switches to turn on and off ModuleInfo generation (ignoring -betterC switch).
  • In DMD export ModuleInfo.
  • In DMD implement a switch to turn on export for all symbols.

So to cap this off: right now DMD is basically not even an option for shared libraries unless you use -betterC or treat it as if it was a non-D library. LDC is possible for non -betterC.

[0] https://github.com/ldc-developers/ldc/issues/3996
[1] https://github.com/dlang/dub/pull/2259
[2] https://issues.dlang.org/show_bug.cgi?id=22367
[3] https://issues.dlang.org/show_bug.cgi?id=9816
[4] https://issues.dlang.org/show_bug.cgi?id=4071

June 07, 2022

On Tuesday, 7 June 2022 at 19:07:53 UTC, Rikki Cattermole wrote:

>

Currently, I'm doing a push towards getting shared library support in D into a much more usable state.

shared libs in D have been fully operational for over a year now with gdc and almost a year with ldc. You can make the druntime/phobos dll and all share it. Even on dmd, there's separate druntimes but they link some things together, enough to kinda work, but the ldc+gdc experience is more reliable (dmd botches typeinfo across the dll boundary, so you're best off using it COM style).

For the plugin model, probably best to just build with the ldc.

June 07, 2022

On Tuesday, 7 June 2022 at 19:07:53 UTC, Rikki Cattermole wrote:

>

Hello!

Currently, I'm doing a push towards getting shared library support in D into a much more usable state.

Thanks for working on this.

June 13, 2022
While fixing one of the bugs I have discovered that when using MSVC linker, shared libraries that have symbols that contain Unicode is currently broken.

It is not on our end. This is a hard blocker for our non-english speaking users.

It will mean that we have to do something to sanitize those symbol names to make them ASCII in nature.

Unfortunately we had an incomplete test case in the testsuite that only covered executables.

https://issues.dlang.org/show_bug.cgi?id=23179
June 12, 2022
On Sunday, 12 June 2022 at 13:52:17 UTC, rikki cattermole wrote:
> While fixing one of the bugs I have discovered that when using MSVC linker, shared libraries that have symbols that contain Unicode is currently broken.

im p sure that's part of the mangling scheme, prolly missed a spot there.
June 13, 2022
On 13/06/2022 1:56 AM, Adam D Ruppe wrote:
> On Sunday, 12 June 2022 at 13:52:17 UTC, rikki cattermole wrote:
>> While fixing one of the bugs I have discovered that when using MSVC linker, shared libraries that have symbols that contain Unicode is currently broken.
> 
> im p sure that's part of the mangling scheme, prolly missed a spot there.

I didn't.

Its a bug that LDC has also encountered in MSVC linker (which they solved by just disabling the test like I did).
June 14, 2022
On 13/06/2022 1:52 AM, rikki cattermole wrote:
> While fixing one of the bugs I have discovered that when using MSVC linker, shared libraries that have symbols that contain Unicode is currently broken.
> 
> It is not on our end. This is a hard blocker for our non-english speaking users.
> 
> It will mean that we have to do something to sanitize those symbol names to make them ASCII in nature.
> 
> Unfortunately we had an incomplete test case in the testsuite that only covered executables.
> 
> https://issues.dlang.org/show_bug.cgi?id=23179

I did manage to create a fix for this bug, but because of how it was discovered we are favoring a wait and see approach to see if it appears in the wild.

So if you experience this, please update the bug report!
November 26, 2022
Wahoooooo joy to the world we have another bug report that could be closed by getting exportation of ModuleInfo fixed.

https://issues.dlang.org/show_bug.cgi?id=6019

Current running total is four if this one issue were to be solved satisfactorily.

https://issues.dlang.org/show_bug.cgi?id=22367
https://issues.dlang.org/show_bug.cgi?id=9816
https://issues.dlang.org/show_bug.cgi?id=4071

This is a set of very high value set of bug reports that could be fixed very easily if someone with knowledge of dmd compiler internals wanted to fix it.

See: https://forum.dlang.org/post/ohflhdaggtapeqpynkjd@forum.dlang.org

Latest attempt of mine to fix this: https://github.com/dlang/dmd/pull/14647