Jump to page: 1 2
Thread overview
How to compile Phobos with other D code to create a shared library?
May 31
kinke
Jun 02
bachmeier
Jun 04
bachmeier
Jun 04
bachmeier
May 31

Hi,

I am trying to compile D code to a shared library to be called by another language (in this case Julia). I can get a basic call to work but I can't call functions that use Phobos (I have no idea how to compile against it). I have found some documentation here and here that lists compilation against specific flags so for instance I have tried:

ldc2 jbasic.d -O3 -link-defaultlib-shared --betterC --boundscheck=off -nogc -shared -of=jbasic.so

But in Julia (and the equivalent thing happens in R), when I attempt to load the function I get an error:

ERROR: could not load library "./jbasic.so"
./jbasic.so: undefined symbol: _D3std6random__T21MersenneTwisterEngineTmVmi64Vmi312Vmi156Vmi31VmN5403634167711393303Vmi29Vmi6148914691236517205Vmi17Vmi8202884508482404352Vmi37VmN2270628950310912Vmi43Vmi6364136223846793005ZQGt6__initZ

I tried the equivalent with dmd:

dmd jbasic.d -O -defaultlib=libphobos2.so -betterC -fPIC -boundscheck=off  -shared -of=jbasic.so

with the same result.

Thank you.

May 31

On Monday, 31 May 2021 at 19:21:52 UTC, data pulverizer wrote:

>

ldc2 jbasic.d -O3 -link-defaultlib-shared --betterC --boundscheck=off -nogc -shared -of=jbasic.so

The problem is almost certainly -betterC, which disables linking against Phobos and druntime.

May 31

On Monday, 31 May 2021 at 20:32:11 UTC, kinke wrote:

>

On Monday, 31 May 2021 at 19:21:52 UTC, data pulverizer wrote:

>

ldc2 jbasic.d -O3 -link-defaultlib-shared --betterC --boundscheck=off -nogc -shared -of=jbasic.so

The problem is almost certainly -betterC, which disables linking against Phobos and druntime.

Thanks - I should have know that one! I've removed the flag but now I'm getting the following error when I try to call the function:

Aborting from src/core/time.d(2131) MonoTimeImpl!(ClockType.normal) failed to get the frequency of the system's monotonic clock.
signal (6): Aborted
in expression starting at REPL[2]:1
gsignal at x86_64-linux-gnu/libc.so.6 (unknown line)
abort at x86_64-linux-gnu/libc.so.6 (unknown line)
_D4core8internal5abortQgFNbNiNfMAyaMQemZv at /lib64/libphobos2.so.0.90 (unknown line)
_D4core4time__T12MonoTimeImplVEQBdQBb9ClockTypei0ZQBj8currTimeFNbNdNiNeZSQCtQCr__TQCpVQCei0ZQCz at dmd/current/lib64/libphobos2.so.0.90 (unknown line)
_D3std6random13bootstrapSeedFNbNiZm at dmd/current/lib64/libphobos2.so.0.90 (unknown line)
_D3std6random__T17unpredictableSeedTmZQwFNbNdNiNeZm at dmd/current/lib64/libphobos2.so.0.90 (unknown line)
_D3std6random17unpredictableSeedFNbNdNiNeZk at dmd/current/lib64/libphobos2.so.0.90 (unknown line)

May 31
On 5/31/21 4:41 PM, data pulverizer wrote:
> On Monday, 31 May 2021 at 20:32:11 UTC, kinke wrote:
>> On Monday, 31 May 2021 at 19:21:52 UTC, data pulverizer wrote:
>>> ldc2 jbasic.d -O3 -link-defaultlib-shared --betterC --boundscheck=off -nogc -shared -of=jbasic.so
>>
>> The problem is almost certainly `-betterC`, which disables linking against Phobos and druntime.
> 
> Thanks - I should have know that one! I've removed the flag but now I'm getting the following error when I try to call the function:
> 
> ```
> Aborting from src/core/time.d(2131) MonoTimeImpl!(ClockType.normal) failed to get the frequency of the system's monotonic clock.
> signal (6): Aborted
> in expression starting at REPL[2]:1
> gsignal at x86_64-linux-gnu/libc.so.6 (unknown line)
> abort at x86_64-linux-gnu/libc.so.6 (unknown line)
> _D4core8internal5abortQgFNbNiNfMAyaMQemZv at /lib64/libphobos2.so.0.90 (unknown line)
> _D4core4time__T12MonoTimeImplVEQBdQBb9ClockTypei0ZQBj8currTimeFNbNdNiNeZSQCtQCr__TQCpVQCei0ZQCz at dmd/current/lib64/libphobos2.so.0.90 (unknown line)
> _D3std6random13bootstrapSeedFNbNiZm at dmd/current/lib64/libphobos2.so.0.90 (unknown line)
> _D3std6random__T17unpredictableSeedTmZQwFNbNdNiNeZm at dmd/current/lib64/libphobos2.so.0.90 (unknown line)
> _D3std6random17unpredictableSeedFNbNdNiNeZk at dmd/current/lib64/libphobos2.so.0.90 (unknown line)
> 
> ```

ticksPerSecond is initialized in the runtime just before static constructors are run. See https://github.com/dlang/druntime/blob/2d8b28da39e8bc3bc3172c69bb96c35d77f40d2a/src/rt/dmain2.d#L130

Are you calling Runtime.initialize()?

-Steve
May 31
On Monday, 31 May 2021 at 21:01:19 UTC, Steven Schveighoffer wrote:
>
> ticksPerSecond is initialized in the runtime just before static constructors are run. See https://github.com/dlang/druntime/blob/2d8b28da39e8bc3bc3172c69bb96c35d77f40d2a/src/rt/dmain2.d#L130
>
> Are you calling Runtime.initialize()?


Nope, I guess I'm supposed to be? if so where do I place the call(s) for initialize() and terminate()?


May 31
On 5/31/21 5:20 PM, data pulverizer wrote:
> On Monday, 31 May 2021 at 21:01:19 UTC, Steven Schveighoffer wrote:
>>
>> ticksPerSecond is initialized in the runtime just before static constructors are run. See https://github.com/dlang/druntime/blob/2d8b28da39e8bc3bc3172c69bb96c35d77f40d2a/src/rt/dmain2.d#L130 
>>
>>
>> Are you calling Runtime.initialize()?
> 
> 
> Nope, I guess I'm supposed to be? if so where do I place the call(s) for initialize() and terminate()?
> 
> 

You need to call it wherever you think it might not have been called yet.

It's reentrant, so if you call it more than once, it will only initialize once, and count how many times you have to call `Runtime.terminate`.

Best to use a `scope(exit)` to call `Runtime.terminate` if you are calling it periodically.

-Steve
May 31

On Monday, 31 May 2021 at 21:26:15 UTC, Steven Schveighoffer wrote:

>

You need to call it wherever you think it might not have been called yet.

It's reentrant, so if you call it more than once, it will only initialize once, and count how many times you have to call Runtime.terminate.

Best to use a scope(exit) to call Runtime.terminate if you are calling it periodically.

Many thanks!

Something interesting is using arrays. I can see that if you instantiate an array within the D function using new, for instance

auto result = new double[n];

you can't return that array to Julia (or whatever) if your function does

scope(exit) Runtime.terminate();

and it segfaults. But while you can do:

auto result = cast(double*)malloc(double.sizeof * n);

you're left with the issue of not being able to free the memory since you've terminated the runtime.

So I guess you have to appropriately pair up initialize and terminate as appropriate to kind of startup and and shutdown the connection with druntime with your allocations if they stay within D?

May 31

On Monday, 31 May 2021 at 21:46:09 UTC, data pulverizer wrote:

>

Something interesting is using arrays. I can see that if you instantiate an array within the D function using new, for instance

Passing one of those to a C function is iffy anyway because the C function can hide it from the garbage collector.

What happens here is Runtime.terminate does a final GC sweep to gracefully clean up as much as it can before it close... and it thinks that array is unused, so it frees it.

GC.addRoot(array) can tell it not to free normally... but i'm not sure if runtime.terminate cares about that since terminate makes it think the whole thing is going down anyway.

>

auto result = cast(double*)malloc(double.sizeof * n);


you're left with the issue of not being able to free the memory since you've terminated the runtime.

You can free still, malloc and free are not controlled by the D runtime.

This is sometimes a better idea anyway when passing it to C function. I don't know how Julia does it though.

>

So I guess you have to appropriately pair up initialize and terminate as appropriate to kind of startup and and shutdown the connection with druntime with your allocations if they stay within D?

But yeah the best way to do it is to initialize once on library load then terminate once when the whole thing is finished.

Typically C plugin systems have some hook for this you can use. Like some onload function they define you can use or something.

June 01
On 5/31/21 5:46 PM, data pulverizer wrote:
> On Monday, 31 May 2021 at 21:26:15 UTC, Steven Schveighoffer wrote:
>>
>> You need to call it wherever you think it might not have been called yet.
>>
>> It's reentrant, so if you call it more than once, it will only initialize once, and count how many times you have to call `Runtime.terminate`.
>>
>> Best to use a `scope(exit)` to call `Runtime.terminate` if you are calling it periodically.
> 
> Many thanks!
> 
> Something interesting is using arrays. I can see that if you instantiate an array within the D function using `new`, for instance
> 
> ```
> auto result = new double[n];
> ```
> 
> you can't return that array to Julia (or whatever) if your function does
> 
> ```
> scope(exit) Runtime.terminate();
> ```
> 
> and it segfaults. But while you can do:
> 
> ```
> auto result = cast(double*)malloc(double.sizeof * n);
> ```
> 
> you're left with the issue of not being able to free the memory since you've terminated the runtime.
> 
> So I guess you have to appropriately pair up `initialize` and `terminate` as appropriate to kind of *startup* and and *shutdown* the connection with druntime with your allocations if they stay within D?
> 

You should use whatever Julia is expecting for memory management. I don't know what that is, but typically you want to use your host language's memory management system to create blocks instead of the D GC.

-Steve
June 01

Doing Runtime.initialize is working with Julia but not yet R, I'm getting a clock/GLIBC error

Error in dyn.load("rbasic.so") :
  unable to load shared object 'code/rbasic.so':
  lib/x86_64-linux-gnu/librt.so.1: undefined symbol: __clock_nanosleep, version GLIBC_PRIVATE

do I need to import anything else?

« First   ‹ Prev
1 2