View mode: basic / threaded / horizontal-split · Log in · Help
September 05, 2007
Cross-platform dynamic libraries (GNU/Linux, Mac OS X, Windows,...)
Hi,

I'm planning to create a multi-platform (at least GNU/Linux, Mac OS X 
and Windows) application in D. One of its key characteristics is that it 
will be based on modules (plug-ins). Ideally, functionalities more 
advanced than just normal shared libraries would be desirable.

I took a look at DDL, but it seems to be targeted only at Windows.

I made some dlopen() tests on Mac OS X 10.4 with gdc. It works, as long 
as the garbage collector isn't involved in the library. Instancing a 
class in the library works only if new() and delete() are defined and 
don't call the gc. When new() and delete() are not defined, instancing a 
class results in a bus error. gdb gives the following message:

Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: KERN_PROTECTION_FAILURE at address: 0x00000000
0x0005389a in _d_newclass ()

I also tried calling std.gc.setGCHandle(), but it also fails:

Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: KERN_PROTECTION_FAILURE at address: 0x00000000
0x00053435 in std.gc.addRange(void*, void*) ()

This may sound like a newbie question, but is there an easy way (or a 
way at all) to develop a cross-platform application in D with shared 
libraries?

Dynamic linking is a very useful feature for software development. 
Including this in the D specification (like threads) would be great.

Hugues De Keyzer

--
The only constant in life is change
   -- www.wikipedia.org
September 05, 2007
Re: Cross-platform dynamic libraries (GNU/Linux, Mac OS X, Windows,...)
Hugues De Keyzer wrote:
> Hi,
> 
> I'm planning to create a multi-platform (at least GNU/Linux, Mac OS X 
> and Windows) application in D. One of its key characteristics is that it 
> will be based on modules (plug-ins). Ideally, functionalities more 
> advanced than just normal shared libraries would be desirable.
> 
> I took a look at DDL, but it seems to be targeted only at Windows.
> 
> I made some dlopen() tests on Mac OS X 10.4 with gdc. It works, as long 
> as the garbage collector isn't involved in the library. Instancing a 
> class in the library works only if new() and delete() are defined and 
> don't call the gc. When new() and delete() are not defined, instancing a 
> class results in a bus error. gdb gives the following message:
> 
> Program received signal EXC_BAD_ACCESS, Could not access memory.
> Reason: KERN_PROTECTION_FAILURE at address: 0x00000000
> 0x0005389a in _d_newclass ()
> 
> I also tried calling std.gc.setGCHandle(), but it also fails:
> 
> Program received signal EXC_BAD_ACCESS, Could not access memory.
> Reason: KERN_PROTECTION_FAILURE at address: 0x00000000
> 0x00053435 in std.gc.addRange(void*, void*) ()
> 
> This may sound like a newbie question, but is there an easy way (or a 
> way at all) to develop a cross-platform application in D with shared 
> libraries?
> 
> Dynamic linking is a very useful feature for software development. 
> Including this in the D specification (like threads) would be great.
> 
> Hugues De Keyzer
> 
> -- 
> The only constant in life is change
>    -- www.wikipedia.org

GDC supports proper .so files on all platforms with .so files. Build the 
.so files without a standard library (-no-stdlib or something like that) 
and then build the host exporting dynamic symbols (default on OS X, 
-rdynamic on everything else). DSSS, of course, does all of this for you.

Windows DLL's will never be capable of full, proper support until phobos 
is in a .dll. Even then, if there was conflicting typeinfo, it could 
still fail. However, DDL ought to cover it there.

 - Gregor Richards
September 05, 2007
Re: Cross-platform dynamic libraries (GNU/Linux, Mac OS X, Windows,...)
On Wed, 05 Sep 2007 20:55:14 +0300, Gregor Richards <Richards@codu.org> wrote:

> Windows DLL's will never be capable of full, proper support until phobos
> is in a .dll. Even then, if there was conflicting typeinfo, it could
> still fail. However, DDL ought to cover it there.

I wonder how hard it would be to develop a native-code cross-platform plug-in architecture for the IA32 platform. The idea is that the "plugins" are compiled to native code, either position-independent or with relocation information, and don't contain any of the standard library. Instead, on initialization, the main, platform-dependant application passes the plug-in a structure filled with function/global var pointers, which have a cross-platform ABI, which will allow the plugins to access the standard library and other host app functionality.

Sounds theoretically feasable? I think there are no problems with the idea as long as there are no platform-dependant data types.

-- 
Best regards,
 Vladimir                          mailto:thecybershadow@gmail.com
September 05, 2007
Re: Cross-platform dynamic libraries (GNU/Linux, Mac OS X, Windows,...)
Vladimir Panteleev wrote:
> On Wed, 05 Sep 2007 20:55:14 +0300, Gregor Richards <Richards@codu.org> wrote:
> 
>> Windows DLL's will never be capable of full, proper support until phobos
>> is in a .dll. Even then, if there was conflicting typeinfo, it could
>> still fail. However, DDL ought to cover it there.
> 
> I wonder how hard it would be to develop a native-code cross-platform plug-in architecture for the IA32 platform. The idea is that the "plugins" are compiled to native code, either position-independent or with relocation information, and don't contain any of the standard library. Instead, on initialization, the main, platform-dependant application passes the plug-in a structure filled with function/global var pointers, which have a cross-platform ABI, which will allow the plugins to access the standard library and other host app functionality.
> 
> Sounds theoretically feasable? I think there are no problems with the idea as long as there are no platform-dependant data types.
> 

So, ELF? I think AT&T developed that around 25 years ago.

 - Gregor Richards
September 05, 2007
Re: Cross-platform dynamic libraries (GNU/Linux, Mac OS X, Windows,...)
On Wed, 05 Sep 2007 22:58:13 +0300, Gregor Richards <Richards@codu.org> wrote:

> Vladimir Panteleev wrote:
>> On Wed, 05 Sep 2007 20:55:14 +0300, Gregor Richards <Richards@codu.org> wrote:
>>
>>> Windows DLL's will never be capable of full, proper support until phobos
>>> is in a .dll. Even then, if there was conflicting typeinfo, it could
>>> still fail. However, DDL ought to cover it there.
>>
>> I wonder how hard it would be to develop a native-code cross-platform plug-in architecture for the IA32 platform. The idea is that the "plugins" are compiled to native code, either position-independent or with relocation information, and don't contain any of the standard library. Instead, on initialization, the main, platform-dependant application passes the plug-in a structure filled with function/global var pointers, which have a cross-platform ABI, which will allow the plugins to access the standard library and other host app functionality.
>>
>> Sounds theoretically feasable? I think there are no problems with the idea as long as there are no platform-dependant data types.
>>
>
> So, ELF? I think AT&T developed that around 25 years ago.

Link to Windows implementation please - preferably something D-friendly :P

-- 
Best regards,
 Vladimir                          mailto:thecybershadow@gmail.com
September 05, 2007
Re: Cross-platform dynamic libraries (GNU/Linux, Mac OS X, Windows,...)
Vladimir Panteleev wrote:
> On Wed, 05 Sep 2007 22:58:13 +0300, Gregor Richards <Richards@codu.org> wrote:
> 
>> Vladimir Panteleev wrote:
>>> On Wed, 05 Sep 2007 20:55:14 +0300, Gregor Richards <Richards@codu.org> wrote:
>>>
>>>> Windows DLL's will never be capable of full, proper support until phobos
>>>> is in a .dll. Even then, if there was conflicting typeinfo, it could
>>>> still fail. However, DDL ought to cover it there.
>>> I wonder how hard it would be to develop a native-code cross-platform plug-in architecture for the IA32 platform. The idea is that the "plugins" are compiled to native code, either position-independent or with relocation information, and don't contain any of the standard library. Instead, on initialization, the main, platform-dependant application passes the plug-in a structure filled with function/global var pointers, which have a cross-platform ABI, which will allow the plugins to access the standard library and other host app functionality.
>>>
>>> Sounds theoretically feasable? I think there are no problems with the idea as long as there are no platform-dependant data types.
>>>
>> So, ELF? I think AT&T developed that around 25 years ago.
> 
> Link to Windows implementation please - preferably something D-friendly :P
> 

http://svn.berlios.de/svnroot/repos/crosslibc/other/winelf/ .

OK, no .so support yet (I'm no ELF expert), but when/if it gets .so 
support. there ya go :P

 - Gregor Richards
September 10, 2007
Re: Cross-platform dynamic libraries (GNU/Linux, Mac OS X, Windows,...)
Gregor Richards wrote:
> Hugues De Keyzer wrote:
>> Hi,
>>
>> I'm planning to create a multi-platform (at least GNU/Linux, Mac OS X 
>> and Windows) application in D. One of its key characteristics is that 
>> it will be based on modules (plug-ins). Ideally, functionalities more 
>> advanced than just normal shared libraries would be desirable.
>>
>> I took a look at DDL, but it seems to be targeted only at Windows.
>>
>> I made some dlopen() tests on Mac OS X 10.4 with gdc. It works, as 
>> long as the garbage collector isn't involved in the library. 
>> Instancing a class in the library works only if new() and delete() are 
>> defined and don't call the gc. When new() and delete() are not 
>> defined, instancing a class results in a bus error. gdb gives the 
>> following message:
>>
>> Program received signal EXC_BAD_ACCESS, Could not access memory.
>> Reason: KERN_PROTECTION_FAILURE at address: 0x00000000
>> 0x0005389a in _d_newclass ()
>>
>> I also tried calling std.gc.setGCHandle(), but it also fails:
>>
>> Program received signal EXC_BAD_ACCESS, Could not access memory.
>> Reason: KERN_PROTECTION_FAILURE at address: 0x00000000
>> 0x00053435 in std.gc.addRange(void*, void*) ()
>>
>> This may sound like a newbie question, but is there an easy way (or a 
>> way at all) to develop a cross-platform application in D with shared 
>> libraries?
>>
>> Dynamic linking is a very useful feature for software development. 
>> Including this in the D specification (like threads) would be great.
>>
>> Hugues De Keyzer
>>
>> -- 
>> The only constant in life is change
>>    -- www.wikipedia.org
> 
> GDC supports proper .so files on all platforms with .so files. Build the 
> ..so files without a standard library (-no-stdlib or something like 
> that) and then build the host exporting dynamic symbols (default on OS 
> X, -rdynamic on everything else). DSSS, of course, does all of this for 
> you.
> 
> Windows DLL's will never be capable of full, proper support until phobos 
> is in a .dll. Even then, if there was conflicting typeinfo, it could 
> still fail. However, DDL ought to cover it there.
> 
>  - Gregor Richards

Thank you for this quick reply. I tested on Mac OS X with -nostdlib and 
had of course undefined references. After some more searching, I 
succeeded in building a dynamic library with the following options:

gdc -dynamiclib -nostdlib -undefined dynamic_lookup -o mylib.so mylib.d

This seems to work. The GC works fine.

Another question: is it safe to directly load a mangled D symbol like this:

void* test = dlsym(handle, "_D8mymodule8MyModule4testFZv");

for static void MyModyle.test()?

It works, but will it work an all platforms?

-- 
The only constant in life is change
   -- www.wikipedia.org
September 10, 2007
Re: Cross-platform dynamic libraries (GNU/Linux, Mac OS X, Windows,...)
Hugues De Keyzer wrote:
> Gregor Richards wrote:
>> Hugues De Keyzer wrote:
>>> Hi,
>>>
>>> I'm planning to create a multi-platform (at least GNU/Linux, Mac OS X 
>>> and Windows) application in D. One of its key characteristics is that 
>>> it will be based on modules (plug-ins). Ideally, functionalities more 
>>> advanced than just normal shared libraries would be desirable.
>>>
>>> I took a look at DDL, but it seems to be targeted only at Windows.
>>>
>>> I made some dlopen() tests on Mac OS X 10.4 with gdc. It works, as 
>>> long as the garbage collector isn't involved in the library. 
>>> Instancing a class in the library works only if new() and delete() 
>>> are defined and don't call the gc. When new() and delete() are not 
>>> defined, instancing a class results in a bus error. gdb gives the 
>>> following message:
>>>
>>> Program received signal EXC_BAD_ACCESS, Could not access memory.
>>> Reason: KERN_PROTECTION_FAILURE at address: 0x00000000
>>> 0x0005389a in _d_newclass ()
>>>
>>> I also tried calling std.gc.setGCHandle(), but it also fails:
>>>
>>> Program received signal EXC_BAD_ACCESS, Could not access memory.
>>> Reason: KERN_PROTECTION_FAILURE at address: 0x00000000
>>> 0x00053435 in std.gc.addRange(void*, void*) ()
>>>
>>> This may sound like a newbie question, but is there an easy way (or a 
>>> way at all) to develop a cross-platform application in D with shared 
>>> libraries?
>>>
>>> Dynamic linking is a very useful feature for software development. 
>>> Including this in the D specification (like threads) would be great.
>>>
>>> Hugues De Keyzer
>>>
>>> -- 
>>> The only constant in life is change
>>>    -- www.wikipedia.org
>>
>> GDC supports proper .so files on all platforms with .so files. Build 
>> the ..so files without a standard library (-no-stdlib or something 
>> like that) and then build the host exporting dynamic symbols (default 
>> on OS X, -rdynamic on everything else). DSSS, of course, does all of 
>> this for you.
>>
>> Windows DLL's will never be capable of full, proper support until 
>> phobos is in a .dll. Even then, if there was conflicting typeinfo, it 
>> could still fail. However, DDL ought to cover it there.
>>
>>  - Gregor Richards
> 
> Thank you for this quick reply. I tested on Mac OS X with -nostdlib and 
> had of course undefined references. After some more searching, I 
> succeeded in building a dynamic library with the following options:
> 
> gdc -dynamiclib -nostdlib -undefined dynamic_lookup -o mylib.so mylib.d
> 
> This seems to work. The GC works fine.
> 
> Another question: is it safe to directly load a mangled D symbol like this:
> 
> void* test = dlsym(handle, "_D8mymodule8MyModule4testFZv");
> 
> for static void MyModyle.test()?
> 
> It works, but will it work an all platforms?
> 

The mangling is the same on all platforms, so that should work, but is 
bound to break eventually. Better would be to make an extern (C) symbol 
in the loaded library.

 - Gregor Richards
September 10, 2007
Re: Cross-platform dynamic libraries (GNU/Linux, Mac OS X, Windows,...)
Gregor Richards wrote:

> Hugues De Keyzer wrote:
>> Another question: is it safe to directly load a mangled D symbol like
>> this:
>> 
>> void* test = dlsym(handle, "_D8mymodule8MyModule4testFZv");
>> 
>> for static void MyModyle.test()?
>> 
>> It works, but will it work an all platforms?
>> 
> 
> The mangling is the same on all platforms, so that should work, but is
> bound to break eventually. Better would be to make an extern (C) symbol
> in the loaded library.
> 
>   - Gregor Richards

There is also the .mangleof property.
September 10, 2007
Re: Cross-platform dynamic libraries (GNU/Linux, Mac OS X, Windows,...)
Tomas Lindquist Olsen wrote:
> Gregor Richards wrote:
> 
>> Hugues De Keyzer wrote:
>>> Another question: is it safe to directly load a mangled D symbol like
>>> this:
>>>
>>> void* test = dlsym(handle, "_D8mymodule8MyModule4testFZv");
>>>
>>> for static void MyModyle.test()?
>>>
>>> It works, but will it work an all platforms?
>>>
>> The mangling is the same on all platforms, so that should work, but is
>> bound to break eventually. Better would be to make an extern (C) symbol
>> in the loaded library.
>>
>>   - Gregor Richards
> 
> There is also the .mangleof property.

Yes, that's the reliable cross-platform etc etc way to do it. I was 
assuming he didn't necessarily have the .d[i] file corresponding to the 
library, but I guess that doesn't make any sense :)

 - Gregor Richards
Top | Discussion index | About this forum | D home