Tried this on D.gnu but didn't get an answer.  Any feedback here?
------------------------------------------------------------------


From what I read on newsgroups, this should be possible with gdc on linux?  Goal is a plugin type solution.  With the following code, I compile intf.d and lib.d together into a shared library, and intf.d and main.d into an executable.  Compiles/links fine.  When I run it, I get a message like:

Null library handle: libxyz.so.1.0.1: undefined symbol: __data_start

I compiled the .d files with -fPIC and the .so with:
gcc -shared -Wl,-soname,libxyz.so.1 -fPIC.

Is this definitely possible, and if so, what am I doing wrong?  I tried gdc 0.24 and the latest off svn with the same results.

TIA.

-------------------------------------------------------------

intf.d:
=========
module intf;
public interface I { public int getId(); }

lib.d:
=========
module lib;
import intf;
class A : I { public int getId() { return 42; } }
extern (C) { I getObj() { return new A(); } }

main.d:
==========
...
void main(char[][] args)
{
    void* hnd = dlopen(toStringz("libxyz.so.1.0.1"), RTLD_NOW);
    if (hnd == null) {
        printf("Null library handle: %s\n", dlerror());
        return 0;
    }
    ...
}

BB wrote:
> Tried this on D.gnu but didn't get an answer.  Any feedback here?

Shared libraries are also supported by the dmd compiler (the -fPIC switch is implemented), but not by phobos, simply because nobody has gotten around to doing it.

To compile a shared library, Linux using gdc:

gdmd -op -oflibxyz.so.1.0.1 lib.d -fPIC -q,-rdynamic,-shared
gdmd -op main.d -fPIC -q,-rdynamic -L-ldl

Compiling a shared library on Windows is a bit more complicated (although possible with dmd and phobos as they are.)  It involves DllMain and def files...

Please note, if you have selinux you may have have to run chcon.

-[Unknown]


BB wrote:
> Tried this on D.gnu but didn't get an answer.  Any feedback here?
> ------------------------------------------------------------------
> 
> 
>  From what I read on newsgroups, this should be possible with gdc on linux?  Goal is a plugin type solution.  With the following code, I compile intf.d and lib.d together into a shared library, and intf.d and main.d into an executable.  Compiles/links fine.  When I run it, I get a message like:
> 
> Null library handle: libxyz.so.1.0.1: undefined symbol: __data_start
> 
> I compiled the .d files with -fPIC and the .so with:
> gcc -shared -Wl,-soname,libxyz.so.1 -fPIC.
> 
> Is this definitely possible, and if so, what am I doing wrong?  I tried gdc 0.24 and the latest off svn with the same results.
> 
> TIA.
> 
> -------------------------------------------------------------
> 
> intf.d:
> =========
> module intf;
> public interface I { public int getId(); }
> 
> lib.d:
> =========
> module lib;
> import intf;
> class A : I { public int getId() { return 42; } }
> extern (C) { I getObj() { return new A(); } }
> 
> main.d:
> ==========
> ...
> void main(char[][] args)
> {
>     void* hnd = dlopen(toStringz("libxyz.so.1.0.1"), RTLD_NOW);
>     if (hnd == null) {
>         printf("Null library handle: %s\n", dlerror());
>         return 0;
>     }
>     ...
> }
> 
> 
> 

Shared libraries are quite popular.  To be honest, I'm no expert with them or I would try my hand (I actually did a bit, but I really don't have much experience with linking problems.)

Would you have any suggestions on what one might do to better understand the problems Phobos currently has with supporting shared libraries?  If I understood the problems, I would be happy to work on a solution.

Also, since it's a bit related - what is your opinion on the concept of ddl?

It (the concept; ddl doesn't work at all for me no matter what I try, and seems overcomplicated in ways) seems like a perfect way to bridge yet another gap between D and other popular dynamic languages like Ruby, Python, Perl, PHP, etc.: ease of importing/using shared code dynamically.

-[Unknown]


Walter Bright wrote:
> BB wrote:
>> Tried this on D.gnu but didn't get an answer.  Any feedback here?
> 
> Shared libraries are also supported by the dmd compiler (the -fPIC switch is implemented), but not by phobos, simply because nobody has gotten around to doing it.

Thanks very much, that worked.

After being able to call into the .so, I see a segmentation fault at the end of main().  Through gdb I see it's crashing in _d_callfinalizer(). Locally explicitly deleting the object allocated by the .so also results in a sigsegv.  However, calling into the .so to delete the object works fine.

What is the reasoning for this - is it that the gc is statically linked into both the .so and the executable, so sharing a reference causes the local gc to try and clean it up?  If so, what are the guidelines/rules to use?

Thanks again.


Unknown W. Brackets wrote:
> To compile a shared library, Linux using gdc:
> 
> gdmd -op -oflibxyz.so.1.0.1 lib.d -fPIC -q,-rdynamic,-shared
> gdmd -op main.d -fPIC -q,-rdynamic -L-ldl
> 
> Compiling a shared library on Windows is a bit more complicated (although possible with dmd and phobos as they are.) It involves DllMain and def files...
> 
> Please note, if you have selinux you may have have to run chcon.
> 
> -[Unknown]
> 
> 
> BB wrote:
>> Tried this on D.gnu but didn't get an answer. Any feedback here?
>> ------------------------------------------------------------------
>>
>>
>> From what I read on newsgroups, this should be possible with gdc on linux? Goal is a plugin type solution. With the following code, I compile intf.d and lib.d together into a shared library, and intf.d and main.d into an executable. Compiles/links fine. When I run it, I get a message like:
>>
>> Null library handle: libxyz.so.1.0.1: undefined symbol: __data_start
>>
>> I compiled the .d files with -fPIC and the .so with:
>> gcc -shared -Wl,-soname,libxyz.so.1 -fPIC.
>>
>> Is this definitely possible, and if so, what am I doing wrong? I tried gdc 0.24 and the latest off svn with the same results.
>>
>> TIA.
>>
>> -------------------------------------------------------------
>>
>> intf.d:
>> =========
>> module intf;
>> public interface I { public int getId(); }
>>
>> lib.d:
>> =========
>> module lib;
>> import intf;
>> class A : I { public int getId() { return 42; } }
>> extern (C) { I getObj() { return new A(); } }
>>
>> main.d:
>> ==========
>> ...
>> void main(char[][] args)
>> {
>> void* hnd = dlopen(toStringz("libxyz.so.1.0.1"), RTLD_NOW);
>> if (hnd == null) {
>> printf("Null library handle: %s\n", dlerror());
>> return 0;
>> }
>> ...
>> }
>>
>>
>>

Assuming you just want to create a shared library of your own, initialization of
the runtime is an issue.  I believe there's a doc page somewhere that describes
how to do this for Phobos, and Tango has rt_init() ad rt_term() routines for this
purpose.  If the issue is putting just Phobos into a shared lib, changes would
have to be made for how the GC is exposed, and on Windows most things
would need to be exported as well.  This is a pretty big and nasty job (for
Win32 anyway).

== Quote from Unknown W. Brackets (unknown@simplemachines.org)'s article
> Shared libraries are quite popular.  To be honest, I'm no expert with
> them or I would try my hand (I actually did a bit, but I really don't
> have much experience with linking problems.)
> Would you have any suggestions on what one might do to better understand
> the problems Phobos currently has with supporting shared libraries?  If
> I understood the problems, I would be happy to work on a solution.
> Also, since it's a bit related - what is your opinion on the concept of ddl?
> It (the concept; ddl doesn't work at all for me no matter what I try,
> and seems overcomplicated in ways) seems like a perfect way to bridge
> yet another gap between D and other popular dynamic languages like Ruby,
> Python, Perl, PHP, etc.: ease of importing/using shared code dynamically.
> -[Unknown]
> Walter Bright wrote:
> > BB wrote:
> >> Tried this on D.gnu but didn't get an answer.  Any feedback here?
> >
> > Shared libraries are also supported by the dmd compiler (the -fPIC switch is implemented), but not by phobos, simply because nobody has gotten around to doing it.

Did you initialize the runtime within the shared library?  As for sharing data between a D shared library and external code, if you want to pass ownership of GCed data to the external code then the GC will have to know either that it should just never collect the data or have a means of scanning external thread stacks.  Tango opted for the second approach, and you can register external threads via thread_attachThis() and thread_detachThis() -- both are declared in the Thread module.

== Quote from BB (b.buderman@gmail.com)'s article
> Thanks very much, that worked.
> After being able to call into the .so, I see a segmentation fault at the
> end of main().  Through gdb I see it's crashing in _d_callfinalizer().
> Locally explicitly deleting the object allocated by the .so also results
> in a sigsegv.  However, calling into the .so to delete the object works
> fine.
> What is the reasoning for this - is it that the gc is statically linked
> into both the .so and the executable, so sharing a reference causes the
> local gc to try and clean it up?  If so, what are the guidelines/rules
> to use?
> Thanks again.
> Unknown W. Brackets wrote:
> > To compile a shared library, Linux using gdc:
> >
> > gdmd -op -oflibxyz.so.1.0.1 lib.d -fPIC -q,-rdynamic,-shared gdmd -op main.d -fPIC -q,-rdynamic -L-ldl
> >
> > Compiling a shared library on Windows is a bit more complicated (although possible with dmd and phobos as they are.) It involves DllMain and def files...
> >
> > Please note, if you have selinux you may have have to run chcon.
> >
> > -[Unknown]
> >
> >
> > BB wrote:
> >> Tried this on D.gnu but didn't get an answer. Any feedback here?
> >> ------------------------------------------------------------------
> >>
> >>
> >> From what I read on newsgroups, this should be possible with gdc on linux? Goal is a plugin type solution. With the following code, I compile intf.d and lib.d together into a shared library, and intf.d and main.d into an executable. Compiles/links fine. When I run it, I get a message like:
> >>
> >> Null library handle: libxyz.so.1.0.1: undefined symbol: __data_start
> >>
> >> I compiled the .d files with -fPIC and the .so with:
> >> gcc -shared -Wl,-soname,libxyz.so.1 -fPIC.
> >>
> >> Is this definitely possible, and if so, what am I doing wrong? I tried gdc 0.24 and the latest off svn with the same results.
> >>
> >> TIA.
> >>
> >> -------------------------------------------------------------
> >>
> >> intf.d:
> >> =========
> >> module intf;
> >> public interface I { public int getId(); }
> >>
> >> lib.d:
> >> =========
> >> module lib;
> >> import intf;
> >> class A : I { public int getId() { return 42; } }
> >> extern (C) { I getObj() { return new A(); } }
> >>
> >> main.d:
> >> ==========
> >> ...
> >> void main(char[][] args)
> >> {
> >> void* hnd = dlopen(toStringz("libxyz.so.1.0.1"), RTLD_NOW);
> >> if (hnd == null) {
> >> printf("Null library handle: %s\n", dlerror());
> >> return 0;
> >> }
> >> ...
> >> }
> >>
> >>
> >>

Windows is a huge ordeal.  I'm more interested right now in fixing it on Linux (I do not write many Windows programs anyway, and the differences between so and dll are many.)

I realize there are runtime issues, but I'm concerned mainly about the reason why dmd on Linux is unable to create shared objects, while gdc can (although there are many complications.)

-[Unknown]


Sean Kelly wrote:
> Assuming you just want to create a shared library of your own, initialization of
> the runtime is an issue.  I believe there's a doc page somewhere that describes
> how to do this for Phobos, and Tango has rt_init() ad rt_term() routines for this
> purpose.  If the issue is putting just Phobos into a shared lib, changes would
> have to be made for how the GC is exposed, and on Windows most things
> would need to be exported as well.  This is a pretty big and nasty job (for
> Win32 anyway).
> 
> == Quote from Unknown W. Brackets (unknown@simplemachines.org)'s article
>> Shared libraries are quite popular.  To be honest, I'm no expert with
>> them or I would try my hand (I actually did a bit, but I really don't
>> have much experience with linking problems.)
>> Would you have any suggestions on what one might do to better understand
>> the problems Phobos currently has with supporting shared libraries?  If
>> I understood the problems, I would be happy to work on a solution.
>> Also, since it's a bit related - what is your opinion on the concept of ddl?
>> It (the concept; ddl doesn't work at all for me no matter what I try,
>> and seems overcomplicated in ways) seems like a perfect way to bridge
>> yet another gap between D and other popular dynamic languages like Ruby,
>> Python, Perl, PHP, etc.: ease of importing/using shared code dynamically.
>> -[Unknown]
>> Walter Bright wrote:
>>> BB wrote:
>>>> Tried this on D.gnu but didn't get an answer.  Any feedback here?
>>> Shared libraries are also supported by the dmd compiler (the -fPIC
>>> switch is implemented), but not by phobos, simply because nobody has
>>> gotten around to doing it.
> 
> 

I seem to remember they, essentially, share a GC.  I do think they can get confused/trip over each other if you're not careful.

Generally, I've just had two functions I call into the so with, which init/deinit the library.  To reduce complications and problems like you're seeing, I keep the plugins as simple as possible.

IIRC, they share the same gc handle already.  It's possible they both may run the gc when they shutdown.  I'd use the functions in std.gc and assembler output to debug...

Unfortunately this is one area where D (mostly in its available tools) is lacking currently...

-[Unknown]


BB wrote:
> Thanks very much, that worked.
> 
> After being able to call into the .so, I see a segmentation fault at the end of main().  Through gdb I see it's crashing in _d_callfinalizer(). Locally explicitly deleting the object allocated by the .so also results in a sigsegv.  However, calling into the .so to delete the object works fine.
> 
> What is the reasoning for this - is it that the gc is statically linked into both the .so and the executable, so sharing a reference causes the local gc to try and clean it up?  If so, what are the guidelines/rules to use?
> 
> Thanks again.
> 
> 
> Unknown W. Brackets wrote:
>> To compile a shared library, Linux using gdc:
>>
>> gdmd -op -oflibxyz.so.1.0.1 lib.d -fPIC -q,-rdynamic,-shared
>> gdmd -op main.d -fPIC -q,-rdynamic -L-ldl
>>
>> Compiling a shared library on Windows is a bit more complicated (although possible with dmd and phobos as they are.) It involves DllMain and def files...
>>
>> Please note, if you have selinux you may have have to run chcon.
>>
>> -[Unknown]
>>
>>
>> BB wrote:
>>> Tried this on D.gnu but didn't get an answer. Any feedback here?
>>> ------------------------------------------------------------------
>>>
>>>
>>> From what I read on newsgroups, this should be possible with gdc on linux? Goal is a plugin type solution. With the following code, I compile intf.d and lib.d together into a shared library, and intf.d and main.d into an executable. Compiles/links fine. When I run it, I get a message like:
>>>
>>> Null library handle: libxyz.so.1.0.1: undefined symbol: __data_start
>>>
>>> I compiled the .d files with -fPIC and the .so with:
>>> gcc -shared -Wl,-soname,libxyz.so.1 -fPIC.
>>>
>>> Is this definitely possible, and if so, what am I doing wrong? I tried gdc 0.24 and the latest off svn with the same results.
>>>
>>> TIA.
>>>
>>> -------------------------------------------------------------
>>>
>>> intf.d:
>>> =========
>>> module intf;
>>> public interface I { public int getId(); }
>>>
>>> lib.d:
>>> =========
>>> module lib;
>>> import intf;
>>> class A : I { public int getId() { return 42; } }
>>> extern (C) { I getObj() { return new A(); } }
>>>
>>> main.d:
>>> ==========
>>> ...
>>> void main(char[][] args)
>>> {
>>> void* hnd = dlopen(toStringz("libxyz.so.1.0.1"), RTLD_NOW);
>>> if (hnd == null) {
>>> printf("Null library handle: %s\n", dlerror());
>>> return 0;
>>> }
>>> ...
>>> }
>>>
>>>
>>>
> 
> 

Unknown W. Brackets wrote:
> Shared libraries are quite popular.  To be honest, I'm no expert with them or I would try my hand (I actually did a bit, but I really don't have much experience with linking problems.)
> 
> Would you have any suggestions on what one might do to better understand the problems Phobos currently has with supporting shared libraries?  If I understood the problems, I would be happy to work on a solution.


I'd start by examining how DLL support is done for Win32 (i.e. having a shared gc and threading system).


> Also, since it's a bit related - what is your opinion on the concept of ddl?
> 
> It (the concept; ddl doesn't work at all for me no matter what I try, and seems overcomplicated in ways) seems like a perfect way to bridge yet another gap between D and other popular dynamic languages like Ruby, Python, Perl, PHP, etc.: ease of importing/using shared code dynamically.

I don't know anything about ddl.