I have difficulties creating a Shared Object (.so) with D. Is it possible? Can I use classes defined in the library from the executable?

Here is my library file:
module test; // file "test.d"
export int testMe() { return 1; }
export class Test
{
    private int n;
    this(int i) { n = i; }
    int get() { return n; }
}

I compile like shown below:
$ dmd -fPIC -c test.d
$ gcc -shared -o libtest.so test.o

I have now a "libtest.so" and
$ nm libtest.so
shows following:

00000860 t
         U _D10ModuleInfo6__vtblZ
         U _D14TypeInfo_Class6__vtblZ
000020b4 V _D20TypeInfo_C4test4Test6__initZ
00002060 D _D4test12__ModuleInfoZ
00000890 T _D4test4Test3getMFZi
00000880 T _D4test4Test6__ctorMFiZC4test4Test
00000904 R _D4test4Test6__initZ
00000924 R _D4test4Test6__vtblZ
00002010 D _D4test4Test7__ClassZ
00000874 T _D4test6testMeFZi
         U _D6Object7__ClassZ
         U _D6object6Object5opCmpMFC6ObjectZi
         U _D6object6Object6toHashMFZk
         U _D6object6Object8opEqualsMFC6ObjectZb
         U _D6object6Object8toStringMFZAya
         U _D9ClassInfo6__vtblZ
         U _D9invariant12_d_invariantFC6ObjectZv
00001f18 a _DYNAMIC
         U _Dmodule_ref
00001ff4 a _GLOBAL_OFFSET_TABLE_
         w _Jv_RegisterClasses
00001f08 d __CTOR_END__
00001f00 d __CTOR_LIST__
00001f10 d __DTOR_END__
00001f0c d __DTOR_LIST__
00000944 r __FRAME_END__
00001f14 d __JCR_END__
00001f14 d __JCR_LIST__
000020c0 A __bss_start
         w __cxa_finalize@@GLIBC_2.1.3
000008b0 t __do_global_ctors_aux
000007a0 t __do_global_dtors_aux
0000200c d __dso_handle
         w __gmon_start__
00000857 t __i686.get_pc_thunk.bx
000020c0 A _edata
000020e0 A _end
000008e8 T _fini
00000724 T _init
000020c0 b completed.6635
000020c4 b dtor_idx.6637
00000820 t frame_dummy


And this is the program:
module main; // file "prog.d"
import std.stdio;
import test;
void main()
{
   writefln("testMe: %d", testMe());
   writefln("Test class: %d", (new Test(3)).get());
}

I compile it with:
$ dmd prog.d -L-L`pwd` -L-ltest

And get:
/usr/bin/ld: dynamic variable `_D4test4Test7__ClassZ' is zero size
/usr/bin/ld: prog.o(.text._Dmain+0x26): unresolvable R_386_32 relocation
against symbol `_D4test4Test7__ClassZ'
/usr/bin/ld: final link failed: Nonrepresentable section on output
collect2: ld returned 1 exit status
--- errorlevel 1

Please note that "_D4test4Test7__ClassZ" is defined in the library.

BTW compiling with following works:
$ dmd test.d prog.d

Thu, 30 Jul 2009 06:52:14 +0000 (UTC), teo wrote:

> I have difficulties creating a Shared Object (.so) with D. Is it possible? Can I use classes defined in the library from the executable?
> 
> Here is my library file:
> module test; // file "test.d"
> export int testMe() { return 1; }
> export class Test
> {
>     private int n;
>     this(int i) { n = i; }
>     int get() { return n; }
> }
> 
> I compile like shown below:
> $ dmd -fPIC -c test.d
> $ gcc -shared -o libtest.so test.o
> 
> [snip]
> 
> And this is the program:
> module main; // file "prog.d"
> import std.stdio;
> import test;
> void main()
> {
>    writefln("testMe: %d", testMe());
>    writefln("Test class: %d", (new Test(3)).get());
> }
> 
> I compile it with:
> $ dmd prog.d -L-L`pwd` -L-ltest
> 
> And get:
> /usr/bin/ld: dynamic variable `_D4test4Test7__ClassZ' is zero size
> /usr/bin/ld: prog.o(.text._Dmain+0x26): unresolvable R_386_32 relocation
> against symbol `_D4test4Test7__ClassZ'
> /usr/bin/ld: final link failed: Nonrepresentable section on output
> collect2: ld returned 1 exit status
> --- errorlevel 1
> 
> Please note that "_D4test4Test7__ClassZ" is defined in the library.
> 
> BTW compiling with following works:
> $ dmd test.d prog.d

I get the same results, i.e. it does not work.  Though with one clarification: from the command:

> $ gcc -shared -o libtest.so test.o

I get a warning:

> /usr/lib/gcc/i686-pc-linux-gnu/4.3.3/../../../../i686-pc-linux-gnu/bin/ld: warning: creating a DT_TEXTREL in object.

Internets say that this is because test.o is *not* position-independent. Could this mean that -fPIC switch does not work?  Could it be the reason linking fails?

Reproduced with both DMD 1.046 and 2.031.  GCC is 4.3.3 as you can see from the warning.

On Fri, 31 Jul 2009 18:47:29 +0400, Sergey Gromov wrote:

> Thu, 30 Jul 2009 06:52:14 +0000 (UTC), teo wrote:
> 
>> I have difficulties creating a Shared Object (.so) with D. Is it possible? Can I use classes defined in the library from the executable?
>> 
>> Here is my library file:
>> module test; // file "test.d"
>> export int testMe() { return 1; }
>> export class Test
>> {
>>     private int n;
>>     this(int i) { n = i; }
>>     int get() { return n; }
>> }
>> 
>> I compile like shown below:
>> $ dmd -fPIC -c test.d
>> $ gcc -shared -o libtest.so test.o
>> 
>> [snip]
>> 
>> And this is the program:
>> module main; // file "prog.d"
>> import std.stdio;
>> import test;
>> void main()
>> {
>>    writefln("testMe: %d", testMe());
>>    writefln("Test class: %d", (new Test(3)).get());
>> }
>> 
>> I compile it with:
>> $ dmd prog.d -L-L`pwd` -L-ltest
>> 
>> And get:
>> /usr/bin/ld: dynamic variable `_D4test4Test7__ClassZ' is zero size
>> /usr/bin/ld: prog.o(.text._Dmain+0x26): unresolvable R_386_32
>> relocation against symbol `_D4test4Test7__ClassZ' /usr/bin/ld: final
>> link failed: Nonrepresentable section on output collect2: ld returned 1
>> exit status
>> --- errorlevel 1
>> 
>> Please note that "_D4test4Test7__ClassZ" is defined in the library.
>> 
>> BTW compiling with following works:
>> $ dmd test.d prog.d
> 
> I get the same results, i.e. it does not work.  Though with one clarification: from the command:
> 
>> $ gcc -shared -o libtest.so test.o
> 
> I get a warning:
> 
>> /usr/lib/gcc/i686-pc-linux-gnu/4.3.3/../../../../i686-pc-linux-gnu/bin/
ld:
>> warning: creating a DT_TEXTREL in object.
> 
> Internets say that this is because test.o is *not* position-independent. Could this mean that -fPIC switch does not work?  Could it be the reason linking fails?
> 
> Reproduced with both DMD 1.046 and 2.031.  GCC is 4.3.3 as you can see from the warning.

I got it working:
0) setup
$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/lib

1) compile the library and move it in /opt/lib
$ dmd -fPIC -c test.d
$ gcc -shared -o libtest.so test.o
$ dmd -c -H -o- test.d

2) compile the program
$ dmd prog.d test.di -L-L/opt/lib -L-ltest

The output of ldd is:
$ ldd prog
	linux-gate.so.1 =>  (0xb8075000)
	libtest.so => /opt/lib/libtest.so (0xb8070000)
	libpthread.so.0 => /lib/tls/i686/cmov/libpthread.so.0 (0xb8053000)
	libm.so.6 => /lib/tls/i686/cmov/libm.so.6 (0xb802c000)
	libc.so.6 => /lib/tls/i686/cmov/libc.so.6 (0xb7ec9000)
	/lib/ld-linux.so.2 (0xb8076000)

My problem was that I thought that DMD will read the .so and will obtain required information from there, but it obviously needs the .di file.

You should try again - this above is the complete procedure.

Fri, 31 Jul 2009 18:50:28 +0000 (UTC), teo wrote:

> On Fri, 31 Jul 2009 18:47:29 +0400, Sergey Gromov wrote:
> 
>> Thu, 30 Jul 2009 06:52:14 +0000 (UTC), teo wrote:
>> 
>>> I have difficulties creating a Shared Object (.so) with D. Is it possible? Can I use classes defined in the library from the executable?
>>> 
>>> Here is my library file:
>>> module test; // file "test.d"
>>> export int testMe() { return 1; }
>>> export class Test
>>> {
>>>     private int n;
>>>     this(int i) { n = i; }
>>>     int get() { return n; }
>>> }
>>> 
>>> I compile like shown below:
>>> $ dmd -fPIC -c test.d
>>> $ gcc -shared -o libtest.so test.o
>>> 
>>> [snip]
>>> 
>>> And this is the program:
>>> module main; // file "prog.d"
>>> import std.stdio;
>>> import test;
>>> void main()
>>> {
>>>    writefln("testMe: %d", testMe());
>>>    writefln("Test class: %d", (new Test(3)).get());
>>> }
>>> 
>>> I compile it with:
>>> $ dmd prog.d -L-L`pwd` -L-ltest
>>> 
>>> And get:
>>> /usr/bin/ld: dynamic variable `_D4test4Test7__ClassZ' is zero size
>>> /usr/bin/ld: prog.o(.text._Dmain+0x26): unresolvable R_386_32
>>> relocation against symbol `_D4test4Test7__ClassZ' /usr/bin/ld: final
>>> link failed: Nonrepresentable section on output collect2: ld returned 1
>>> exit status
>>> --- errorlevel 1
>>> 
>>> Please note that "_D4test4Test7__ClassZ" is defined in the library.
>>> 
>>> BTW compiling with following works:
>>> $ dmd test.d prog.d
>> 
>> I get the same results, i.e. it does not work.  Though with one clarification: from the command:
>> 
>>> $ gcc -shared -o libtest.so test.o
>> 
>> I get a warning:
>> 
>>> /usr/lib/gcc/i686-pc-linux-gnu/4.3.3/../../../../i686-pc-linux-gnu/bin/
> ld:
>>> warning: creating a DT_TEXTREL in object.
>> 
>> Internets say that this is because test.o is *not* position-independent. Could this mean that -fPIC switch does not work?  Could it be the reason linking fails?
>> 
>> Reproduced with both DMD 1.046 and 2.031.  GCC is 4.3.3 as you can see from the warning.
> 
> I got it working:
> 0) setup
> $ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/lib
> 
> 1) compile the library and move it in /opt/lib
> $ dmd -fPIC -c test.d
> $ gcc -shared -o libtest.so test.o
> $ dmd -c -H -o- test.d
> 
> 2) compile the program
> $ dmd prog.d test.di -L-L/opt/lib -L-ltest
> 
> The output of ldd is:
> $ ldd prog
> 	linux-gate.so.1 =>  (0xb8075000)
> 	libtest.so => /opt/lib/libtest.so (0xb8070000)
> 	libpthread.so.0 => /lib/tls/i686/cmov/libpthread.so.0 (0xb8053000)
> 	libm.so.6 => /lib/tls/i686/cmov/libm.so.6 (0xb802c000)
> 	libc.so.6 => /lib/tls/i686/cmov/libc.so.6 (0xb7ec9000)
> 	/lib/ld-linux.so.2 (0xb8076000)
> 
> My problem was that I thought that DMD will read the .so and will obtain required information from there, but it obviously needs the .di file.
> 
> You should try again - this above is the complete procedure.

My guess is that test.di is exactly the same as test.d because all the functions are small.  Therefore compiling 'dmd prog.d test.di' resolves all symbols statically and the .so is simply not linked.  Does your 'prog' have any unresolved symbols in its symbol table?

On Sun, 02 Aug 2009 02:18:28 +0400, Sergey Gromov wrote:

> My guess is that test.di is exactly the same as test.d because all the functions are small.  Therefore compiling 'dmd prog.d test.di' resolves all symbols statically and the .so is simply not linked.  Does your 'prog' have any unresolved symbols in its symbol table?

It looks like this is the case. I found following in the symbols:
0804b8cc T _D4test6testMeFZi
0804b8d8 T _D4test9testClassFiZC4test4Test

The whole nm output is too long, but if you want I can send it to you.

When I use the *--undefined-only* option I get:
$ nm -u prog
         w _Jv_RegisterClasses
         U __assert_fail@@GLIBC_2.0
         U __cxa_atexit@@GLIBC_2.1.3
         U __errno_location@@GLIBC_2.0
         w __gmon_start__
         U __libc_start_main@@GLIBC_2.0
         U _main
         U _pthread_cleanup_push@@GLIBC_2.0
         U calloc@@GLIBC_2.0
         U clearerr@@GLIBC_2.0
         U fclose@@GLIBC_2.1
         U feof@@GLIBC_2.0
         U ferror@@GLIBC_2.0
         U fflush@@GLIBC_2.0
         U fgetwc_unlocked@@GLIBC_2.2
         U fileno@@GLIBC_2.0
         U flockfile@@GLIBC_2.0
         U fopen64@@GLIBC_2.1
         U fputc_unlocked@@GLIBC_2.0
         U fputwc_unlocked@@GLIBC_2.2
         U free@@GLIBC_2.0
         U fseek@@GLIBC_2.0
         U ftell@@GLIBC_2.0
         U funlockfile@@GLIBC_2.0
         U fwide@@GLIBC_2.2
         U fwrite@@GLIBC_2.0
         U getdelim@@GLIBC_2.0
         U malloc@@GLIBC_2.0
         U memchr@@GLIBC_2.0
         U memcmp@@GLIBC_2.0
         U memcpy@@GLIBC_2.0
         U memmove@@GLIBC_2.0
         U memset@@GLIBC_2.0
         U mmap@@GLIBC_2.0
         U munmap@@GLIBC_2.0
         U nanosleep@@GLIBC_2.0
         U popen@@GLIBC_2.1
         U printf@@GLIBC_2.0
         U pthread_attr_init@@GLIBC_2.1
         U pthread_attr_setdetachstate@@GLIBC_2.0
         U pthread_attr_setstacksize@@GLIBC_2.1
         U pthread_create@@GLIBC_2.1
         U pthread_detach@@GLIBC_2.0
         U pthread_getschedparam@@GLIBC_2.0
         U pthread_getspecific@@GLIBC_2.0
         U pthread_join@@GLIBC_2.0
         U pthread_key_create@@GLIBC_2.0
         U pthread_kill@@GLIBC_2.0
         U pthread_mutex_destroy@@GLIBC_2.0
         U pthread_mutex_init@@GLIBC_2.0
         U pthread_mutex_lock@@GLIBC_2.0
         U pthread_mutex_unlock@@GLIBC_2.0
         U pthread_mutexattr_destroy@@GLIBC_2.0
         U pthread_mutexattr_init@@GLIBC_2.0
         U pthread_mutexattr_settype@@GLIBC_2.1
         U pthread_self@@GLIBC_2.0
         U pthread_setschedparam@@GLIBC_2.0
         U pthread_setspecific@@GLIBC_2.0
         U realloc@@GLIBC_2.0
         U rewind@@GLIBC_2.0
         U sched_get_priority_max@@GLIBC_2.0
         U sched_get_priority_min@@GLIBC_2.0
         U sched_yield@@GLIBC_2.0
         U sem_init@@GLIBC_2.1
         U sem_post@@GLIBC_2.1
         U sem_wait@@GLIBC_2.1
         U setvbuf@@GLIBC_2.0
         U sigaction@@GLIBC_2.0
         U sigdelset@@GLIBC_2.0
         U sigfillset@@GLIBC_2.0
         U sigsuspend@@GLIBC_2.0
         U strerror@@GLIBC_2.0
         U strerror_r@@GLIBC_2.0
         U strlen@@GLIBC_2.0
         U sysconf@@GLIBC_2.0
         U tmpfile@@GLIBC_2.1
         U vsnprintf@@GLIBC_2.0
         U write@@GLIBC_2.0

Sun, 2 Aug 2009 11:18:24 +0000 (UTC), teo wrote:

> On Sun, 02 Aug 2009 02:18:28 +0400, Sergey Gromov wrote:
> 
>> My guess is that test.di is exactly the same as test.d because all the functions are small.  Therefore compiling 'dmd prog.d test.di' resolves all symbols statically and the .so is simply not linked.  Does your 'prog' have any unresolved symbols in its symbol table?
> 
> It looks like this is the case. I found following in the symbols:
> 0804b8cc T _D4test6testMeFZi
> 0804b8d8 T _D4test9testClassFiZC4test4Test
> 
> The whole nm output is too long, but if you want I can send it to you.
> 
> When I use the *--undefined-only* option I get:
> $ nm -u prog
> [snip]

All unresolved symbols seem to be in glibc.

I've filed a bug: http://d.puremagic.com/issues/show_bug.cgi?id=3226

On Tue, 04 Aug 2009 18:41:50 +0400, Sergey Gromov wrote:

> Sun, 2 Aug 2009 11:18:24 +0000 (UTC), teo wrote:
> 
>> On Sun, 02 Aug 2009 02:18:28 +0400, Sergey Gromov wrote:
>> 
>>> My guess is that test.di is exactly the same as test.d because all the functions are small.  Therefore compiling 'dmd prog.d test.di' resolves all symbols statically and the .so is simply not linked. Does your 'prog' have any unresolved symbols in its symbol table?
>> 
>> It looks like this is the case. I found following in the symbols:
>> 0804b8cc T _D4test6testMeFZi
>> 0804b8d8 T _D4test9testClassFiZC4test4Test
>> 
>> The whole nm output is too long, but if you want I can send it to you.
>> 
>> When I use the *--undefined-only* option I get: $ nm -u prog [snip]
> 
> All unresolved symbols seem to be in glibc.
> 
Yes.

> I've filed a bug: http://d.puremagic.com/issues/show_bug.cgi?id=3226

I don't have the warning you mention.

Wed, 5 Aug 2009 20:46:53 +0000 (UTC), teo wrote:

> On Tue, 04 Aug 2009 18:41:50 +0400, Sergey Gromov wrote:
> 
>> Sun, 2 Aug 2009 11:18:24 +0000 (UTC), teo wrote:
>> 
>>> On Sun, 02 Aug 2009 02:18:28 +0400, Sergey Gromov wrote:
>>> 
>>>> My guess is that test.di is exactly the same as test.d because all the functions are small.  Therefore compiling 'dmd prog.d test.di' resolves all symbols statically and the .so is simply not linked. Does your 'prog' have any unresolved symbols in its symbol table?
>>> 
>>> It looks like this is the case. I found following in the symbols:
>>> 0804b8cc T _D4test6testMeFZi
>>> 0804b8d8 T _D4test9testClassFiZC4test4Test
>>> 
>>> The whole nm output is too long, but if you want I can send it to you.
>>> 
>>> When I use the *--undefined-only* option I get: $ nm -u prog [snip]
>> 
>> All unresolved symbols seem to be in glibc.
>> 
> Yes.
> 
>> I've filed a bug: http://d.puremagic.com/issues/show_bug.cgi?id=3226
> 
> I don't have the warning you mention.

Try to run

objdump -r test.o

It'll print relocation table for the object file.  In my case almost all of them are of type R_386_32 which are load-time relocations.  A position-independent code should not contain them.

On Thu, 06 Aug 2009 04:24:35 +0400, Sergey Gromov wrote:

> Wed, 5 Aug 2009 20:46:53 +0000 (UTC), teo wrote:
> 
>> On Tue, 04 Aug 2009 18:41:50 +0400, Sergey Gromov wrote:
>> 
>>> Sun, 2 Aug 2009 11:18:24 +0000 (UTC), teo wrote:
>>> 
>>>> On Sun, 02 Aug 2009 02:18:28 +0400, Sergey Gromov wrote:
>>>> 
>>>>> My guess is that test.di is exactly the same as test.d because all the functions are small.  Therefore compiling 'dmd prog.d test.di' resolves all symbols statically and the .so is simply not linked. Does your 'prog' have any unresolved symbols in its symbol table?
>>>> 
>>>> It looks like this is the case. I found following in the symbols:
>>>> 0804b8cc T _D4test6testMeFZi
>>>> 0804b8d8 T _D4test9testClassFiZC4test4Test
>>>> 
>>>> The whole nm output is too long, but if you want I can send it to you.
>>>> 
>>>> When I use the *--undefined-only* option I get: $ nm -u prog [snip]
>>> 
>>> All unresolved symbols seem to be in glibc.
>>> 
>> Yes.
>> 
>>> I've filed a bug: http://d.puremagic.com/issues/show_bug.cgi?id=3226
>> 
>> I don't have the warning you mention.
> 
> Try to run
> 
> objdump -r test.o
> 
> It'll print relocation table for the object file.  In my case almost all of them are of type R_386_32 which are load-time relocations.  A position-independent code should not contain them.

Correct. This is exactly the case. And my environment is identical to
yours:
$ dmd | head -n1
Digital Mars D Compiler v2.031
$ gcc --version | head -n1
gcc (Ubuntu 4.3.3-5ubuntu4) 4.3.3
$ uname -a
Linux dev-phobos 2.6.28-14-server #47-Ubuntu SMP Sat Jul 25 01:18:34 UTC
2009 i686 GNU/Linux

So it is reproducible with DMD2 as well.

teo 写道:
> I have difficulties creating a Shared Object (.so) with D. Is it possible? Can I use classes defined in the library from the executable?
> 

All the steps have been done successfully on FreeBSD 7.2 with DMD v1.046 + gcc v  4.2.1 20070719.

The running of this line
    writefln("testMe: %d", testMe());
is OK.

However, when running this line
  writefln("Test class: %d", (new Test(3)).get());
I got core dumped. The dumped file is so huge nearly 1G.

Maybe I should report this bug?