Hi,

I can't look at this today as I am at Groovy and Grails eXchange 2012 and have to give a talk which needs some reworking in the light of happenings yesterday.  Ping me a couple of times over the weekend to make sure I get this looked into by Monday morning.

Thanks.

On Thu, 2012-12-13 at 14:49 -0800, H. S. Teoh wrote:
> Hi Russel,
> 
> I've been using your BitBucket scons_d_tooling version of SCons for my D projects, and it's been great! However, I needed to make a static library today and I'm having some trouble with it.  Here's a reduced testcase:
> 
> 	env = Environment(
> 		DC = '/usr/src/d/dmd/src/dmd'
> 	)
> 	env.StaticLibrary('mylib', Split("""
> 		a.d
> 		b.d
> 	"""))
> 
> Here's the output:
> 
> 	scons: Reading SConscript files ...
> 	scons: done reading SConscript files.
> 	scons: Building targets ...
> 	/usr/src/d/dmd/src/dmd -I. -c -ofa.o a.d
> 	/usr/src/d/dmd/src/dmd -I. -c -ofb.o b.d
> 	lib -c libmylib.a a.o b.o
> 	sh: 1: lib: not found
> 	scons: *** [libmylib.a] Error 127
> 	scons: building terminated because of errors.
> 
> The compilation steps work fine, but when it should be running ar to create the library archive, it runs a non-existent 'lib' instead, which fails.
> 
> I've tracked down the problem to the presub command $SMART_ARCOM, but it appears to be a function disguised as a magical variable, so I've no idea how to go further.
> 
> Am I missing some setting in the Environment? How can I convince it to use 'ar' (as it should) instead of 'lib'?
> 
> Thanks!
> 
> 
> T
> 

-- 
Russel. ============================================================================= Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder@ekiga.net 41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel@winder.org.uk London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder

On Thu, 2012-12-13 at 14:49 -0800, H. S. Teoh wrote:
> Hi Russel,
> 
> I've been using your BitBucket scons_d_tooling version of SCons for my D projects, and it's been great! However, I needed to make a static library today and I'm having some trouble with it.  Here's a reduced testcase:
> 
> 	env = Environment(
> 		DC = '/usr/src/d/dmd/src/dmd'
> 	)
> 	env.StaticLibrary('mylib', Split("""
> 		a.d
> 		b.d
> 	"""))

You are the first person to try doing this since I severely restructured the code!

> Here's the output:
> 
> 	scons: Reading SConscript files ...
> 	scons: done reading SConscript files.
> 	scons: Building targets ...
> 	/usr/src/d/dmd/src/dmd -I. -c -ofa.o a.d
> 	/usr/src/d/dmd/src/dmd -I. -c -ofb.o b.d
> 	lib -c libmylib.a a.o b.o
> 	sh: 1: lib: not found
> 	scons: *** [libmylib.a] Error 127
> 	scons: building terminated because of errors.
> 
> The compilation steps work fine, but when it should be running ar to create the library archive, it runs a non-existent 'lib' instead, which fails.

lib is the Windows equivalent of ar, I failed to take this into account.

> I've tracked down the problem to the presub command $SMART_ARCOM, but it appears to be a function disguised as a magical variable, so I've no idea how to go further.
> 
> Am I missing some setting in the Environment? How can I convince it to use 'ar' (as it should) instead of 'lib'?

I entered a variant of your code as a test and made the smallest possible change to make things work on Linux. This is untested on OS X or Windows.

The LDC test fails for reasons I cannot suss just now, the DMD and GDC tests pass.

I have no doubt this is a hack patch, it all needs to be sorted properly.

-- 
Russel. ============================================================================= Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder@ekiga.net 41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel@winder.org.uk London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder

On Sat, Dec 15, 2012 at 06:42:45PM +0000, Russel Winder wrote:
> On Thu, 2012-12-13 at 14:49 -0800, H. S. Teoh wrote:
> > Hi Russel,
> > 
> > I've been using your BitBucket scons_d_tooling version of SCons for my D projects, and it's been great! However, I needed to make a static library today and I'm having some trouble with it.  Here's a reduced testcase:
[...]
[...]
> I entered a variant of your code as a test and made the smallest possible change to make things work on Linux. This is untested on OS X or Windows.
> 
> The LDC test fails for reasons I cannot suss just now, the DMD and GDC tests pass.
> 
> I have no doubt this is a hack patch, it all needs to be sorted properly.
[...]

Thanks! I can build the static library now.

But I have trouble when I try to link to it. For some reason, the dmd link command isn't picking up the value of LIBPATH, so the linker can't find the library. Here's a reduced test case:

	#!/usr/src/scons/russel/scons_d_tooling/bootstrap.py -f
	env = Environment(
		DC = '/usr/src/d/dmd/src/dmd',
	)
	env.Library('mylib', 'mylib.d')

	prog_env = env.Clone(
		LIBS = ['mylib'],
		LIBPATH = '#'
	)
	prog_env.Program('prog', 'prog.d')

Output:

	scons: Reading SConscript files ...
	scons: done reading SConscript files.
	scons: Building targets ...
	/usr/src/d/dmd/src/dmd -I. -c -ofmylib.o mylib.d
	ar cr libmylib.a mylib.o
	ranlib libmylib.a
	/usr/src/d/dmd/src/dmd -I. -c -ofprog.o prog.d
	/usr/src/d/dmd/src/dmd -ofprog prog.o -L-lmylib
	/usr/bin/ld: cannot find -lmylib
	collect2: error: ld returned 1 exit status
	--- errorlevel 1
	scons: *** [prog] Error 1
	scons: building terminated because of errors.

This works correctly when using the C compiler (SCons correctly inserts
a "-L." in the link command).

//

Also, an unrelated issue: if DC isn't specified and no D compiler is found in $PATH, it produces a rather odd command line:

	scons: Reading SConscript files ...
	scons: done reading SConscript files.
	scons: Building targets ...
	I. -c -ofprog.o prog.d
	sh: 1: I.: not found
	ofprog prog.o -L-lmylib
	sh: 1: ofprog: not found
	scons: done building targets.

This is not a big deal, but it'd be nice if the tool gave a more helpful message along the lines of "I can't find a D compiler, please specify one", instead of producing a mangled command. :-)


T

-- 
It's amazing how careful choice of punctuation can leave you hanging:

On Sat, 2012-12-15 at 11:19 -0800, H. S. Teoh wrote:
[…]
> But I have trouble when I try to link to it. For some reason, the dmd link command isn't picking up the value of LIBPATH, so the linker can't find the library. Here's a reduced test case:
> 
> 	#!/usr/src/scons/russel/scons_d_tooling/bootstrap.py -f
> 	env = Environment(
> 		DC = '/usr/src/d/dmd/src/dmd',
> 	)
> 	env.Library('mylib', 'mylib.d')
> 
> 	prog_env = env.Clone(
> 		LIBS = ['mylib'],
> 		LIBPATH = '#'
> 	)
> 	prog_env.Program('prog', 'prog.d')
> 
> Output:
> 
> 	scons: Reading SConscript files ...
> 	scons: done reading SConscript files.
> 	scons: Building targets ...
> 	/usr/src/d/dmd/src/dmd -I. -c -ofmylib.o mylib.d
> 	ar cr libmylib.a mylib.o
> 	ranlib libmylib.a
> 	/usr/src/d/dmd/src/dmd -I. -c -ofprog.o prog.d
> 	/usr/src/d/dmd/src/dmd -ofprog prog.o -L-lmylib
> 	/usr/bin/ld: cannot find -lmylib
> 	collect2: error: ld returned 1 exit status
> 	--- errorlevel 1
> 	scons: *** [prog] Error 1
> 	scons: building terminated because of errors.
> 
> This works correctly when using the C compiler (SCons correctly inserts
> a "-L." in the link command).

Can you set out that case and the correct command line? I have to admit DMD/LDC flags often confuse me, I must be a GDC/GCC person ;-)

> //
> 
> Also, an unrelated issue: if DC isn't specified and no D compiler is found in $PATH, it produces a rather odd command line:
> 
> 	scons: Reading SConscript files ...
> 	scons: done reading SConscript files.
> 	scons: Building targets ...
> 	I. -c -ofprog.o prog.d
> 	sh: 1: I.: not found
> 	ofprog prog.o -L-lmylib
> 	sh: 1: ofprog: not found
> 	scons: done building targets.
> 
> This is not a big deal, but it'd be nice if the tool gave a more helpful message along the lines of "I can't find a D compiler, please specify one", instead of producing a mangled command. :-)

Actually this is a big deal. If none of dmd, gdc, ldc2 are found then the build should fails before trying a command.  Isn't this why exists exists.  Also the generate should fail if there is no D compiler.

-- 
Russel. ============================================================================= Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder@ekiga.net 41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel@winder.org.uk London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder

On Mon, Dec 17, 2012 at 06:32:57PM +0000, Russel Winder wrote:
> On Sat, 2012-12-15 at 11:19 -0800, H. S. Teoh wrote:
> […]
> > But I have trouble when I try to link to it. For some reason, the dmd link command isn't picking up the value of LIBPATH, so the linker can't find the library. Here's a reduced test case:
> > 
> > 	#!/usr/src/scons/russel/scons_d_tooling/bootstrap.py -f
> > 	env = Environment(
> > 		DC = '/usr/src/d/dmd/src/dmd',
> > 	)
> > 	env.Library('mylib', 'mylib.d')
> > 
> > 	prog_env = env.Clone(
> > 		LIBS = ['mylib'],
> > 		LIBPATH = '#'
> > 	)
> > 	prog_env.Program('prog', 'prog.d')
> > 
> > Output:
> > 
> > 	scons: Reading SConscript files ...
> > 	scons: done reading SConscript files.
> > 	scons: Building targets ...
> > 	/usr/src/d/dmd/src/dmd -I. -c -ofmylib.o mylib.d
> > 	ar cr libmylib.a mylib.o
> > 	ranlib libmylib.a
> > 	/usr/src/d/dmd/src/dmd -I. -c -ofprog.o prog.d
> > 	/usr/src/d/dmd/src/dmd -ofprog prog.o -L-lmylib
> > 	/usr/bin/ld: cannot find -lmylib
> > 	collect2: error: ld returned 1 exit status
> > 	--- errorlevel 1
> > 	scons: *** [prog] Error 1
> > 	scons: building terminated because of errors.
> > 
> > This works correctly when using the C compiler (SCons correctly inserts
> > a "-L." in the link command).
> 
> Can you set out that case and the correct command line? I have to admit DMD/LDC flags often confuse me, I must be a GDC/GCC person ;-)

I'm a GDC/GCC person as well. :-) Except that GDC for my platform is horribly outdated, so I've had to resort to DMD for my newer projects. Plus, sometimes I'm testing fixes for Phobos, which often requires the latest git dmd to even compile.

Anyway.  The command is missing a "-L-L." option. It should read:

	/usr/src/d/dmd/src/dmd -ofprog prog.o -L-L. -L-lmylib

which passes a "-L." option to the linker to make it search the current directory for the library.


[...]
> > Also, an unrelated issue: if DC isn't specified and no D compiler is found in $PATH, it produces a rather odd command line:
> > 
> > 	scons: Reading SConscript files ...
> > 	scons: done reading SConscript files.
> > 	scons: Building targets ...
> > 	I. -c -ofprog.o prog.d
> > 	sh: 1: I.: not found
> > 	ofprog prog.o -L-lmylib
> > 	sh: 1: ofprog: not found
> > 	scons: done building targets.
> > 
> > This is not a big deal, but it'd be nice if the tool gave a more helpful message along the lines of "I can't find a D compiler, please specify one", instead of producing a mangled command. :-)
> 
> Actually this is a big deal. If none of dmd, gdc, ldc2 are found then the build should fails before trying a command.  Isn't this why exists exists.  Also the generate should fail if there is no D compiler.
[....]

True.


T

-- 
Question authority. Don't ask why, just do it.

On Mon, Dec 17, 2012 at 11:08:00AM -0800, H. S. Teoh wrote: [...]
> Anyway.  The command is missing a "-L-L." option. It should read:
> 
> 	/usr/src/d/dmd/src/dmd -ofprog prog.o -L-L. -L-lmylib
> 
> which passes a "-L." option to the linker to make it search the current directory for the library.
[...]

Clarification: I guess what I meant was that for each $path in LIBPATH, there should be a corresponding option "-L-L$path" in the linker command. It just happened to be "." for my project. But in general, one would need to generate a -L-L option for each path in LIBPATH so that the linker knows where to look for libraries.

Basically, passing "-L-L$path" to DMD causes it to pass "-L$path" to the linker, which is how one specifies linker search paths.


T

-- 
Tech-savvy: euphemism for nerdy.

On Mon, 2012-12-17 at 11:08 -0800, H. S. Teoh wrote:
[…]

> I'm a GDC/GCC person as well. :-) Except that GDC for my platform is horribly outdated, so I've had to resort to DMD for my newer projects. Plus, sometimes I'm testing fixes for Phobos, which often requires the latest git dmd to even compile.

I switched to LDC as DMD broke in the 2.059 → 2.060 change.

> Anyway.  The command is missing a "-L-L." option. It should read:
> 
> 	/usr/src/d/dmd/src/dmd -ofprog prog.o -L-L. -L-lmylib
> 
> which passes a "-L." option to the linker to make it search the current directory for the library.

I tried this on spec earlier and got:

|> dmd -ofprog prog.o -L-L. -L-lmylib
/usr/lib/x86_64-linux-gnu/libphobos2.a(dmain2_45f_1a5.o): In function `_D2rt6dmain24mainUiPPaZi7runMainMFZv':
src/rt/dmain2.d:(.text._D2rt6dmain24mainUiPPaZi7runMainMFZv+0x18): undefined reference to `_Dmain'
/usr/lib/x86_64-linux-gnu/libphobos2.a(thread_192_1b8.o): In function `_D4core6thread6Thread6__ctorMFZC4core6thread6Thread':
src/core/thread.d:(.text._D4core6thread6Thread6__ctorMFZC4core6thread6Thread+0x26): undefined reference to `_tlsend'
src/core/thread.d:(.text._D4core6thread6Thread6__ctorMFZC4core6thread6Thread+0x31): undefined reference to `_tlsstart'
/usr/lib/x86_64-linux-gnu/libphobos2.a(thread_1a2_6e4.o): In function `thread_attachThis':
src/core/thread.d:(.text.thread_attachThis+0xf4): undefined reference to `_tlsstart'
src/core/thread.d:(.text.thread_attachThis+0xff): undefined reference to `_tlsend'
/usr/lib/x86_64-linux-gnu/libphobos2.a(thread_180_1b8.o): In function `_D4core6thread6Thread6__ctorMFPFZvmZC4core6thread6Thread':
src/core/thread.d:(.text._D4core6thread6Thread6__ctorMFPFZvmZC4core6thread6Thread+0x2b): undefined reference to `_tlsend'
src/core/thread.d:(.text._D4core6thread6Thread6__ctorMFPFZvmZC4core6thread6Thread+0x36): undefined reference to `_tlsstart'
/usr/lib/x86_64-linux-gnu/libphobos2.a(thread_181_1b8.o): In function `_D4core6thread6Thread6__ctorMFDFZvmZC4core6thread6Thread':
src/core/thread.d:(.text._D4core6thread6Thread6__ctorMFDFZvmZC4core6thread6Thread+0x37): undefined reference to `_tlsend'
src/core/thread.d:(.text._D4core6thread6Thread6__ctorMFDFZvmZC4core6thread6Thread+0x42): undefined reference to `_tlsstart'
/usr/lib/x86_64-linux-gnu/libphobos2.a(deh2_441_525.o): In function `_D2rt4deh213__eh_finddataFPvZPS2rt4deh29FuncTable':
src/rt/deh2.d:(.text._D2rt4deh213__eh_finddataFPvZPS2rt4deh29FuncTable+0xa): undefined reference to `_deh_beg'
src/rt/deh2.d:(.text._D2rt4deh213__eh_finddataFPvZPS2rt4deh29FuncTable+0x14): undefined reference to `_deh_beg'
src/rt/deh2.d:(.text._D2rt4deh213__eh_finddataFPvZPS2rt4deh29FuncTable+0x1e): undefined reference to `_deh_end'
src/rt/deh2.d:(.text._D2rt4deh213__eh_finddataFPvZPS2rt4deh29FuncTable+0x45): undefined reference to `_deh_end'
/usr/lib/x86_64-linux-gnu/libphobos2.a(thread_17d_713.o): In function `thread_entryPoint':
src/core/thread.d:(.text.thread_entryPoint+0xa1): undefined reference to `_tlsend'
src/core/thread.d:(.text.thread_entryPoint+0xac): undefined reference to `_tlsstart'
collect2: error: ld returned 1 exit status
--- errorlevel 1

The problem here I think relates to the name of the library. Should it be libmylib.as or mylib.a on Linux?

-- 
Russel. ============================================================================= Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder@ekiga.net 41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel@winder.org.uk London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder

On Monday, 17 December 2012 at 19:29:04 UTC, Russel Winder wrote:
> I tried this on spec earlier and got:
>
> |> dmd -ofprog prog.o -L-L. -L-lmylib
> /usr/lib/x86_64-linux-gnu/libphobos2.a(dmain2_45f_1a5.o): In function `_D2rt6dmain24mainUiPPaZi7runMainMFZv':
> src/rt/dmain2.d:(.text._D2rt6dmain24mainUiPPaZi7runMainMFZv+0x18): undefined reference to `_Dmain'

Your program seems to be missing a main() function, the rest is just noise (DMD emits certain symbols to the main module only).

David

On Mon, Dec 17, 2012 at 07:28:53PM +0000, Russel Winder wrote:
> On Mon, 2012-12-17 at 11:08 -0800, H. S. Teoh wrote:
> […]
> 
> > I'm a GDC/GCC person as well. :-) Except that GDC for my platform is horribly outdated, so I've had to resort to DMD for my newer projects.  Plus, sometimes I'm testing fixes for Phobos, which often requires the latest git dmd to even compile.
> 
> I switched to LDC as DMD broke in the 2.059 → 2.060 change.

I haven't tried LDC yet, though I've heard good things about it. Maybe I should check it out sometime.


> > Anyway.  The command is missing a "-L-L." option. It should read:
> > 
> > 	/usr/src/d/dmd/src/dmd -ofprog prog.o -L-L. -L-lmylib
> > 
> > which passes a "-L." option to the linker to make it search the current directory for the library.
> 
> I tried this on spec earlier and got:
> 
> |> dmd -ofprog prog.o -L-L. -L-lmylib
> /usr/lib/x86_64-linux-gnu/libphobos2.a(dmain2_45f_1a5.o): In function `_D2rt6dmain24mainUiPPaZi7runMainMFZv':
> src/rt/dmain2.d:(.text._D2rt6dmain24mainUiPPaZi7runMainMFZv+0x18): undefined reference to `_Dmain'
> /usr/lib/x86_64-linux-gnu/libphobos2.a(thread_192_1b8.o): In function `_D4core6thread6Thread6__ctorMFZC4core6thread6Thread':
> src/core/thread.d:(.text._D4core6thread6Thread6__ctorMFZC4core6thread6Thread+0x26): undefined reference to `_tlsend'
> src/core/thread.d:(.text._D4core6thread6Thread6__ctorMFZC4core6thread6Thread+0x31): undefined reference to `_tlsstart'
[...]

Did you define main() in prog.d (or somewhere)? The second line of the output here seems to indicate that you didn't. I think the latest git dmd is patched so that a more helpful error message is provided in this case, but this unhelpful screenful of errors is what the current release dmd gives when main() isn't defined.


T

-- 
Try to keep an open mind, but not so open your brain falls out. -- theboz

On Mon, 2012-12-17 at 11:37 -0800, H. S. Teoh wrote:

[…]
> I haven't tried LDC yet, though I've heard good things about it. Maybe I should check it out sometime.

I checked out the Git repository and build master/HEAD, but then I am an unstable, bleeding-edge kind of person.

> Did you define main() in prog.d (or somewhere)? The second line of the output here seems to indicate that you didn't. I think the latest git dmd is patched so that a more helpful error message is provided in this case, but this unhelpful screenful of errors is what the current release dmd gives when main() isn't defined.

I will try and hide the fact that I was a complete muppet and failed to have a main function in prog.d.

-- 
Russel. ============================================================================= Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder@ekiga.net 41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel@winder.org.uk London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder