As a fan of both D and the nixpkgs package manager I wanted to try and help include GDC in nixpkgs, but I have been running into some problems.

If anyone could help by taking a look at my pull request at https://github.com/NixOS/nixpkgs/pull/69144 that would be greatly appreciated.

Current issues include:

* When compiling without forcing --disable-shared, there are linker errors about needing to compile with -fPIC. It seems that some files are built using that flag, but not all, and I'm not sure how to go about forcing it for the rest.
* When invoking nix-build without specifying --cores 1 there are problems when building libphobos; it looks like object files are moved before they are generated. This problem goes away when I build on a single core, but of course everything takes a very long time to try out.
* I have disabled bootstrapping for now to speed up the debugging process, hopefully it can be enabled again later on.

I have taken the compiler flags to be used from https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=gdc-git, which was linked in another thread on this forum, but maybe this is too specific for Arch?

Thanks in advance.

Hi Stefan,

Am Fri, 20 Sep 2019 10:29:10 +0000 schrieb Stefan Frijters:

> As a fan of both D and the nixpkgs package manager I wanted to try and help include GDC in nixpkgs, but I have been running into some problems.
> 

Thank you a lot for looking at this and helping the D ecosystem!

> If anyone could help by taking a look at my pull request at https://github.com/NixOS/nixpkgs/pull/69144 that would be greatly appreciated.
> 
> Current issues include:
> 
> * When compiling without forcing --disable-shared, there are linker errors about needing to compile with -fPIC. It seems that some files are built using that flag, but not all, and I'm not sure how to go about forcing it for the rest.

This should not happen, -fPIC should be added automatically where required. Maybe it's caused by --enable-default-pie or any of the other flags which are not really needed?

> * When invoking nix-build without specifying --cores 1 there are problems when building libphobos; it looks like object files are moved before they are generated. This problem goes away when I build on a single core, but of course everything takes a very long time to try out.

Is this a problem during the make step or during make install? I don't think make install is multi-thread safe, but make should actually be fine.

> * I have disabled bootstrapping for now to speed up the debugging process, hopefully it can be enabled again later on.
> 
> I have taken the compiler flags to be used from https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=gdc-git, which was linked in another thread on this forum, but maybe this is too specific for Arch?

Yes, these are actually arch and not gdc specific. You do not need any special flags for gdc, just add d to --enable-languages= and you should be fine. --with-target-system-zlib=yes is the only D specific flag in your github pull request, and it forces libphobos to link against the system zlib instead of including a copy. You'll have to decide whether this is the behavior you want.


-- 
Johannes

Hi Johannes,

Thank you for the feedback. I've posted some more below so we can try to find out what I'm doing wrong. I've added some blank lines here and there to make things more readable, but it's all very spammy I'm afraid.

On Friday, 20 September 2019 at 17:18:20 UTC, Johannes Pfau wrote:
> Hi Stefan,
>
> Am Fri, 20 Sep 2019 10:29:10 +0000 schrieb Stefan Frijters:
>
>> As a fan of both D and the nixpkgs package manager I wanted to try and help include GDC in nixpkgs, but I have been running into some problems.
>> 
>
> Thank you a lot for looking at this and helping the D ecosystem!
>
>> If anyone could help by taking a look at my pull request at https://github.com/NixOS/nixpkgs/pull/69144 that would be greatly appreciated.
>> 
>> Current issues include:

>> * I have disabled bootstrapping for now to speed up the debugging process, hopefully it can be enabled again later on.
>> 
>> I have taken the compiler flags to be used from https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=gdc-git, which was linked in another thread on this forum, but maybe this is too specific for Arch?
>
> Yes, these are actually arch and not gdc specific. You do not need any special flags for gdc, just add d to --enable-languages= and you should be fine. --with-target-system-zlib=yes is the only D specific flag in your github pull request, and it forces libphobos to link against the system zlib instead of including a copy. You'll have to decide whether this is the behavior you want.

Ok, I actually started out with only adding d as an enabled language, but then I just started adding other flags hoping that they would to the trick and at least let the build get to the end. --with-target-system-zlib=yes seems to be necessary though for my situation.

This is what I get from config.log if I remove all the extra flags (except --disable-shared and --disable-bootstrap), but keeping whatever nixpkgs gives me by default:

  $ ../gcc-9.2.0/configure --prefix=/nix/store/crh7b7jill2hh8gg1i59i34qvmiq2mfa-gdc-9.2.0 --with-gmp-include=/nix/store/jljw9i7377gi55gbnipsqanqirdzs41p-gmp-6.1.2-dev/include --with-gmp-lib=/nix/store/j2p1qjbajqfb95ba3dqgjbsknipknikk-gmp-6.1.2/lib --with-mpfr-include=/nix/store/676gyak02v4zsgjx23f5arhlnd3j02ny-mpfr-4.0.2-dev/include --with-mpfr-lib=/nix/store/n5m42hdi36gn9yvbcjyihj4fxjn9bgjf-mpfr-4.0.2/lib --with-mpc=/nix/store/4abiapxk8wng6viks962wyr3ag04xnz4-libmpc-1.1.0 --with-libelf=/nix/store/hgx581smqcl2zi76lqzyk1j9pv8cf85p-libelf-0.8.13 --with-native-system-header-dir=/nix/store/7b67hqgxavm47qizrkya55qvyp1f65s4-glibc-2.27-dev/include --enable-lto --disable-libstdcxx-pch --without-included-gettext --with-system-zlib --enable-static --enable-languages=d --disable-multilib --enable-plugin --with-isl=/nix/store/wak0rkw1xvpzh01hk9a77h56cx2czhgi-isl-0.17.1 --disable-bootstrap --with-target-system-zlib=yes --disable-shared --build=x86_64-unknown-linux-gnu --host=x86_64-unknown-linux-gnu

Does any of that look weird to you?

And just to check: is it okay to build GDC9 with GCC8, or should I use GCC9?

>> * When compiling without forcing --disable-shared, there are linker errors about needing to compile with -fPIC. It seems that some files are built using that flag, but not all, and I'm not sure how to go about forcing it for the rest.
>
> This should not happen, -fPIC should be added automatically where required. Maybe it's caused by --enable-default-pie or any of the other flags which are not really needed?

Without all the other flags I still get this problem (it was pretty much the reason I started adding things in the first place). Does this mean it's 'just' libphobos itself that's missing the -fPIC flag?

libtool: link: /build/build/./gcc/gdc -B/build/build/./gcc/ -O2 -I/nix/store/7b67hqgxavm47qizrkya55qvyp1f65s4-glibc-2.27-dev/include -B/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib/ -idirafter /nix/store/7b67hqgxavm47qizrkya55qvyp1f65s4-glibc-2.27-dev/include -idirafter /nix/store/pcs8pq4a5rkym1hzibqz7da45fxkmig7-gcc-8.3.0/lib/gcc/x86_64-unknown-linux-gnu/8.3.0/include-fixed -Wl,-rpath,/nix/store/dwmw9jilsa5fmvpj1z568kxbnchzyvx2-gdc-9.2.0-lib/lib -Wl,-L/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib -Wl,-rpath -Wl,/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib -Wl,-L/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib    -shared -fversion=Shared  core/.libs/atomic.o core/.libs/attribute.o core/.libs/bitop.o core/.libs/checkedint.o core/.libs/cpuid.o core/.libs/demangle.o core/.libs/exception.o core/internal/.libs/abort.o core/internal/.libs/arrayop.o core/internal/.libs/convert.o core/internal/.libs/hash.o core/internal/.libs/spinlock.o core/internal/.libs/string.o core/internal/.libs/traits.o core/.libs/math.o core/.libs/memory.o core/.libs/runtime.o core/.libs/simd.o core/stdc/.libs/assert_.o core/stdc/.libs/complex.o core/stdc/.libs/config.o core/stdc/.libs/ctype.o core/stdc/.libs/errno.o core/stdc/.libs/fenv.o core/stdc/.libs/float_.o core/stdc/.libs/inttypes.o core/stdc/.libs/limits.o core/stdc/.libs/locale.o core/stdc/.libs/math.o core/stdc/.libs/signal.o core/stdc/.libs/stdarg.o core/stdc/.libs/stddef.o core/stdc/.libs/stdint.o core/stdc/.libs/stdio.o core/stdc/.libs/stdlib.o core/stdc/.libs/string.o core/stdc/.libs/tgmath.o core/stdc/.libs/time.o core/stdc/.libs/wchar_.o core/stdc/.libs/wctype.o core/sync/.libs/barrier.o core/sync/.libs/condition.o core/sync/.libs/config.o core/sync/.libs/exception.o core/sync/.libs/mutex.o core/sync/.libs/rwmutex.o core/sync/.libs/semaphore.o core/.libs/thread.o core/.libs/time.o core/.libs/vararg.o gcc/.libs/attribute.o gcc/.libs/backtrace.o gcc/.libs/builtins.o gcc/.libs/deh.o gcc/.libs/emutls.o gcc/.libs/gthread.o gcc/sections/.libs/android.o gcc/sections/.libs/elf_shared.o gcc/sections/.libs/osx.o gcc/sections/.libs/package.o gcc/sections/.libs/win32.o gcc/sections/.libs/win64.o gcc/unwind/.libs/arm.o gcc/unwind/.libs/arm_common.o gcc/unwind/.libs/c6x.o gcc/unwind/.libs/generic.o gcc/unwind/.libs/package.o gcc/unwind/.libs/pe.o .libs/object.o rt/.libs/aApply.o rt/.libs/aApplyR.o rt/.libs/aaA.o rt/.libs/adi.o rt/.libs/arrayassign.o rt/.libs/arraycast.o rt/.libs/arraycat.o rt/.libs/cast_.o rt/.libs/config.o rt/.libs/critical_.o rt/.libs/deh.o rt/.libs/dmain2.o rt/.libs/invariant.o rt/.libs/lifetime.o rt/.libs/memory.o rt/.libs/minfo.o rt/.libs/monitor_.o rt/.libs/obj.o rt/.libs/qsort.o rt/.libs/sections.o rt/.libs/switch_.o rt/.libs/tlsgc.o rt/typeinfo/.libs/ti_Acdouble.o rt/typeinfo/.libs/ti_Acfloat.o rt/typeinfo/.libs/ti_Acreal.o rt/typeinfo/.libs/ti_Adouble.o rt/typeinfo/.libs/ti_Afloat.o rt/typeinfo/.libs/ti_Ag.o rt/typeinfo/.libs/ti_Aint.o rt/typeinfo/.libs/ti_Along.o rt/typeinfo/.libs/ti_Areal.o rt/typeinfo/.libs/ti_Ashort.o rt/typeinfo/.libs/ti_C.o rt/typeinfo/.libs/ti_byte.o rt/typeinfo/.libs/ti_cdouble.o rt/typeinfo/.libs/ti_cent.o rt/typeinfo/.libs/ti_cfloat.o rt/typeinfo/.libs/ti_char.o rt/typeinfo/.libs/ti_creal.o rt/typeinfo/.libs/ti_dchar.o rt/typeinfo/.libs/ti_delegate.o rt/typeinfo/.libs/ti_double.o rt/typeinfo/.libs/ti_float.o rt/typeinfo/.libs/ti_idouble.o rt/typeinfo/.libs/ti_ifloat.o rt/typeinfo/.libs/ti_int.o rt/typeinfo/.libs/ti_ireal.o rt/typeinfo/.libs/ti_long.o rt/typeinfo/.libs/ti_n.o rt/typeinfo/.libs/ti_ptr.o rt/typeinfo/.libs/ti_real.o rt/typeinfo/.libs/ti_short.o rt/typeinfo/.libs/ti_ubyte.o rt/typeinfo/.libs/ti_ucent.o rt/typeinfo/.libs/ti_uint.o rt/typeinfo/.libs/ti_ulong.o rt/typeinfo/.libs/ti_ushort.o rt/typeinfo/.libs/ti_void.o rt/typeinfo/.libs/ti_wchar.o rt/util/.libs/array.o rt/util/container/.libs/array.o rt/util/container/.libs/common.o rt/util/container/.libs/hashtab.o rt/util/container/.libs/treap.o rt/util/.libs/random.o rt/util/.libs/typeinfo.o rt/util/.libs/utf.o core/stdc/.libs/libgdruntime_la-errno_.o gc/.libs/bits.o gc/.libs/config.o gc/.libs/gcinterface.o gc/impl/conservative/.libs/gc.o gc/impl/manual/.libs/gc.o gc/.libs/os.o gc/.libs/pooltable.o gc/.libs/proxy.o core/sys/posix/.libs/aio.o core/sys/posix/arpa/.libs/inet.o core/sys/posix/.libs/config.o core/sys/posix/.libs/dirent.o core/sys/posix/.libs/dlfcn.o core/sys/posix/.libs/fcntl.o core/sys/posix/.libs/grp.o core/sys/posix/.libs/iconv.o core/sys/posix/.libs/inttypes.o core/sys/posix/.libs/libgen.o core/sys/posix/.libs/mqueue.o core/sys/posix/net/.libs/if_.o core/sys/posix/.libs/netdb.o core/sys/posix/netinet/.libs/in_.o core/sys/posix/netinet/.libs/tcp.o core/sys/posix/.libs/poll.o core/sys/posix/.libs/pthread.o core/sys/posix/.libs/pwd.o core/sys/posix/.libs/sched.o core/sys/posix/.libs/semaphore.o core/sys/posix/.libs/setjmp.o core/sys/posix/.libs/signal.o core/sys/posix/.libs/spawn.o core/sys/posix/.libs/stdio.o core/sys/posix/.libs/stdlib.o core/sys/posix/sys/.libs/filio.o core/sys/posix/sys/.libs/ioccom.o core/sys/posix/sys/.libs/ioctl.o core/sys/posix/sys/.libs/ipc.o core/sys/posix/sys/.libs/mman.o core/sys/posix/sys/.libs/msg.o core/sys/posix/sys/.libs/resource.o core/sys/posix/sys/.libs/select.o core/sys/posix/sys/.libs/shm.o core/sys/posix/sys/.libs/socket.o core/sys/posix/sys/.libs/stat.o core/sys/posix/sys/.libs/statvfs.o core/sys/posix/sys/.libs/time.o core/sys/posix/sys/.libs/ttycom.o core/sys/posix/sys/.libs/types.o core/sys/posix/sys/.libs/uio.o core/sys/posix/sys/.libs/un.o core/sys/posix/sys/.libs/utsname.o core/sys/posix/sys/.libs/wait.o core/sys/posix/.libs/syslog.o core/sys/posix/.libs/termios.o core/sys/posix/.libs/time.o core/sys/posix/.libs/ucontext.o core/sys/posix/.libs/unistd.o core/sys/posix/.libs/utime.o core/sys/linux/.libs/config.o core/sys/linux/.libs/dlfcn.o core/sys/linux/.libs/elf.o core/sys/linux/.libs/epoll.o core/sys/linux/.libs/errno.o core/sys/linux/.libs/execinfo.o core/sys/linux/.libs/fcntl.o core/sys/linux/.libs/ifaddrs.o core/sys/linux/.libs/link.o core/sys/linux/netinet/.libs/in_.o core/sys/linux/netinet/.libs/tcp.o core/sys/linux/.libs/sched.o core/sys/linux/.libs/stdio.o core/sys/linux/sys/.libs/auxv.o core/sys/linux/sys/.libs/eventfd.o core/sys/linux/sys/.libs/file.o core/sys/linux/sys/.libs/inotify.o core/sys/linux/sys/.libs/mman.o core/sys/linux/sys/netinet/.libs/tcp.o core/sys/linux/sys/.libs/prctl.o core/sys/linux/sys/.libs/signalfd.o core/sys/linux/sys/.libs/socket.o core/sys/linux/sys/.libs/sysinfo.o core/sys/linux/sys/.libs/time.o core/sys/linux/sys/.libs/xattr.o core/sys/linux/.libs/termios.o core/sys/linux/.libs/time.o core/sys/linux/.libs/timerfd.o core/sys/linux/.libs/tipc.o core/sys/linux/.libs/unistd.o config/x86/.libs/libgdruntime_la-switchcontext.o gcc/.libs/config.o gcc/.libs/libbacktrace.o  -Wl,--whole-archive ../../libatomic/.libs/libatomic_convenience.a ../../libbacktrace/.libs/libbacktrace.a -Wl,--no-whole-archive  -lm -lpthread -ldl -lz  -Wl,-rpath -Wl,/nix/store/dwmw9jilsa5fmvpj1z568kxbnchzyvx2-gdc-9.2.0-lib/lib -Wl,-L/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib -Wl,-rpath -Wl,/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib -Wl,-L/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib -Wl,-rpath -Wl,/nix/store/dwmw9jilsa5fmvpj1z568kxbnchzyvx2-gdc-9.2.0-lib/lib -Wl,-L/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib -Wl,-rpath -Wl,/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib -Wl,-L/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib -nophoboslib -dstartfiles -B../src -Bgcc -Wl,-rpath -Wl,/nix/store/dwmw9jilsa5fmvpj1z568kxbnchzyvx2-gdc-9.2.0-lib/lib -Wl,-L/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib -Wl,-rpath -Wl,/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib -Wl,-L/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib   -pthread -Wl,-soname -Wl,libgdruntime.so.76 -o .libs/libgdruntime.so.76.0.3



/nix/store/cl1i6bfqnx48ipakj4px7pb1babzs23j-binutils-2.31.1/bin/ld: core/.libs/atomic.o: relocation R_X86_64_32S against hidden symbol `gdc.dso_slot' can not be used when making a shared object
/nix/store/cl1i6bfqnx48ipakj4px7pb1babzs23j-binutils-2.31.1/bin/ld: core/.libs/attribute.o: relocation R_X86_64_32S against hidden symbol `gdc.dso_slot' can not be used when making a shared object
/nix/store/cl1i6bfqnx48ipakj4px7pb1babzs23j-binutils-2.31.1/bin/ld: core/.libs/bitop.o: relocation R_X86_64_32 against `.rodata.str1.8' can not be used when making a shared object; recompile with -fPIC
/nix/store/cl1i6bfqnx48ipakj4px7pb1babzs23j-binutils-2.31.1/bin/ld: core/.libs/checkedint.o: relocation R_X86_64_32S against hidden symbol `gdc.dso_slot' can not be used when making a shared object
/nix/store/cl1i6bfqnx48ipakj4px7pb1babzs23j-binutils-2.31.1/bin/ld: core/.libs/cpuid.o: relocation R_X86_64_32S against symbol `_D4core5cpuid18getcacheinfoCPUID2FNbNiNeZ14decipherCpuid2MFNbNihZ3idsyG63h' can not be used when making a shared object; recompile with -fPIC
/nix/store/cl1i6bfqnx48ipakj4px7pb1babzs23j-binutils-2.31.1/bin/ld: core/.libs/demangle.o: relocation R_X86_64_32 against symbol `_D11TypeInfo_xm6__initZ' can not be used when making a shared object; recompile with -fPIC
/nix/store/cl1i6bfqnx48ipakj4px7pb1babzs23j-binutils-2.31.1/bin/ld: core/.libs/exception.o: relocation R_X86_64_32 against symbol `_D12TypeInfo_Aya6__initZ' can not be used when making a shared object; recompile with -fPIC
/nix/store/cl1i6bfqnx48ipakj4px7pb1babzs23j-binutils-2.31.1/bin/ld: core/internal/.libs/abort.o: relocation R_X86_64_32S against `.rodata' can not be used when making a shared object; recompile with -fPIC
[... and some more screens of similar errors ... ]

Looks like the shared version is getting -fversion=Shared, but not -fPIC:

/nix/store/l6h4ya0wzb4b8mr0y58k2gh2nhfql4sn-bash-4.4-p23/bin/bash ../libtool --tag=D   --mode=compile /build/build/./gcc/gdc -B/build/build/./gcc/ -O2 -I/nix/store/7b67hqgxavm47qizrkya55qvyp1f65s4-glibc-2.27-dev/include -B/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib/ -idirafter /nix/store/7b67hqgxavm47qizrkya55qvyp1f65s4-glibc-2.27-dev/include -idirafter /nix/store/pcs8pq4a5rkym1hzibqz7da45fxkmig7-gcc-8.3.0/lib/gcc/x86_64-unknown-linux-gnu/8.3.0/include-fixed -Wl,-rpath,/nix/store/dwmw9jilsa5fmvpj1z568kxbnchzyvx2-gdc-9.2.0-lib/lib -Wl,-L/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib -Wl,-rpath -Wl,/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib -Wl,-L/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib     -O2 -g  -nostdinc -I ../../../../gcc-9.2.0/libphobos/libdruntime -I . -c -o core/sys/linux/sys/socket.lo ../../../../gcc-9.2.0/libphobos/libdruntime/core/sys/linux/sys/socket.d

libtool: compile:  /build/build/./gcc/gdc -B/build/build/./gcc/ -O2 -I/nix/store/7b67hqgxavm47qizrkya55qvyp1f65s4-glibc-2.27-dev/include -B/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib/ -idirafter /nix/store/7b67hqgxavm47qizrkya55qvyp1f65s4-glibc-2.27-dev/include -idirafter /nix/store/pcs8pq4a5rkym1hzibqz7da45fxkmig7-gcc-8.3.0/lib/gcc/x86_64-unknown-linux-gnu/8.3.0/include-fixed -Wl,-rpath,/nix/store/dwmw9jilsa5fmvpj1z568kxbnchzyvx2-gdc-9.2.0-lib/lib -Wl,-L/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib -Wl,-rpath -Wl,/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib -Wl,-L/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib -O2 -g -nostdinc -I ../../../../gcc-9.2.0/libphobos/libdruntime -I . -c ../../../../gcc-9.2.0/libphobos/libdruntime/core/sys/linux/sys/socket.d -fversion=Shared

d21: warning: command line option '-idirafter /nix/store/7b67hqgxavm47qizrkya55qvyp1f65s4-glibc-2.27-dev/include' is valid for C/C++/Fortran/ObjC/ObjC++ but not for D

d21: warning: command line option '-idirafter /nix/store/pcs8pq4a5rkym1hzibqz7da45fxkmig7-gcc-8.3.0/lib/gcc/x86_64-unknown-linux-gnu/8.3.0/include-fixed' is valid for C/C++/Fortran/ObjC/ObjC++ but not for D

libtool: compile: mv -f "socket.o" "core/sys/linux/sys/.libs/socket.o"

libtool: compile:  /build/build/./gcc/gdc -B/build/build/./gcc/ -O2 -I/nix/store/7b67hqgxavm47qizrkya55qvyp1f65s4-glibc-2.27-dev/include -B/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib/ -idirafter /nix/store/7b67hqgxavm47qizrkya55qvyp1f65s4-glibc-2.27-dev/include -idirafter /nix/store/pcs8pq4a5rkym1hzibqz7da45fxkmig7-gcc-8.3.0/lib/gcc/x86_64-unknown-linux-gnu/8.3.0/include-fixed -Wl,-rpath,/nix/store/dwmw9jilsa5fmvpj1z568kxbnchzyvx2-gdc-9.2.0-lib/lib -Wl,-L/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib -Wl,-rpath -Wl,/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib -Wl,-L/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib -O2 -g -nostdinc -I ../../../../gcc-9.2.0/libphobos/libdruntime -I . -c ../../../../gcc-9.2.0/libphobos/libdruntime/core/sys/linux/sys/socket.d >/dev/null 2>&1

libtool: compile: mv -f "socket.o" "core/sys/linux/sys/socket.o"


>> * When invoking nix-build without specifying --cores 1 there are problems when building libphobos; it looks like object files are moved before they are generated. This problem goes away when I build on a single core, but of course everything takes a very long time to try out.
>
> Is this a problem during the make step or during make install? I don't think make install is multi-thread safe, but make should actually be fine.

It's happening during an invocation of libtool - pasted an example of lines grepped for time\.[od] below, but I've seen it happen for other files as well (e.g. array.o). Also note that in the example for socket.o above, which was built using a single core, the compile / mv steps are nicely interleaved, unlike below:

/nix/store/l6h4ya0wzb4b8mr0y58k2gh2nhfql4sn-bash-4.4-p23/bin/bash ../libtool --tag=D   --mode=compile /build/build/./gcc/gdc -B/build/build/./gcc/ -O2 -I/nix/store/7b67hqgxavm47qizrkya55qvyp1f65s4-glibc-2.27-dev/include -B/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib/ -idirafter /nix/store/7b67hqgxavm47qizrkya55qvyp1f65s4-glibc-2.27-dev/include -idirafter /nix/store/pcs8pq4a5rkym1hzibqz7da45fxkmig7-gcc-8.3.0/lib/gcc/x86_64-unknown-linux-gnu/8.3.0/include-fixed -Wl,-rpath,/nix/store/1cfsf8mqph03gz3r9qh9fbqv99vdg0dy-gdc-9.2.0-lib/lib -Wl,-L/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib -Wl,-rpath -Wl,/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib -Wl,-L/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib     -O2 -g  -nostdinc -I ../../../../gcc-9.2.0/libphobos/libdruntime -I . -c -o core/sys/linux/sys/time.lo ../../../../gcc-9.2.0/libphobos/libdruntime/core/sys/linux/sys/time.d

/nix/store/l6h4ya0wzb4b8mr0y58k2gh2nhfql4sn-bash-4.4-p23/bin/bash ../libtool --tag=D   --mode=compile /build/build/./gcc/gdc -B/build/build/./gcc/ -O2 -I/nix/store/7b67hqgxavm47qizrkya55qvyp1f65s4-glibc-2.27-dev/include -B/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib/ -idirafter /nix/store/7b67hqgxavm47qizrkya55qvyp1f65s4-glibc-2.27-dev/include -idirafter /nix/store/pcs8pq4a5rkym1hzibqz7da45fxkmig7-gcc-8.3.0/lib/gcc/x86_64-unknown-linux-gnu/8.3.0/include-fixed -Wl,-rpath,/nix/store/1cfsf8mqph03gz3r9qh9fbqv99vdg0dy-gdc-9.2.0-lib/lib -Wl,-L/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib -Wl,-rpath -Wl,/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib -Wl,-L/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib     -O2 -g  -nostdinc -I ../../../../gcc-9.2.0/libphobos/libdruntime -I . -c -o core/sys/linux/time.lo ../../../../gcc-9.2.0/libphobos/libdruntime/core/sys/linux/time.d

libtool: compile:  /build/build/./gcc/gdc -B/build/build/./gcc/ -O2 -I/nix/store/7b67hqgxavm47qizrkya55qvyp1f65s4-glibc-2.27-dev/include -B/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib/ -idirafter /nix/store/7b67hqgxavm47qizrkya55qvyp1f65s4-glibc-2.27-dev/include -idirafter /nix/store/pcs8pq4a5rkym1hzibqz7da45fxkmig7-gcc-8.3.0/lib/gcc/x86_64-unknown-linux-gnu/8.3.0/include-fixed -Wl,-rpath,/nix/store/1cfsf8mqph03gz3r9qh9fbqv99vdg0dy-gdc-9.2.0-lib/lib -Wl,-L/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib -Wl,-rpath -Wl,/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib -Wl,-L/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib -O2 -g -nostdinc -I ../../../../gcc-9.2.0/libphobos/libdruntime -I . -c ../../../../gcc-9.2.0/libphobos/libdruntime/core/sys/linux/sys/time.d

libtool: compile:  /build/build/./gcc/gdc -B/build/build/./gcc/ -O2 -I/nix/store/7b67hqgxavm47qizrkya55qvyp1f65s4-glibc-2.27-dev/include -B/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib/ -idirafter /nix/store/7b67hqgxavm47qizrkya55qvyp1f65s4-glibc-2.27-dev/include -idirafter /nix/store/pcs8pq4a5rkym1hzibqz7da45fxkmig7-gcc-8.3.0/lib/gcc/x86_64-unknown-linux-gnu/8.3.0/include-fixed -Wl,-rpath,/nix/store/1cfsf8mqph03gz3r9qh9fbqv99vdg0dy-gdc-9.2.0-lib/lib -Wl,-L/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib -Wl,-rpath -Wl,/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib -Wl,-L/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib -O2 -g -nostdinc -I ../../../../gcc-9.2.0/libphobos/libdruntime -I . -c ../../../../gcc-9.2.0/libphobos/libdruntime/core/sys/linux/time.d

libtool: compile: mv -f "time.o" "core/sys/linux/sys/time.o"
libtool: compile: mv -f "time.o" "core/sys/linux/time.o"
mv: cannot stat 'time.o': No such file or directory

This is particulary annoying since I have a 16-core Ryzen so I lose a *lot* of time debugging with a single core...

Thanks for helping!

On Sunday, 22 September 2019 at 10:29:24 UTC, Stefan Frijters wrote:
> Hi Johannes,
>
> Thank you for the feedback. I've posted some more below so we can try to find out what I'm doing wrong. I've added some blank lines here and there to make things more readable, but it's all very spammy I'm afraid.
>
> [...]

Well, I've made some progress:

* I still don't know what causes libtool to misbehave in parallel, but I've forced libphobos to be built with -j1, so that's at least a working workaround that doesn't slow down the rest of the build.
* The nixpkgs wrappers for gcc introduce the -idirafter flag, which is not accepted by gdc. It looks like this causes the check for the '-fPIC' flag to fail, and the flag to be dropped. This is the cause of the linker error for the shared libs. I've hacked around for a bit and got rid of the flag at the point of the check, so libphobos now builds correctly (but still with a bunch of warnings).
* I've added some nix code to generate a wrapper for gdc, like the ones that are generated for the rest of the GNU compiler family.

With these changes I can now build gdc + libraries and I've successfully built the example on the dlang.org front page from a nix shell.

The PR will need some reviewing from nixpkgs folks for sure since I did some nasty things.

On Wed, 25 Sep 2019 at 21:30, Stefan Frijters via D.gnu <d.gnu@puremagic.com> wrote:
>
> On Sunday, 22 September 2019 at 10:29:24 UTC, Stefan Frijters wrote:
> > Hi Johannes,
> >
> > Thank you for the feedback. I've posted some more below so we can try to find out what I'm doing wrong. I've added some blank lines here and there to make things more readable, but it's all very spammy I'm afraid.
> >
> > [...]
>
> Well, I've made some progress:
>
> * I still don't know what causes libtool to misbehave in
> parallel, but I've forced libphobos to be built with -j1, so
> that's at least a working workaround that doesn't slow down the
> rest of the build.
> * The nixpkgs wrappers for gcc introduce the -idirafter flag,
> which is not accepted by gdc. It looks like this causes the check
> for the '-fPIC' flag to fail, and the flag to be dropped. This is
> the cause of the linker error for the shared libs. I've hacked
> around for a bit and got rid of the flag at the point of the
> check, so libphobos now builds correctly (but still with a bunch
> of warnings).
> * I've added some nix code to generate a wrapper for gdc, like
> the ones that are generated for the rest of the GNU compiler
> family.
>

How does it go about building Ada or Go?  Surely those two frontends will run into the same problem also.

> With these changes I can now build gdc + libraries and I've successfully built the example on the dlang.org front page from a nix shell.
>
> The PR will need some reviewing from nixpkgs folks for sure since I did some nasty things.

If you need any other eyes to have a look over, don't be afraid to ping.

-- 
Iain

On Monday, 21 October 2019 at 19:26:15 UTC, Iain Buclaw wrote:
> On Wed, 25 Sep 2019 at 21:30, Stefan Frijters via D.gnu <d.gnu@puremagic.com> wrote:
>>[...]
>
> How does it go about building Ada or Go?  Surely those two frontends will run into the same problem also.
>
>> [...]
>
> If you need any other eyes to have a look over, don't be afraid to ping.

It took some time, but my PR to get gdc into nixpkgs has been approved and merged.
It will take some more time to trickle down to the stable branch, but it's available at the bleeding edge at least.

Stefan

On 24/05/2020 13:02, Stefan Frijters via D.gnu wrote:
> On Monday, 21 October 2019 at 19:26:15 UTC, Iain Buclaw wrote:
>> On Wed, 25 Sep 2019 at 21:30, Stefan Frijters via D.gnu <d.gnu@puremagic.com> wrote:
>>> [...]
>>
>> How does it go about building Ada or Go?  Surely those two frontends will run into the same problem also.
>>
>>> [...]
>>
>> If you need any other eyes to have a look over, don't be afraid to ping.
> 
> It took some time, but my PR to get gdc into nixpkgs has been approved and merged.
> It will take some more time to trickle down to the stable branch, but it's available at the bleeding edge at least.
> 

Thanks for your persistence.

Iain.