Jump to page: 1 2
Thread overview
[Issue 15391] Problems loading libcurl.so and running datetime unittest on NixOS package build
[Issue 15391] Deadlock in std.net.curl unittest
Nov 30, 2015
Thomas Mader
[Issue 15391] Deadlock in std.net.curl unittest in checkPhase of NixOS package build
Nov 30, 2015
Thomas Mader
Nov 30, 2015
Thomas Mader
Nov 30, 2015
Thomas Mader
Nov 30, 2015
Thomas Mader
Nov 30, 2015
Jonathan M Davis
Nov 30, 2015
Martin Nowak
Dec 12, 2015
Thomas Mader
Dec 12, 2015
Thomas Mader
Jul 31, 2017
Thomas Mader
November 30, 2015
https://issues.dlang.org/show_bug.cgi?id=15391

--- Comment #1 from Thomas Mader <thomas.mader@gmail.com> ---
I need to add that I try to run the unittests in the checkPhase of a package
build of NixOS and the network is restricted in this case I guess.
But the tests for std.socket were successful:

0.518s PASS debug64 std.socket

--
November 30, 2015
https://issues.dlang.org/show_bug.cgi?id=15391

Thomas Mader <thomas.mader@gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Summary|Deadlock in std.net.curl    |Deadlock in std.net.curl
                   |unittest                    |unittest in checkPhase of
                   |                            |NixOS package build

--
November 30, 2015
https://issues.dlang.org/show_bug.cgi?id=15391

--- Comment #2 from Thomas Mader <thomas.mader@gmail.com> ---
Running the testsuite in the shell instead of inside the package creation process gives me an Exception and the process deadlocks in the same place. So the libcurl dependency cannot be resolved.

****** FAIL debug64 std.net.curl std.net.curl.CurlException@std/net/curl.d(3885): Failed to load curl, tried "libcurl.so", "libcurl.so.4", "libcurl-gnutls.so.4", "libcurl-nss.so.4", "libcurl.so.3".

--
November 30, 2015
https://issues.dlang.org/show_bug.cgi?id=15391

Thomas Mader <thomas.mader@gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Summary|Deadlock in std.net.curl    |Problems loading libcurl.so
                   |unittest in checkPhase of   |and running datetime
                   |NixOS package build         |unittest on NixOS package
                   |                            |build

--
November 30, 2015
https://issues.dlang.org/show_bug.cgi?id=15391

--- Comment #3 from Thomas Mader <thomas.mader@gmail.com> ---
I was able to hack around the issue by patching the curl.d sourcefile.

      #Ugly hack so the dlopen call has a chance to succeed.
      substituteInPlace src/phobos/std/net/curl.d --replace libcurl.so
${curl}/lib/libcurl.so


That's in the package description for the dmd package on NixOS. That's
necessary because in NixOS every package is stored in isolation to everything
else.
So libcurl on my system resides in
/nix/store/v5a69m1b823zm1yy8yvhrd2zi2w385b1-curl-7.44.0/lib/libcurl.so.

This also makes problems in the unittest of datetime.d. The path to the zoneinfo is hardcoded so I needed to hack around it in a similar fashion.


      #Ugly hack to fix the hardcoded path to zoneinfo in the source file.
      substituteInPlace src/phobos/std/datetime.d --replace
/usr/share/zoneinfo/ ${tzdata}/share/zoneinfo/

I think it should be possible to specify those paths somehow via the Makefile to get a clean build.

--
November 30, 2015
https://issues.dlang.org/show_bug.cgi?id=15391

Jonathan M Davis <issues.dlang@jmdavisProg.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |issues.dlang@jmdavisProg.co
                   |                            |m

--- Comment #4 from Jonathan M Davis <issues.dlang@jmdavisProg.com> ---
What are you doing differently from normal that makes it so that the normal path for the time zone files doesn't work?

--
November 30, 2015
https://issues.dlang.org/show_bug.cgi?id=15391

Martin Nowak <code@dawg.eu> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |code@dawg.eu

--- Comment #5 from Martin Nowak <code@dawg.eu> ---
(In reply to Thomas Mader from comment #3)
> I was able to hack around the issue by patching the curl.d sourcefile.
> 
>       #Ugly hack so the dlopen call has a chance to succeed.
>       substituteInPlace src/phobos/std/net/curl.d --replace libcurl.so
> ${curl}/lib/libcurl.so
> 
> 
> That's in the package description for the dmd package on NixOS. That's
> necessary because in NixOS every package is stored in isolation to
> everything else.
> So libcurl on my system resides in
> /nix/store/v5a69m1b823zm1yy8yvhrd2zi2w385b1-curl-7.44.0/lib/libcurl.so.

Maybe try to  LD_LIBRARY_PATH instead?
As mentioned here https://dlang.org/changelog/2.069.0.html#curl-dynamic-loading
you can also link against libcurl (make sure to use no-as-needed), then the
symbols will get loaded from your executable.

--
December 12, 2015
https://issues.dlang.org/show_bug.cgi?id=15391

--- Comment #6 from Thomas Mader <thomas.mader@gmail.com> ---
(In reply to Jonathan M Davis from comment #4)
> What are you doing differently from normal that makes it so that the normal path for the time zone files doesn't work?

NixOS is using a new and different approach to package and system management as
normal Linux distributions do and one aspect of that is that there is no such
thing as standard paths.
In my next reply I will attach a link to a better explanation about the
reasoning behind the NixOS ideas.

--
December 12, 2015
https://issues.dlang.org/show_bug.cgi?id=15391

--- Comment #7 from Thomas Mader <thomas.mader@gmail.com> ---
(In reply to Martin Nowak from comment #5)
> (In reply to Thomas Mader from comment #3)
> > I was able to hack around the issue by patching the curl.d sourcefile.
> > 
> >       #Ugly hack so the dlopen call has a chance to succeed.
> >       substituteInPlace src/phobos/std/net/curl.d --replace libcurl.so
> > ${curl}/lib/libcurl.so
> > 
> > 
> > That's in the package description for the dmd package on NixOS. That's
> > necessary because in NixOS every package is stored in isolation to
> > everything else.
> > So libcurl on my system resides in
> > /nix/store/v5a69m1b823zm1yy8yvhrd2zi2w385b1-curl-7.44.0/lib/libcurl.so.
> 
> Maybe try to  LD_LIBRARY_PATH instead?
> As mentioned here
> https://dlang.org/changelog/2.069.0.html#curl-dynamic-loading you can also
> link against libcurl (make sure to use no-as-needed), then the symbols will
> get loaded from your executable.

The LD_LIBRARY_PATH solution isn't viable for NixOS and even if the libcurl linking is solved, two issues remain. (Path to zoneinfo and using gcc's linker)

I think I need to explain the idea of NixOS a little bit. Since I am new to
NixOS I am not the perfect candidate to explain in detail. Therefore it's
better to read
https://www.domenkozar.com/2014/03/11/why-puppet-chef-ansible-arent-good-enough-and-we-can-do-better/
for a brief overview and the reasoning behind the NixOS idea.

I see the patching I need to do now to package dmd on NixOS as a Hack. There
needs to be a better solution.
Wouldn't it be reasonable to provide a way to define the exact path to every
external dependency?
I thought about optional Makefile arguments to set the paths which are
propageted into the build as compiler switches?
If it is considered a viable solution, I could even try to do a PR.

--
July 31, 2017
https://issues.dlang.org/show_bug.cgi?id=15391

--- Comment #8 from Thomas Mader <thomas.mader@gmail.com> ---
(In reply to Thomas Mader from comment #3)
> I was able to hack around the issue by patching the curl.d sourcefile.
> 
>       #Ugly hack so the dlopen call has a chance to succeed.
>       substituteInPlace src/phobos/std/net/curl.d --replace libcurl.so
> ${curl}/lib/libcurl.so
> 
> 
> That's in the package description for the dmd package on NixOS. That's
> necessary because in NixOS every package is stored in isolation to
> everything else.
> So libcurl on my system resides in
> /nix/store/v5a69m1b823zm1yy8yvhrd2zi2w385b1-curl-7.44.0/lib/libcurl.so.

With dmd 2.075.0 I also needed to patch the Makefile of phobos to include the full path to the shared lib:

    substituteInPlace phobos/posix.mak \
        --replace "-soname=libcurl.so.4" "-soname=${curl.out}/lib/libcurl.so.4"

--
« First   ‹ Prev
1 2