Hello all,

As some have you may have followed, I've been working on snap-packaging LDC.  However, I've run into an issue when it comes to programs that use std.net.curl.

Here's a simple example:

------------------------
void main ()
{
    import std.net.curl : get;
    auto website = "http://dlang.org/".get;
}
------------------------

When I compile with my snap-packaged LDC, the program builds, but segfaults when I run it.  Here's the gdb backtrace:

Thread 1 "curlget" received signal SIGSEGV, Segmentation fault.
0x000000000046b048 in gc.gc.Gcx.smallAlloc(ubyte, ref ulong, uint) ()
(gdb) bt
#0  0x000000000046b048 in gc.gc.Gcx.smallAlloc(ubyte, ref ulong, uint) ()
#1  0x000000000046918a in gc.gc.GC.malloc(ulong, uint, ulong*, const(TypeInfo)) ()
#2  0x00000000004682bc in gc_qalloc ()
#3  0x000000000046293b in core.memory.GC.qalloc(ulong, uint, const(TypeInfo)) ()
#4  0x00000000004534f2 in std.array.Appender!(char[]).Appender.ensureAddable(ulong) ()
#5  0x00000000004530c5 in std.uni.toCase!(std.uni.toLowerIndex(dchar), 1043, std.uni.toLowerTab(ulong), char[]).toCase(char[]).__foreachbody2(ref ulong, ref dchar) ()
#6  0x000000000046dc96 in _aApplycd2 ()
#7  0x000000000044d997 in std.net.curl.HTTP.Impl.onReceiveHeader(void(const(char[]), const(char[])) delegate).__lambda2(const(char[])) ()
#8  0x0000000000451d78 in std.net.curl.Curl._receiveHeaderCallback(const(char*), ulong, ulong, void*) ()
#9  0x00007ffff6f7695d in ?? () from /usr/lib/x86_64-linux-gnu/libcurl.so
#10 0x00007ffff6f74ed0 in ?? () from /usr/lib/x86_64-linux-gnu/libcurl.so
#11 0x00007ffff6f8cce0 in ?? () from /usr/lib/x86_64-linux-gnu/libcurl.so
#12 0x00007ffff6f96b22 in ?? () from /usr/lib/x86_64-linux-gnu/libcurl.so
#13 0x00007ffff6f97986 in curl_multi_perform () from /usr/lib/x86_64-linux-gnu/libcurl.so
#14 0x00007ffff6f8e65b in curl_easy_perform () from /usr/lib/x86_64-linux-gnu/libcurl.so
#15 0x000000000044f971 in std.net.curl.HTTP.perform(std.typecons.Flag!("throwOnError").Flag) ()
#16 0x000000000040db01 in std.net.curl._basicHTTP!(char)._basicHTTP(const(char)[], const(void)[], std.net.curl.HTTP) ()
#17 0x00000000004031ff in std.net.curl.get!(std.net.curl.HTTP, char).get(const(char)[], std.net.curl.HTTP) ()
#18 0x0000000000403020 in std.net.curl.get!(std.net.curl.AutoProtocol, char).get(const(char)[], std.net.curl.AutoProtocol) ()
#19 0x0000000000402f98 in D main ()

Since I'm not super-familiar with libcurl, it's a bit un-obvious what's going on here.  If I understand right, libcurl is lazily loaded at runtime rather than explicitly linked.  libcurl4-gnutls-dev is installed on my machine, so it can't just be lack of availability.  (I've also tried uninstalling libcurl4-gnutls-dev and installing libcurl4-openssl-dev instead, with no difference in the result.)

The segfault would suggest to me that either the loading of the library fails or that there's some resource phobos expects to find which it can't access.  Can anyone advise what could be going on here?

In particular, is there anything that happens at compile-time which could affect the ability of the resulting executable to locate libcurl.so or any other resources it needs?  My suspicion is that the snap-package containerization of LDC could mean that expectations about system-path locations are interfered with.

Can anyone advise?

Thanks & best wishes,

    -- Joe

On Thursday, 22 September 2016 at 23:19:27 UTC, Joseph Rushton Wakeling wrote:
>
> The segfault would suggest to me that either the loading of the library fails or that there's some resource phobos expects to find which it can't access.  Can anyone advise what could be going on here?
>     -- Joe

> /usr/lib/x86_64-linux-gnu/libcurl.so
> #13 0x00007ffff6f97986 in curl_multi_perform () from /usr/lib/x86_64-linux-gnu/libcurl.so
> #14 0x00007ffff6f8e65b in curl_easy_perform () from /usr/lib/x86_64-linux-gnu/libcurl.so

This suggests that libcurl is loaded.
could you compile with -g ?
and then post the output ?

Also the call seems to fail during allocation.

On Friday, 23 September 2016 at 00:55:43 UTC, Stefan Koch wrote:
> This suggests that libcurl is loaded.
> could you compile with -g ?
> and then post the output ?

Thanks Stefan!  It was compiled with -g, but I was missing the libcurl3-dbg package.  Here's the results:

#0  0x000000000046b048 in gc.gc.Gcx.smallAlloc(ubyte, ref ulong, uint) ()
#1  0x000000000046918a in gc.gc.GC.malloc(ulong, uint, ulong*, const(TypeInfo)) ()
#2  0x00000000004682bc in gc_qalloc ()
#3  0x000000000046293b in core.memory.GC.qalloc(ulong, uint, const(TypeInfo)) ()
#4  0x00000000004534f2 in std.array.Appender!(char[]).Appender.ensureAddable(ulong) ()
#5  0x00000000004530c5 in std.uni.toCase!(std.uni.toLowerIndex(dchar), 1043, std.uni.toLowerTab(ulong), char[]).toCase(char[]).__foreachbody2(ref ulong, ref dchar) ()
#6  0x000000000046dc96 in _aApplycd2 ()
#7  0x000000000044d997 in std.net.curl.HTTP.Impl.onReceiveHeader(void(const(char[]), const(char[])) delegate).__lambda2(const(char[])) ()
#8  0x0000000000451d78 in std.net.curl.Curl._receiveHeaderCallback(const(char*), ulong, ulong, void*) ()
#9  0x00007ffff6f75a1d in Curl_client_chop_write (conn=conn@entry=0x741fd0, type=type@entry=2,
    ptr=0x72d3a0 "Server: Apache/2.4.17 (FreeBSD) OpenSSL/1.0.2d PHP/5.6.16\r\n", len=59) at sendf.c:454
#10 0x00007ffff6f75b9d in Curl_client_write (conn=conn@entry=0x741fd0, type=type@entry=2,
    ptr=<optimised out>, len=<optimised out>) at sendf.c:511
#11 0x00007ffff6f73f90 in Curl_http_readwrite_headers (data=data@entry=0x72fb60,
    conn=conn@entry=0x741fd0, nread=nread@entry=0x7fffffffd370,
    stop_reading=stop_reading@entry=0x7fffffffd36f) at http.c:3739
#12 0x00007ffff6f8be30 in readwrite_data (done=0x7fffffffd40b, didwhat=<synthetic pointer>, k=0x72fbd8,
    conn=0x741fd0, data=0x72fb60) at transfer.c:492
#13 Curl_readwrite (conn=0x741fd0, data=data@entry=0x72fb60, done=done@entry=0x7fffffffd40b)
    at transfer.c:1074
#14 0x00007ffff6f95f42 in multi_runsingle (multi=multi@entry=0x7388b0, now=...,
    data=data@entry=0x72fb60) at multi.c:1544
#15 0x00007ffff6f96ddd in curl_multi_perform (multi_handle=multi_handle@entry=0x7388b0,
    running_handles=running_handles@entry=0x7fffffffd5a4) at multi.c:1821
#16 0x00007ffff6f8d84b in easy_transfer (multi=0x7388b0) at easy.c:724
#17 easy_perform (events=false, data=0x72fb60) at easy.c:812
#18 curl_easy_perform (easy=0x72fb60) at easy.c:831
#19 0x000000000044f971 in std.net.curl.HTTP.perform(std.typecons.Flag!("throwOnError").Flag) ()
#20 0x000000000040db01 in std.net.curl._basicHTTP!(char)._basicHTTP(const(char)[], const(void)[], std.net.curl.HTTP) ()
#21 0x00000000004031ff in std.net.curl.get!(std.net.curl.HTTP, char).get(const(char)[], std.net.curl.HTTP) ()
#22 0x0000000000403020 in std.net.curl.get!(std.net.curl.AutoProtocol, char).get(const(char)[], std.net.curl.AutoProtocol) ()
#23 0x0000000000402f98 in D main ()


> Also the call seems to fail during allocation.

Yup.  And that allocation seems to occur during use of an Appender while converting unicode text to lowercase ... ?

On Saturday, 24 September 2016 at 19:11:52 UTC, Joseph Rushton Wakeling wrote:
> On Friday, 23 September 2016 at 00:55:43 UTC, Stefan Koch wrote:
>> This suggests that libcurl is loaded.
>> could you compile with -g ?
>> and then post the output ?

Further to earlier remarks: I now think this may be a general problem of LDC 1.0.0 and not a problem of the snap package.

I tried building my simple curl-using program using an LDC 1.0.0 build and installed from source in the standard cmake && make && make install fashion.  The same segfault occurs.

On Saturday, 24 September 2016 at 19:27:31 UTC, Joseph Rushton Wakeling wrote:
> Further to earlier remarks: I now think this may be a general problem of LDC 1.0.0 and not a problem of the snap package.
>
> I tried building my simple curl-using program using an LDC 1.0.0 build and installed from source in the standard cmake && make && make install fashion.  The same segfault occurs.

More on this: ldc 0.17.1 (the version packaged with Ubuntu 16.04) is based on dmd 2.068.2, which still linked against libcurl.  It's only from v2.069.0+ that libcurl is loaded dynamically:
https://dlang.org/changelog/2.069.0.html#curl-dynamic-loading

ldc 1.0.0 is based on 2.070.2.

On Saturday, 24 September 2016 at 19:42:11 UTC, Joseph Rushton Wakeling wrote:
> On Saturday, 24 September 2016 at 19:27:31 UTC, Joseph Rushton Wakeling wrote:
>> Further to earlier remarks: I now think this may be a general problem of LDC 1.0.0 and not a problem of the snap package.
>>
>> I tried building my simple curl-using program using an LDC 1.0.0 build and installed from source in the standard cmake && make && make install fashion.  The same segfault occurs.
>
> More on this: ldc 0.17.1 (the version packaged with Ubuntu 16.04) is based on dmd 2.068.2, which still linked against libcurl.  It's only from v2.069.0+ that libcurl is loaded dynamically:
> https://dlang.org/changelog/2.069.0.html#curl-dynamic-loading
>
> ldc 1.0.0 is based on 2.070.2.

Downloaded a pre-built copy of ldc 1.0.0 from here (I used the Linux x86_64 tarball):
https://github.com/ldc-developers/ldc/releases/tag/v1.0.0

... and this runs my little curl-based program without any trouble.  Maybe something about how the build was carried out?