Hi forums, I have what should be a simple question.  I have an application which I would like to distribute to the cloud onto a linux system which I will not have root access.  I thought it should be easier for me to perform static linking of this D app instead of packing the application with library dependencies.

If I try to do this, using my most simple example below, with ldc2 (1.15.0) using the --static flag, I get various warnings: warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
warning: Using 'gethostbyaddr' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking

If I use gdc (4.9.0 2.065) for this, I get linker errors, such as: undefined reference to `curl_easy_setopt'

I'll go back and review the best practices for using curl with phobos.  I still am struggling to make this work, as something that would be relatively simple in C to perform (statically link to any libraries I would like).  Without being an expert on phobos library construction, what is the recommended practice for app distribution?  Does anyone have a great sample project to point to?  I've looked at the dlang-tour https://github.com/dlang-tour/core/blob/master/dub.sdl , and this indeed seems like the right track, however, I still get the curl warnings and the app won't run.

I have also tried building and dynamically linking, which works fine, but I've had trouble pulling the dependent libraries for distribution, and namely having trouble with finding curl at runtime.

```
import std.net.curl, std.stdio;
int main() {
  auto http = HTTP("dlang.org");
  http.onReceiveHeader = (in char[] key, in char[] value) { writeln(key, ": ", value);  };
  http.onReceive = (ubyte[] data) { /+ drop +/ return data.length;  };
  http.perform();
  return 0;
}
```

On Tuesday, 8 October 2019 at 18:28:14 UTC, David J Kordsmeier wrote:
> I'll go back and review the best practices for using curl with phobos.  I still am struggling to make this work, as something that would be relatively simple in C to perform (statically link to any libraries I would like).

Phobos 2.069 switched to loading libcurl at runtime, see https://dlang.org/changelog/2.069.0.html#curl-dynamic-loading. It mentions that it's still possible to link it statically into the executable.

On Tuesday, 8 October 2019 at 18:28:14 UTC, David J Kordsmeier wrote:
> Hi forums, I have what should be a simple question.  I have an application which I would like to distribute to the cloud onto a linux system which I will not have root access.  I thought it should be easier for me to perform static linking of this D app instead of packing the application with library dependencies.
>
> [...]

What do you need from Curl? There is also the option to not use Curl but e.g. the http client from here https://github.com/adamdruppe/arsd/blob/master/http2.d

There is also vibe-d and several other http implementations available.

Kind regards
Andre

On Tuesday, 8 October 2019 at 18:28:14 UTC, David J Kordsmeier wrote:
> warning: Using 'gethostbyaddr' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking

This is a well-known problem with GNU libc. Your best bets are

1. accepting dynamic linking on the target platform, and setting up a
build system that's similar enough to it that deployment is easy.
Maybe setting LD_LIBRARY_PATH to point to deployed libraries that you
can't get installed on the taget.

2. using something like flatpak or appimage.

3. using DNS resolution from a library outside of libc, and hacking
phobos to use that instead of gethostbyaddr.

On Tuesday, 8 October 2019 at 18:28:14 UTC, David J Kordsmeier wrote:
> Hi forums, I have what should be a simple question.  I have an application which I would like to distribute to the cloud onto a linux system which I will not have root access.  I thought it should be easier for me to perform static linking of this D app instead of packing the application with library dependencies.
>
> [...]

Take a look at https://github.com/ikod/dlang-requests

It offers more features than the libcurl wrapper in std.net.curl.

On Tuesday, 8 October 2019 at 19:35:03 UTC, kinke wrote:
> On Tuesday, 8 October 2019 at 18:28:14 UTC, David J Kordsmeier wrote:
>> I'll go back and review the best practices for using curl with phobos.  I still am struggling to make this work, as something that would be relatively simple in C to perform (statically link to any libraries I would like).
>
> Phobos 2.069 switched to loading libcurl at runtime, see https://dlang.org/changelog/2.069.0.html#curl-dynamic-loading. It mentions that it's still possible to link it statically into the executable.

The TSV Utilities have used static linking for Linux prebuilt release binaries for a couple years now without any problems. The tools do not use curl or gethostbyaddr, but there are still plenty of warning messages. (Example: https://travis-ci.org/eBay/tsv-utils/jobs/593432965)

Static linking was suggested by Jacob Carlborg, who pointed out that the pre-built binaries wouldn't run on all Linux distributions. Static linking fixed it, with no subsequent problems reported (https://github.com/eBay/tsv-utils/issues/67).

On Tuesday, 8 October 2019 at 22:23:39 UTC, Jon Degenhardt wrote:
> On Tuesday, 8 October 2019 at 19:35:03 UTC, kinke wrote:
>> On Tuesday, 8 October 2019 at 18:28:14 UTC, David J Kordsmeier wrote:
>>> I'll go back and review the best practices for using curl with phobos.  I still am struggling to make this work, as something that would be relatively simple in C to perform (statically link to any libraries I would like).
>>
>> Phobos 2.069 switched to loading libcurl at runtime, see https://dlang.org/changelog/2.069.0.html#curl-dynamic-loading. It mentions that it's still possible to link it statically into the executable.
>
> The TSV Utilities have used static linking for Linux prebuilt release binaries for a couple years now without any problems. The tools do not use curl or gethostbyaddr, but there are still plenty of warning messages. (Example: https://travis-ci.org/eBay/tsv-utils/jobs/593432965)
>
> Static linking was suggested by Jacob Carlborg, who pointed out that the pre-built binaries wouldn't run on all Linux distributions. Static linking fixed it, with no subsequent problems reported (https://github.com/eBay/tsv-utils/issues/67).

Thank you all forum folks for the replies.  My use case for curl is simply HTTP/HTTPS, not libcurl specifically.  That said, I was making a best effort to not use external dependencies per my requirements.  My feeling is that a platform should provide needs for modern computing, and web is a part of that.  I hope this question "goes away" in the future and there becomes a modern library alternative to the std.net for http as standard part of phobos.

So a number of great responses here, and I am going to explore the export option as well as the http2 (from arsd) option.  John, I'll check out the tsv-utils build as well, your CI server set up looks really great.  I am very interested to learn the best practices of distribution of prebuilt D apps and libraries.

On Wednesday, 9 October 2019 at 01:10:23 UTC, David J Kordsmeier wrote:
> HTTP/HTTPS
>
> as well as the http2 (from arsd) option.


So let me warn you ahead of time that my http2.d still has an outside dependency: OpenSSL. And that can sometimes be tricky as openssl broke binary compatibility between 1.1 and 1.0. I was planning on making it dynamically load that to work around it :S

so your -static build MIGHT work just I'm not sure. TLS/SSL is one of the few things I don't DIY and feel a third-party dependency is justified on, I just try to make it seamless and... it usually works on Linux but not always.

Helper switches: -version=older_openssl if you get linker errors and have 1.0 on the system or -version=newer_openssl if you get linker errors and have 1.1 on.

I *have* statically linked OpenSSL with this library before so I know it works, just I also had to jump through a few hoops to make it work. However I was on Windows then so Linux might be easier... or might be harder. I don't know. but if it does work that also gives a nice way around the 1.0/1.1 problem!

if you do try it lemme know how it goes plz.

On Wednesday, 9 October 2019 at 01:29:59 UTC, Adam D. Ruppe wrote:
> On Wednesday, 9 October 2019 at 01:10:23 UTC, David J Kordsmeier wrote:
>> HTTP/HTTPS
>>
>> as well as the http2 (from arsd) option.
>
>
> So let me warn you ahead of time that my http2.d still has an outside dependency: OpenSSL. And that can sometimes be tricky as openssl broke binary compatibility between 1.1 and 1.0. I was planning on making it dynamically load that to work around it :S
>
> so your -static build MIGHT work just I'm not sure. TLS/SSL is one of the few things I don't DIY and feel a third-party dependency is justified on, I just try to make it seamless and... it usually works on Linux but not always.
>
> Helper switches: -version=older_openssl if you get linker errors and have 1.0 on the system or -version=newer_openssl if you get linker errors and have 1.1 on.
>
> I *have* statically linked OpenSSL with this library before so I know it works, just I also had to jump through a few hoops to make it work. However I was on Windows then so Linux might be easier... or might be harder. I don't know. but if it does work that also gives a nice way around the 1.0/1.1 problem!
>
> if you do try it lemme know how it goes plz.


Thank you Adam for your work in the D community.  Thanks for the heads up.  Yes, definitely OpenSSL (or any security library) isn't had for free in a secure Internet and I will need to make some effort to ensure I get that working.  In my limited use case (does not need to run ANYWHERE, but in a specific environment) it should be possible.  Will update as soon as I know.

On Tuesday, 8 October 2019 at 18:28:14 UTC, David J Kordsmeier wrote:
> Hi forums, I have what should be a simple question.  I have an application which I would like to distribute to the cloud onto a linux system which I will not have root access.  I thought it should be easier for me to perform static linking of this D app instead of packing the application with library dependencies.

Alternatively you can build inside a Docker container with the same distribution and version as the machine running in the cloud. Then you should be able to do dynamic linking and run the binary in the cloud without any problem.

--
/Jacob Carlborg