Lately I've been looking at Zig programming language. Besides other interesting features like error handling and automatically importing C headers, one feature made me really excited. Zig includes a minified version of musl's, glibc's and mingw's CRT, so it can cross compile stuff very easy. And for whatever reason it includes a C cross compiler as well (I guess they already had libclang bundled for importing C headers, so it was kinda easy to use it to compile C).

As far as I understand there are few things that forbids LDC from cross compiling as easy as Zig:
1) Cross linker. Zig uses LLVM's linker which is called LLD. Can LDC use LLD to target all suppored operating systems?
2) Cross libc. Zig bundles minified version of musl, glibc and mingw (I didn't quite understood if it can cross compile to macosx). Do you think LDC could do the same or reuse the work done by the Zig team?
3) Cross druntime and phobos. I guess LDC could distribute prebuilt versions of druntime and phobos for most used operating system. Or just build and cache it the first time the user tries to cross compile to new target the first time. What do you think?

On Saturday, 1 August 2020 at 10:26:12 UTC, Rel wrote:
> Lately I've been looking at Zig programming language. Besides other interesting features like error handling and automatically importing C headers, one feature made me really excited. Zig includes a minified version of musl's, glibc's and mingw's CRT, so it can cross compile stuff very easy. And for whatever reason it includes a C cross compiler as well (I guess they already had libclang bundled for importing C headers, so it was kinda easy to use it to compile C).
>
> As far as I understand there are few things that forbids LDC from cross compiling as easy as Zig:
> 1) Cross linker. Zig uses LLVM's linker which is called LLD. Can LDC use LLD to target all suppored operating systems?
> 2) Cross libc. Zig bundles minified version of musl, glibc and mingw (I didn't quite understood if it can cross compile to macosx). Do you think LDC could do the same or reuse the work done by the Zig team?
> 3) Cross druntime and phobos. I guess LDC could distribute prebuilt versions of druntime and phobos for most used operating system. Or just build and cache it the first time the user tries to cross compile to new target the first time. What do you think?

IIRC the windows versions of the devel tools such as D-Scanner or DCD, available for download, are generated on CI services that run on Ubtuntu VMs and based on cross LDC abilities to cross-compile.

So this might be mostly "ok" already. Maybe not for musl but cross compilation works to some extent.

On 2020-08-01 12:26, Rel wrote:
> Lately I've been looking at Zig programming language. Besides other interesting features like error handling and automatically importing C headers, one feature made me really excited. Zig includes a minified version of musl's, glibc's and mingw's CRT, so it can cross compile stuff very easy. And for whatever reason it includes a C cross compiler as well (I guess they already had libclang bundled for importing C headers, so it was kinda easy to use it to compile C).

I've been thinking of the same thing myself. I've talked with kinke (via private email) about this. He wasn't sold on the idea. But I've started to work on a project for this [1].

> As far as I understand there are few things that forbids LDC from cross compiling as easy as Zig:
> 1) Cross linker. Zig uses LLVM's linker which is called LLD. Can LDC use LLD to target all suppored operating systems?

There are some things that are missing for LLD to work for D applications for targeting Apple platforms.

My idea was to use the official Apple linker (ld64). Someone has already ported ld64 to other Posix platforms, including MinGW. I've setup a CI job to build ld64 for Linux [2].

> 2) Cross libc. Zig bundles minified version of musl, glibc and mingw (I didn't quite understood if it can cross compile to macosx). Do you think LDC could do the same or reuse the work done by the Zig team?

It can cross-compiler to macOS, by using, in my opinion, an ugly hack. The Zig compiler adds a flag to the linker to ignore undefined symbols. Since the whole SDK is dynamically linked on Apple platforms, the only thing you need the SDK for, is to verify that the symbols the application is using exist. It cannot cross-compiler C/C++ code to macOS because it doesn't include the header files.

My solution to this problem was to generate SDK in a CI job. These days the libraries in the SDK don't contain any code, since everything is dynamically linked. Instead, the SDK consists of .tbd files, which are basically YAML files. It more or less only contains the symbol names. Apple provides a tool to generate these files based on the actual library files. I've built a tool [3] that automatically generates the SDK. Then I have a separate CI job that uses that tool to generate the SDK [4] and publish it on GitHub. I'm not sure what the Xcode license say about this. But I created the files myself and not copied any existing files.

> 3) Cross druntime and phobos. I guess LDC could distribute prebuilt versions of druntime and phobos for most used operating system. Or just build and cache it the first time the user tries to cross compile to new target the first time. What do you think?

It could, but my idea was to have more or less an archive for each target available online. The first time the target is used it will automatically download the archive, unpack it and install it so it can be used.

[1] https://github.com/d-cross-compiler

[2] https://github.com/d-cross-compiler/cctools-port/releases/tag/cctools-949.0.1-ld64-530

[3] https://github.com/d-cross-compiler/apple-sdk-generator

[4] https://github.com/d-cross-compiler/sdk-apple/releases/tag/v0.0.1

-- 
/Jacob Carlborg

On Saturday, 1 August 2020 at 10:26:12 UTC, Rel wrote:
>
> As far as I understand there are few things that forbids LDC from cross compiling as easy as Zig:
> 1) Cross linker. Zig uses LLVM's linker which is called LLD. Can LDC use LLD to target all suppored operating systems?

Potentially yes. You can use GNU linker as well.

> 2) Cross libc. Zig bundles minified version of musl, glibc and mingw (I didn't quite understood if it can cross compile to macosx). Do you think LDC could do the same or reuse the work done by the Zig team?

druntime already supports musl and probably glibc as it is usually default for Linux. I don't think we should bundle various C-libs in the D distribution as it would just baggage the D distribution and far from all would use it. Also I don't see the benefit of a "minimized" version. The C-library should be installed as part of the system with the full version. For embedded targets things starts to become more complicated as it's full of special cases.

> 3) Cross druntime and phobos. I guess LDC could distribute prebuilt versions of druntime and phobos for most used operating system. Or just build and cache it the first time the user tries to cross compile to new target the first time. What do you think?

LDC has made a compile druntime/phobos tool and this works quite well for the supported targets.


I regulary cross compile using LDC but using the GNU linker because LLD is a bit immature and this worked well for me without much hazzle. The big problem is the structure of the druntime/phobos so that it can be adapted to new systems more easily. This has been discussed in this forum previously. I see no problems supporting more types of C-libraries but unfortunately all of them have their own quirks so it is not just a plug and play operation. Before that I think we must get rid of

version(this) { } else version(that) {}

in druntime because as we support more targets it is going to become a merge hell.

On Saturday, 1 August 2020 at 10:26:12 UTC, Rel wrote:
> 1) Cross linker. Zig uses LLVM's linker which is called LLD. Can LDC use LLD to target all suppored operating systems?

Almost, as already mentioned (proper Apple targets support is underway AFAIK, and our custom Android TLS emulation doesn't work with LLD out of the box either). LLD is integrated in official LDC builds (-link-internally), but Posix systems require quite a lot of extra flags when invoking the linker directly instead of a preconfigured C compiler as linker driver.

> 2) Cross libc. Zig bundles minified version of musl, glibc and mingw (I didn't quite understood if it can cross compile to macosx). Do you think LDC could do the same or reuse the work done by the Zig team?

We have been doing so for a while with the Windows package, which includes MinGW-based libs and makes cross-compilation to Windows a trivial thing on all hosts. For Posix targets, I clearly prefer a C cross-toolchain (à la `apt-get install gcc-aarch64-linux-gnu`).

> 3) Cross druntime and phobos. I guess LDC could distribute prebuilt versions of druntime and phobos for most used operating system.

That's what we do, see https://wiki.dlang.org/Cross-compiling_with_LDC. You just need to download the according package, extract the libs and extend the ldc2.conf file. Automating these last steps (doable manually in 1-2 mins once you've already done it once) has been discussed (multiple times) before.