So, okey, bare with me here. As I once told here before the only one thing I love about Golang is the ability to easily cross-compile code from any supported OS targeting any supported OS. So I was thinking what actually stops DMD from doing the same thing? DMD has own backends targeting X86 and x64 already, it seems to work well with LLD (at least for me) that was integrated not so long ago, and LLD has the ability to cross-link.

So I'm not a compiler expert here, and I'm not familiar with the DMD code base, but from a user perspective the one huge showstopper for bringing the same level of cross-compilation to DMD like Golang has seems to be the dependency on C-library. As of 2.079 we can write code that is not dependent on the druntime and libc, so technically we can reimplement the needed parts of cruntime in D and make it the part of druntime itself. What do you think about it?

Also we will need some libraries that would generate bindings from extern functions to actual OS library code, like for example kernel32.lib on Windows. The library doesn't have actual code, it just binds the symbol to the executable import table. Pascal programming language doesn't seem to have the same notion, when you declare extern function you just declare it with the actual library name (like kernel32.dll on windows) and the compiler generates the proper imports. Can the behavior like this be implemented for DMD? How we can solve this problem otherwise?

And any other thought on the topic is welcomed, I like talking about compilers and stuff with some wise and experienced people. Thanks!

On 2018-04-27 12:56, Rel wrote:
> So, okey, bare with me here. As I once told here before the only one thing I love about Golang is the ability to easily cross-compile code from any supported OS targeting any supported OS. So I was thinking what actually stops DMD from doing the same thing? DMD has own backends targeting X86 and x64 already, it seems to work well with LLD (at least for me) that was integrated not so long ago, and LLD has the ability to cross-link.
> 
> So I'm not a compiler expert here, and I'm not familiar with the DMD code base, but from a user perspective the one huge showstopper for bringing the same level of cross-compilation to DMD like Golang has seems to be the dependency on C-library. 

DMD can cross-compile between 32-bit and 64-bit on the same platform. To targeting a different platform than the host the code in DMD needs to be reorganized a bit. When compiling the compiler it will only include support for targeting the same platform as you're compiling on. Currently it's using #ifdefs to decide which target should be supported. This occurs the compile time of the compiler. This needs to occur at runtime instead. It doesn't depend on anything expect for the standard C library so it should be fairly straight forward to fix.

> As of 2.079 we can write code that is not dependent on the druntime and libc, so technically we can reimplement the needed parts of cruntime in D and make it the part of druntime itself. What do you think about it?

That would work, but it requires a lot of effort. It's not only depending on the C standard library, it also depends on other system functionality that are not part of the kernel. For example, the thread local storage implementation depends on the platform. If we're only using the kernel that would be needed to be implemented as well.

It's possible to use the libraries provided by the target platform when cross-compiling. I've done that with LDC and created two Dockerfiles, one targeting Windows [1] and one targeting macOS [2]. Note, the SDKs for macOS and Windows are downloaded from Dropbox accounts.

[1] https://github.com/jacob-carlborg/docker-ldc-windows/blob/master/Dockerfile
[2] https://github.com/jacob-carlborg/docker-ldc-darwin/blob/master/Dockerfile

-- 
/Jacob Carlborg

On Friday, 27 April 2018 at 15:31:37 UTC, Jacob Carlborg wrote:
> DMD can cross-compile between 32-bit and 64-bit on the same platform. To targeting a different platform than the host the code in DMD needs to be reorganized a bit. When compiling the compiler it will only include support for targeting the same platform as you're compiling on. Currently it's using #ifdefs to decide which target should be supported. This occurs the compile time of the compiler. This needs to occur at runtime instead. It doesn't depend on anything expect for the standard C library so it should be fairly straight forward to fix.

Well, that's a good news.

> That would work, but it requires a lot of effort. It's not only depending on the C standard library, it also depends on other system functionality that are not part of the kernel. For example, the thread local storage implementation depends on the platform. If we're only using the kernel that would be needed to be implemented as well.

Something like memcpy and similar stuff is easy to implement.
If we are talking about Linux stuff like open, malloc and etc
can be implemented either by using syscalls or by generating
the binding to the libc.so of some minimal version that we
decide to support (like Pascal compiler do). If we are talking
about Windows all of the needed functions can be reimplemented
using API provided by kernel32.dll and friends.

> It's possible to use the libraries provided by the target platform when cross-compiling. I've done that with LDC and created two Dockerfiles, one targeting Windows [1] and one targeting macOS [2]. Note, the SDKs for macOS and Windows are downloaded from Dropbox accounts.

Downloading some SDK's from dropbox accounts is the thing I'd
like to avoid, I'd like the Dlang compiler to be a self-contained
toolchain. This doesn't mean something, just a personal preference.

On Friday, 27 April 2018 at 10:56:50 UTC, Rel wrote:
> So I was thinking what actually stops DMD from doing the same thing?

IIRC:

1. Linking. DMD uses platform linkers, and the only thing it can "link" by itself is a library (with -lib).
2. Import libraries, for Windows COFF targets. (OMF import libraries are in the Windows distribution.)
3. Cross-platform C compilation Platform headers (some parts of Phobos/Druntime are in C), with all its dependencies, i.e. C compiler support, platform and libc headers...
4. When we link to the libc statically (i.e. on Windows), that is also needed.
5. CTFE needs to emulate the target architecture, which creates some issues with things like pointer size and floating-point calculations.

Probably the closest thing to cross-platform building would be through LDC, as LLVM already includes everything for C cross compilation.

On 2018-04-28 10:49, Rel wrote:

> Something like memcpy and similar stuff is easy to implement.
> If we are talking about Linux stuff like open, malloc and etc
> can be implemented either by using syscalls or by generating
> the binding to the libc.so of some minimal version that we
> decide to support (like Pascal compiler do). If we are talking
> about Windows all of the needed functions can be reimplemented
> using API provided by kernel32.dll and friends.

For macOS I'm thinking functions that are implemented in the dynamic loader [1] or other platform specific functions [2]. For macOS the thread local implementation is in the dynamic loader. It will setup TLS variables before C main function is called. That would need to be reimplemented.

> Downloading some SDK's from dropbox accounts is the thing I'd
> like to avoid, 

If you have access to a machine running the target system you can copy/extract the SDK yourself. If you don't have there's Travis CI which has support for macOS and AppVeyor which supports Windows which you can copy the SDK from.

When it comes to the SDK for macOS we might be able to ship our own SDK. Most of the libraries are dynamic libraries. The SDK doesn't actually contain any real libraries, only YAML files containing some metadata and the symbols the library defines. We could generate those YAML files ourselves based on the real libraries.

I don't know what the SDK EULA says about this. But since we created the files I think we should be able to distribute them.

> I'd like the Dlang compiler to be a self-contained
> toolchain.
That would be ideal yes.

[1] https://github.com/dlang/druntime/blob/master/src/core/sys/darwin/mach/dyld.d

[2] https://github.com/dlang/druntime/blob/master/src/core/sys/darwin/mach/getsect.d

-- 
/Jacob Carlborg

On 2018-04-28 12:02, Vladimir Panteleev wrote:

> Probably the closest thing to cross-platform building would be through LDC, as LLVM already includes everything for C cross compilation.

Except for a C standard library.

-- 
/Jacob Carlborg