Hi everyone,

I am mainly a Linux developer and find working on Windows rather complicated. My most recent try confirmed that; I wanted to write a D application using GraphicsMagick's C interface -- now I have a mixture of compilers on my Windows machine (MinGW GCC, Msys2 GCC, MSVC, LDC, DMD) and I still get linking errors.

That only for the context. Since I previously had good experience in
using a Linux -> Windows crosscompiler for regular C/C++ (MXE[1]) I thought
it could be possible to have something similar for D. Until now I have
been using dmd, but Internet research told me dmd cannot be built as a
crosscompiler. GDC appears to be a one-man show and is still pending
upstream GCC approval, so I decided to go with LDC.

Thus, is it possible to build an LDC crosscompiler that is hosted on x86_64 Linux and targets i686/x86_64 Windows? If so, how? I didn't find any information on the topic, people appeareantly only ever want to target ARM (cell phones?), which is not what I am interested in.

Greetings
Marvin

[1]: http://mxe.cc

-- 
Blog: https://www.guelkerdev.de
PGP/GPG ID: F1D8799FBCC8BC4F

On Monday, 15 May 2017 at 09:27:36 UTC, Marvin Gülker wrote:
> Hi everyone,
>
> I am mainly a Linux developer and find working on Windows rather complicated. My most recent try confirmed that; I wanted to write a D application using GraphicsMagick's C interface -- now I have a mixture of compilers on my Windows machine (MinGW GCC, Msys2 GCC, MSVC, LDC, DMD) and I still get linking errors.

Are you mixing object files produced by different D compilers?  That won't work, you need to compile all your D code with a single D compiler.

> That only for the context. Since I previously had good experience in
> using a Linux -> Windows crosscompiler for regular C/C++ (MXE[1]) I thought
> it could be possible to have something similar for D. Until now I have
> been using dmd, but Internet research told me dmd cannot be built as a
> crosscompiler. GDC appears to be a one-man show and is still pending
> upstream GCC approval, so I decided to go with LDC.

It can be done with dmd, but it would require some work on the compiler itself.

> Thus, is it possible to build an LDC crosscompiler that is hosted on x86_64 Linux and targets i686/x86_64 Windows? If so, how? I didn't find any information on the topic, people appeareantly only ever want to target ARM (cell phones?), which is not what I am interested in.

Of course, by default ldc is a cross-compiler for all other platforms it officially supports.  You simply need to pass the right llvm triple to ldc, similar to how it's done for the ARM cross-compiler.

The bigger issue is getting the cross-compiled stdlib and a cross-linker, that will link for Windows but run on linux.  You can probably just take the pre-built stdlib from the ldc for Windows package, but you're on your own for finding a cross-linker.  Maybe lld would work?

http://lld.llvm.org

On Monday, 15 May 2017 at 09:27:36 UTC, Marvin Gülker wrote:
> Hi everyone,
>
> I am mainly a Linux developer and find working on Windows rather complicated. My most recent try confirmed that; I wanted to write a D application using GraphicsMagick's C interface -- now I have a mixture of compilers on my Windows machine (MinGW GCC, Msys2 GCC, MSVC, LDC, DMD) and I still get linking errors.
>
> That only for the context. Since I previously had good experience in
> using a Linux -> Windows crosscompiler for regular C/C++ (MXE[1]) I thought
> it could be possible to have something similar for D. Until now I have
> been using dmd, but Internet research told me dmd cannot be built as a
> crosscompiler. GDC appears to be a one-man show and is still pending
> upstream GCC approval, so I decided to go with LDC.
>
> Thus, is it possible to build an LDC crosscompiler that is hosted on x86_64 Linux and targets i686/x86_64 Windows? If so, how? I didn't find any information on the topic, people appeareantly only ever want to target ARM (cell phones?), which is not what I am interested in.
>
> Greetings
> Marvin
>
> [1]: http://mxe.cc

It should indeed be possible today. For LDC, MinGW was once supported but isn't anymore [although fixing that shouldn't be that hard]. We chose to focus on the 'native' path on Windows, centered around Microsoft's Visual C++ (its C runtime and its toolchain). So if you are fine with that, all you basically need is

a) an MS-compatible cross-linker (LLVM's lld),
b) druntime and Phobos compiled for Windows (you could just copy them from a Windows release package), and
c) a bunch of MS libs (WinSDK + Visual C++ ones)

while specifying something like `-mtriple=x86_64-pc-windows-msvc` in the LDC command line.
There's one catch though: you currently cannot specify an alternative external linker to be used for MSVC targets (`link.exe` is hardcoded), so you'd have to link manually or hack the LDC source. Generating static libs only is supported for any target starting with LDC 1.3.
Oh and lld isn't able to produce .pdb debuginfos at the moment. The plan is to integrate lld in LDC at some point, see https://github.com/ldc-developers/ldc/issues/2028.

Hi,

On Mon, May 15, 2017 at 05:58:22PM +0000, kinke via digitalmars-d-ldc wrote:
> It should indeed be possible today. For LDC, MinGW was once supported but isn't anymore [although fixing that shouldn't be that hard]. We chose to focus on the 'native' path on Windows, centered around Microsoft's Visual C++ (its C runtime and its toolchain). So if you are fine with that,

Well, "fine" does not exactly fit it. I am impressed by the pure size of the installation as such. It took over three hours to install MSVC (kinda slow hard drive) on my Windows 7 system, and it now crashes on startup with a COM exception whenever I open an SLN file. And all I want is just a compiler+linker...

> all you basically need is
> 
> a) an MS-compatible cross-linker (LLVM's lld),

That looks doable.

> b) druntime and Phobos compiled for Windows (you could just copy them from a
> Windows release package),

Okay. Ideally, of course, they could be crosscompiled as well.

> and
> c) a bunch of MS libs (WinSDK + Visual C++ ones)

Is there a list which libraries are required? Is it just those listed when I run LDC with the "-v" switch?

> while specifying something like `-mtriple=x86_64-pc-windows-msvc` in the LDC command line.

I tried, and it correctly output an .obj file that file(1) classifies as a COFF library file. Naturally, it does not do linking, most likely to the problem you described here:

> There's one catch though: you currently cannot specify an alternative external linker to be used for MSVC targets (`link.exe` is hardcoded),

The command silently fails, i.e., it does not even print an error message:

    % ldc2 -mtriple=i686-pc-windows-msvc hello.d
    % echo $?
    255

At least a short note that link(.exe) could not be found would be
appropriate I think.

As mentioned, the compilation as such appears to be successful and results in an obj file to be created:

    % ls -hl
    insgesamt 16K
    -rw-r--r-- 1 user users   73 16. Mai 08:50 hello.d
    -rw-r--r-- 1 user users 8,4K 16. Mai 08:51 hello.obj
    % file hello.obj
    hello.obj: Intel 80386 COFF object file, not stripped, 29 sections,
    symbol offset=0x1124, 111 symbols

(hello.d is a simple hello world programme in D)

> so you'd have to link manually or hack the LDC source. Generating static libs only is supported for any target starting with LDC 1.3.

For reference, I used this ldc for my tests:

    % ldc2 -version
    LDC - the LLVM D compiler (1.2.0):
      based on DMD v2.072.2 and LLVM 3.9.1
      built with DMD64 D Compiler v2.072.2
      Default target: x86_64-pc-linux-gnu
      Host CPU: ivybridge
      http://dlang.org - http://wiki.dlang.org/LDC

      Registered Targets:
        amdgcn  - AMD GCN GPUs
        bpf     - BPF (host endian)
        bpfeb   - BPF (big endian)
        bpfel   - BPF (little endian)
        nvptx   - NVIDIA PTX 32-bit
        nvptx64 - NVIDIA PTX 64-bit
        r600    - AMD GPUs HD2XXX-HD6XXX
        x86     - 32-bit X86: Pentium-Pro and above
        x86-64  - 64-bit X86: EM64T and AMD64

> Oh and lld isn't able to produce .pdb debuginfos at the moment. The plan is to integrate lld in LDC at some point, see https://github.com/ldc-developers/ldc/issues/2028.

Will the integration of lld remove the dependency on MSVC's link.exe? In that case, crosscompilation to Windows would probably be easier.

Thank you both for your suggestions. I may fiddle with the topic further by investigating lld, but if it is about to be integrated into LDC anyway, I may also just wait for that to happen. I however realised that there is an additional problem; I would have to crosscompile my C dependencies as well, targetting MSVC. That is not possible as of now to my knowledge. One can crosscompile targetting MinGW (MXE does a great job here), but not targetting MSVC. Even if it would be, I bet that people's build systems do not cover that case and thereby cause a lot of work on each dependant project's build system for me, which I don't have the time for.

Greetings
Marvin

-- 
Blog: https://www.guelkerdev.de
PGP/GPG ID: F1D8799FBCC8BC4F

On Mon, May 15, 2017 at 05:37:30PM +0000, Joakim via digitalmars-d-ldc wrote:
> Are you mixing object files produced by different D compilers?  That won't work, you need to compile all your D code with a single D compiler.

I'm sorry if I was unclear. No, I don't do that. I have, for the attempts on native Windows, however tried mixing MinGW/MSYS2 GCC object files for C code with LDC ones for D code. And I have tried mixing MSVC object files with LDC ones for D code. The latter worked, but GraphicsMagick fails to compile with MSVC 2017, and MSVC 2015 is only crashing for me. I also tried mixing MinGW/MSYS2 GCC object files for C code with DMD object files for D code, but haven't yet gotten around to trying it with MSVC C object files since MSVC has stopped working for me meanwhile.

As a non-Windows dev, this is a quite frustrating experience for me. The most annoying thing is that whatever one does with the MSVC setup installer, it takes really, really, really long. It doesn't do anything under an hour. Why is it distributed as an ISO file that I have to burn to a DVD first? That slows things further down... Since a few weeks, I finally have an acceptable Internet connection (was a max 16MBit aDSL line before), so at least downloading this insane amount of data went acceptably fast.

I have concluded from that that D+C on Windows doesn't work for me, mostly because MSVC doesn't.

I'm sorry if I am only complaining all the time. I'm an open-source developer myself, so I know that users complain all the time about this or that not fitting their exotic edgecase. I would have to work on LDC to support MinGW if I wanted it to be otherwise, but given that I have never worked on a compiler this is going to be difficult (and does not fit my time constraints).

> It can be done with dmd, but it would require some work on the compiler itself.

Likewise.

> Of course, by default ldc is a cross-compiler for all other platforms it officially supports.  You simply need to pass the right llvm triple to ldc, similar to how it's done for the ARM cross-compiler.

See my reply to the other post; that worked indeed pretty seemless.

> The bigger issue is getting the cross-compiled stdlib and a cross-linker, that will link for Windows but run on linux.  You can probably just take the pre-built stdlib from the ldc for Windows package, but you're on your own for finding a cross-linker.  Maybe lld would work?

Likewise.

Greetings
Marvin

-- 
Blog: https://www.guelkerdev.de
PGP/GPG ID: F1D8799FBCC8BC4F

On Tuesday, 16 May 2017 at 07:16:37 UTC, Marvin Gülker wrote:
> I however realised that there is an additional problem; I would have to crosscompile my C dependencies as well, targetting MSVC. That is not possible as of now to my knowledge. One can crosscompile targetting MinGW (MXE does a great job here), but not targetting MSVC. Even if it would be, I bet that people's build systems do not cover that case and thereby cause a lot of work on each dependant project's build system for me, which I don't have the time for.

Yeah, that's what I meant by whether you are fine with the toolchain. clang does an excellent job at compiling gcc-style code (incl. pragmas and special attributes) for MSVC; I used it a few years back to compile parts of the MinGW C runtime itself. Just saying that it would be possible. ;)

On Tuesday, 16 May 2017 at 07:40:59 UTC, Marvin Gülker wrote:
> On Mon, May 15, 2017 at 05:37:30PM +0000, Joakim via digitalmars-d-ldc wrote:
>> Are you mixing object files produced by different D compilers?
>>  That won't work, you need to compile all your D code with a single D compiler.
>
> I'm sorry if I was unclear. No, I don't do that. I have, for the attempts on native Windows, however tried mixing MinGW/MSYS2 GCC object files for C code with LDC ones for D code. And I have tried mixing MSVC object files with LDC ones for D code. The latter worked, but GraphicsMagick fails to compile with MSVC 2017, and MSVC 2015 is only crashing for me. I also tried mixing MinGW/MSYS2 GCC object files for C code with DMD object files for D code, but haven't yet gotten around to trying it with MSVC C object files since MSVC has stopped working for me meanwhile.

OK, sometimes mixing D compilers is the problem, so thought I'd mention it.

> As a non-Windows dev, this is a quite frustrating experience for me. The most annoying thing is that whatever one does with the MSVC setup installer, it takes really, really, really long. It doesn't do anything under an hour. Why is it distributed as an ISO file that I have to burn to a DVD first? That slows things further down... Since a few weeks, I finally have an acceptable Internet connection (was a max 16MBit aDSL line before), so at least downloading this insane amount of data went acceptably fast.

How fast is your connection now?  I've been using two connections between 2-8 Mbps until recently, so even 16 Mbps seems fast to me. :)

> I have concluded from that that D+C on Windows doesn't work for me, mostly because MSVC doesn't.

Sounds like it is the C that is the problem. ;) An alternative to Visual Studio is to download a Windows SDK that included the compiler and linker, like the Windows 7.1 SDK that I used years ago:

http://forum.dlang.org/post/hvdyyutbgehlefluvsup@forum.dlang.org

I don't know if they still include the compiler/linker in later versions of the Windows SDK, maybe one of the newer ones would work too.

> I'm sorry if I am only complaining all the time. I'm an open-source developer myself, so I know that users complain all the time about this or that not fitting their exotic edgecase. I would have to work on LDC to support MinGW if I wanted it to be otherwise, but given that I have never worked on a compiler this is going to be difficult (and does not fit my time constraints).

I don't know that you'd have to work on the compiler itself, as ldc used to support MinGW at one point.  My guess is that you'd just have to update some MinGW stuff in druntime, but I haven't looked at it.

>> The bigger issue is getting the cross-compiled stdlib and a cross-linker, that will link for Windows but run on linux.  You can probably just take the pre-built stdlib from the ldc for Windows package, but you're on your own for finding a cross-linker.  Maybe lld would work?
>
> Likewise.

This is likely not so bad, you simply have to use -v to see the linker command ldc tries to invoke and replace it by manually running with lld instead, especially since it's only needed once for the final executable.

On Tuesday, 16 May 2017 at 08:06:10 UTC, Joakim wrote:
> On Tuesday, 16 May 2017 at 07:40:59 UTC, Marvin Gülker wrote:
>> [...]
>
> OK, sometimes mixing D compilers is the problem, so thought I'd mention it.
>
>> [...]
>
> How fast is your connection now?  I've been using two connections between 2-8 Mbps until recently, so even 16 Mbps seems fast to me. :)
>
>> [...]
>
> Sounds like it is the C that is the problem. ;) An alternative to Visual Studio is to download a Windows SDK that included the compiler and linker, like the Windows 7.1 SDK that I used years ago:
>
> http://forum.dlang.org/post/hvdyyutbgehlefluvsup@forum.dlang.org
>
> I don't know if they still include the compiler/linker in later versions of the Windows SDK, maybe one of the newer ones would work too.
>
>> [...]
>
> I don't know that you'd have to work on the compiler itself, as ldc used to support MinGW at one point.  My guess is that you'd just have to update some MinGW stuff in druntime, but I haven't looked at it.
>
>> [...]
>
> This is likely not so bad, you simply have to use -v to see the linker command ldc tries to invoke and replace it by manually running with lld instead, especially since it's only needed once for the final executable.

I successfully cross compiled a simple "hello world" application on my Mac using something like:

   ldc2 -c -mtriple=x86_64-pc-windows-msvc main.d
   LIB=vc/lib lld-link main.obj ldc2_win/lib64/phobos2-ldc.lib ldc2_win/lib64/druntime-ldc.lib vc/lib/msvcrt.lib

where I've copied a bunch of libraries in vc/lib, like the mscvrt.lib, shell32.lib etc, coming out from a previous wine attempt to use the microsoft linker in a dockerised environment.

/Paolo

On Tue, May 16, 2017 at 07:49:53AM +0000, kinke via digitalmars-d-ldc wrote:
> Yeah, that's what I meant by whether you are fine with the toolchain.

Heh. After running MSVC Setup's "Repair" program (which, again, took an hour or so) Visual Studio now starts. And then closes with the message "Your evaluation time span has expired" (although I never even used it). I don't think I'm going to feel familiar with MSVC at any point in time. What a crude world Windows is.

> clang does an excellent job at compiling gcc-style code (incl. pragmas and special attributes) for MSVC; I used it a few years back to compile parts of the MinGW C runtime itself. Just saying that it would be possible. ;)

Yeah, I know, it's possible :-). It's just a question of how much time I want to investigate. But honestly, for this one programme I want to run on Windows, it will take less time if I write it in C/C++ and crosscompile it with MXE (or compile it natively with MSYS2), even if D is a nicer language. If I find fun in developing on LDC, that's something different, but as only an intermediate goal I'm not willing to investigate that much.

Greetings
Marvin

-- 
Blog: https://www.guelkerdev.de
PGP/GPG ID: F1D8799FBCC8BC4F

On Tue, May 16, 2017 at 08:06:10AM +0000, Joakim via digitalmars-d-ldc wrote:
> How fast is your connection now?  I've been using two connections between 2-8 Mbps until recently, so even 16 Mbps seems fast to me. :)

That's indeed fairly slow; the >5 GiB download of MSVC would take very long on that connection. I'm on 100 MBit/s now (at least they say...). With such downloads I'm quite happy about that now.

> > I have concluded from that that D+C on Windows doesn't work for me, mostly because MSVC doesn't.
> 
> Sounds like it is the C that is the problem. ;)

Yep. GraphicsMagick wants to be built via a VisualStudio SLN file, and
appearently doesn't work with Visual Studio 2017, because Microsoft
changed some default values on the linker. Visual Studio 2015 has
decided to not let me use it (see my other reply), so all I can do now
is compile it with MinGW/MSYS2's GCC (which worked out of the
box). However, that one in turn does not include GraphicsMagick's
dependencies; copying them out of the MinGW system was possible
(renaming the .a files to .lib), but linking in things like libgomp
fails again (I would have been surprised if linking in GCC internals
with the MSVC linker worked anyway). Then, one shouldn't have to copy
around these files and rename them anyway; it's hackish. In total, I
have invested nearly three days into the issue now, and as much as I
like the D language, I am coming to a point where I conclude that it's
simply not worth further pursueing that. Just doing C/C++ is easier.

Suggestion for LDC: Accept .a files instead of rejecting them only based on the file extension. Renaming them to .lib usually makes things work as the MSVC linker link.exe understands them. That would at least prevent the copy+rename procedure.

(.a is the extension used for static libraries on Unixish systems, and is used by MinGW/MSYS2 as well)

Next thing I should probably do is report a bug against GraphicsMagick that it doesn't built with MSVC 2017, but I am too frustrated with the issue at the moment and they use SourceForge, which I don't want to sign up at as they recently decided to include Malware into popular open-source programme's installers.

> An alternative to Visual
> Studio is to download a Windows SDK that included the compiler and linker,
> like the Windows 7.1 SDK that I used years ago:
> 
> http://forum.dlang.org/post/hvdyyutbgehlefluvsup@forum.dlang.org
> 
> I don't know if they still include the compiler/linker in later versions of the Windows SDK, maybe one of the newer ones would work too.

That certainly is an interesting suggestion, but since GraphicsMagick insists on using an SLN file when compiling for MSVC, I don't see how I could work around installing a full-blown Visual Studio. Once upon a time, there was an nmake programme that was used for easy commandline compilation of MSVC projects, but it appears to have become unpopular (although it is still included in Visual Studio I saw).

> I don't know that you'd have to work on the compiler itself, as ldc used to support MinGW at one point.  My guess is that you'd just have to update some MinGW stuff in druntime, but I haven't looked at it.

As I explained in my other reply, I could certainly do that, but only as a project of its own, not as an intermediate step while aiming for simply developing an ordinary Windows application. I have withdrawn from my main open-source project[1] recently due to time constraints, and I'm not able to replace that with another open-source project at the moment.

Greetings
Marvin

[1]: https://secretchronicles.org/en/

-- 
Blog: https://www.guelkerdev.de
PGP/GPG ID: F1D8799FBCC8BC4F