Hi,

Visual Studio has a batch file which sets all needed environment variables for the Microsoft linker (LIB environment variables points to all necessary folders). Even the folder containing the link.exe for the specified architecture (x86, amd64) is added to the path variable as first path.

> vsdevcmd -arch=amd64

As the sc.ini targets the windows development (visuals studio) after calling the developer command prompt, dmd should be able to compile -m32mscoff, -m64 out of the box without the need to change anything within sc.ini.

There is even a comment in sc.ini which points in this direction:
; Add the library subdirectories of all VC and Windows SDK versions so things
; just work for users using dmd from the VS Command Prompt

Unfortunately it does not work for various reasons:
- optlink has the filename link.exe which conflicts with the filename of the microsoft linker
- from my investigation I have the assumption that even if an environment variable LIB exists, the value is discarded by the sc.ini logic. I tried hard, but it seems existing environment variable LIB cannot be used in sc.ini

If you can use the windows installer, it will set the right values in sc.ini. But there are scenarios in which you cannot use the windows installer but have to use the dmd zip archive (build infrastructure scripts).

In my scenario it seems the only possibility is to adapt my build infrastructure scripts to:
- extract dmd archive
- delete link.exe (optlink)
- modify the sc.ini

That is a solution for my case, but my gut feeling is, it should be solved in general
to ease the usage of the microsoft linker.

I thought it was planned to rename the optlink link.exe to optlink.exe?

Kind regards
André

On Friday, 3 November 2017 at 11:52:10 UTC, Andre Pany wrote:
> Hi,
>
> Visual Studio has a batch file which sets all needed environment variables for the Microsoft linker (LIB environment variables points to all necessary folders). Even the folder containing the link.exe for the specified architecture (x86, amd64) is added to the path variable as first path.
>
>> vsdevcmd -arch=amd64
>
> As the sc.ini targets the windows development (visuals studio) after calling the developer command prompt, dmd should be able to compile -m32mscoff, -m64 out of the box without the need to change anything within sc.ini.
>
> There is even a comment in sc.ini which points in this direction:
> ; Add the library subdirectories of all VC and Windows SDK versions so things
> ; just work for users using dmd from the VS Command Prompt
>
> Unfortunately it does not work for various reasons:
> - optlink has the filename link.exe which conflicts with the filename of the microsoft linker
> - from my investigation I have the assumption that even if an environment variable LIB exists, the value is discarded by the sc.ini logic. I tried hard, but it seems existing environment variable LIB cannot be used in sc.ini
>
> If you can use the windows installer, it will set the right values in sc.ini. But there are scenarios in which you cannot use the windows installer but have to use the dmd zip archive (build infrastructure scripts).
>
> In my scenario it seems the only possibility is to adapt my build infrastructure scripts to:
> - extract dmd archive
> - delete link.exe (optlink)
> - modify the sc.ini
>
> That is a solution for my case, but my gut feeling is, it should be solved in general
> to ease the usage of the microsoft linker.
>
> I thought it was planned to rename the optlink link.exe to optlink.exe?
>
> Kind regards
> André

I have an idea which solves several problems.

Current state:
The dmd windows archive has a folder "bin" with a 32 bit dmd compiler which is able to create x86 and x86_64 applications. By default the dmd compiler creates 32 bit applications using the optlink link.exe contained also in this folder.

Idea:
Within the folder "bin64" the 64 bit dmd compiler could be shipped within the dmd windows archive. I assume the 64 bit dmd compiler is able to create x86 and x86_64 applications. In folder "bin64" no optlink link.exe should be available.
The usage of either the microsoft link.exe or another linker (maybe the one from LDC) is anticipated. The 64 bit dmd compiler could have the option -m64 set as default.

The user can decide whether he wants to use the "stable" dmd compiler with usage of optlink or he want to use the 64 bit dmd compiler which is open for other linkers than optlink by simply setting his path variable to bin or to bin64.

Using this approach:
- DMD compiler is working out of the box for the VS Developer Command Prompt - no need to rename the optlink link.exe
- A 64 bit dmd compiler is shipped which solves issues on large D projects

What do you think?

Kind regards
André

On 03.11.2017 21:51, Andre Pany wrote:
> On Friday, 3 November 2017 at 11:52:10 UTC, Andre Pany wrote:
>> Hi,
>>
>> Visual Studio has a batch file which sets all needed environment variables for the Microsoft linker (LIB environment variables points to all necessary folders). Even the folder containing the link.exe for the specified architecture (x86, amd64) is added to the path variable as first path.
>>
>>> vsdevcmd -arch=amd64
>>
>> As the sc.ini targets the windows development (visuals studio) after calling the developer command prompt, dmd should be able to compile -m32mscoff, -m64 out of the box without the need to change anything within sc.ini.
>>
>> There is even a comment in sc.ini which points in this direction:
>> ; Add the library subdirectories of all VC and Windows SDK versions so things
>> ; just work for users using dmd from the VS Command Prompt
>>
>> Unfortunately it does not work for various reasons:
>> - optlink has the filename link.exe which conflicts with the filename of the microsoft linker
>> - from my investigation I have the assumption that even if an environment variable LIB exists, the value is discarded by the sc.ini logic. I tried hard, but it seems existing environment variable LIB cannot be used in sc.ini
>>
>> If you can use the windows installer, it will set the right values in sc.ini. But there are scenarios in which you cannot use the windows installer but have to use the dmd zip archive (build infrastructure scripts).
>>
>> In my scenario it seems the only possibility is to adapt my build infrastructure scripts to:
>> - extract dmd archive
>> - delete link.exe (optlink)
>> - modify the sc.ini
>>
>> That is a solution for my case, but my gut feeling is, it should be solved in general
>> to ease the usage of the microsoft linker.
>>
>> I thought it was planned to rename the optlink link.exe to optlink.exe?
>>
>> Kind regards
>> André
> 
> I have an idea which solves several problems.
> 
> Current state:
> The dmd windows archive has a folder "bin" with a 32 bit dmd compiler which is able to create x86 and x86_64 applications. By default the dmd compiler creates 32 bit applications using the optlink link.exe contained also in this folder.
> 
> Idea:
> Within the folder "bin64" the 64 bit dmd compiler could be shipped within the dmd windows archive. I assume the 64 bit dmd compiler is able to create x86 and x86_64 applications. In folder "bin64" no optlink link.exe should be available.
> The usage of either the microsoft link.exe or another linker (maybe the one from LDC) is anticipated. The 64 bit dmd compiler could have the option -m64 set as default.
> 
> The user can decide whether he wants to use the "stable" dmd compiler with usage of optlink or he want to use the 64 bit dmd compiler which is open for other linkers than optlink by simply setting his path variable to bin or to bin64.
> 
> Using this approach:
> - DMD compiler is working out of the box for the VS Developer Command Prompt - no need to rename the optlink link.exe
> - A 64 bit dmd compiler is shipped which solves issues on large D projects
> 
> What do you think?
> 
> Kind regards
> André

Finding the wrong linker seems to happen quite often, but I wonder why:

sc.ini usually has the absolute path to the linker set via the LINKCMD variable, so finding the correct executable should not be a problem if the installer has set things up correctly. If extracted from the zip there are also defaults that might not work but should not find the wrong linker.

Is it the result of trying to fix the sc.ini manually?

The problem with overriding the LIB environment variable is that optlink and the MS linker both use it, but stumble over the libraries for the other linker due to different file formats.

I think the best option forward is to not rely on the installer to find the correct setup at installation time, but detect the available linker and its environment before invoking it:

- if the environment is already set up via vcvarsall, i.e. VCINSTALLDIR set, use that to call the approriate linker

- if VCINSTALLDIR is not set, use the registry to find the best fit, i.e. highest available VS version

- some way to override the automatism should be available, too.

We might also be able to provide some fallback if no VS installation is found, based on the LLVM linker, but with limited C runtime/platform library support.

On Saturday, 4 November 2017 at 09:39:02 UTC, Rainer Schuetze wrote:
>
>
> On 03.11.2017 21:51, Andre Pany wrote:
>> On Friday, 3 November 2017 at 11:52:10 UTC, Andre Pany wrote:
>>> [...]
>> 
>> I have an idea which solves several problems.
>> 
>> Current state:
>> The dmd windows archive has a folder "bin" with a 32 bit dmd compiler which is able to create x86 and x86_64 applications. By default the dmd compiler creates 32 bit applications using the optlink link.exe contained also in this folder.
>> 
>> Idea:
>> Within the folder "bin64" the 64 bit dmd compiler could be shipped within the dmd windows archive. I assume the 64 bit dmd compiler is able to create x86 and x86_64 applications. In folder "bin64" no optlink link.exe should be available.
>> The usage of either the microsoft link.exe or another linker (maybe the one from LDC) is anticipated. The 64 bit dmd compiler could have the option -m64 set as default.
>> 
>> The user can decide whether he wants to use the "stable" dmd compiler with usage of optlink or he want to use the 64 bit dmd compiler which is open for other linkers than optlink by simply setting his path variable to bin or to bin64.
>> 
>> Using this approach:
>> - DMD compiler is working out of the box for the VS Developer Command Prompt - no need to rename the optlink link.exe
>> - A 64 bit dmd compiler is shipped which solves issues on large D projects
>> 
>> What do you think?
>> 
>> Kind regards
>> André
>
> Finding the wrong linker seems to happen quite often, but I wonder why:
>
> sc.ini usually has the absolute path to the linker set via the LINKCMD variable, so finding the correct executable should not be a problem if the installer has set things up correctly. If extracted from the zip there are also defaults that might not work but should not find the wrong linker.
>
> Is it the result of trying to fix the sc.ini manually?
>
> The problem with overriding the LIB environment variable is that optlink and the MS linker both use it, but stumble over the libraries for the other linker due to different file formats.
>
> I think the best option forward is to not rely on the installer to find the correct setup at installation time, but detect the available linker and its environment before invoking it:
>
> - if the environment is already set up via vcvarsall, i.e. VCINSTALLDIR set, use that to call the approriate linker
>
> - if VCINSTALLDIR is not set, use the registry to find the best fit, i.e. highest available VS version
>
> - some way to override the automatism should be available, too.
>
> We might also be able to provide some fallback if no VS installation is found, based on the LLVM linker, but with limited C runtime/platform library support.

I would say, by executing vcvarsall the user showed his intention to use the coff libraries and the Microsoft linker for the actual console session instead of the omf libraries.

I see a disadvantage by detecting the vc installation directory / lib directories. All paths are specific for the vs version. If there is x new vs version we are always behind.

But Microsoft has a solution for this problem. Vcvarsall sets the linker as first element of the path variable and the lib env variable points to the right folder. This solution will work for old vs version and any upcoming version.

I set linkcmd to "link.exe". Unfortunately the link.exe in the dmd bin folder has precedence over the path env variable.

Kind regards
Andre

On 04.11.2017 12:16, Andre Pany wrote:
> On Saturday, 4 November 2017 at 09:39:02 UTC, Rainer Schuetze wrote:
>>
>>
>> On 03.11.2017 21:51, Andre Pany wrote:
>>> On Friday, 3 November 2017 at 11:52:10 UTC, Andre Pany wrote:
>>>> [...]
>>>
>>> I have an idea which solves several problems.
>>>
>>> Current state:
>>> The dmd windows archive has a folder "bin" with a 32 bit dmd compiler which is able to create x86 and x86_64 applications. By default the dmd compiler creates 32 bit applications using the optlink link.exe contained also in this folder.
>>>
>>> Idea:
>>> Within the folder "bin64" the 64 bit dmd compiler could be shipped within the dmd windows archive. I assume the 64 bit dmd compiler is able to create x86 and x86_64 applications. In folder "bin64" no optlink link.exe should be available.
>>> The usage of either the microsoft link.exe or another linker (maybe the one from LDC) is anticipated. The 64 bit dmd compiler could have the option -m64 set as default.
>>>
>>> The user can decide whether he wants to use the "stable" dmd compiler with usage of optlink or he want to use the 64 bit dmd compiler which is open for other linkers than optlink by simply setting his path variable to bin or to bin64.
>>>
>>> Using this approach:
>>> - DMD compiler is working out of the box for the VS Developer Command Prompt - no need to rename the optlink link.exe
>>> - A 64 bit dmd compiler is shipped which solves issues on large D projects
>>>
>>> What do you think?
>>>
>>> Kind regards
>>> André
>>
>> Finding the wrong linker seems to happen quite often, but I wonder why:
>>
>> sc.ini usually has the absolute path to the linker set via the LINKCMD variable, so finding the correct executable should not be a problem if the installer has set things up correctly. If extracted from the zip there are also defaults that might not work but should not find the wrong linker.
>>
>> Is it the result of trying to fix the sc.ini manually?
>>
>> The problem with overriding the LIB environment variable is that optlink and the MS linker both use it, but stumble over the libraries for the other linker due to different file formats.
>>
>> I think the best option forward is to not rely on the installer to find the correct setup at installation time, but detect the available linker and its environment before invoking it:
>>
>> - if the environment is already set up via vcvarsall, i.e. VCINSTALLDIR set, use that to call the approriate linker
>>
>> - if VCINSTALLDIR is not set, use the registry to find the best fit, i.e. highest available VS version
>>
>> - some way to override the automatism should be available, too.
>>
>> We might also be able to provide some fallback if no VS installation is found, based on the LLVM linker, but with limited C runtime/platform library support.
> 
> I would say, by executing vcvarsall the user showed his intention to use the coff libraries and the Microsoft linker for the actual console session instead of the omf libraries.
> 
> I see a disadvantage by detecting the vc installation directory / lib directories. All paths are specific for the vs version. If there is x new vs version we are always behind.
> 
> But Microsoft has a solution for this problem. Vcvarsall sets the linker as first element of the path variable and the lib env variable points to the right folder. This solution will work for old vs version and any upcoming version.

That's what I wrote above, too: the settings from vcvarsall should take precedence. The library path for the phobos library must be added, though.

> 
> I set linkcmd to "link.exe". Unfortunately the link.exe in the dmd bin folder has precedence over the path env variable.

This is caused by CreateProcess looking in the directory of the calling process first. This could be avoided in dmd by searching the PATH folders explicitly before invoking the linker.