Thread overview
Specify LDC to compile specific files
Mar 02, 2019
Michelle Long
Mar 02, 2019
Rainer Schuetze
Mar 02, 2019
Jacob Carlborg
Mar 03, 2019
Michelle Long
Mar 03, 2019
Rainer Schuetze
Mar 04, 2019
Jacob Carlborg
Mar 05, 2019
Michelle Long
Mar 05, 2019
Jacob Carlborg
Mar 05, 2019
Michelle Long
Mar 08, 2019
Jacob Carlborg
March 02, 2019
The problem with LDC is that it is very slow to compile. About 10x slower than dmd. Unfortunately certain things need ldc to function such as dcompute.

It would be nice if one could specify certain files to compile for ldc only.


I guess the only way to accomplish this is to create a new project in the solution and use LDC for that?
March 02, 2019

On 02/03/2019 05:49, Michelle Long wrote:
> The problem with LDC is that it is very slow to compile. About 10x slower than dmd. Unfortunately certain things need ldc to function such as dcompute.

I think this varies quite a bit with the layout of the project and the size of the source files. For example, a debug build of dmd itself compiles a bit slower with LDC than with DMD, but the release build actually compiles faster with LDC.

> 
> It would be nice if one could specify certain files to compile for ldc only.
> 
> 
> I guess the only way to accomplish this is to create a new project in the solution and use LDC for that?

Yes, you could try to build a library with LDC. I would not bet on it to be ABI compatible with DMD, though. For example, real is 80-bit for DMD, but 64-bit with LDC.
March 02, 2019
On 2019-03-02 05:49, Michelle Long wrote:
> The problem with LDC is that it is very slow to compile. About 10x slower than dmd. Unfortunately certain things need ldc to function such as dcompute.
> 
> It would be nice if one could specify certain files to compile for ldc only.
> 
> 
> I guess the only way to accomplish this is to create a new project in the solution and use LDC for that?

Perhaps this is a question specific for Visual Studio and I'm not sure exactly what you're looking for. But generally speaking you could add `version (LDC):` at the top of a file and it would only be compiled if LDC is the compiler.

If you're using Dub you can do something like:

"sourceFiles-ldc": ["some_file.d"]

-- 
/Jacob Carlborg
March 03, 2019
On Saturday, 2 March 2019 at 17:03:20 UTC, Jacob Carlborg wrote:
> On 2019-03-02 05:49, Michelle Long wrote:
>> The problem with LDC is that it is very slow to compile. About 10x slower than dmd. Unfortunately certain things need ldc to function such as dcompute.
>> 
>> It would be nice if one could specify certain files to compile for ldc only.
>> 
>> 
>> I guess the only way to accomplish this is to create a new project in the solution and use LDC for that?
>
> Perhaps this is a question specific for Visual Studio and I'm not sure exactly what you're looking for. But generally speaking you could add `version (LDC):` at the top of a file and it would only be compiled if LDC is the compiler.
>

But one can only select one compiler using the build configuration. So in Visual D one has to use one compiler per project(one can switch at a click but that is it).

So version is useless to be able to do both.

Visual D could have a combo but it would then require versioning all the files.


If they are not ABI it might be a real problem getting it to work ;/


> If you're using Dub you can do something like:
>
> "sourceFiles-ldc": ["some_file.d"]


And doing so will use dmd for all the others, compile some_file.d using ldc and then work all the details out?

This is essentially what I'm asking for in Visual D so to speak. To mark a file to use LDC and it will effectively do what dub is doing(if it is doing it as I stated).

March 03, 2019

On 03/03/2019 17:44, Michelle Long wrote:
> On Saturday, 2 March 2019 at 17:03:20 UTC, Jacob Carlborg wrote:
>> On 2019-03-02 05:49, Michelle Long wrote:
>>> The problem with LDC is that it is very slow to compile. About 10x slower than dmd. Unfortunately certain things need ldc to function such as dcompute.
>>>
>>> It would be nice if one could specify certain files to compile for ldc only.
>>>
>>>
>>> I guess the only way to accomplish this is to create a new project in the solution and use LDC for that?
>>
>> Perhaps this is a question specific for Visual Studio and I'm not sure exactly what you're looking for. But generally speaking you could add `version (LDC):` at the top of a file and it would only be compiled if LDC is the compiler.
>>
> 
> But one can only select one compiler using the build configuration. So in Visual D one has to use one compiler per project(one can switch at a click but that is it).
> 
> So version is useless to be able to do both.
> 
> Visual D could have a combo but it would then require versioning all the files.
> 
> 
> If they are not ABI it might be a real problem getting it to work ;/
> 
> 
>> If you're using Dub you can do something like:
>>
>> "sourceFiles-ldc": ["some_file.d"]
> 
> 
> And doing so will use dmd for all the others, compile some_file.d using ldc and then work all the details out?
> 
> This is essentially what I'm asking for in Visual D so to speak. To mark a file to use LDC and it will effectively do what dub is doing(if it is doing it as I stated).
> 

Reading the documentation at https://dub.pm/package-format-json.html#build-settings I assume it adds source files specific to the compiler selection, but won't mix different compilers.

Maybe a mixture works if you only declare extern(C) functions and use -betterC, but that doesn't seem like a common use case.

With a bit of manual configuration, you could try to build the specific files with the "custom" build tool, but you'll have to setup the full command line and dependencies yourself.
March 04, 2019
On 2019-03-03 17:44, Michelle Long wrote:

> But one can only select one compiler using the build configuration. So in Visual D one has to use one compiler per project(one can switch at a click but that is it).
> 
> So version is useless to be able to do both.
> 
> Visual D could have a combo but it would then require versioning all the files.

Aha, you mean like that.

> If they are not ABI it might be a real problem getting it to work ;/

The ABIs of DMD and LDC don't exactly the same. You should not expect linking object files compiled with both compilers to work.

>> If you're using Dub you can do something like:
>>
>> "sourceFiles-ldc": ["some_file.d"]
> 
> 
> And doing so will use dmd for all the others, compile some_file.d using ldc and then work all the details out?

No, it will compile "some_file.d" if the compiler being used is LDC, otherwise it will not be compiled.

> This is essentially what I'm asking for in Visual D so to speak. To mark a file to use LDC and it will effectively do what dub is doing(if it is doing it as I stated).

I misunderstood your question.

-- 
/Jacob Carlborg
March 05, 2019
On Monday, 4 March 2019 at 16:59:10 UTC, Jacob Carlborg wrote:
> On 2019-03-03 17:44, Michelle Long wrote:
>
>> But one can only select one compiler using the build configuration. So in Visual D one has to use one compiler per project(one can switch at a click but that is it).
>> 
>> So version is useless to be able to do both.
>> 
>> Visual D could have a combo but it would then require versioning all the files.
>
> Aha, you mean like that.
>
>> If they are not ABI it might be a real problem getting it to work ;/
>
> The ABIs of DMD and LDC don't exactly the same. You should not expect linking object files compiled with both compilers to work.
>
>>> If you're using Dub you can do something like:
>>>
>>> "sourceFiles-ldc": ["some_file.d"]
>> 
>> 
>> And doing so will use dmd for all the others, compile some_file.d using ldc and then work all the details out?
>
> No, it will compile "some_file.d" if the compiler being used is LDC, otherwise it will not be compiled.
>
>> This is essentially what I'm asking for in Visual D so to speak. To mark a file to use LDC and it will effectively do what dub is doing(if it is doing it as I stated).
>
> I misunderstood your question.

It's quite simple:

LDC takes about 10x longer to compile a project. But not all of a project needs LDC's optimization. Hence if one could easily split up parts and optimize those parts that need it then it would be better.

Ideally it might go like this:

pragma(compiler, LDC)
auto foo()


and the compiler would compile foo using ldc(somehow) and everything would work...

I realize that is a pipe dream but that would be a nice goal...

or have LDC itself be the main compiler but it would invoke dmd, which could be done by default, and then a pragma or file specifier would cause it to use the ldc compiler...

but if things are not ABI compatible I guess it won't work.

I don't know enough about the ABI specifics but if functions could be specified  such as the pragma and the machine could could directly be replaced then it shouldn't be a problem. If it would depend on parameters and globals then they could be checked for ABI compatibility and if it fails it won't compile, else it would.

The goal would be to avoid having to maintain multiple projects to do simple optimizations of functions and deal with all the those problems(essentially having to use libs/dlls(which still has the ABI issues anyways).

After all, really we are just talking about the specific machine code generated but they are functionally the same(or should be) so it shouldn't matter too much except for stuff passed in and out.

Know what the specific ABI issues are?
March 05, 2019
On 2019-03-05 17:40, Michelle Long wrote:

> Know what the specific ABI issues are?

LDC has a slightly modified version of druntime. When using "new" on a class, the compiler lowers that to a function call implemented in druntime. The DMD and LDC versions of this function is different. LDC uses a function called "_d_allocclass" and DMD a function called "_d_newclass". The LDC function only allocates memory (I believe), without doing any initialization.

-- 
/Jacob Carlborg
March 05, 2019
On Tuesday, 5 March 2019 at 18:52:45 UTC, Jacob Carlborg wrote:
> On 2019-03-05 17:40, Michelle Long wrote:
>
>> Know what the specific ABI issues are?
>
> LDC has a slightly modified version of druntime. When using "new" on a class, the compiler lowers that to a function call implemented in druntime. The DMD and LDC versions of this function is different. LDC uses a function called "_d_allocclass" and DMD a function called "_d_newclass". The LDC function only allocates memory (I believe), without doing any initialization.


If that is all then it should be no problem to compile a function with one compiler and "insert" it in to the code of another?

auto foo(T...)(T p) { .... }


This just then gets mapped to different machine code based on which compiler is used, but the outside world doesn't care... only the inputs and the outputs must make sense. Since the calling convention is the same, no allocations occur "on" the call, and if all the types past are compatible it should work.

For example

One should in theory be able to compile a simple module with a function in it using DMD and LDC then extract the machine code from one and replace it in the other and everything work fine(assuming any global addresses are corrected for).


If this is actually true for all(or most cases) then it should be a relatively easy problem to solve.

The main issue I imagine would be that to compile the functions marked one might still have to compile quite a bit of code(Ideally it would be lazily compiled so only those symbols used outside the function would be searched).

If this is essentially correct then one could ideally create a master compiler that one can set versions and pragmas to control what gets compiled where and the master compiler simply handles all the match making and everything just works. This allows one to mix debug and release and performance(ldc) with compile time performance(dmd) possibly with globally overriding for specifying a homogeneous process(e.g., all dmd debug). (and caching could be done to speed up compiling further since if a model or function does not change in source it won't change in binary output and no need to recompile)


The goal, of course, is again to provide targeted performance for those areas that matter such as math routines that do not need debugging features or dmd behavior. This then speeds up the overall program since it is more tailored for it's purpose rather than having to use a blanket set of rules that are not optimal in the real world.

March 08, 2019
On 2019-03-05 21:48, Michelle Long wrote:

> If that is all then it should be no problem to compile a function with one compiler and "insert" it in to the code of another?

That's not all, it's just one example I happen to know about. The problem is that when linking, if you're linking against druntime compiled for DMD then the LDC compiled code won't find the function "_d_allocclass".

-- 
/Jacob Carlborg