Thread overview
[Help Needed] - Debugging compilation time
Oct 21, 2022
Hipreme
Oct 21, 2022
H. S. Teoh
Oct 21, 2022
ryuukk_
Oct 22, 2022
Hipreme
Oct 22, 2022
ryuukk_
Oct 21, 2022
ryuukk_
Oct 21, 2022
apz28
October 21, 2022

Hey guys, I have been complaining a lot of time right now from D compilation speed at least for my project.

I have:

  • Underused CTFE
  • Underused Templates
  • Avoided importing standard libraries
  • Created a multi module projects for better code reuse

Those are all the techniques I have tried to maintain my compilation times lower, yet, It still builds slow even when using the --combined (which does a single compiler run on dub)

So, doesn't matter if all is imported at once or not, it seems the problem seems to be somewhere in my project which I've been unable to find. I wanted to show someone who is more experienced on the time trace from LDC to check what I've been doing wrong.

This time trace took 9 seconds. DMD takes 7 seconds to build my project. Adam has built his entire arsd in 2.5 seconds, while my PC is faster and arsd is much bigger than my project, this does not make any sense. So I would gladly wish you guys help:

My repository: https://github.com/MrcSnm/HipremeEngine/
My time trace: https://ufile.io/gmvw1wlu (This time trace will be in air for 30 days only)

October 21, 2022
On Fri, Oct 21, 2022 at 04:32:17PM +0000, Hipreme via Digitalmars-d-learn wrote:
> Hey guys, I have been complaining a lot of time right now from D compilation speed at least for my project.
> 
> I have:
> - Underused CTFE
> - Underused Templates
> - Avoided importing standard libraries
> - Created a multi module projects for better code reuse
> 
> Those are all the techniques I have tried to maintain my compilation times lower, yet, It still builds slow even when using the --combined (which does a single compiler run on dub)

Using dub immediately slows you down by at least several seconds. There may be some options to skip the time-consuming dependency graph resolution and network access.  But my personal preference is to use an offline build system with less overhead.


> So, doesn't matter if all is imported at once or not, it seems the problem seems to be somewhere in my project which I've been unable to find. I wanted to show someone who is more experienced on the time trace from LDC to check what I've been doing wrong.

LDC generally tends to be quite a bit slower than DMD because of the time spent in aggressive optimizations.  For development builds, consider using DMD instead.


> This time trace took 9 seconds. DMD takes 7 seconds to build my project.  Adam has built his entire arsd in 2.5 seconds, while my PC is faster and arsd is much bigger than my project, this does not make any sense.

It could be caused by a number of different things:

- Using many nested templates will slow down compilation.
- Doing excessive CTFE computations will slow things down.
- Using unusually-large static arrays in some cases may trigger slow
  paths in the front-end, consuming lots of compiler memory.
- Using string imports may slow things down as the compiler parses the
  imported file(s).
- Having excessively-large function bodies may trigger some O(n^2) paths
  in the compiler that significantly slows things down.
- Using excessive mixins may also slow things down due to copying of the
  AST.

OTOH, uninstantiated templates are cheap (the only cost is parsing them, which is very fast): just avoiding templates along may not save you much time if it's being spent elsewhere.  I suspect a good chunk of Adam's code is in uninstantiated templates, so a straight comparison isn't really fair, if those templates aren't actually being instantiated in the first place.

I also notice you use mixin templates a lot; it might be worth investigating whether they might be the cause of your issue.


> So I would gladly wish you guys help:
> 
> My repository: https://github.com/MrcSnm/HipremeEngine/
> My time trace: https://ufile.io/gmvw1wlu (This time trace will be in air for
> 30 days only)

Sorry, no time to look into it in detail, but I did skim over a few files and note that you use mixin templates a lot.  I haven't had much experience with mixin templates but it might be worth investigating whether they might be causing compile time slowdowns.


T

-- 
They say that "guns don't kill people, people kill people." Well I think the gun helps. If you just stood there and yelled BANG, I don't think you'd kill too many people. -- Eddie Izzard, Dressed to Kill
October 21, 2022

I tried your project:

Linux x64

git clone https://github.com/MrcSnm/HipremeEngine.git
cd HipremeEngine
dub build (once to download dependencies if any)
time dub build -f


real	0m4.604s
user	0m3.686s
sys	0m0.900s

4.6 sec for a FULL rebuild doesn't seem that bad

and

real	0m1.730s
user	0m1.480s
sys	0m0.245s

after editing one module, not bad at all

As other people say:

LDC for release with optimizations
DMD for development

THat's what i personally do

October 21, 2022

Make sure you have the latest version of DMD

Make sure your antivirus isn't blocking your files (scanning), it's a common thing with Windows, whitelist dmd folder, your dev folder and dub folder

October 21, 2022

On Friday, 21 October 2022 at 16:32:17 UTC, Hipreme wrote:

>

Hey guys, I have been complaining a lot of time right now from D compilation speed at least for my project.

I have:

  • Underused CTFE
  • Underused Templates
  • Avoided importing standard libraries
  • Created a multi module projects for better code reuse

The slow is from executing CTFE (template constraints, enum expression, static if ...). The culprit is this function: Expression.ctfeInterpret() in this module dmd.expression.d
Even for compiling DMD, that function will take 1/3 of total times

Try to reduce CTFE usage than it will be fast again. D should analyze to make executing CTFE faster

Cheers

October 22, 2022

On Friday, 21 October 2022 at 18:10:39 UTC, ryuukk_ wrote:

>

I tried your project:

Linux x64

git clone https://github.com/MrcSnm/HipremeEngine.git
cd HipremeEngine
dub build (once to download dependencies if any)
time dub build -f


real	0m4.604s
user	0m3.686s
sys	0m0.900s

4.6 sec for a FULL rebuild doesn't seem that bad

and

real	0m1.730s
user	0m1.480s
sys	0m0.245s

after editing one module, not bad at all

As other people say:

LDC for release with optimizations
DMD for development

THat's what i personally do

I'm not using anti virus :c

Seems that Linux really is significantly faster for building , 1.7secs for a rebuild seems pretty nice for me, though I can't say it happens the same on my ground

October 22, 2022

On Saturday, 22 October 2022 at 12:27:21 UTC, Hipreme wrote:

>

On Friday, 21 October 2022 at 18:10:39 UTC, ryuukk_ wrote:

>

I tried your project:

Linux x64

git clone https://github.com/MrcSnm/HipremeEngine.git
cd HipremeEngine
dub build (once to download dependencies if any)
time dub build -f


real	0m4.604s
user	0m3.686s
sys	0m0.900s

4.6 sec for a FULL rebuild doesn't seem that bad

and

real	0m1.730s
user	0m1.480s
sys	0m0.245s

after editing one module, not bad at all

As other people say:

LDC for release with optimizations
DMD for development

THat's what i personally do

I'm not using anti virus :c

Seems that Linux really is significantly faster for building , 1.7secs for a rebuild seems pretty nice for me, though I can't say it happens the same on my ground

What exactly takes time? do you notice a specific module taking longer than others? or is it the linker?

A module that is windows specific might be causing the issue you are having

Try to output the mixin file and see what generates lot of code: https://dlang.org/dmd-linux.html#switch-mixin

I'll see if i can test on Windows later today