November 28, 2022
On Mon, Nov 28, 2022 at 04:27:03AM +0000, Basile B. via Digitalmars-d wrote: [...]
> For better compile times and with LDC people should also always use the undocumented option `--disable-verify` (for a DUB recipe this would go in the dlags-ldc2 array for example).
> 
> By default ldc2 verifies the IR produced but that verification is mostly useful to detect bugs in the AST to IR translation, so unlikely to detected any problems for a project like LDC, that's well settled, and has the main drawback to be very slow, especially with functions with bad a cyclomatic complexity. For example for my old iz library, 12KSLOCs of D (per D-Scanner critetions), the gain measured with `--disable-verify` goes from 150 to 300ms, depending on the run.

Hmm.  I just tested `--disable-verify` on one of my medium-complexity projects (just under 40 .d files, compiled in a single command); didn't measure any significant speed difference.  Both with and without `--disable-verify` it took about 20 seconds for a full build (generate Linux & Windows executables + run unittests).


T

-- 
Javascript is what you use to allow third party programs you don't know anything about and doing you know not what to run on your computer. -- Charles Hixson
November 29, 2022
On 28/11/2022 5:38 AM, rikki cattermole wrote:
> So what is interesting about this is MSVC link is taking about ~15s by itself, LLVM is 15s which means that the frontend is actually taking only like 1s at most.

Very interestingly it is indeed not 15s at all, but ~4s.

Thanks to the nifty /TIME switch on MSVC link! Welp, guess something else is doing it.
November 28, 2022

On Sunday, 27 November 2022 at 09:29:29 UTC, FeepingCreature wrote:

>

(snip)

Hey, good on you for reaching out to the guy!! That's really cool. I had thought there would be some obvious reason why the compile times would be so bad, but I guess there's not.

Maybe it's time to revisit Stefan's type functions? Or, even though it won't exactly help the template slowness, work on getting newCTFE finished up?

November 29, 2022
On 29/11/2022 9:12 AM, rikki cattermole wrote:
> On 28/11/2022 5:38 AM, rikki cattermole wrote:
>> So what is interesting about this is MSVC link is taking about ~15s by itself, LLVM is 15s which means that the frontend is actually taking only like 1s at most.
> 
> Very interestingly it is indeed not 15s at all, but ~4s.
> 
> Thanks to the nifty /TIME switch on MSVC link! Welp, guess something else is doing it.

LDC is doing a full link each time, while with incremental linking you can get this far lower (like dmd is doing), which is not accurate unfortunately. If you did use incremental linking you unfortunately must remove the extra unused import libraries that LDC is adding (which don't make enough of a difference to matter with a full link.

TLDR: LDC is correct, DMD is giving not entirely correct results. But there should be some wins here if different choices were made.
November 28, 2022
On 11/27/2022 8:12 AM, Steven Schveighoffer wrote:
> Phobos does not make this easy either. Things like std.format are so insanely complex because you can just reach for a bunch of sub-templates. It's easy to write the code, but it increases compile times significantly.

I once tracked just what was being instantiated to format an integer into a string. The layers of templates are literally 10 deep. One template forwards to the next, which forwards to the next, which forwards to the next, 10 layers deep.

This is not D's fault. It's poor design of the conversion code.


> I still have some hope that there are ways to decrease the template cost that will just improve performance across the board. Maybe that needs a new frontend compiler, I don't know.

Phobos2 needs to take a hard look at all the template forwarding going on.

I've also noticed that many templates can be replaced with 2 or 3 ordinary function overloads.

November 28, 2022

On Monday, 28 November 2022 at 22:27:34 UTC, Walter Bright wrote:

>

Phobos2 needs to take a hard look at all the template forwarding going on.

I've also noticed that many templates can be replaced with 2 or 3 ordinary function overloads.

Finally, we are starting realize the cost of these issues in terms of lost productivity during incremental development.

November 29, 2022
Early into my initial scoping of value type exceptions, I looked into std.format.

There is no reason formattedWrite should allocate right? So why isn't it already working with -betterC.

Well, the answer is quite simple, sooooo many exceptions are strewn throughout ready to be fired off. I kinda gave up any hope that it could ever be usable in even the harshest of scenarios.

But if we are thinking about doing a full rewrite of it, it would certainly be good to ditch the class based exception mechanism for error handling!
November 28, 2022

On 11/28/22 5:27 PM, Walter Bright wrote:

>

On 11/27/2022 8:12 AM, Steven Schveighoffer wrote:

>

I still have some hope that there are ways to decrease the template cost that will just improve performance across the board. Maybe that needs a new frontend compiler, I don't know.

Phobos2 needs to take a hard look at all the template forwarding going on.

I've also noticed that many templates can be replaced with 2 or 3 ordinary function overloads.

Sure, you can look at this as "templates are bad, we shouldn't use them as much", but I see it more of a problem that "templates are bad, we should make them less bad". I am also a firm believer that running ordinary functions instead of templates can be much easier to write, easier to debug, and maybe easier to optimize with a new CTFE engine. Perhaps it is just a case of using the wrong tool for the job. But let's also see if there's anything we can do about template performance also. And we have to make it more pleasant to use such things (type functions would be nice to have).

I did a test on something I was working on for my talk, and I'm going to write a blog post about it, because I'm kind of stunned at the results. But in essence, the template ReturnType!fun adds 60KB permanently to the RAM usage of the compiler, even if the function is just a temporary lambda used to check a constraint, and it adds a non-significant amount of compile time vs. just is(typeof(fun()) T). The compile time difference is hard to measure though, let's say it's 500µs.

I think we need to start picking apart how these things are being processed in the compiler, and realize that while it doesn't add that much, all those little 60kb and 500µs add up when you are generating significant tonnage of templates and CTFE.

D's core strength is compile-time metaprogramming and code generation. It shouldn't also be the thing that drives you away because of compile times and memory usage. In other words, we shouldn't have to say "oh you did it wrong because you used too much of D's cool unique features".

Maybe I'm wrong, maybe we just have to tell people not to use these things. But then they really shouldn't be in phobos...

-Steve

P.S., when I say "we" should make them better, I'm shamefully aware that I am too ignorant to be part of that we, it's like the compiler devs are my sports team and I refer to them and me as "we" like I'm on the team! I appreciate all you guys do!

November 29, 2022

On Tuesday, 29 November 2022 at 01:58:48 UTC, Steven Schveighoffer wrote:

>

such things (type functions would be nice to have).

>

D's core strength is compile-time metaprogramming and code generation. -Steve

If there is no metaprogramming for D, why not use C++?
The author leaves D, which is also the compile time performance of D, currently cannot meet the needs of heavy metaprogramming.

November 29, 2022

On Monday, 28 November 2022 at 21:45:37 UTC, TheGag96 wrote:

>

Maybe it's time to revisit Stefan's type functions? Or, even though it won't exactly help the template slowness, work on getting newCTFE finished up?

In a world where language competition is fierce, any improvement is worth it.