Martin ran some benchmarks recently that showed that ddmd compiled with dmd was about 30% slower than when compiled with gdc/ldc. This seems to be fairly typical.

I'm interested in ways to reduce that gap.

There are 3 broad kinds of optimizations that compilers do:

1. source translations like rewriting x*2 into x<<1, and function inlining

2. instruction selection patterns like should one generate:

    SETC AL
    MOVZ EAX,AL

or:
    SBB EAX
    NEG EAX

3. data flow analysis optimizations like constant propagation, dead code elimination, register allocation, loop invariants, etc.

Modern compilers (including dmd) do all three.

So if you're comparing code generated by dmd/gdc/ldc, and notice something that dmd could do better at (1, 2 or 3), please let me know. Often this sort of thing is low hanging fruit that is fairly easily inserted into the back end.

For example, recently I improved the usage of the SETcc instructions.

https://github.com/D-Programming-Language/dmd/pull/4901
https://github.com/D-Programming-Language/dmd/pull/4904

A while back I improved usage of BT instructions, the way switch statements were implemented, and fixed integer divide by a constant with multiply by its reciprocal.

On Tuesday, 18 August 2015 at 10:45:49 UTC, Walter Bright wrote:
> So if you're comparing code generated by dmd/gdc/ldc, and notice something that dmd could do better at (1, 2 or 3), please let me know. Often this sort of thing is low hanging fruit that is fairly easily inserted into the back end.

I think someone mentioned how other compilers unroll loops at more than 2 levels.

Other than that, there was a recent Java vs D thread which showed it orders of magnitude faster on vtable calls. So I think the most amazing feature would be to allow profiling & sampling to compile with samples and select which functions to inline or do some magic around vtable pointers like what Java is doing.

Finally, I'm going to write this down here and haven't had time to look more into it but I've never been able to compile Botan with optimizations on DMD64 Win64 VS2013 (https://github.com/etcimon/botan), it's really strange having a crypto library that you can't optimize, building -O -g also gives me a ccog.c ICE error. I think it might be something about `asm pure` that uses some locals, does that eliminate the function call parameters?

On Tuesday, 18 August 2015 at 10:45:49 UTC, Walter Bright wrote:
> So if you're comparing code generated by dmd/gdc/ldc, and notice something that dmd could do better at (1, 2 or 3), please let me know. Often this sort of thing is low hanging fruit that is fairly easily inserted into the back end.

Hi,

From my experience reducing regressions, I have noticed that backend changes in general have a very high chance of introducing code generation regressions. Codegen bugs are nasty: they are occasionally difficult to reduce, and since software is rarely tested with its "release" build, have a habit of sneaking into published releases of otherwise bug-free software.

IIRC, I have had three releases affected by optimization/inlining DMD bugs (two of Digger and one of RABCDAsm). These do not speak well for D when end-users ask me what the cause of the bug is, and I have to say "Yeah, it's a bug in the official D compiler".

I think stability of the DMD backend is a goal of much higher value than the performance of the code it emits. DMD is never going to match the code generation quality of LLVM and GCC, which have had many, many man-years invested in them. Working on DMD optimizations is essentially duplicating this work, and IMHO I think it's not only a waste of time, but harmful to D because of the risk of regressions.

I suggest that we revamp the compiler download page again. The lead should be a "select your compiler" which lists the advantages and disadvantages of each of DMD, LDC and GDC.

On Tuesday, 18 August 2015 at 12:32:17 UTC, Etienne Cimon wrote:
> a crypto library that you can't optimize, building -O -g also gives me a ccog.c ICE error. I think it might be something about `asm pure` that uses some locals, does that eliminate the function call parameters?

Sorry that was cgcod.c

Internal error: backend\cgcod.c 2311
FAIL .dub\build\__test__full__-unittest-windows-x86_64-dmd_2068-8073079C502FEEB927744150233D4046\ __test__full__ executa
ble

I'll try and file a bugzilla about this. I think stability should be first concern.

On Tuesday, 18 August 2015 at 12:37:37 UTC, Vladimir Panteleev wrote:
> I think stability of the DMD backend is a goal of much higher value than the performance of the code it emits. DMD is never going to match the code generation quality of LLVM and GCC, which have had many, many man-years invested in them. Working on DMD optimizations is essentially duplicating this work, and IMHO I think it's not only a waste of time, but harmful to D because of the risk of regressions.

+1

On Tuesday, 18 August 2015 at 10:45:49 UTC, Walter Bright wrote:
> Martin ran some benchmarks recently that showed that ddmd compiled with dmd was about 30% slower than when compiled with gdc/ldc. This seems to be fairly typical.
>
> I'm interested in ways to reduce that gap.
>
> There are 3 broad kinds of optimizations that compilers do:
>
> 1. source translations like rewriting x*2 into x<<1, and function inlining
>
> 2. instruction selection patterns like should one generate:
>
>     SETC AL
>     MOVZ EAX,AL
>
> or:
>     SBB EAX
>     NEG EAX
>
> 3. data flow analysis optimizations like constant propagation, dead code elimination, register allocation, loop invariants, etc.
>
> Modern compilers (including dmd) do all three.
>
> So if you're comparing code generated by dmd/gdc/ldc, and notice something that dmd could do better at (1, 2 or 3), please let me know. Often this sort of thing is low hanging fruit that is fairly easily inserted into the back end.
>
> For example, recently I improved the usage of the SETcc instructions.
>
> https://github.com/D-Programming-Language/dmd/pull/4901
> https://github.com/D-Programming-Language/dmd/pull/4904
>
> A while back I improved usage of BT instructions, the way switch statements were implemented, and fixed integer divide by a constant with multiply by its reciprocal.

I've often looked at the assembly output of ICC.

One thing that was striking to me is that it by and large it doesn't use PUSH, POP, and SETcc. Actually I don't remember such an instruction being emitted by it.

And indeed using PUSH/POP/SETcc in assembly were often slower than the alternative. Which is _way_ different that the old x86 where each of these things would gain speed.

Instead of PUSH/POP it would spill all registers to an RBP-based location the (perhaps taking advantage of the register renamer?).

---------------

That said: I entirely agree with Vladimir about the codegen risk. DMD will always be used anyway because it compiles faster.

On Tuesday, 18 August 2015 at 10:45:49 UTC, Walter Bright wrote:
> ...
> 3. data flow analysis optimizations like constant propagation, dead code elimination, register allocation, loop invariants, etc.
>
> Modern compilers (including dmd) do all three.
>
> So if you're comparing code generated by dmd/gdc/ldc, and notice something that dmd could do better at (1, 2 or 3), please let me know. Often this sort of thing is low hanging fruit that is fairly easily inserted into the back end.
> ...

I've once tried to trace the slowdown cause in a simple program and reduced it to https://issues.dlang.org/show_bug.cgi?id=11821, I think if falls under point 3 in your post (redundant instruction in a simple loop).  Despite 1.5 years have passed, the issue still stands with 2.068.0.

That's for -m32.  The -m64 version of the loop does not look as having a redundant instruction to me, but is still longer than the output of GDC or LDC.

On Tuesday, 18 August 2015 at 12:37:37 UTC, Vladimir Panteleev wrote:
> I think stability of the DMD backend is a goal of much higher value than the performance of the code it emits. DMD is never going to match the code generation quality of LLVM and GCC, which have had many, many man-years invested in them. Working on DMD optimizations is essentially duplicating this work, and IMHO I think it's not only a waste of time, but harmful to D because of the risk of regressions.

Well, you have to admit that it's pretty impressive that dmd's backend gets within 30% of those monumental backends despite having pretty much only Walter working on it sporadically.  If it's a waste of time to work on compiler optimizations because of existing work, you could have said the same to the llvm devs when they tried to take on gcc.  As ponce said, people are always going to use dmd because of it's speed, no reason not to make its codegen better also.

Also, soon the dmd compiler backend will be the only one written in D. :) No reason to not also make it better. Of course, Walter is the only one who can decide the best use of his time.

On Tuesday, 18 August 2015 at 15:22:15 UTC, Joakim wrote:
> Also, soon the dmd compiler backend will be the only one written in D. :)

Soon the front end will be written in D. And the front end is shared among dmd, gdc, ldc. Walter has expressed a desire to port the back end to D, too [1]. But that's not going to happen "soon".

[1] http://forum.dlang.org/post/mohrrs$1pu7$1@digitalmars.com

On Tuesday, 18 August 2015 at 15:45:25 UTC, anonymous wrote:
> On Tuesday, 18 August 2015 at 15:22:15 UTC, Joakim wrote:
>> Also, soon the dmd compiler backend will be the only one written in D. :)
>
> Soon the front end will be written in D. And the front end is shared among dmd, gdc, ldc. Walter has expressed a desire to port the back end to D, too [1]. But that's not going to happen "soon".
>
> [1] http://forum.dlang.org/post/mohrrs$1pu7$1@digitalmars.com

Yes, that's why I said the dmd _backend_ will be the only one written in D.  Not sure how you know what the timeline for such a backend port is, seems like he really wants to get everything in D soon.