On 18/07/2024 3:51 AM, Walter Bright wrote:
> On 7/15/2024 11:20 AM, Richard (Rikki) Andrew Cattermole wrote:
>> Right, but it's dmd we are talking about.
>>
>> A compiler that no matter what switches are set internally or externally cannot compete with ldc or gdc.
> 
> People benchmark it anyway.

And get bad results.

I really need you to understand just how much things have changed, and this bit of experience no longer translates to this lesson. I do accept and enjoy the history of the switch when it did translate to this lesson however!

This is a real benchmark that came from a Go article that made the rounds a while ago, here is my version of it with results I just reran:

https://news.ycombinator.com/item?id=39106972

https://www.reddit.com/r/golang/comments/199u7np/from_slow_to_simd_a_go_optimization_story/

```
$ dmd -O -release -Iinterface main.d -run attempt.d
27033600000     12 secs, 689 ms, 420 ╬╝s, and 5 hnsecs

$ dmd -O -release -Iinterface main.d -run attempt2.d
27033600000     15 secs, 764 ms, 182 ╬╝s, and 9 hnsecs

$ ldc2 -O3 -Iinterface main.d -run attempt.d
27033600000     6 secs, 28 ms, 747 μs, and 4 hnsecs

$ ldc2 -O3 -Iinterface main.d -run attempt2.d
27033600000     14 secs, 939 ms, 722 μs, and 3 hnsecs
```

attempt.d is completely naive, attempt2 does an explicit loop unroll with static foreach.

If I turn on release for ldc (or swap to .ptr on the slices):

```
$ ldc2 -O3 -release -Iinterface main.d -run attempt.d
27033600000     6 secs, 70 ms, 191 μs, and 4 hnsecs

$ ldc2 -O3 -release -Iinterface main.d -run attempt2.d
27033600000     5 hnsecs
```

Dmd simply does not have the optimizations to get anywhere close to a modern backend. It _cannot_ come close to winning benchmarks.

Note: attempt2 with the second optimization is as good as hand rolled assembly by the author of the article.

>> It is better to slap a message both in the version/help message of the compiler and on the site that says "don't use dmd for reviewing of performance of D use ldc/gdc instead".
> 
> People doing benchmarks do not read the site. That's the whole problem.

And D goes to the bottom of the benchmark.

Except in cases where the D community takes over, like here:

https://github.com/jinyus/related_post_gen/tree/main

A lot of effort went into that by quite a few people.

If anything all of this is making me go: "drop dmd from download page". Problem solved.

>> Given this, it is better to simply remove the switch as it fools people into doing unsafe things that give them a very bad day.
> 
> People don't develop code using a -release switch. Those spending time coding in D (rather than spending as little time as possible writing a benchmark) will be likely to spend time reading what a switch does before using it.

They don't.

We the community catch it and tell them not to use it.

Repeatedly over the last 12 years it has been a very common occurrence.

This is a lesson that has been learned by many people and keeps coming up as not what they wanted to use.

> I've been in this business a long time. What people should do and what they do do is rarely aligned. People who do comparative benchmarks spend as little time as possible benchmarking a language/compiler they are not familiar with.

Yes that is what happened in the past, I agree it's a good story for showing how things use to be. For benchmarks that make the rounds today that isn't what happens.

I linked two different benchmarks that did make the rounds in the last year. One was benchmarking Go's backend, the other accepted contributions and we won because we could drive LLVM better.

Neither was benchmarking the frontend or language itself.

On Wednesday, July 17, 2024 9:51:29 AM MDT Walter Bright via Digitalmars-d wrote:
> On 7/15/2024 11:20 AM, Richard (Rikki) Andrew Cattermole wrote:
> > Right, but it's dmd we are talking about.
> >
> > A compiler that no matter what switches are set internally or externally cannot compete with ldc or gdc.
>
> People benchmark it anyway.
>
> > It is better to slap a message both in the version/help message of the compiler and on the site that says "don't use dmd for reviewing of performance of D use ldc/gdc instead".
>
> People doing benchmarks do not read the site. That's the whole problem.
>
> > Given this, it is better to simply remove the switch as it fools people into doing unsafe things that give them a very bad day.
>
> People don't develop code using a -release switch. Those spending time coding in D (rather than spending as little time as possible writing a benchmark) will be likely to spend time reading what a switch does before using it.
>
> I've been in this business a long time. What people should do and what they do do is rarely aligned. People who do comparative benchmarks spend as little time as possible benchmarking a language/compiler they are not familiar with.
>
> Have you ever read the descriptions of all the compiler switches on gcc or VC?
>
> Me neither.

While your approach here might make good sense if -release actually provided an optimized build, it really doesn't. While it will help somewhat with giving more performant code, since it does strip out some code, it doesn't include any actual optimization passes, meaning that it's really not going to make dmd look much better in benchmarks. Maybe if it included -O, it would help in the way that you're looking for, but as things stand, I don't think that it really does. Rather, in practice, what happens is that a number of programmers assume that -release is what they're supposed to use for release builds, and that's what they use it for. Folks who are are more involved with D and/or who use it professionally are more likely to have taken the time to figure out exactly which switches they want to be using, but a lot of programmers won't, and dub is set up to use -release as part of its default release target, meaning that D programmers in general are going to get it unless they go to extra effort to avoid it (which is obviously an issue with dub and not the compiler, but it goes to show that whoever made that choice thought that -release was for release builds, and it affects everyone using dub).

Regardless, arguably, the main problem here is that -release turns off bounds checking for non-@safe code, and in practice, that's a _lot_ of D code. It's that behavior that sometimes causes folks like Steven to tell people to not use -release. The fact that assertions and contracts are removed is probably just fine, particularly since that's usually what folks are looking for with a release build, whereas arguably, removing those bounds checks is enough of an @safety issue that it shouldn't be happening without the programmer explicitly asking for it (and lumping it into a switch that does a bunch of other stuff makes it very easy to ask for it without intending to). And no, -release doesn't remove _all_ bounds checking, since @safe code still has bounds checking with -release, but a _lot_ of code doesn't, since a lot of code isn't @safe. So, we're arguably providing the wrong defaults with -release right now.

So, I'm not sure that I agree with Steven that we should remove -release (though I'd certainly be fine with that), but I _do_ think that we should seriously consider changing exactly what it does if we're going to keep it.

- Jonathan M Davis

On 7/17/24 21:12, Mike Shah wrote:
> In D Release Candidate to the public: "dmd -O -release -inline -boundscheck=0ff ..."

Well, this is the one with potential buffer overrun exploits, so not really fit for consumption by the public.

On Wednesday, 17 July 2024 at 19:21:02 UTC, Timon Gehr wrote:
> On 7/17/24 21:12, Mike Shah wrote:
>> In D Release Candidate to the public: "dmd -O -release -inline -boundscheck=0ff ..."
>
> Well, this is the one with potential buffer overrun exploits, so not really fit for consumption by the public.

The description of 'fastest performance' is precise though 🙂 (and what you'd want for a benchmark if measuring maximum possible performance even when trading safety).

This is why we probably need some wiki page on best practices for builds -- I'll put in my work queue to make a video to explain compiler flags in the next few weeks then regardless to help with some efforts.

On Wed, Jul 17, 2024 at 08:25:31PM +0000, Mike Shah via Digitalmars-d wrote:
> On Wednesday, 17 July 2024 at 19:21:02 UTC, Timon Gehr wrote:
> > On 7/17/24 21:12, Mike Shah wrote:
> > > In D Release Candidate to the public: "dmd -O -release -inline -boundscheck=0ff ..."
> > 
> > Well, this is the one with potential buffer overrun exploits, so not really fit for consumption by the public.
> 
> The description of 'fastest performance' is precise though 🙂 (and what you'd want for a benchmark if measuring maximum possible performance even when trading safety).
> 
> This is why we probably need some wiki page on best practices for builds -- I'll put in my work queue to make a video to explain compiler flags in the next few weeks then regardless to help with some efforts.

Seriously, if you guys wanna be serious about -release actually doing something useful, make it run `ldc -O2`.  I'm dead serious.  IME no amount of -release, -inline, -O with dmd gives me anywhere near satisfactory performance.  If any of my coworkers were to use dmd and try to benchmark it with -release, they'd laugh D out of the room.  Make it run `ldc -O2`, however, and you might actually raise a few eyebrows.

I know it's not nice to admit that dmd does not generate optimal executables.  I wish I could say otherwise, but the facts are the facts. I've consistently gotten 30% performance boosts in my program just by compiling with ldc2 instead of dmd (not even with -O2, just plain ldc2 is enough to give you a significant boost). With ldc2 -O2, I get about 40% performance boost, sometimes up to 50% depending on what the program does.  If you're trying to win benchmarks, having people use dmd is the sure way NOT to win them.  You want them to use ldc2, period. You do NOT want them to use dmd, except when you're trying to compete in the compile-speed category.

Say what you may about Adam Ruppe and his fork, but he got this one thing right: have normal dev builds use dmd for compile speeds, and -release drop dmd altogether and compile with ldc2 instead. This is what will give newcomers and benchmarkers the best impression of D.

Trying to hack dmd -release to do this or not do that, all of that is futile, wasted effort. Just ship ldc2 with dmd by default, and have dmd -release redirect itself to ldc2.  End of story.

(If you don't believe any of the above, don't take my word for it, test it for yourself. Take any benchmark you wish, compile it with dmd -release, or dmd -release -O -inline, or any combination of flags you wish in dmd, really. Run it and save the results.  Now recompile it with ldc2 -O2.  Compare the results.  The facts speak for themselves.)


T

-- 
Famous last words: I *think* this will work...

On Wed, Jul 17, 2024 at 11:56:44PM +0000, Quirin Schroll via Digitalmars-d wrote:
> On Wednesday, 17 July 2024 at 21:19:31 UTC, H. S. Teoh wrote:
> > I know it's not nice to admit that dmd does not generate optimal executables.  I wish I could say otherwise, but the facts are the facts.  I've consistently gotten 30% performance boosts in my program just by compiling with ldc2 instead of dmd (not even with -O2, just plain ldc2 is enough to give you a significant boost). With ldc2 -O2, I get about 40% performance boost, sometimes up to 50% depending on what the program does.  If you're trying to win benchmarks, having people use dmd is the sure way NOT to win them. You want them to use ldc2, period. You do NOT want them to use dmd, except when you're trying to compete in the compile-speed category.
> 
> If that is true, DMD should focus on development and debugging experience.
[...]

Don't take my word for it.  Measure it yourself.  Take any D benchmark and compile with dmd -release -O -inline, then compile with ldc2 -O2. Compare the difference.  You'll see.


T

-- 
When you breathe, you inspire. When you don't, you expire. -- The Weekly Reader