It's great that the community steps up to ensure we get good benchmark results!

Lots of times benchmarks are done informally and internally by companies, and we never know about them and never have a chance.

On 7/17/2024 10:52 AM, Jonathan M Davis wrote:
> Maybe if it included -O,

It used to, along with -inline, but as I mentioned earlier at some point that was removed. It should be put back.

On Saturday, 13 July 2024 at 05:35:53 UTC, Walter Bright wrote:
> Why -release is the way it is:
>
> I got sick of journalists running benchmarks using compilers they weren't familiar with. They never spent any effort learning what the switches were.
>
> The big vendors would wine and dine them and show them exactly how to run their compilers. With mine, they never talked to me. I wouldn't even know they did a review until it hit the news stand.
>
> They'd use the wrong switches, the code would run slow, and I'd get a bad review.
>
> The -release switch means "make the fastest code". Make it easy for the journalists to do the right thing.
>
> Nobody new to D will know to use -O -boundscheck-safeonly -inline. They'll just get a "D is slow" result and move on.

I'm not sure what are you talking about honestly.
First of all 'D benchmarker' here. So which impressions/observations I have based on my experience:
1) No journalists are interested in D... at all.
And 'D is slow' usually not because of the compiler or flags, but because Phobos/runtime/library implementation.
2) People sometimes adding D to their benchmarks, but most usually it is somebody from the community.
3) Community pretty aware of 'not using dmd in any benchmarks', but the problem not in this.
4) As many people already said the problem is in 'misleading' name. Because benchmarkers also sometime want to use 'the flags that will be used in production'. And many other languages have 'release' flags exactly for these purposes:

Zig: "Standard optimization options allow the person running zig build to select between Debug, ReleaseSafe, ReleaseFast, and ReleaseSmall. By default none of the release options are considered the preferable choice by the build script, and the user must make a decision in order to create a release build."

Rust: "
$ cargo build
    Finished dev [unoptimized + debuginfo] target(s) in 0.0s
$ cargo build --release
    Finished release [optimized] target(s) in 0.0s
The dev and release are these different profiles used by the compiler."

So Dub is doing fine with 'release' build option (having 'release-nobounds' in case it is what developer wants) and also it is possible to configure the release build flags.

On Monday, 15 July 2024 at 18:14:39 UTC, Walter Bright wrote:
> On 7/14/2024 1:44 PM, Steven Schveighoffer wrote:
>> The biggest problem I have, I guess, is that it's called "release". It fools people into thinking you should use it for released code.
>
> It's an issue I've run into repeatedly. Reviewers not familiar with a compiler look for a "release" switch for benchmarking.

So…
How about calling it -benchmark?