On Sunday, 14 February 2016 at 17:43:01 UTC, Ola Fosheim Grøstad wrote:
> On Sunday, 14 February 2016 at 17:38:54 UTC, artemalive wrote:
>> Hi Adam, I'll check the influence of enabled bounds check on benchmark result. Did not try this before.
>
> If you do, then you should use bounds checks in C++ too. (STL container.at(index) )

Thanks, I'll take this into account.

At the moment I don't think bounds checking will be enabled.

I believe that real world usage when you have a render farm is to have the fastest code possible, because any performance penalty is too costly. When new rendering algorithm is developed sometimes we run infinite rendering with all debug/logic check enabled to find the bugs. There could be even dedicated computer for this. But for production when thousands of cores do the same ray processing the fastest code is needed.

On Sunday, 14 February 2016 at 17:49:10 UTC, Johan Engelen wrote:
> On Saturday, 13 February 2016 at 19:26:39 UTC, artemalive wrote:
>> On Saturday, 13 February 2016 at 19:19:46 UTC, Johan Engelen wrote:
>>>
>>> Could you add the compiler versions to the outputted .txt file, e.g. `dmd --version`? (the example output files don't have it)
>>
>> These files are just for demonstration of expected output. The benchmark should work fine with the latest versions of D compilers. No guarantee for older versions;) I tested with dmd v2.069.2.
>
> What I meant to say is that the framework should output the compiler version, instead of "READY". At least for the D compilers, the performance depends so much on which version is used, that the output is unusable without version numbers, I feel.

Thanks. Good suggestion. I'll check if the version information can be retrieved automatically for all compilers. If that's the case then version information will be added soon. The thing I don't want to do is to ask user manually to specify compiler version in config.py, since it's easy to forget to updated it and also I like to keep config.py as simple as possible.

At the moment DigitalWhip is just for personal usage, the user knows which compilers are installed. But version information will help with archiving/sharing results.

On Sunday, 14 February 2016 at 17:31:37 UTC, artemalive wrote:
> From ldc output:
> "-release - Disables asserts, invariants, contracts and boundscheck".

We (LDC team) should clarify this description. In D2, -release does not disable bounds-checking for @safe code anymore.

> -singleobj really does the magic. I'll commit only this option. Thanks for the suggestion and for the pull request. It's my first pull request;)

I'm not convinced that it makes for a good comparison to use different optimization settings for one of the compilers…

 — David

On Sunday, 14 February 2016 at 18:12:03 UTC, David Nadlinger wrote:
> On Sunday, 14 February 2016 at 17:31:37 UTC, artemalive wrote:
>> From ldc output:
>> "-release - Disables asserts, invariants, contracts and boundscheck".
>
> We (LDC team) should clarify this description. In D2, -release does not disable bounds-checking for @safe code anymore.
>
>> -singleobj really does the magic. I'll commit only this option. Thanks for the suggestion and for the pull request. It's my first pull request;)
>
> I'm not convinced that it makes for a good comparison to use different optimization settings for one of the compilers…
>
>  — David

Sure, we need the same optimization for all compilers if there is direct correspondence.
Your pull request has been merged.

On Saturday, 13 February 2016 at 21:10:11 UTC, Adam D. Ruppe wrote:
> On Saturday, 13 February 2016 at 20:45:41 UTC, David Nadlinger wrote:
>> Your scripts had bounds checking enabled for LDC but not the other two D compilers.
>
> I strongly recommend people to always keep bounds checking enabled in real world programs because it is so useful in keeping programs sane.

I agree - but not for all situations.

Sometimes it's fine to let programs crash, if performance is important, and the environment is configured to handle that gracefully, or where there is no risk of data loss, such as video games.

I think that in the context of a render farm, disabling bounds checking is completely reasonable. Bugs will manifest as crashes or rendering artifacts, and there is no risk of code execution exploits.

On Sunday, 14 February 2016 at 19:29:54 UTC, Vladimir Panteleev wrote:
> I think that in the context of a render farm, disabling bounds checking is completely reasonable. Bugs will manifest as crashes or rendering artifacts, and there is no risk of code execution exploits.

But nobody would write a render-engine in Go. Most likely you would write it for GPU support and certainly use SIMD heavily. That's what current engines do. So if it is a benchmark, then it makes perfect sense to use the same feature set for all languages.

But optimization should be set to the highest possible, including whole program optimization where available. It's a benchmark...

On Sunday, 14 February 2016 at 18:07:00 UTC, artemalive wrote:
>
> Thanks. Good suggestion. I'll check if the version information can be retrieved automatically for all compilers. If that's the case then version information will be added soon.

I would simply print the output of "--version" for all compilers.

>  But version information will help with archiving/sharing results.

Exactly.

On Saturday, 13 February 2016 at 21:10:11 UTC, Adam D. Ruppe wrote:
> We should run benchmarks with bounds checking enabled to better reflect real world results. Yes, it might "lose" to C

Like for like comparisons are the best approach, making it clear what a given result is for. The most effective story for D is that it's fast.

On Saturday, 13 February 2016 at 19:19:46 UTC, Johan Engelen wrote:
> On Saturday, 13 February 2016 at 18:48:12 UTC, artemalive wrote:
>>
>> DigitalWhip is a performance benchmark of statically typed programming languages that
>> compile to native code: https://github.com/artemalive/DigitalWhip
>
> Could you add the compiler versions to the outputted .txt file, e.g. `dmd --version`? (the example output files don't have it)

DigitalWhip 1.2.0 has been released:
https://github.com/artemalive/DigitalWhip/releases/tag/v1.2.0

Compiler version information is reported now.