Thread overview | |||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
November 15, 2019 Language performance benchmark be updated 2019/11/09 | ||||
---|---|---|---|---|
| ||||
| Language | Time, s | Memory, MiB | | --------------- | ------- | ----------- | | Kotlin | 2.01 | 37.6 | | Nim Gcc | 2.17 | 0.7 | | C++ Gcc | 2.41 | 1.7 | | OCaml | 2.50 | 4.4 | | Go | 2.94 | 1.5 | | Java | 3.05 | 37.2 | | Crystal | 3.06 | 2.7 | | ML MLton | 3.22 | 0.7 | | Go Gcc | 3.30 | 19.2 | | Rust | 3.43 | 0.8 | | Nim Clang | 3.43 | 1.0 | | D Ldc | 3.57 | 1.4 | | D Gdc | 3.72 | 5.8 | | C# .NET Core | 4.30 | 24.3 | | Scala | 4.30 | 136.3 | | F# .NET Core | 4.35 | 24.5 | | D Dmd | 4.74 | 3.3 | | Haskell (MArray)| 6.88 | 3.5 | | C# Mono | 6.88 | 17.6 | | Javascript Node | 6.97 | 31.5 | | V Gcc | 7.30 | 0.8 | | V Clang | 9.06 | 1.0 | | Racket | 10.49 | 77.4 | | LuaJIT | 10.99 | 2.1 | | Python PyPy | 21.51 | 95.4 | | Chez Scheme | 24.72 | 29.2 | | Haskell | 29.14 | 3.4 | | Ruby truffle | 32.52 | 613.3 | | Ruby JRuby | 180.65 | 241.7 | | Ruby | 191.36 | 13.1 | | Lua 5.3 | 201.26 | 1.4 | | Elixir | 279.03 | 48.9 | | Python3 | 388.22 | 7.8 | | Python | 399.75 | 6.2 | | Tcl (FP) | 494.78 | 4.3 | | Perl | 769.17 | 5.2 | | Tcl (OO) | 1000.55 | 4.3 | https://github.com/kostya/benchmarks |
November 15, 2019 Re: Language performance benchmark be updated 2019/11/09 | ||||
---|---|---|---|---|
| ||||
Posted in reply to zoujiaqing | On Friday, 15 November 2019 at 03:31:24 UTC, zoujiaqing wrote:
> https://github.com/kostya/benchmarks
Sadly, the benchmark entries appears to use different algorithms... despite the site claiming otherwise. As far as I can tell...
|
November 15, 2019 Re: Language performance benchmark be updated 2019/11/09 | ||||
---|---|---|---|---|
| ||||
Posted in reply to Ola Fosheim Grøstad | On Friday, 15 November 2019 at 08:25:26 UTC, Ola Fosheim Grøstad wrote:
> On Friday, 15 November 2019 at 03:31:24 UTC, zoujiaqing wrote:
>> https://github.com/kostya/benchmarks
>
> Sadly, the benchmark entries appears to use different algorithms... despite the site claiming otherwise. As far as I can tell...
You mean for sorting one uses quick sort while another uses bubble or something to that affect? Did you check a fair amount and found them all different? (I haven't looked yet obviously, and trying to avoid it depending on how much you peaked :) )
|
November 15, 2019 Re: Language performance benchmark be updated 2019/11/09 | ||||
---|---|---|---|---|
| ||||
Posted in reply to aliak | On Friday, 15 November 2019 at 18:05:35 UTC, aliak wrote:
> You mean for sorting one uses quick sort while another uses bubble or something to that affect? Did you check a fair amount and found them all different?
The testset are very limited.
Matrix multiplication for one...
If you benchmark a language that calls into a C-implementation library... and get twice as good results as the C-benchmark... then you know something is not right! :-D
|
November 16, 2019 Re: Language performance benchmark be updated 2019/11/09 | ||||
---|---|---|---|---|
| ||||
Posted in reply to aliak | On Friday, 15 November 2019 at 18:05:35 UTC, aliak wrote:
> On Friday, 15 November 2019 at 08:25:26 UTC, Ola Fosheim Grøstad wrote:
>> On Friday, 15 November 2019 at 03:31:24 UTC, zoujiaqing wrote:
>>> https://github.com/kostya/benchmarks
>>
>> Sadly, the benchmark entries appears to use different algorithms... despite the site claiming otherwise. As far as I can tell...
>
> You mean for sorting one uses quick sort while another uses bubble or something to that affect? Did you check a fair amount and found them all different? (I haven't looked yet obviously, and trying to avoid it depending on how much you peaked :) )
The Json test uses very different parser implementations. There are even multiple implementations for D. One of them uses the fast I/O library while the other one ises std.json, I think. I haven't checked the others, but I expect them to have a similar spread.
|
November 16, 2019 Re: Language performance benchmark be updated 2019/11/09 | ||||
---|---|---|---|---|
| ||||
Posted in reply to zoujiaqing | On Friday, 15 November 2019 at 03:31:24 UTC, zoujiaqing wrote: > https://github.com/kostya/benchmarks Sorry, but have you tried it by yourself? I'm running this benchmark outside of Docker, and the numbers I see are very interesting. First of all, the time measurement script is quite questionable. It's a Ruby script, and I found that the running time depends on the shell and environment. For example, when I run it under VSCode's console window, I get 20% worse results in time for all binaries than when I run it under the regular terminal emulator window. Second, the numbers are, hmmm, how to say... bullshit? Pardon my French! Here is what I get for the Brainfuck2 mandelbrot benchmark (a simple Brainfuck interpreter implemented in different languages): C++ gcc version 7.4.0 (g++ -flto -O3 -o bin_cpp bf.cpp): ``` $ ../xtime.rb ./bin_cpp mandel.b # ... here is the Mandelbrot set ASCII art printed ... 18.05s, 3.6Mb ``` D LDC2 1.18.0 (ldc2 -ofbin_d_ldc -O5 -release -boundscheck=off bf.d) ``` $ ../xtime.rb ./bin_d_ldc mandel.b # ... here is the Mandelbrot set ASCII art printed ... 19.53s, 3.6Mb ``` Nim 1.0.2 (nim c -o:bin_nim_gcc -d:danger --cc:gcc --verbosity:0 bf.nim) ``` $ ../xtime.rb ./bin_nim_gcc mandel.b # ... here is the Mandelbrot set ASCII art printed ... 25.07s, 2.2Mb ``` Kotlin kotlinc-jvm 1.3.50 (JRE 1.8.0_201-b09) (kotlinc bf2.kt -include-runtime -d bf2-kt.jar) ``` $ ../xtime.rb java -jar bf2-kt.jar mandel.b JIT warming up time: 1.25s run # ... here is the Mandelbrot set ASCII art printed ... 26.81s, 36.6Mb ``` Golang go1.13.4 linux/amd64 (go build -o bin_go bf.go) ``` $ ../xtime.rb ./bin_go mandel.b # ... here is the Mandelbrot set ASCII art printed ... 38.73s, 2.9Mb ``` ============================================ So the results for Braunfuck2 mandel.b are: ``` C++ gcc: 18.05s, 3.6Mb D LDC2: 19.53s, 3.6Mb Nim: 25.07s, 2.2Mb Kotlin: 26.81s, 36.6Mb ``` Please note that I've added `boundscheck=off` to LDC2 command line there. Also, Kotlin is always printing `JIT warming up` and takes about 1 to 2 seconds to warm up, so the results for Brainfuck2 `bench.b` are VERY different. Kotlin is not the first one obviously. I'm running everything on my laptop with Intel(R) Core(TM) i7-8550U CPU @ 1.80GHz As I mentioned, no Docker containers, just using the included scripts to build the binaries. Haven't tried other tests, but I feel like I'll get very interesting results for them also. Would be great if someone else is able to confirm these results because these benchmarks look very manipulative. |
November 16, 2019 Re: Language performance benchmark be updated 2019/11/09 | ||||
---|---|---|---|---|
| ||||
Posted in reply to Jacob Shtokolov | On Saturday, 16 November 2019 at 16:07:29 UTC, Jacob Shtokolov wrote:
> Haven't tried other tests
Just tried to compile and run Base64
The results are:
```
C: 1.46s, 1.9Mb
Rust: 1.49s, 2.4Mb
D LDC2: 1.98s, 4.2Mb
Golang: 2.89s, 10.9Mb
C++: 3.18s, 4.8Mb
```
This test is closer to the author's numbers, but Rust implementation isn't faster than the C implementation on my machine.
Golang here is faster than C++. The D version was built with bounds checks this time.
|
November 16, 2019 Re: Language performance benchmark be updated 2019/11/09 | ||||
---|---|---|---|---|
| ||||
Posted in reply to Jacob Shtokolov | On Saturday, 16 November 2019 at 16:34:58 UTC, Jacob Shtokolov wrote:
> Just tried to compile and run Base64
The Havlak test is closer to reality:
```
Nim: 12.24s, 477.8Mb
C++: 17.33s, 179.3Mb
Golang: 21.58s, 358.0Mb
D LDC2: 23.55s, 460.4Mb
D DMD: 29.04s, 461.9Mb
```
Nim is the winner.
But here I would look into the code: what makes LDC produce such poorly optimized binary.
|
November 16, 2019 Re: Language performance benchmark be updated 2019/11/09 | ||||
---|---|---|---|---|
| ||||
Posted in reply to Jacob Shtokolov | On Saturday, 16 November 2019 at 16:34:58 UTC, Jacob Shtokolov wrote:
> On Saturday, 16 November 2019 at 16:07:29 UTC, Jacob Shtokolov wrote:
>> Haven't tried other tests
>
> Just tried to compile and run Base64
> The results are:
>
> ```
> C: 1.46s, 1.9Mb
> Rust: 1.49s, 2.4Mb
> D LDC2: 1.98s, 4.2Mb
> Golang: 2.89s, 10.9Mb
> C++: 3.18s, 4.8Mb
> ```
>
> This test is closer to the author's numbers, but Rust implementation isn't faster than the C implementation on my machine.
>
> Golang here is faster than C++. The D version was built with bounds checks this time.
Why is C++ doing so badly? Is it because of inefficient usage of buffers and that it doesn't natively support slices?
|
November 16, 2019 Re: Language performance benchmark be updated 2019/11/09 | ||||
---|---|---|---|---|
| ||||
Posted in reply to IGotD- | On Saturday, 16 November 2019 at 16:47:40 UTC, IGotD- wrote:
> Why is C++ doing so badly?
Looks like that's because they're using some libcrypto APIs (like the BIO). Also, my C++ compiler is not the latest one - 7.4.0. In the benchmark, it's claimed as GCC 9.2.1.
But the rest compilers are up to date and actually the same versions as in the benchmark README file
|
Copyright © 1999-2021 by the D Language Foundation