To see what performance advantage D would give me over using a scripting language, I made a small benchmark. It consists of this code:

>    auto L = iota(0.0, 10000000.0);
>    auto L2 = map!"a / 2"(L);
>    auto L3 = map!"a + 2"(L2);
>    auto V = reduce!"a + b"(L3);

It runs in 281 ms on my computer.

The same code in Lua (using LuaJIT) runs in 23 ms.

That's about 10 times faster. I would have expected D to be faster. Did I do something wrong?

The first Lua version uses a simplified design. I thought maybe that is unfair to ranges, which are more complicated. You could argue ranges have more features and do more work. To make it fair, I made a second Lua version of the above benchmark that emulates ranges. It is still 29 ms fast.

The full D version is here: http://pastebin.com/R5AGHyPx
The Lua version: http://pastebin.com/Sa7rp6uz
Lua version that emulates ranges: http://pastebin.com/eAKMSWyr

Could someone help me solving this mystery?

Or is D, unlike I thought, not suitable for high performance computing? What should I do?

> Or is D, unlike I thought, not suitable for high performance computing? What should I do?

LuaJIT seems to have a really good backend.
You better compare with ldc or gdc.

On Wed, 22 Dec 2010 17:04:21 -0500, Andreas Mayer <spam@bacon.eggs> wrote:

> To see what performance advantage D would give me over using a scripting language, I made a small benchmark. It consists of this code:
>
>>    auto L = iota(0.0, 10000000.0);
>>    auto L2 = map!"a / 2"(L);
>>    auto L3 = map!"a + 2"(L2);
>>    auto V = reduce!"a + b"(L3);
>
> It runs in 281 ms on my computer.
>
> The same code in Lua (using LuaJIT) runs in 23 ms.
>
> That's about 10 times faster. I would have expected D to be faster. Did I do something wrong?
>
> The first Lua version uses a simplified design. I thought maybe that is unfair to ranges, which are more complicated. You could argue ranges have more features and do more work. To make it fair, I made a second Lua version of the above benchmark that emulates ranges. It is still 29 ms fast.
>
> The full D version is here: http://pastebin.com/R5AGHyPx
> The Lua version: http://pastebin.com/Sa7rp6uz
> Lua version that emulates ranges: http://pastebin.com/eAKMSWyr
>
> Could someone help me solving this mystery?
>
> Or is D, unlike I thought, not suitable for high performance computing? What should I do?

Without any imperical testing, I would guess this has something to do with the lack of inlining for algorithmic functions.  This is due primarily to uses of enforce, which use lazy parameters, which are currently not inlinable (also, ensure you use -O -release -inline for the most optimized code).

I hope that someday this is solved, because it doesn't look very good for D...

-Steve

On 22/12/2010 23:06, Steven Schveighoffer wrote:
> (also, ensure you use -O -release -inline for the most optimized code).

quote...
// D version, with std.algorithm
// ~ 281 ms, using dmd 2.051 (dmd -O -release -inline)

Sometimes having a look at the source helps :)

Trass3r Wrote:

> LuaJIT seems to have a really good backend.
> You better compare with ldc or gdc.

Maybe someone could do that for me? I don't have ldc or gdc here. There are some Debian packages, but they are D version 1 only?

Andreas Mayer Wrote:

> To see what performance advantage D would give me over using a scripting language, I made a small benchmark. It consists of this code:
> 
> >    auto L = iota(0.0, 10000000.0);
> >    auto L2 = map!"a / 2"(L);
> >    auto L3 = map!"a + 2"(L2);
> >    auto V = reduce!"a + b"(L3);

First note: this is a synthetic toy benchmark. Take it with a grain of salt. It represent in no way the true state of D.

> 
> It runs in 281 ms on my computer.
> 
> The same code in Lua (using LuaJIT) runs in 23 ms.

Your mp3 player or file system was doing stuff while executing the benchmark. You probably don't know how to run the test many times and use the average/minimum result for both languages. For example D does not have JIT startup cost so take the minimum result for D, JIT has varying startup speed so take the average or slowest result for Luajit. Compare these. More fair for native code D.

> That's about 10 times faster. I would have expected D to be faster. Did I do something wrong?
> 
> The first Lua version uses a simplified design. I thought maybe that is unfair to ranges, which are more complicated. You could argue ranges have more features and do more work. To make it fair, I made a second Lua version of the above benchmark that emulates ranges. It is still 29 ms fast.
> 
> The full D version is here: http://pastebin.com/R5AGHyPx
> The Lua version: http://pastebin.com/Sa7rp6uz
> Lua version that emulates ranges: http://pastebin.com/eAKMSWyr
> 
> Could someone help me solving this mystery?

My guesses are:

1) you didn't even test this and didn't use optimizations. -> User error
2) whenever doing benchmarks you must compare the competing stuff against all D compilers, cut and paste the object code of different compilers and manually build the fastest executable.
3) you didn't use inline assembler or profiler for D
4) you were using unstable Phobos functions. There is no doubt the final Phobos 2.0 will beat Luajit. D *is* a compiler statical language, Luajit just a joke.
5) you were using old d runtime garbage collector. One fellow here made a precise state of the art GC which beats even Java's 20 year old GC and C#. Patch your dmd to use this instead.

Not intending to start a religious war but if your native code runs slower than *JIT* code, you're doing something wrong. D will always beat JIT. Lua is also a joke language, D is for high performance servers and operating systems. In the worst case, disassemble the luajit program, steal its codes and write it using inline assembler in D. D must win these performance battles. It's technically superior.

> 
> Or is D, unlike I thought, not suitable for high performance computing? What should I do?

It is. D a) takes time to mature and b) you didn't fully utilize the compiler.

On 22/12/2010 23:31, Gary Whatmore wrote:
> Not intending to start a religious war but if your native code runs slower than*JIT*  code, you're doing something wrong. D will always beat JIT.

You talk like a prayer, don't you ? No need to measure.  I believe ...

Anyway I don't care about LUA. The dynamic ASM generation on the other hand is something which should be seriously be considered for D templates. Why ? What comes immediately in mind..no more Bloated executables.
my 2 euro cents

== Quote from Andreas Mayer (spam@bacon.eggs)'s article
> To see what performance advantage D would give me over using a scripting
language, I made a small benchmark. It consists of this code:

One corner case doesn't amount to an end-all be-all proof to anything.

> >    auto L = iota(0.0, 10000000.0);
> >    auto L2 = map!"a / 2"(L);
> >    auto L3 = map!"a + 2"(L2);
> >    auto V = reduce!"a + b"(L3);
> It runs in 281 ms on my computer.
> The same code in Lua (using LuaJIT) runs in 23 ms.
> That's about 10 times faster. I would have expected D to be faster. Did I do
something wrong?
> The first Lua version uses a simplified design. I thought maybe that is unfair
to ranges, which are more complicated. You could argue ranges have more features and do more work. To make it fair, I made a second Lua version of the above benchmark that emulates ranges. It is still 29 ms fast.

As has been already echoed, the lack of inlining algorithmic functions may be one reason for the added cost to runtime. Another may be simply that there is a lot more going on behind the scenes than what you give credit for in D.

Regards :~)

Lua is a proven, robust language

Lua has been used in many industrial applications (e.g., Adobe's Photoshop Lightroom), with an emphasis on embedded systems (e.g., the Ginga middleware for digital TV in Brazil) and games (e.g., World of Warcraft). Lua is currently the leading scripting language in games. Lua has a solid reference manual and there are several books about it. Several versions of Lua have been released and used in real applications since its creation in 1993. Lua featured in HOPL III, the Third ACM SIGPLAN History of Programming Languages Conference, in June 2007.

On Wed, 22 Dec 2010 17:04:21 -0500
Andreas Mayer <spam@bacon.eggs> wrote:

> To see what performance advantage D would give me over using a scripting language, I made a small benchmark. It consists of this code:
> 
> >    auto L = iota(0.0, 10000000.0);
> >    auto L2 = map!"a / 2"(L);
> >    auto L3 = map!"a + 2"(L2);
> >    auto V = reduce!"a + b"(L3);
> 
> It runs in 281 ms on my computer.
> 
> The same code in Lua (using LuaJIT) runs in 23 ms.
> 
> That's about 10 times faster. I would have expected D to be faster. Did I do something wrong?
> 
> The first Lua version uses a simplified design. I thought maybe that is unfair to ranges, which are more complicated. You could argue ranges have more features and do more work. To make it fair, I made a second Lua version of the above benchmark that emulates ranges. It is still 29 ms fast.
> 
> The full D version is here: http://pastebin.com/R5AGHyPx
> The Lua version: http://pastebin.com/Sa7rp6uz
> Lua version that emulates ranges: http://pastebin.com/eAKMSWyr
> 
> Could someone help me solving this mystery?
> 
> Or is D, unlike I thought, not suitable for high performance computing? What should I do?

Dunno why D seems slow. But Lua is a very fast dynamic language, very simple & rather low level (both on language & implementation sides). Benchmark trials in Lua often run much faster than python or ruby equivalents (often 10 X). Depending on the domain, LuaJIT often adds a speed factor of an order of magnitude. This alltogether brings comparable performance to some compiled languages using high-level features such as virtual funcs, GC,... higher-order funcs, ranges ;-)

denis
-- -- -- -- -- -- --
vit esse estrany ☣

spir.wikidot.com