| Thread overview | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
December 13, 2008 Basic benchmark | ||||
|---|---|---|---|---|
| ||||
I have adapted another small benchmark to D. This benchmark is less interesting than the other ones because it mostly tests the optimizations done by the back-end. This means it's not a problem of the D language or its front-end, so even if DMD here shows to be not much efficient, LDC once finished may show significant improvements. As usual I may have done several errors, so keep your eyes open. D code: /* original code copyright 2004 Christopher W. Cowell-Shah http://www.cowell-shah.com/research/benchmark/code other code portions copyright http://dada.perl.it/shootout/ and Doug Bagley http://www.bagley.org/~doug/shootout combined, modified and fixed by Thomas Bruckschlegel - http://www.tommti-systems.com */ import std.c.stdio: printf; import std.c.time: clock, CLOCKS_PER_SEC, clock_t; void longArithmetic(long longMin, long longMax) { clock_t startTime = clock(); long longResult = longMin; long i = longMin; while (i < longMax) { longResult -= i++; longResult += i++; longResult *= i++; longResult /= i++; } clock_t stopTime = clock(); double elapsedTime = (stopTime - startTime) / (CLOCKS_PER_SEC / 1000.0); printf("Long arithmetic elapsed time: %1.0f ms with longMax %ld\n", elapsedTime, longMax); printf(" i: %ld\n", i); printf(" longResult: %ld\n", longResult); } void nested_loops(int n) { clock_t startTime = clock(); int a, b, c, d, e, f; int x=0; for (a=0; a<n; a++) for (b=0; b<n; b++) for (c=0; c<n; c++) for (d=0; d<n; d++) for (e=0; e<n; e++) for (f=0; f<n; f++) x+=a+b+c+d+e+f; clock_t stopTime = clock(); double elapsedTime = (stopTime - startTime) / (CLOCKS_PER_SEC / 1000.0); printf("Nested Loop elapsed time: %1.0f ms %d\n", elapsedTime, x); } int main() { long longMin = 10_000_000_000L; long longMax = 11_000_000_000L; longArithmetic(longMin, longMax); nested_loops(40); return 0; } ------------------------ C code, almost the same (you may need to change LL_FORMAT to make it run correctly): /* original code copyright 2004 Christopher W. Cowell-Shah http://www.cowell-shah.com/research/benchmark/code other code portions copyright http://dada.perl.it/shootout/ and Doug Bagley http://www.bagley.org/~doug/shootout combined, modified and fixed by Thomas Bruckschlegel - http://www.tommti-systems.com */ #include "time.h" #include "stdio.h" // accopding to your compiler #define LL_FORMAT "%I64d" //#define LL_FORMAT "%ld" void longArithmetic(long long longMin, long long longMax) { clock_t startTime = clock(); long long longResult = longMin; long long i = longMin; while (i < longMax) { longResult -= i++; longResult += i++; longResult *= i++; longResult /= i++; } clock_t stopTime = clock(); double elapsedTime = (stopTime - startTime) / (CLOCKS_PER_SEC / 1000.0); printf("Long arithmetic elapsed time: %1.0f ms with longMax "LL_FORMAT"\n", elapsedTime, longMax); printf(" i: "LL_FORMAT"\n", i); printf(" longResult: "LL_FORMAT"\n", longResult); } void nested_loops(int n) { clock_t startTime = clock(); int a, b, c, d, e, f; int x=0; for (a=0; a<n; a++) for (b=0; b<n; b++) for (c=0; c<n; c++) for (d=0; d<n; d++) for (e=0; e<n; e++) for (f=0; f<n; f++) x+=a+b+c+d+e+f; clock_t stopTime = clock(); double elapsedTime = (stopTime - startTime) / (CLOCKS_PER_SEC / 1000.0); printf("Nested Loop elapsed time: %1.0f ms %d\n", elapsedTime, x); } int main() { long long longMin = 10000000000LL; long long longMax = 11000000000LL; longArithmetic(longMin, longMax); nested_loops(40); return 0; } ------------------- I have compiled it with GCC and DMD with: gcc version 4.2.1-dw2 (mingw32-2) -O3 -s DMD v1.037 -O -release -inline --------------------- Timings: C gcc: Long arithmetic: 11.15 s Nested Loops: 0.11 s D dmd: Long arithmetic: 63.7 s Nested Loops: 6.17 s Bye, bearophile | ||||
December 13, 2008 Re: Basic benchmark | ||||
|---|---|---|---|---|
| ||||
Posted in reply to bearophile | bearophile wrote: > I have adapted another small benchmark to D. This benchmark is less interesting than the other ones because it mostly tests the optimizations done by the back-end. This means it's not a problem of the D language or its front-end, so even if DMD here shows to be not much efficient, LDC once finished may show significant improvements. > As usual I may have done several errors, so keep your eyes open. > ..snip.. > > Timings: > > C gcc: > Long arithmetic: 11.15 s > Nested Loops: 0.11 s > > D dmd: > Long arithmetic: 63.7 s > Nested Loops: 6.17 s > > Bye, > bearophile I tried this out with Tango + DMD 1.033, Tango + LDC r847 and GCC 4.3.2, my timings are as follows, best of three: $ dmd bench.d -O -release -inline long arith: 55630 ms nested loop: 5090 ms $ ldc bench.d -O3 -release -inline long arith: 13870 ms nested loop: 120 ms $ gcc bench.c -O3 -s -fomit-frame-pointer long arith: 13600 ms nested loop: 170 ms My cpu is: Athlon64 X2 3800+ | |||
December 13, 2008 Re: Basic benchmark | ||||
|---|---|---|---|---|
| ||||
Posted in reply to Tomas Lindquist Olsen | On Sat, Dec 13, 2008 at 11:16 AM, Tomas Lindquist Olsen <tomas@famolsen.dk> wrote:
> I tried this out with Tango + DMD 1.033, Tango + LDC r847 and GCC 4.3.2, my timings are as follows, best of three:
>
> $ dmd bench.d -O -release -inline
> long arith: 55630 ms
> nested loop: 5090 ms
>
>
> $ ldc bench.d -O3 -release -inline
> long arith: 13870 ms
> nested loop: 120 ms
>
>
> $ gcc bench.c -O3 -s -fomit-frame-pointer
> long arith: 13600 ms
> nested loop: 170 ms
>
>
> My cpu is: Athlon64 X2 3800+
>
Go LDC!
I hope bearophile will eventually understand that DMD is not good at optimizing code, and so comparing its output to GCC's is ultimately meaningless.
| |||
December 13, 2008 Re: Basic benchmark | ||||
|---|---|---|---|---|
| ||||
Posted in reply to Jarrett Billingsley | == Quote from Jarrett Billingsley (jarrett.billingsley@gmail.com)'s article > On Sat, Dec 13, 2008 at 11:16 AM, Tomas Lindquist Olsen <tomas@famolsen.dk> wrote: > > I tried this out with Tango + DMD 1.033, Tango + LDC r847 and GCC 4.3.2, my timings are as follows, best of three: > > > > $ dmd bench.d -O -release -inline > > long arith: 55630 ms > > nested loop: 5090 ms > > > > > > $ ldc bench.d -O3 -release -inline > > long arith: 13870 ms > > nested loop: 120 ms > > > > > > $ gcc bench.c -O3 -s -fomit-frame-pointer > > long arith: 13600 ms > > nested loop: 170 ms > > > > > > My cpu is: Athlon64 X2 3800+ > > > Go LDC! > I hope bearophile will eventually understand that DMD is not good at > optimizing code, and so comparing its output to GCC's is ultimately > meaningless. Speaking of LDC, any chance that the exception handling on Win32 gets fixed in the near future? I'd like to start using it, but I work on Windows. | |||
December 13, 2008 Re: Basic benchmark | ||||
|---|---|---|---|---|
| ||||
Posted in reply to Jarrett Billingsley | Jarrett Billingsley wrote:
> I hope bearophile will eventually understand that DMD is not good at optimizing code, and so comparing its output to GCC's is ultimately meaningless.
Personally, I appreciate seeing this stuff from bearophile. I use D in ways where speed really does count. One of my draws to D was that it was a systems language that could be faster than something like Java. I also was sick of C++ and its problems, such as code that requires workarounds for compiler bugs or lack of compiler optimization. It's really sad to see D requiring the same kind of stuff. For D to become as mainstream as C++, all of this stuff that bearophile posts must be fixed.
| |||
December 13, 2008 Re: Basic benchmark | ||||
|---|---|---|---|---|
| ||||
Posted in reply to Jason House | On Sat, Dec 13, 2008 at 12:55 PM, Jason House <jason.james.house@gmail.com> wrote:
> Jarrett Billingsley wrote:
>
>> I hope bearophile will eventually understand that DMD is not good at optimizing code, and so comparing its output to GCC's is ultimately meaningless.
>
> Personally, I appreciate seeing this stuff from bearophile. I use D in ways where speed really does count. One of my draws to D was that it was a systems language that could be faster than something like Java. I also was sick of C++ and its problems, such as code that requires workarounds for compiler bugs or lack of compiler optimization. It's really sad to see D requiring the same kind of stuff. For D to become as mainstream as C++, all of this stuff that bearophile posts must be fixed.
>
Walter is the only one who can make DMD faster, and I think his time is much better spent on designing and maintaining the language. The reference compiler is just supposed to be _correct_, not necessarily _fast_. If Walter spent all his time working on making the the DMDFE optimizer better and making DMD backend produce faster code, he wouldn't have time to work on the language anymore, and it would be duplicated effort since GDC and LDC already do it better.
| |||
December 13, 2008 Re: Basic benchmark | ||||
|---|---|---|---|---|
| ||||
Posted in reply to Jarrett Billingsley | On 2008-12-13 19:07:09 +0100, "Jarrett Billingsley" <jarrett.billingsley@gmail.com> said: > On Sat, Dec 13, 2008 at 12:55 PM, Jason House > <jason.james.house@gmail.com> wrote: >> Jarrett Billingsley wrote: >> >>> I hope bearophile will eventually understand that DMD is not good at >>> optimizing code, and so comparing its output to GCC's is ultimately >>> meaningless. >> >> Personally, I appreciate seeing this stuff from bearophile. I use D in ways where speed really does count. One of my draws to D was that it was a systems language that could be faster than something like Java. I also was sick of C++ and its problems, such as code that requires workarounds for compiler bugs or lack of compiler optimization. It's really sad to see D requiring the same kind of stuff. For D to become as mainstream as C++, all of this stuff that bearophile posts must be fixed. >> > > Walter is the only one who can make DMD faster, and I think his time > is much better spent on designing and maintaining the language. The > reference compiler is just supposed to be _correct_, not necessarily > _fast_. If Walter spent all his time working on making the the DMDFE > optimizer better and making DMD backend produce faster code, he > wouldn't have time to work on the language anymore, and it would be > duplicated effort since GDC and LDC already do it better. I fully agree, and it is not that DMD is necessarily slow, but does not perform some kinds of optimizations. For example for the nested loops it does not float the operations out of the internal loop to as high up as possible. I would like for this to be the case (for example my multidimensional array library would profit from this), but if you really see that in your code it becomes and issue (looking at profiling) then normally it is quite easy to rewrite it so that it is fast. Just looking at very specific benchmarks that test one kind of optimization can be very misleading. It is good to have benchmarks and know where the weaknesses of a compiler are, but for real code the situation is different. At least for the code that I write, and typical code I have seen DMD is reasonably competitive. (this does not mean that it can't and shouldn't be improved ;) Fawzi | |||
December 13, 2008 Re: Basic benchmark | ||||
|---|---|---|---|---|
| ||||
On Sun, Dec 14, 2008 at 3:07 AM, Jarrett Billingsley <jarrett.billingsley@gmail.com> wrote: > On Sat, Dec 13, 2008 at 12:55 PM, Jason House <jason.james.house@gmail.com> wrote: >> Jarrett Billingsley wrote: >> >>> I hope bearophile will eventually understand that DMD is not good at optimizing code, and so comparing its output to GCC's is ultimately meaningless. >> >> Personally, I appreciate seeing this stuff from bearophile. I use D in ways where speed really does count. One of my draws to D was that it was a systems language that could be faster than something like Java. I also was sick of C++ and its problems, such as code that requires workarounds for compiler bugs or lack of compiler optimization. It's really sad to see D requiring the same kind of stuff. For D to become as mainstream as C++, all of this stuff that bearophile posts must be fixed. >> > > Walter is the only one who can make DMD faster, and I think his time is much better spent on designing and maintaining the language. I think the point is not to convince Walter to spend time working on DMD's optimizer, but to convince him that the DMD optimizer is hopelessly obsolete and thus should be abandoned in favor of another, like LDC. There's also the 64-bit issue. I don't see Walter ever making the current toolchain 64-bit capable (at least not on Windows). This is going to become an increasingly ridiculous limitation for a supposed "systems programming language" as time marches on. At some point something has to change. > The reference compiler is just supposed to be _correct_, not necessarily _fast_. Fortunately it's not an either/or situation. If Walter chooses to move the reference compiler to a mainstream compiler infrastructure, then *he* can work on making the reference compiler correct, while many *other people* (including many who don't know anything about D) work on making the compiler fast. > If Walter spent all his time working on making the the DMDFE optimizer better and making DMD backend produce faster code, he wouldn't have time to work on the language anymore, Agreed. That would be like putting lipstick on the proverbial pig. > and it would be > duplicated effort since GDC and LDC already do it better. I guess it's possible to imagine a world where Walter cranks out DMDFE code coupled to a sub-par DMD backend that no one uses, since everyone has moved on to LDC or something. But why go there? LDC is completely open source. There's no reason the reference D compiler can't also be the fast D compiler. And become more open in the process, too. That reference compiler / fast compiler dichotomy might have been ok for C++ back in the old "cfront" days, but in those days people everywhere were dying for something a little more high-level than C. Today they aren't. In those days the big corps took notice of C++ and most vendors were maintaining their own cfront-based compilers for their own platforms with their own custom back-end optimizations. There's nothing like that happening with D today. Today the big corps have C++ and if that's not high-level enough then they have 32-dozen scripting languages and VM hosted byte-compiled languages to choose from. So for a niche language like D, making the default compiler be a sucky compiler is very bad marketing in my opinion. And talk about duplicating efforts -- every time Walter releases a new reference compiler, the developers on the fast compiler have to scramble to incorporate those changes, when they could be working on bug fixes and other useful performance improvements. And downstream bugfixes is another area of duplicated efforts -- already LDC developers have fixed various bugs in the DMDFE, and these must then be posted to bugzilla for Walter to eventually put back into his version of DMDFE. That said, LDC isn't quite there yet, especially on Windows, but it would be very encouraging to see Walter take at least a little interest in it. The transition would be a little painful for a while, but much less painful than trying to write a new back end from scratch, and in the end I believe it would make D a much more viable platform going forward. --bb | ||||
December 13, 2008 Re: Basic benchmark | ||||
|---|---|---|---|---|
| ||||
Posted in reply to Bill Baxter | Bill Baxter wrote:
> On Sun, Dec 14, 2008 at 3:07 AM, Jarrett Billingsley
> <jarrett.billingsley@gmail.com> wrote:
>> On Sat, Dec 13, 2008 at 12:55 PM, Jason House
>> <jason.james.house@gmail.com> wrote:
>>> Jarrett Billingsley wrote:
>>>
>>>> I hope bearophile will eventually understand that DMD is not good at
>>>> optimizing code, and so comparing its output to GCC's is ultimately
>>>> meaningless.
>>> Personally, I appreciate seeing this stuff from bearophile. I use D in ways where speed really does count. One of my draws to D was that it was a systems language that could be faster than something like Java. I also was sick of C++ and its problems, such as code that requires workarounds for compiler bugs or lack of compiler optimization. It's really sad to see D requiring the same kind of stuff. For D to become as mainstream as C++, all of this stuff that bearophile posts must be fixed.
>>>
>> Walter is the only one who can make DMD faster, and I think his time
>> is much better spent on designing and maintaining the language.
>
> I think the point is not to convince Walter to spend time working on
> DMD's optimizer, but to convince him that the DMD optimizer is
> hopelessly obsolete and thus should be abandoned in favor of another,
> like LDC. There's also the 64-bit issue. I don't see Walter ever
> making the current toolchain 64-bit capable (at least not on Windows).
> This is going to become an increasingly ridiculous limitation for a
> supposed "systems programming language" as time marches on.
>
> At some point something has to change.
>
>> The reference compiler is just supposed to be _correct_, not necessarily
>> _fast_.
>
> Fortunately it's not an either/or situation. If Walter chooses to
> move the reference compiler to a mainstream compiler infrastructure,
> then *he* can work on making the reference compiler correct, while
> many *other people* (including many who don't know anything about D)
> work on making the compiler fast.
>
>> If Walter spent all his time working on making the the DMDFE
>> optimizer better and making DMD backend produce faster code, he
>> wouldn't have time to work on the language anymore,
>
> Agreed. That would be like putting lipstick on the proverbial pig.
>
>> and it would be
>> duplicated effort since GDC and LDC already do it better.
>
> I guess it's possible to imagine a world where Walter cranks out DMDFE
> code coupled to a sub-par DMD backend that no one uses, since everyone
> has moved on to LDC or something. But why go there? LDC is
> completely open source. There's no reason the reference D compiler
> can't also be the fast D compiler. And become more open in the
> process, too.
>
> That reference compiler / fast compiler dichotomy might have been ok
> for C++ back in the old "cfront" days, but in those days people
> everywhere were dying for something a little more high-level than C.
> Today they aren't. In those days the big corps took notice of C++ and
> most vendors were maintaining their own cfront-based compilers for
> their own platforms with their own custom back-end optimizations.
> There's nothing like that happening with D today. Today the big corps
> have C++ and if that's not high-level enough then they have 32-dozen
> scripting languages and VM hosted byte-compiled languages to choose
> from.
>
> So for a niche language like D, making the default compiler be a sucky
> compiler is very bad marketing in my opinion. And talk about
> duplicating efforts -- every time Walter releases a new reference
> compiler, the developers on the fast compiler have to scramble to
> incorporate those changes, when they could be working on bug fixes and
> other useful performance improvements. And downstream bugfixes is
> another area of duplicated efforts -- already LDC developers have
> fixed various bugs in the DMDFE, and these must then be posted to
> bugzilla for Walter to eventually put back into his version of DMDFE.
>
> That said, LDC isn't quite there yet, especially on Windows, but it
> would be very encouraging to see Walter take at least a little
> interest in it. The transition would be a little painful for a while,
> but much less painful than trying to write a new back end from
> scratch, and in the end I believe it would make D a much more viable
> platform going forward.
>
> --bb
After having seen GDC fail to live up to expectations and become abandonware, it's unsurprising that Walter's unwilling to invest any emotional energy into LDC just yet. In six months the story may be completely different.
| |||
December 13, 2008 Re: Basic benchmark | ||||
|---|---|---|---|---|
| ||||
Posted in reply to bearophile | bearophile wrote:
> Timings:
>
> C gcc:
> Long arithmetic: 11.15 s
> Nested Loops: 0.11 s
>
> D dmd:
> Long arithmetic: 63.7 s
> Nested Loops: 6.17 s
I suggest running obj2asm on the resulting obj files and see what the real difference is.
| |||
Copyright © 1999-2021 by the D Language Foundation
Permalink
Reply