Hi all,

I have set up some benchmarks for dmd, ldc and gdc at http://dbench.octarineparrot.com/.

There are currently only 6 tests all from http://shootout.alioth.debian.org/gp4/d.php. My knowledge of phobos is not great enough to port the others to tango (I've chosen tango as ldc does not support phobos currently, so it make sense to choose tango as all compilers support it). If you would like to contribute new tests or improve on the current ones let me know and I'll include them next time I run them.

All source code can be found at http://hg.octarineparrot.com/dbench/file/tip.

Let me know if you have any ideas for how I can improve the benchmarks, I currently plan to add compile times, size of the final executable and memory usage (if anyone knows an easy way to get the memory usage of a process in D, let me know :D).

1. Could you add "some" bench that Alioth currently run on q6600 ubuntu?
2. A gnu c++ for a reference would be great.
I'm very eager to port C++ entries to D :)

Robert Clipsham wrote:
> Hi all,
> 
> I have set up some benchmarks for dmd, ldc and gdc at http://dbench.octarineparrot.com/.
> 
> There are currently only 6 tests all from http://shootout.alioth.debian.org/gp4/d.php. My knowledge of phobos is not great enough to port the others to tango (I've chosen tango as ldc does not support phobos currently, so it make sense to choose tango as all compilers support it). If you would like to contribute new tests or improve on the current ones let me know and I'll include them next time I run them.
> 
> All source code can be found at http://hg.octarineparrot.com/dbench/file/tip.
> 
> Let me know if you have any ideas for how I can improve the benchmarks, I currently plan to add compile times, size of the final executable and memory usage (if anyone knows an easy way to get the memory usage of a process in D, let me know :D).

3. Do you use multithread or single thread?

Robert Clipsham wrote:
> ...
> 
> (if anyone knows an easy way to get the memory usage of a
> process in D, let me know :D).

There's a way to do it in Phobos;

> import gc=std.gc;
> import gcstats;
>
> void main()
> {
>     GCStats gcst;
>     gc.getStats(gcst);
> }


I went hunting through Tango, and it looks like it has the same method; it just isn't exposed.  Probably because of this comment:

// NOTE: This routine is experimental.  The stats or function name may //       change before it is made officially available.

None the less, if you want it now, you could try adding this to your code somewhere:

> struct GCStats
> {
>     size_t poolsize;        // total size of pool
>     size_t usedsize;        // bytes allocated
>     size_t freeblocks;      // number of blocks marked FREE
>     size_t freelistsize;    // total of memory on free lists
>     size_t pageblocks;      // number of blocks marked PAGE
> }
>
> extern(C) GCStats gc_stats();

You should probably whack this in a module so you can replace it easily if and when it changes.


  -- Daniel

Daniel Keep wrote:
> 
> Robert Clipsham wrote:
>> ...
>>
>> (if anyone knows an easy way to get the memory usage of a
>> process in D, let me know :D).
> 
> There's a way to do it in Phobos;
> 
>> import gc=std.gc;
>> import gcstats;
>>
>> void main()
>> {
>>     GCStats gcst;
>>     gc.getStats(gcst);
>> }
> 
> 
> I went hunting through Tango, and it looks like it has the same method;
> it just isn't exposed.  Probably because of this comment:
> 
> // NOTE: This routine is experimental.  The stats or function name may
> //       change before it is made officially available.
> 
> None the less, if you want it now, you could try adding this to your
> code somewhere:
> 
>> struct GCStats
>> {
>>     size_t poolsize;        // total size of pool
>>     size_t usedsize;        // bytes allocated
>>     size_t freeblocks;      // number of blocks marked FREE
>>     size_t freelistsize;    // total of memory on free lists
>>     size_t pageblocks;      // number of blocks marked PAGE
>> }
>>
>> extern(C) GCStats gc_stats();
> 
> You should probably whack this in a module so you can replace it easily
> if and when it changes.
> 
> 
>   -- Daniel
I was thinking more a way of getting the memory usage using run.d (the app I'm using for benchmarking, it's in the repository if you're interested). It's rather difficult to get the memory usage in run.d so I can put it straight into the stats page if it's being got from within benchmark, and doing this would probably affect the benchmark some amount which I would ideally like to avoid.

The Anh Tran wrote:
> 1. Could you add "some" bench that Alioth currently run on q6600 ubuntu?
> 2. A gnu c++ for a reference would be great.
> I'm very eager to port C++ entries to D :)

1. All the benchmarks currently up are just tango ports of tests from alioth, if that's what you mean?
2. I wasn't planning on adding in C,C++ etc benchmarks as then it will just become a clone of the shootout. I don't mind adding a reference in for each test from C/C++ if there is enough demand, I would rather avoid it and make it purely for D benchmarks if possible.

Please feel free to port benchmarks to D/tango, I'll be more than happy to incorporate them into the suite (which is currently fairly minimal).

The Anh Tran wrote:
> 3. Do you use multithread or single thread?

I'm not sure what you mean here. All the current benchmarks are single threaded as the multi threaded benchmarks use std.thread, and my knowledge of phobos is not good enough to port them. If you mean the machine itself, it does support multithreading, so tests could benefit from that.

Robert Clipsham wrote:
> I was thinking more a way of getting the memory usage using run.d (the app I'm using for benchmarking, it's in the repository if you're interested). It's rather difficult to get the memory usage in run.d so I can put it straight into the stats page if it's being got from within benchmark, and doing this would probably affect the benchmark some amount which I would ideally like to avoid.

Ah.

From the alioth FAQ:

> How did you measure memory use?
>
> By sampling GTop proc_mem for the program and it's child processes every 0.2 seconds. Obviously those measurements are unlikely to be reliable for programs that run for less than 0.2 seconds.

Probably best to ensure this sampling thread is running on a different hardware thread to the tested program...

  -- Daniel

Could you provide D compiler version that you're using?
Download-able compiler packages if you don't mind :D. I'm spoiled by EasyD.

Incidentally, this might be of assistance:

http://shootout.alioth.debian.org/u32q/faq.php#measurementscripts

  -- Daniel