View mode: basic / threaded / horizontal-split · Log in · Help
April 08, 2012
std.benchmark ready for review. Manager sought after
Hello,


I finally found the time to complete std.benchmark. I got to a very 
simple API design, starting where I like it: one line of code.

Code is in the form of a pull request at 
https://github.com/D-Programming-Language/phobos/pull/529. (There's some 
noise in there caused by my git n00biness). Documentation is at 
http://erdani.com/d/web/phobos-prerelease/std_benchmark.html.

If reasonable and at all possible, I'd like to bump the priority of this 
proposal. Clearly D's user base is highly interested in efficiency, and 
many of the upcoming libraries have efficiency a virtual part of their 
design. So we should get std.benchmark in soon and require that new 
addition come with benchmarks for their essential functionality.

My vision is that in the future Phobos will have a significant 
benchmarks battery, which will help improving Phobos and porting to new 
platforms.


Andrei
April 08, 2012
Re: std.benchmark ready for review. Manager sought after
On Sunday, 8 April 2012 at 03:25:16 UTC, Andrei Alexandrescu 
wrote:
> Hello,
>
>
> I finally found the time to complete std.benchmark. I got to a 
> very simple API design, starting where I like it: one line of 
> code.

Nice, some comments:

1) Is it possible to do something about the "kilonanoseconds", 
and print the two-letter SI time abbreviation instead 
(μs/ns/ms/s/...)?

2) The meaning of "epoch" in this context is unfamiliar to me:

> Measurement is done in epochs. For each function benchmarked, 
> the smallest time is taken over all epochs.

3) "benchmark_relative_file read" should be replaced with a 
language construct. E.g. a function call like 
relativeBenchmark("file read"), or an enum value like getopt's.

4) There is a TODO in benchmarkSuspend's example.
April 08, 2012
Re: std.benchmark ready for review. Manager sought after
On 4/7/12 11:45 PM, Vladimir Panteleev wrote:
> On Sunday, 8 April 2012 at 03:25:16 UTC, Andrei Alexandrescu wrote:
>> Hello,
>>
>>
>> I finally found the time to complete std.benchmark. I got to a very
>> simple API design, starting where I like it: one line of code.
>
> Nice, some comments:
>
> 1) Is it possible to do something about the "kilonanoseconds", and print
> the two-letter SI time abbreviation instead (μs/ns/ms/s/...)?

Cool.

> 2) The meaning of "epoch" in this context is unfamiliar to me:
>
>> Measurement is done in epochs. For each function benchmarked, the
>> smallest time is taken over all epochs.

To me too :o).

> 3) "benchmark_relative_file read" should be replaced with a language
> construct. E.g. a function call like relativeBenchmark("file read"), or
> an enum value like getopt's.

No can do. Need a function name-based convention so we can automate 
scheduleForBenchmarking.

> 4) There is a TODO in benchmarkSuspend's example.

Will destroy.


Andrei
April 08, 2012
Re: std.benchmark ready for review. Manager sought after
On Sunday, 8 April 2012 at 05:41:11 UTC, Andrei Alexandrescu 
wrote:
>> 3) "benchmark_relative_file read" should be replaced with a 
>> language
>> construct. E.g. a function call like relativeBenchmark("file 
>> read"), or
>> an enum value like getopt's.
>
> No can do. Need a function name-based convention so we can 
> automate scheduleForBenchmarking.

Hmm, maybe there's a misunderstanding, but what I meant was:

The "benchmark_relative_" prefix makes sense for function names 
(for scheduleForBenchmarking), but not so much for string 
literals for benchmark names. The string literal 
"benchmark_relative_file read" looks like the words "benchmark 
relative file" are grouped together, with "read" added on. So, my 
suggestion would be to wrap the "benchmark_relative_" prefix - 
when used with benchmark name strings - into a semantical 
function / enum / etc. In my example above, relativeBenchmark 
would be:

string relativeBenchmark(string s) { return "benchmark_relative_" 
~ s; }

I suppose it can be summed up as a tradeoff between complexity 
(you need to explain both the function name usage and the 
relativeBenchmark wrapper usage) vs. code prettiness.
April 08, 2012
Re: std.benchmark ready for review. Manager sought after
Le 08/04/2012 05:25, Andrei Alexandrescu a écrit :
> Hello,
> 
> 
> I finally found the time to complete std.benchmark. I got to a very
> simple API design, starting where I like it: one line of code.
> 
> Code is in the form of a pull request at
> https://github.com/D-Programming-Language/phobos/pull/529. (There's some
> noise in there caused by my git n00biness). Documentation is at
> http://erdani.com/d/web/phobos-prerelease/std_benchmark.html.
> 
> If reasonable and at all possible, I'd like to bump the priority of this
> proposal. Clearly D's user base is highly interested in efficiency, and
> many of the upcoming libraries have efficiency a virtual part of their
> design. So we should get std.benchmark in soon and require that new
> addition come with benchmarks for their essential functionality.
> 
> My vision is that in the future Phobos will have a significant
> benchmarks battery, which will help improving Phobos and porting to new
> platforms.
> 
> 
> Andrei

Like it. Would it be a good idea to add a column with an average memory
used ?
April 08, 2012
Re: std.benchmark ready for review. Manager sought after
Andrei Alexandrescu wrote:
> I finally found the time to complete std.benchmark. I got to a very
> simple API design, starting where I like it: one line of code.
>
> Code is in the form of a pull request at
> https://github.com/D-Programming-Language/phobos/pull/529. (There's some
> noise in there caused by my git n00biness). Documentation is at
> http://erdani.com/d/web/phobos-prerelease/std_benchmark.html.

For algorithms that process sequences, it would be nice to have results 
represented in cycles per item, This should give more consistent results 
across different CPU familes and different clock speeds.

Specifically, I think about cycles per byte, see 
http://en.wikipedia.org/wiki/Cycles_per_byte
Example: http://www.cryptopp.com/benchmarks.html
April 08, 2012
Re: std.benchmark ready for review. Manager sought after
On 4/8/12 2:02 AM, Vladimir Panteleev wrote:
> The "benchmark_relative_" prefix makes sense for function names (for
> scheduleForBenchmarking), but not so much for string literals for
> benchmark names. The string literal "benchmark_relative_file read" looks
> like the words "benchmark relative file" are grouped together, with
> "read" added on. So, my suggestion would be to wrap the
> "benchmark_relative_" prefix - when used with benchmark name strings -
> into a semantical function / enum / etc. In my example above,
> relativeBenchmark would be:
>
> string relativeBenchmark(string s) { return "benchmark_relative_" ~ s; }
>
> I suppose it can be summed up as a tradeoff between complexity (you need
> to explain both the function name usage and the relativeBenchmark
> wrapper usage) vs. code prettiness.

I understand, thanks.

Andrei
April 08, 2012
Re: std.benchmark ready for review. Manager sought after
On 4/8/12 3:16 AM, Somedude wrote:
> Like it. Would it be a good idea to add a column with an average memory
> used ?

Interesting idea. I saw 
http://stackoverflow.com/questions/1674652/c-c-memory-usage-api-in-linux-windows 
and it looks like it's not an easy problem. Should we make this part of 
the initial release?

Andrei
April 08, 2012
Re: std.benchmark ready for review. Manager sought after
On 4/8/12 5:46 AM, Piotr Szturmaj wrote:
> Andrei Alexandrescu wrote:
>> I finally found the time to complete std.benchmark. I got to a very
>> simple API design, starting where I like it: one line of code.
>>
>> Code is in the form of a pull request at
>> https://github.com/D-Programming-Language/phobos/pull/529. (There's some
>> noise in there caused by my git n00biness). Documentation is at
>> http://erdani.com/d/web/phobos-prerelease/std_benchmark.html.
>
> For algorithms that process sequences, it would be nice to have results
> represented in cycles per item, This should give more consistent results
> across different CPU familes and different clock speeds.
>
> Specifically, I think about cycles per byte, see
> http://en.wikipedia.org/wiki/Cycles_per_byte
> Example: http://www.cryptopp.com/benchmarks.html

The framework aims at generality and simplicity. I'd rather have it 
simple and universal rather than subtle.

Andrei
April 08, 2012
Re: std.benchmark ready for review. Manager sought after
On 08.04.2012 12:16, Somedude wrote:
[snip]
>
> Like it. Would it be a good idea to add a column with an average memory
> used ?

In general it's next to impossible and/or entirely OS-specific.
What can be done I think is adding a query function to GC interface that 
returns amount of RAM currently allocated on it's heap.

So adding GC heap usage can be done albeit it's not really a stable 
metric. This way one gets a nice hint on why something is slow ;)

-- 
Dmitry Olshansky
« First   ‹ Prev
1 2 3 4 5
Top | Discussion index | About this forum | D home