avgtime - Small D util for your everyday benchmarking needs (page 4)

On Tuesday, 27 March 2012 at 00:58:26 UTC, Ary Manzana wrote: > On 3/23/12 4:11 PM, Juan Manuel Cabo wrote: >> On Friday, 23 March 2012 at 06:51:48 UTC, James Miller wrote: >> >>> Dude, this is awesome. I tend to just use time, but if I was doing >>> anything more complicated, I'd use this. I would suggest changing the >>> name while you still can. avgtime is not that informative a name given >>> that it now does more than just "Average" times. >>> >>> -- >>> James Miller >> >> >>> Dude, this is awesome. >> >> Thanks!! I appreciate your feedback! >> >>> I would suggest changing the name while you still can. >> >> Suggestions welcome!! >> >> --jm >> > > give_me_d_average Hahahah, naahh, prefiero avgtime o timestats, porque times<tab> autocompletaría a timestats. Qué hacés tanto tiempo? Gracias por mencionarme D hace años. Me quedó en la cabeza, y el año pasado cuando empecé un laburo nuevo tuve oportunidad de meterme con D. Saludos Ary, espero que andes bien!! --jm

On Tuesday, 27 March 2012 at 01:19:22 UTC, Juan Manuel Cabo wrote: > On Tuesday, 27 March 2012 at 00:58:26 UTC, Ary Manzana wrote: >> On 3/23/12 4:11 PM, Juan Manuel Cabo wrote: >>> On Friday, 23 March 2012 at 06:51:48 UTC, James Miller wrote: >>> >>>> Dude, this is awesome. I tend to just use time, but if I was doing >>>> anything more complicated, I'd use this. I would suggest changing the >>>> name while you still can. avgtime is not that informative a name given >>>> that it now does more than just "Average" times. >>>> >>>> -- >>>> James Miller >>> >>> >>>> Dude, this is awesome. >>> >>> Thanks!! I appreciate your feedback! >>> >>>> I would suggest changing the name while you still can. >>> >>> Suggestions welcome!! >>> >>> --jm >>> >> >> give_me_d_average > > > Hahahah, naahh, prefiero avgtime o timestats, porque times<tab> > autocompletaría a timestats. > > Qué hacés tanto tiempo? Gracias por mencionarme D hace años. > Me quedó en la cabeza, y el año pasado cuando empecé un laburo > nuevo tuve oportunidad de meterme con D. > > Saludos Ary, espero que andes bien!! > --jm El nombre lo dije en broma :-P Me sorprendió muchísimo verte en la lista! Pensé "Juanma?". Qué loco que te guste D. A mí me gusta también, pero tiene algunas cosas feas y que lamentablemente no veo que vayan a cambiar pronto... (o nunca). So you are using D for work?

On Thursday, 22 March 2012 at 17:13:58 UTC, Manfred Nowak wrote: > Juan Manuel Cabo wrote: > >> like the unix 'time' command > > `version linux' is missing. > > -manfred Done!, it works in windows now too. (release 0.5 in github). --jm

On Tuesday, 27 March 2012 at 03:39:56 UTC, Ary Manzana wrote: > On Tuesday, 27 March 2012 at 01:19:22 UTC, Juan Manuel Cabo wrote: >> On Tuesday, 27 March 2012 at 00:58:26 UTC, Ary Manzana wrote: >>> On 3/23/12 4:11 PM, Juan Manuel Cabo wrote: >>>> On Friday, 23 March 2012 at 06:51:48 UTC, James Miller wrote: >>>> >>>>> Dude, this is awesome. I tend to just use time, but if I was doing >>>>> anything more complicated, I'd use this. I would suggest changing the >>>>> name while you still can. avgtime is not that informative a name given >>>>> that it now does more than just "Average" times. >>>>> >>>>> -- >>>>> James Miller >>>> >>>> >>>>> Dude, this is awesome. >>>> >>>> Thanks!! I appreciate your feedback! >>>> >>>>> I would suggest changing the name while you still can. >>>> >>>> Suggestions welcome!! >>>> >>>> --jm >>>> >>> >>> give_me_d_average >> >> >> Hahahah, naahh, prefiero avgtime o timestats, porque times<tab> >> autocompletaría a timestats. >> >> Qué hacés tanto tiempo? Gracias por mencionarme D hace años. >> Me quedó en la cabeza, y el año pasado cuando empecé un laburo >> nuevo tuve oportunidad de meterme con D. >> >> Saludos Ary, espero que andes bien!! >> --jm > > El nombre lo dije en broma :-P [...] ahhaha, ya se que lo dijiste en broma! --jm

March 27, 2012

Re: avgtime - Small D util for your everyday benchmarking needs

Posted by Don Clugston
in reply to Andrei Alexandrescu

Permalink

Don Clugston

Posted in reply to Andrei Alexandrescu

Permalink

On 23/03/12 16:25, Andrei Alexandrescu wrote:
> On 3/23/12 12:51 AM, Manfred Nowak wrote:
>> Andrei Alexandrescu wrote:
>>
>>> You may want to also print the mode of the distribution,
>>> nontrivial but informative
>>
>> In case of this implementation and according to the given link: trivial
>> and noninformative, because
>>
>> | For samples, if it is known that they are drawn from a symmetric
>> | distribution, the sample mean can be used as an estimate of the
>> | population mode.
>>
>> and the program computes the variance as if the values of the sample
>> follow a normal distribution, which is symmetric.
>>
>> Therefore the mode of the sample is of interest only, when the variance
>> is calculated wrongly.
>
> Again, benchmarks I've seen are always asymmetric. Not sure why those
> shown here are symmetric. The mode should be very close to the minimum
> (and in fact I think taking the minimum is a pretty good approximation
> of the sought-after time).
>
> Andrei

Agreed, I think situations where you would get a normal distribution are rare in benchmarking code.
Small sections of code always have a best-case scenario, where there are no cache misses.
If there are task switches, the best case is zero task switches.

If you use the CPU performance counters, you can identify the *cause* of performance variations. When I've done this, I've always been able to get very stable numbers

On Wed, Mar 21, 2012 at 6:32 PM, Juan Manuel Cabo <juanmanuel.cabo@gmail.com > wrote: > This is a small util I wrote in D which is like the unix 'time' command but can repeat the command N times and show median, average, standard deviation, minimum and maximum. > > As you all know, it is not proper to conclude that > a program is faster than another program by running > them just once. > > It's BOOST and is in github: > > https://github.com/jmcabo/**avgtime <https://github.com/jmcabo/avgtime> > > Example: > > > avgtime -r 10 -q ls -lR /etc > > ------------------------ > Total time (ms): 933.742 > Repetitions : 10 > Median time : 90.505 > Avg time : 93.3742 > Std dev. : 4.66808 > Minimum : 88.732 > Maximum : 101.225 > > The -q argument pipes stderr and stdout of the program under test to /dev/null > > I put more info in the github page. > > > HAVE FUN!! > > --jm > > > Nice tool. I used it here: http://www.reddit.com/r/programming/comments/rif9x/uniform_function_call_syntax_for_the_d/c46bjs7?context=2 It'd be neat if it could create comparison statistics between the execution of two different programs so you could compare the performance of changes or alternative approaches to the same problem as I was doing. Regards, Brad Anderson

Forums