Which way of measuring time in std.datetime should I used if I want to benchmark a function by the amount of time spent in the current thread, not wall-clock time?

On Tuesday, November 01, 2016 16:53:51 Nordlöw via Digitalmars-d-learn wrote:
> Which way of measuring time in std.datetime should I used if I want to benchmark a function by the amount of time spent in the current thread, not wall-clock time?

That's what std.datetime.benchmark is for, though unfortunately, it still uses core.time.TickDuration rather using core.time.MonoTime and core.time.Duration.

Alternatively, you can always do something like

immutable before = MonoTime.currTime();
// do stuf...
immutable after = MonoTime.currTime();
Duration timeSpent = after - before;

though that just times how long something took rather than running it over and over again like benchmark does.

I really need to get replacements for the TickDuration-related stuff in std.datetime into std.experimental so that we can move towards replacing it and finally getting rid of TickDuration, but I haven't gotten to it yet.

- Jonathan M Davis

On Tuesday, 1 November 2016 at 18:39:01 UTC, Jonathan M Davis wrote:
> Alternatively, you can always do something like
>
> immutable before = MonoTime.currTime();
> // do stuf...
> immutable after = MonoTime.currTime();
> Duration timeSpent = after - before;

MonoTime has about 5-10 % fluctuations on my laptop. Is this as good as it gets?

On Tuesday, 1 November 2016 at 20:19:31 UTC, Nordlöw wrote:
> MonoTime has about 5-10 % fluctuations on my laptop. Is this as good as it gets?

Is

https://dlang.org/phobos/core_time.html#.ClockType.threadCPUTime

what I should use if I'm on Linux?

On Tuesday, November 01, 2016 20:19:31 Nordlöw via Digitalmars-d-learn wrote:
> On Tuesday, 1 November 2016 at 18:39:01 UTC, Jonathan M Davis
>
> wrote:
> > Alternatively, you can always do something like
> >
> > immutable before = MonoTime.currTime();
> > // do stuf...
> > immutable after = MonoTime.currTime();
> > Duration timeSpent = after - before;
>
> MonoTime has about 5-10 % fluctuations on my laptop. Is this as good as it gets?

How precise it is depends on your system. MonoTime uses the system's monotonic clock at whatever precision it has. And even if it's perfectly precise, you're going to see fluctuations in how long it takes your code to run, since it's not the only thing running on your computer. That's part of why std.datetime.benchmark runs the function over and over again rather than only running it only once. That way, you get the average performance rather than the performance for a single run.

- Jonathan m Davis

On Tuesday, November 01, 2016 20:21:32 Nordlöw via Digitalmars-d-learn wrote:
> On Tuesday, 1 November 2016 at 20:19:31 UTC, Nordlöw wrote:
> > MonoTime has about 5-10 % fluctuations on my laptop. Is this as good as it gets?
>
> Is
>
> https://dlang.org/phobos/core_time.html#.ClockType.threadCPUTime
>
> what I should use if I'm on Linux?

The default (ClockType.normal) unless you have a really good reason to do otherwise. And if you're most worried about having high precision (as is presumably the case if you're benchmarking), ClockType.normal is the best you get on Linux. You can use ClockType.precise, but on Linux, it's the same as ClockType.normal. It's just FreeBSD that has a more precise clock. And really, ClockType.normal is plenty precise. IIRC, it's usually about a tick per microsecond on Linux.

- Jonathan M Davis

On 11/1/16 4:19 PM, Nordlöw wrote:
> On Tuesday, 1 November 2016 at 18:39:01 UTC, Jonathan M Davis wrote:
>> Alternatively, you can always do something like
>>
>> immutable before = MonoTime.currTime();
>> // do stuf...
>> immutable after = MonoTime.currTime();
>> Duration timeSpent = after - before;
>
> MonoTime has about 5-10 % fluctuations on my laptop. Is this as good as
> it gets?

This is not clear. What is this a percentage of?

If you are hoping that you run code and the total time taken is going to be consistent within a few nanoseconds, then I'm sorry to say it's not going to happen. If you mean it's varying by 5-10 seconds, then this is a different story, but still could be expected.

MonoTime is going to be very accurate. What isn't true is that on a laptop being used with a non-realtime OS, you will have exactly consistent results for userspace programs.

As Jonathan mentioned, the way around this is to average several runs of the same test.

-Steve

On Tuesday, 1 November 2016 at 20:54:34 UTC, Steven Schveighoffer wrote:
>> MonoTime has about 5-10 % fluctuations on my laptop. Is this as good as
>> it gets?
>
> This is not clear. What is this a percentage of?

It percentage of execution time.

15 ms +- 2 ms

That seems doesn't look like what you describe.

Benchmark code here:

https://github.com/nordlow/phobos-next/blob/master/src/benchmarkAppend.d

On Tuesday, 1 November 2016 at 22:55:36 UTC, Nordlöw wrote:
> On Tuesday, 1 November 2016 at 20:54:34 UTC, Steven Schveighoffer wrote:
>>> MonoTime has about 5-10 % fluctuations on my laptop. Is this as good as
>>> it gets?
>>
>> This is not clear. What is this a percentage of?
>
> It percentage of execution time.
>
> 15 ms +- 2 ms
>
> That seems doesn't look like what you describe.
>
> Benchmark code here:
>
> https://github.com/nordlow/phobos-next/blob/master/src/benchmarkAppend.d

I supposed you could not have done it with less templates :)
Since you are hitting gc-code all the time it should be expected to have varing times.

On 11/1/16 6:55 PM, Nordlöw wrote:
> On Tuesday, 1 November 2016 at 20:54:34 UTC, Steven Schveighoffer wrote:
>>> MonoTime has about 5-10 % fluctuations on my laptop. Is this as good as
>>> it gets?
>>
>> This is not clear. What is this a percentage of?
>
> It percentage of execution time.
>
> 15 ms +- 2 ms

2ms variance is not horrendous or unexpected on a non-realtime system.

-Steve