As some of you know, I've been working on a datetime module for possible inclusion in Phobos, and I posted the initial code for review on the Phobos list about a week ago. I have updated the code per the suggestions in that thread. Andrei requested that I move the review to the main D list, so I'm posting my code here this time around. The code and html documentation can be found in this compressed tar file here: http://is.gd/g6em1

It is intended to replace std.date and the nascent std.datetime. The code currently in std.datetime has already been integrated into my code.

My datetime module is modeled after Boost's date/time API but does not use any of its code. I've attempted to D-ify it in various ways (including making it use ranges instead of iterators), but the API is still fairly close to Boost's. One of the big differences from Boost is that I removed the special time points (posifitive infinity, negative infinity, and invalid), since they complicated the code considerably and didn't help at all with infinite ranges, since infinite ranges must be known to be infinite at compile time whereas Boost determines infinity based on the end points (which are only known at runtime). I also made it so the the interval and duration types are generic, so you don't have to deal with different types of intervals and durations depending on the type of the time point.

The major time point types are Date, TimeOfDay, DateTime, and SysTime. Date, TimeOfDay, and DateTime are optimized for calendar-based operations and have no concept of time zone, whereas SysTime is used for the system time and does have a time zone but is not optimized for calendar-based operations. SysTime holds the time in hecto-nanoseconds (100 ns) from midnight, January 1st 1 A.D. UTC which avoids all conversion errors due to DST (conversions to the time zone for the SysTime is done in the getters). All date types use the Proleptic Gregorian Calendar (so they use the Gregorian leap year calculations for their whole length) and use year 0 for 1 B.C. as per ISO 8601. Unlike Boost, they cover from tens-of-thousands of years B.C. to tens-of-thousands of years A.D. (the exact range depending on the type) instead of just some subset of A.D. dates.

When using SysTime, you can ignore time zones if you wish (in most places, LocalTime is used if no TimeZone object is given), but you can also specificy which time zone to use. LocalTime implements local time, UTC implements UTC, SimpleTimeZone allows you to construct a time zone with an offset from UTC but no DST, and PosixTimeZone and WindowsTimeZone allow you to select arbitrary time zones on posix and windows systems respectively (you can also use PosixTimeZone on Windows if you provide the time zone database files). Unlike my first submission, PosixTimeZone is now complete, but WindowsTimeZone is still just stubbed out.

Durations have been simplified since my previous submission. There are now only two duration types: Duration and TickDuration (TickDuration being used by SHOO's stopwatch code and internally for getting the time, whereas Duration is used pretty much everywhere else). You can no longer have durations of months or years due to the issue of converting between months and smaller units (because months and years have a variable number of days). Rather, functions have been provided on the various types for handling months and years directly.

There are interval types (Interval, PosInfInterval, and NegInfInterval) for dealing with intervals (including positively and negatively infinite intervals) as well as corresponding range types. Ranges are constructed from an interval and range-generating function which generates each successive time point in a range (IRange and Interval's fwdRange() are good places to look for more details an that), so they are extremely flexible.

I have included a text file, changes.txt, which contains a list of the majority of changes that I've made since I last posted my code, so if you looked at the first round of code, it should give you a good idea of what has been changed.

The source files are in Src, and the html doc files are in Doc. I attempted to make the doc files look more like Phobos this time around, so I used a slight variation on std.ddoc (so that it looked for the css files in the same directory), and have included the css files with them, so they should be much easier to read (though honestly, the link list at the top of the page is horrible for this module, since it makes no attempt to split out module functions from struct or class functions, and many of the struct and class functions have the same name and yet only one of them gets listed).

Hopefully the documentation is enough to properly understand the module, and of course, the source is there to examine as well. So, review away.

- Jonathan M Davis

On 2010-11-26 03:03:15 -0500, Jonathan M Davis <jmdavisProg@gmx.com> said:

> Most recent code: http://is.gd/hO85C
> 
> I added code for Mac OS X which should use its monotonic clock functions.
> Unfortunately, as I don't own a Mac, it's completed untested, but in theory,
> what's there should work, and someone with a Mac can fix it later if need be.

Just tested it. There's an error in the definition of mach_timebase_info (takes a mach_timebase_info_t* argument, but it should be mach_timebase_info_t, no pointer) and the mach_timebase_info_t alias is missing. Here's a fixed version of the definitions:

	extern(C)
	{
		struct mach_timebase_info_data_t
		{
		    uint numer;
		    uint denom;
		}
		alias mach_timebase_info_data_t* mach_timebase_info_t;

		kern_return_t mach_timebase_info(mach_timebase_info_t);

		ulong mach_absolute_time();
	}

I tested it with this small program:

	extern(C) void sleep(uint sec);

	void main()
	{
	    auto start = TickDuration.currSystemTick;
	    sleep(20);
	    auto end = TickDuration.currSystemTick;
	    writeln(to!string(end-start));
	}

And this is the result (more or less a few thousands on each run):

	TickDuration(20000046269)

So it seems all good.

One thing I wonder about... what is the expected behaviour if you put the computer to sleep in the middle of the above program? With mach_absolute_time, the clock stops counting while the computer is put to sleep. Does TickDuration.currSystemTick works like that on all systems? If there's a difference in behaviour perhaps it should be documented.


-- 
Michel Fortin
michel.fortin@michelf.com
http://michelf.com/

On Friday 26 November 2010 04:38:47 Michel Fortin wrote:
> On 2010-11-26 03:03:15 -0500, Jonathan M Davis <jmdavisProg@gmx.com> said:
> > Most recent code: http://is.gd/hO85C
> > 
> > I added code for Mac OS X which should use its monotonic clock functions. Unfortunately, as I don't own a Mac, it's completed untested, but in theory, what's there should work, and someone with a Mac can fix it later if need be.
> 
> Just tested it. There's an error in the definition of mach_timebase_info (takes a mach_timebase_info_t* argument, but it should be mach_timebase_info_t, no pointer) and the mach_timebase_info_t alias is missing. Here's a fixed version of the definitions:
> 
> 	extern(C)
> 	{
> 		struct mach_timebase_info_data_t
> 		{
> 		    uint numer;
> 		    uint denom;
> 		}
> 		alias mach_timebase_info_data_t* mach_timebase_info_t;
> 
> 		kern_return_t mach_timebase_info(mach_timebase_info_t);
> 
> 		ulong mach_absolute_time();
> 	}
> 
> I tested it with this small program:
> 
> 	extern(C) void sleep(uint sec);
> 
> 	void main()
> 	{
> 	    auto start = TickDuration.currSystemTick;
> 	    sleep(20);
> 	    auto end = TickDuration.currSystemTick;
> 	    writeln(to!string(end-start));
> 	}
> 
> And this is the result (more or less a few thousands on each run):
> 
> 	TickDuration(20000046269)
> 
> So it seems all good.

Thanks! It's hard enough to test for Windows, but doing any Mac testing is essentially impossible for me.

> One thing I wonder about... what is the expected behaviour if you put the computer to sleep in the middle of the above program? With mach_absolute_time, the clock stops counting while the computer is put to sleep. Does TickDuration.currSystemTick works like that on all systems? If there's a difference in behaviour perhaps it should be documented.

Really? That is... not good. Unacceptable in fact. Even if it were acceptable when dealing with stop watch and benchmarking code (which is questionable), it totally fries Clock.currAppTick(), which tells you how long the application has been running. The documentation for QueryPermanceCounter() and clock_gettime() give no indication that the clock stops counting when a thread is sleeping.

Do you know if there's another way to get a monotonic clock on the Mac - one which isn't affected by sleep? Much as I'd hate to do it, it would be better to use gettimeofday() and forgo the monotonic clock then have a monotonic clock which is affected by sleep. But I'd _really_ like to have a monotonic clock if I can.

- Jonathan M Davis

On Friday 26 November 2010 05:59:58 Jonathan M Davis wrote:
> On Friday 26 November 2010 04:38:47 Michel Fortin wrote:
> > On 2010-11-26 03:03:15 -0500, Jonathan M Davis <jmdavisProg@gmx.com> said:
> > > Most recent code: http://is.gd/hO85C
> > > 
> > > I added code for Mac OS X which should use its monotonic clock functions. Unfortunately, as I don't own a Mac, it's completed untested, but in theory, what's there should work, and someone with a Mac can fix it later if need be.
> > 
> > Just tested it. There's an error in the definition of mach_timebase_info (takes a mach_timebase_info_t* argument, but it should be mach_timebase_info_t, no pointer) and the mach_timebase_info_t alias is missing. Here's a fixed version of the
> > 
> > definitions:
> > 	extern(C)
> > 	{
> > 
> > 		struct mach_timebase_info_data_t
> > 		{
> > 
> > 		    uint numer;
> > 		    uint denom;
> > 
> > 		}
> > 		alias mach_timebase_info_data_t* mach_timebase_info_t;
> > 
> > 		kern_return_t mach_timebase_info(mach_timebase_info_t);
> > 
> > 		ulong mach_absolute_time();
> > 
> > 	}
> > 
> > I tested it with this small program:
> > 	extern(C) void sleep(uint sec);
> > 
> > 	void main()
> > 	{
> > 
> > 	    auto start = TickDuration.currSystemTick;
> > 	    sleep(20);
> > 	    auto end = TickDuration.currSystemTick;
> > 	    writeln(to!string(end-start));
> > 
> > 	}
> > 
> > And this is the result (more or less a few thousands on each run):
> > 	TickDuration(20000046269)
> > 
> > So it seems all good.
> 
> Thanks! It's hard enough to test for Windows, but doing any Mac testing is essentially impossible for me.
> 
> > One thing I wonder about... what is the expected behaviour if you put the computer to sleep in the middle of the above program? With mach_absolute_time, the clock stops counting while the computer is put to sleep. Does TickDuration.currSystemTick works like that on all systems? If there's a difference in behaviour perhaps it should be documented.
> 
> Really? That is... not good. Unacceptable in fact. Even if it were
> acceptable when dealing with stop watch and benchmarking code (which is
> questionable), it totally fries Clock.currAppTick(), which tells you how
> long the application has been running. The documentation for
> QueryPermanceCounter() and clock_gettime() give no indication that the
> clock stops counting when a thread is sleeping.
> 
> Do you know if there's another way to get a monotonic clock on the Mac - one which isn't affected by sleep? Much as I'd hate to do it, it would be better to use gettimeofday() and forgo the monotonic clock then have a monotonic clock which is affected by sleep. But I'd _really_ like to have a monotonic clock if I can.

This page ( http://is.gd/hP6Zr ) claims that mach_absolute_time() is not affected by sleeping.

- Jonathan M Davis

Jonathan M Davis Wrote:

> On Friday 26 November 2010 04:38:47 Michel Fortin wrote:

> > One thing I wonder about... what is the expected behaviour if you put the computer to sleep in the middle of the above program? With mach_absolute_time, the clock stops counting while the computer is put to sleep. Does TickDuration.currSystemTick works like that on all systems? If there's a difference in behaviour perhaps it should be documented.
> 
> Really? That is... not good. Unacceptable in fact. Even if it were acceptable when dealing with stop watch and benchmarking code (which is questionable), it totally fries Clock.currAppTick(), which tells you how long the application has been running. The documentation for QueryPermanceCounter() and clock_gettime() give no indication that the clock stops counting when a thread is sleeping.
> 
> Do you know if there's another way to get a monotonic clock on the Mac - one which isn't affected by sleep? Much as I'd hate to do it, it would be better to use gettimeofday() and forgo the monotonic clock then have a monotonic clock which is affected by sleep. But I'd _really_ like to have a monotonic clock if I can.

You may have misunderstood. It's when the *computer* is put to sleep, not the thread. It's still an issue, but not as bad I think.

-Steve

On 2010-11-26 08:59:58 -0500, Jonathan M Davis <jmdavisProg@gmx.com> said:

> On Friday 26 November 2010 04:38:47 Michel Fortin wrote:
> 
>> One thing I wonder about... what is the expected behaviour if you put
>> the computer to sleep in the middle of the above program? With
>> mach_absolute_time, the clock stops counting while the computer is put
>> to sleep. Does TickDuration.currSystemTick works like that on all
>> systems? If there's a difference in behaviour perhaps it should be
>> documented.
> 
> Really? That is... not good. Unacceptable in fact. Even if it were acceptable
> when dealing with stop watch and benchmarking code (which is questionable), it
> totally fries Clock.currAppTick(), which tells you how long the application has
> been running. The documentation for QueryPermanceCounter() and clock_gettime()
> give no indication that the clock stops counting when a thread is sleeping.

No no no... not when a *thread* is sleeping. When the *computer* is sleeping; when the OS is suspended while you close the lid on a laptop for instance. So I start my program, it enters the sleep() function, I close the lid, and one minute later I reopen it and when the sleep() function returns the tick duration is as if the clock was stopped while the lid was closed, despite that the real elapsed time was far much. Closing the lid puts the *computer* to sleep and stops the clock, not the sleap() function.

I understand my example using the sleep() function was misleading because the term sleep had two meanings.


-- 
Michel Fortin
michel.fortin@michelf.com
http://michelf.com/

On Friday 26 November 2010 06:14:29 Michel Fortin wrote:
> On 2010-11-26 08:59:58 -0500, Jonathan M Davis <jmdavisProg@gmx.com> said:
> > On Friday 26 November 2010 04:38:47 Michel Fortin wrote:
> >> One thing I wonder about... what is the expected behaviour if you put the computer to sleep in the middle of the above program? With mach_absolute_time, the clock stops counting while the computer is put to sleep. Does TickDuration.currSystemTick works like that on all systems? If there's a difference in behaviour perhaps it should be documented.
> > 
> > Really? That is... not good. Unacceptable in fact. Even if it were
> > acceptable when dealing with stop watch and benchmarking code (which is
> > questionable), it totally fries Clock.currAppTick(), which tells you how
> > long the application has been running. The documentation for
> > QueryPermanceCounter() and clock_gettime() give no indication that the
> > clock stops counting when a thread is sleeping.
> 
> No no no... not when a *thread* is sleeping. When the *computer* is sleeping; when the OS is suspended while you close the lid on a laptop for instance. So I start my program, it enters the sleep() function, I close the lid, and one minute later I reopen it and when the sleep() function returns the tick duration is as if the clock was stopped while the lid was closed, despite that the real elapsed time was far much. Closing the lid puts the *computer* to sleep and stops the clock, not the sleap() function.
> 
> I understand my example using the sleep() function was misleading because the term sleep had two meanings.

Oh. Well that isn't the same magnitude of problem at all. I wouldn't have thought that that the computer going to sleep/hibernating to RAM would cause that sort of problem, but I guess that it could. I have no idea if Windows or Linux would be affected in the same way. They might be. I don't have any way to test it. Regardless, it's still a bit problematic, but acceptable I think. If it could be verified whether Linux or Windows have the same problem, that would be good. It probably should be put in the documentation.

It really never would have occurred to me that putting the computer to sleep/hibernating to RAM would affect the clock. Of course, since I don't have a working laptop (and I've hated having it go to sleep when I've had one), I certainly wouldn't run into it personally.

- Jonathan M Davis

Jonathan M Davis Wrote:

> totally fries Clock.currAppTick(), which tells you how long the application has been running. The documentation for QueryPermanceCounter() and clock_gettime() give no indication that the clock stops counting when a thread is sleeping.

I think, QueryPermanceCounter just queries processor's builtin counter (HPET). So it's hardware implementation defined. HPET is not present on older hardware, and future hardware can implement something different.

On Friday 26 November 2010 06:14:29 Michel Fortin wrote:
> On 2010-11-26 08:59:58 -0500, Jonathan M Davis <jmdavisProg@gmx.com> said:
> > On Friday 26 November 2010 04:38:47 Michel Fortin wrote:
> >> One thing I wonder about... what is the expected behaviour if you put the computer to sleep in the middle of the above program? With mach_absolute_time, the clock stops counting while the computer is put to sleep. Does TickDuration.currSystemTick works like that on all systems? If there's a difference in behaviour perhaps it should be documented.
> > 
> > Really? That is... not good. Unacceptable in fact. Even if it were
> > acceptable when dealing with stop watch and benchmarking code (which is
> > questionable), it totally fries Clock.currAppTick(), which tells you how
> > long the application has been running. The documentation for
> > QueryPermanceCounter() and clock_gettime() give no indication that the
> > clock stops counting when a thread is sleeping.
> 
> No no no... not when a *thread* is sleeping. When the *computer* is sleeping; when the OS is suspended while you close the lid on a laptop for instance. So I start my program, it enters the sleep() function, I close the lid, and one minute later I reopen it and when the sleep() function returns the tick duration is as if the clock was stopped while the lid was closed, despite that the real elapsed time was far much. Closing the lid puts the *computer* to sleep and stops the clock, not the sleap() function.
> 
> I understand my example using the sleep() function was misleading because the term sleep had two meanings.

By the way, did you run the full unit tests on Mac OS X, or did you just test and fix the Mac-specific changes that I made? I assume that the unit tests pass on Mac OS X, but I don't know.

- Jonathan M Davis

On 2010-11-26 20:50:16 -0500, Jonathan M Davis <jmdavisProg@gmx.com> said:

> By the way, did you run the full unit tests on Mac OS X, or did you just test
> and fix the Mac-specific changes that I made? I assume that the unit tests pass on
> Mac OS X, but I don't know.

The unit tests in unittests.d and time.d pass fine, but datetime.d does not compile:

datetime.d(26977): Error: undefined identifier tzset
datetime.d(27035): Error: template unittests.assertEqual(alias pred = "a == b",T,U) if (__traits(compiles,binaryFun!(pred)(actual,expected)) && isPrintable!(T) && isPrintable!(U)) does not match any function template declaration
... and the last error repeated a couple of time


-- 
Michel Fortin
michel.fortin@michelf.com
http://michelf.com/