Regarding ticket 14 in druntime:

http://www.dsource.org/projects/druntime/ticket/14

I've started looking at the best way to annotate all the pure/nothrow functions in druntime.  A lot of druntime functionality calls functions from the C standard library or the Windows/POSIX API.  I figure the best way to get started is to deal w/ these first, and then worry about the higher level stuff that's built on to of it.  These functions are obviously nothrow, since C doesn't even support exceptions.  First of all, is there a decent automated way to fix all of the declarations for these functions so that they're nothrow?

Secondly, since we don't have the sources to the C libraries for which the function prototypes are declared, we can't be absolutely sure about their purity.  A good portion of the ones that seem like they could be pure, mostly math stuff like sqrt, log, etc., are probably messing w/ errno, and theoretically they could be memoizing stuff, etc.  What's a reasonable course of action for these?  It seems like pure is most useful for mathematical functions, , so it would really be a shame not to use it here.

dsimcha wrote:
> I've started looking at the best way to annotate all the pure/nothrow
> functions in druntime.  A lot of druntime functionality calls functions from
> the C standard library or the Windows/POSIX API.  I figure the best way to get
> started is to deal w/ these first, and then worry about the higher level stuff
> that's built on to of it.  These functions are obviously nothrow, since C
> doesn't even support exceptions.  First of all, is there a decent automated
> way to fix all of the declarations for these functions so that they're nothrow?

I did think of making all functions that are extern(C) automatically nothrow, but was concerned that it would result in a lot of bugs and broken code from ones that did throw.

> Secondly, since we don't have the sources to the C libraries for which the
> function prototypes are declared, we can't be absolutely sure about their
> purity.  A good portion of the ones that seem like they could be pure, mostly
> math stuff like sqrt, log, etc., are probably messing w/ errno, and
> theoretically they could be memoizing stuff, etc.  What's a reasonable course
> of action for these?  It seems like pure is most useful for mathematical
> functions, , so it would really be a shame not to use it here.

Functions that modify errno cannot be pure. What needs to be done is go through the definition of each. For example, strlen, strcmp, etc. can be marked as pure. printf and strtok cannot.

The math functions can be a problem since earlier incarnations often messed with global state, but with the advent of thread safe C runtime libraries, this shouldn't be an issue anymore.

== Quote from Walter Bright (newshound1@digitalmars.com)'s article
> dsimcha wrote:
> > I've started looking at the best way to annotate all the pure/nothrow functions in druntime.  A lot of druntime functionality calls functions from the C standard library or the Windows/POSIX API.  I figure the best way to get started is to deal w/ these first, and then worry about the higher level stuff that's built on to of it.  These functions are obviously nothrow, since C doesn't even support exceptions.  First of all, is there a decent automated way to fix all of the declarations for these functions so that they're nothrow?
> I did think of making all functions that are extern(C) automatically nothrow, but was concerned that it would result in a lot of bugs and broken code from ones that did throw.

I assume, when referring to the ones that do throw, you mean functions written in C++ or D, but declared w/ C linkage.  If so, you could make this a per-module setting that defaults to not assuming nothrow.  For example, let's say you made this pragma(Linkage, nothrow).  Then, if this statement is seen at the top of a module, everything declared with extern(Linkage) in that module is assumed to be nothrow.  For standard C, Windows and POSIX API functions and for any library written in pure C, I believe (correct me if I'm wrong) this would be a safe assumption.  At any rate, it would make nothrow a heck of a lot more usable.


> Functions that modify errno cannot be pure. What needs to be done is go
> through the definition of each. For example, strlen, strcmp, etc. can be
> marked as pure. printf and strtok cannot.
> The math functions can be a problem since earlier incarnations often
> messed with global state, but with the advent of thread safe C runtime
> libraries, this shouldn't be an issue anymore.

Isn't errno defined in some implementations to be thread-local?  If so, I guess we still have a problem.  Then again, in the long run it probably makes sense to reimplement a lot of the math stuff that still uses the C std lib in pure D anyhow so that things like CTFE work on it, but in the sort run I'm sure that's not anyone's top priority.

dsimcha wrote:
> I assume, when referring to the ones that do throw, you mean functions written in
> C++ or D, but declared w/ C linkage.  If so, you could make this a per-module
> setting that defaults to not assuming nothrow.  For example, let's say you made
> this pragma(Linkage, nothrow).  Then, if this statement is seen at the top of a
> module, everything declared with extern(Linkage) in that module is assumed to be
> nothrow.  For standard C, Windows and POSIX API functions and for any library
> written in pure C, I believe (correct me if I'm wrong) this would be a safe
> assumption.  At any rate, it would make nothrow a heck of a lot more usable.

This will do it:

nothrow:
... rest of module ...


> Isn't errno defined in some implementations to be thread-local?

Yes, but thread local doesn't mean pure.

> If so, I guess we
> still have a problem.  Then again, in the long run it probably makes sense to
> reimplement a lot of the math stuff that still uses the C std lib in pure D anyhow
> so that things like CTFE work on it, but in the sort run I'm sure that's not
> anyone's top priority.

Don has already reimplemented most of them in D, this was done to:

1. ensure a minimum level of performance and accuracy; some C ones are crappily done

2. properly support all the D floating point types and overloading rules

3. support NAN and INFINITY correctly

Walter Bright wrote:
> dsimcha wrote:
>> I assume, when referring to the ones that do throw, you mean functions written in
>> C++ or D, but declared w/ C linkage.  If so, you could make this a per-module
>> setting that defaults to not assuming nothrow.  For example, let's say you made
>> this pragma(Linkage, nothrow).  Then, if this statement is seen at the top of a
>> module, everything declared with extern(Linkage) in that module is assumed to be
>> nothrow.  For standard C, Windows and POSIX API functions and for any library
>> written in pure C, I believe (correct me if I'm wrong) this would be a safe
>> assumption.  At any rate, it would make nothrow a heck of a lot more usable.
> 
> This will do it:
> 
> nothrow:
> ... rest of module ...
> 
> 
>> Isn't errno defined in some implementations to be thread-local?
> 
> Yes, but thread local doesn't mean pure.
> 
>> If so, I guess we
>> still have a problem.  Then again, in the long run it probably makes sense to
>> reimplement a lot of the math stuff that still uses the C std lib in pure D anyhow
>> so that things like CTFE work on it, but in the sort run I'm sure that's not
>> anyone's top priority.
> 
> Don has already reimplemented most of them in D, this was done to:
> 
> 1. ensure a minimum level of performance and accuracy; some C ones are crappily done
> 
> 2. properly support all the D floating point types and overloading rules
> 
> 3. support NAN and INFINITY correctly

tango.math doesn't use the C library at all, except when inline asm is unavailable. Of they differ from the C functions, in that none of them set errno!
One really annoying issue still remains, though -- the floating point flags in the CPU. They are entirely deterministic, but are they considered to be part of the return value of the function? Or would we allow them to be ignored?
A compiler could check the exception flags before allowing memoisation. But one could also do the same thing for 'errno'.

Likewise, floating point rounding modes. Essentially, the floating point   status register is a hidden global variable, read from# and written to during every floating point operation.

# - only the rounding mode and truncation affect the return value. We could deal with it by regarding that as a whole-program setting. But (depending on the CPU), the old exception flags generally get ORed with the new exception flags.

Also, you can set the flags to allow any floating point function to throw a hardware exception. It's difficult for any function using floating point to claim to be nothrow under ANY circumstances; but that's a horrible limitation.

Don wrote:
> tango.math doesn't use the C library at all, except when inline asm is unavailable. Of they differ from the C functions, in that none of them set errno!
> One really annoying issue still remains, though -- the floating point flags in the CPU. They are entirely deterministic, but are they considered to be part of the return value of the function? Or would we allow them to be ignored?
> A compiler could check the exception flags before allowing memoisation. But one could also do the same thing for 'errno'.
> 
> Likewise, floating point rounding modes. Essentially, the floating point   status register is a hidden global variable, read from# and written to during every floating point operation.
> 
> # - only the rounding mode and truncation affect the return value. We could deal with it by regarding that as a whole-program setting. But (depending on the CPU), the old exception flags generally get ORed with the new exception flags.

Those are good points. I don't know what the answer is. My inclination is to say if your program relies on changing the rounding mode or fiddles with the exception flags, it's undefined behavior.

> Also, you can set the flags to allow any floating point function to throw a hardware exception. It's difficult for any function using floating point to claim to be nothrow under ANY circumstances; but that's a horrible limitation.

I would say that is not supported by D. I've never heard of a use for them.

Walter Bright wrote:
> Don wrote:
>> tango.math doesn't use the C library at all, except when inline asm is unavailable. Of they differ from the C functions, in that none of them set errno!
>> One really annoying issue still remains, though -- the floating point flags in the CPU. They are entirely deterministic, but are they considered to be part of the return value of the function? Or would we allow them to be ignored?
>> A compiler could check the exception flags before allowing memoisation. But one could also do the same thing for 'errno'.
>>
>> Likewise, floating point rounding modes. Essentially, the floating point   status register is a hidden global variable, read from# and written to during every floating point operation.
>>
>> # - only the rounding mode and truncation affect the return value. We could deal with it by regarding that as a whole-program setting. But (depending on the CPU), the old exception flags generally get ORed with the new exception flags.
> 
> Those are good points. I don't know what the answer is. My inclination is to say if your program relies on changing the rounding mode or fiddles with the exception flags, it's undefined behavior.

One form of error analysis is to run the program with different rounding modes, and compare the results.

You can also use rounding modes to implement interval arithmetic, but this would normally be restricted to low-level functions. The rounding mode would not escape from those functions.

I normally use the exception flags for debugging.

>> Also, you can set the flags to allow any floating point function to throw a hardware exception. It's difficult for any function using floating point to claim to be nothrow under ANY circumstances; but that's a horrible limitation.
> 
> I would say that is not supported by D. I've never heard of a use for them.

What happens if a nothrow function throws an exception? IMHO a satisfactory response would be to abort the program with an error message/ drop you into a debugger -- anyway, that's the only thing I use when running with FP exceptions enabled.

I guess it's reasonable to argue that using the floating-point flags is sufficiently hard-core that pure and nothrow should pretend that they don't exist.

Still, some functions (especially correctly-rounded floating-point i/o) go to a lot of trouble to support them.  I have a suspicion that it's not worth the effort.

On 2009-01-25 14:39:52 -0500, Walter Bright <newshound1@digitalmars.com> said:

> I did think of making all functions that are extern(C) automatically nothrow, but was concerned that it would result in a lot of bugs and broken code from ones that did throw.

It's better that way, because it allows you to write proper wrappers for C++ functions that may throw exceptions. Like this:

	extern "C"
	int _doSomething_wrapper(int value)
	{
		try
		{
			return doSomething(value);
		}
		catch (const std::exception & e)
		{
			_d_throw_exception(e.what());
		}
	}

with _d_throw_exception written in D and throwing an exception.

-- 
Michel Fortin
michel.fortin@michelf.com
http://michelf.com/

Don wrote:
> Walter Bright wrote:
>> Don wrote:
>>> tango.math doesn't use the C library at all, except when inline asm is unavailable. Of they differ from the C functions, in that none of them set errno!
>>> One really annoying issue still remains, though -- the floating point flags in the CPU. They are entirely deterministic, but are they considered to be part of the return value of the function? Or would we allow them to be ignored?
>>> A compiler could check the exception flags before allowing memoisation. But one could also do the same thing for 'errno'.
>>>
>>> Likewise, floating point rounding modes. Essentially, the floating point   status register is a hidden global variable, read from# and written to during every floating point operation.
>>>
>>> # - only the rounding mode and truncation affect the return value. We could deal with it by regarding that as a whole-program setting. But (depending on the CPU), the old exception flags generally get ORed with the new exception flags.
>>
>> Those are good points. I don't know what the answer is. My inclination is to say if your program relies on changing the rounding mode or fiddles with the exception flags, it's undefined behavior.
> 
> One form of error analysis is to run the program with different rounding modes, and compare the results.
> 
> You can also use rounding modes to implement interval arithmetic, but this would normally be restricted to low-level functions. The rounding mode would not escape from those functions.
> 
> I normally use the exception flags for debugging.
> 
>>> Also, you can set the flags to allow any floating point function to throw a hardware exception. It's difficult for any function using floating point to claim to be nothrow under ANY circumstances; but that's a horrible limitation.
>>
>> I would say that is not supported by D. I've never heard of a use for them.
> 
> What happens if a nothrow function throws an exception? IMHO a satisfactory response would be to abort the program with an error message/ drop you into a debugger -- anyway, that's the only thing I use when running with FP exceptions enabled.

I'd be ok with saying throwing fp exceptions is a non-recoverable error, like a seg fault or stack overflow, and is acceptable in a nothrow function.


> I guess it's reasonable to argue that using the floating-point flags is sufficiently hard-core that pure and nothrow should pretend that they don't exist.
> 
> Still, some functions (especially correctly-rounded floating-point i/o) go to a lot of trouble to support them.  I have a suspicion that it's not worth the effort.

So we have two options. One is to say that floating point arithmetic cannot be made pure. The other is to ignore the problem (saying it's undefined behavior).

On 2009-01-26 14:21:18 -0500, Walter Bright <newshound1@digitalmars.com> said:

>> I guess it's reasonable to argue that using the floating-point flags is sufficiently hard-core that pure and nothrow should pretend that they don't exist.
>> 
>> Still, some functions (especially correctly-rounded floating-point i/o) go to a lot of trouble to support them.  I have a suspicion that it's not worth the effort.
> 
> So we have two options. One is to say that floating point arithmetic cannot be made pure. The other is to ignore the problem (saying it's undefined behavior).

I see another: the compiler keeps track of the state of floating point flags inside a function, and prevent pure optimisations across those boundaries. Of course, the language would have to expose a mean to change these flags in order for it to work... and prevent any such change from leaking after the lifetime of a function.

-- 
Michel Fortin
michel.fortin@michelf.com
http://michelf.com/