View mode: basic / threaded / horizontal-split · Log in · Help
May 30, 2011
Helping the compiler with assumptions while using exceptions
I'm having some problems trying to get the best of both worlds here.

void f(Class c) {
  assert(c != null);
  // use c
}

In this example, we tell the compiler that c is never able to be null. 
The compiler can use assertions like this for optimizations (not sure if 
dmd does this though).

But assert is only a debugging tool.
Say we wanted to have this check at runtime too - just in case - so we 
can fail where the problem is.

So we do this:

void f(Class c) {
  enforce(c != null);
  // use c
}

But now the compiler has no idea c will never be null later on (or does 
it...?).

We could always do this:

void f(Class c) {
  assert(c != null);
  enforce(c != null);
  // use c
}

But this is overly verbose.

Or is this not a problem at all? E.g. Use enforce for runtime checks - 
the compiler understands them/won't use asserts for optimizations anyway?
May 30, 2011
Re: Helping the compiler with assumptions while using exceptions
simendsjo:

> void f(Class c) {
>    assert(c != null);
>    // use c
> }
> 
> In this example, we tell the compiler that c is never able to be null. 
> The compiler can use assertions like this for optimizations (not sure if 
> dmd does this though).

I think currently DMD is not using this information, and probably it will keep not using it for some more time.
Note: generally asserts go into Contracts. Here it goes in the precondition of f:

void f(Class c)
in {
   assert(c != null);
}
body {
   // use c
}


> void f(Class c) {
>    assert(c != null);
>    enforce(c != null);
>    // use c
> }

This is not meaningful. assert and enforce have different purposes, you don't use both of them at the same time. If f is a inner function/method of your system, then you add asserts inside Contracts to assert your code contains no bugs. If your f is a function/method that's on an "interface" border, so it's used by other users, then you may want to use enforce instead.

Bye,
bearophile
May 30, 2011
Re: Helping the compiler with assumptions while using exceptions
On 2011-05-30 12:49, simendsjo wrote:
> I'm having some problems trying to get the best of both worlds here.
> 
> void f(Class c) {
>    assert(c != null);
>    // use c
> }
> 
> In this example, we tell the compiler that c is never able to be null.
> The compiler can use assertions like this for optimizations (not sure if
> dmd does this though).
> 
> But assert is only a debugging tool.
> Say we wanted to have this check at runtime too - just in case - so we
> can fail where the problem is.
> 
> So we do this:
> 
> void f(Class c) {
>    enforce(c != null);
>    // use c
> }
> 
> But now the compiler has no idea c will never be null later on (or does
> it...?).
> 
> We could always do this:
> 
> void f(Class c) {
>    assert(c != null);
>    enforce(c != null);
>    // use c
> }
> 
> But this is overly verbose.
> 
> Or is this not a problem at all? E.g. Use enforce for runtime checks -
> the compiler understands them/won't use asserts for optimizations anyway?

assert and enforce serve two very different purposes. assert is used for code 
verification. You can't assume that it's always be there, because it might be 
compiled out. As such, the compiler is not going to use it for any 
optimizations. If an assertion fails, there's a bug in your program. It's used 
to detect bugs.

enforce and exceptions, on the other hand, are _not_ intended for bug 
detection. They are for error handling. They're generally used in cases where 
you want to ensure that something is true but where you can't guarantee it 
(such as when dealing with user input or the file system). You throw an 
exception if the condition isn't true, and then you deal with it however is 
appropriate. Generally-speaking, an exception does _not_ indicate a bug. It 
indicates an error, but that doesn't mean that there's anything wrong with 
your program. Rather, there was a problem with its input.

As such, if you were to use assert in the above code, then you're saying that 
it's a bug in your program if c is null and having that check go away in 
release mode is fine, because you're done looking for bugs. It's assumed that 
the code is correct enough that c will never be null. If, you use enforce, on 
the other hand, you're saying that it could be perfectly valid for c to be 
null, but you don't want your f function to ever execute with a null c, so you 
throw if c is null. Then whatever calls f can handle it in whatever way is 
appropriate, possibly resulting in an error being printed to the user or 
another attempt to call f with another value for c which isn't null or 
whatever is appropriate for your program. By using enforce, you're saying that 
it's not a bug for c to be null but that it _is_ an error (likely caused by 
input in some manner) if it occurs.

What assert and enforce are doing here are completely different. You shouldn't 
be choosing between assert and enforce based on whether they'll be around in 
release mode or not. They're two completely different types of error handling.

Now, if it's a bug in your code if c is null, and you want to be absolutely 
certain that c is never null - even in release mode - then you can use 
assert(0). e.g.

if(c is null)
	assert(0, "c cannot be null!");

assert(0) is translated to a HLT instruction in release mode. So, it'll kill 
your program if it's hit. It's intended for cases which should _never_ happen 
and you want to be absolutely sure that they never do (such as a default case 
statement or else clause which should never be hit as long as your logic is 
correct). It's still for bug detection like normal assert, but it's going the 
extra step of saying that if this line of code is ever reached in release 
mode, kill the program. It's still completely different from enforce and 
exceptions which are runtime errors which are potentially handleable.

- Jonathan M Davis
May 30, 2011
Re: Helping the compiler with assumptions while using exceptions
Jonathan M Davis wrote:
> On 2011-05-30 12:49, simendsjo wrote:
>> I'm having some problems trying to get the best of both worlds here.
>>
>> void f(Class c) {
>>    assert(c != null);
>>    // use c
>> }
>>
>> In this example, we tell the compiler that c is never able to be null.
>> The compiler can use assertions like this for optimizations (not sure if
>> dmd does this though).
>>
>> But assert is only a debugging tool.
>> Say we wanted to have this check at runtime too - just in case - so we
>> can fail where the problem is.
>>
>> So we do this:
>>
>> void f(Class c) {
>>    enforce(c != null);
>>    // use c
>> }
>>
>> But now the compiler has no idea c will never be null later on (or does
>> it...?).
>>
>> We could always do this:
>>
>> void f(Class c) {
>>    assert(c != null);
>>    enforce(c != null);
>>    // use c
>> }
>>
>> But this is overly verbose.

Actually, what you have written does not make sense. Correct would be:
void f(Class c) {
  enforce(c != null);
  assert(c != null); //if c is null, control can never reach this instruction.
  // use c
}

You cannot move assert into the contract in your use case, because it is not a
contract. It will be valid to call your function with a null reference, it will
just throw an exception.

>>
>> Or is this not a problem at all? E.g. Use enforce for runtime checks -
>> the compiler understands them/won't use asserts for optimizations anyway?

Currently, DMD does not use assertions or exceptions for optimization of code, but
it could.

>
> assert and enforce serve two very different purposes. assert is used for code
> verification. You can't assume that it's always be there, because it might be
> compiled out. As such, the compiler is not going to use it for any
> optimizations. If an assertion fails, there's a bug in your program. It's used
> to detect bugs.

And to give hints to the compiler. This is what his question was about.

>
> enforce and exceptions, on the other hand, are _not_ intended for bug
> detection. They are for error handling. They're generally used in cases where
> you want to ensure that something is true but where you can't guarantee it
> (such as when dealing with user input or the file system). You throw an
> exception if the condition isn't true, and then you deal with it however is
> appropriate. Generally-speaking, an exception does _not_ indicate a bug. It
> indicates an error, but that doesn't mean that there's anything wrong with
> your program. Rather, there was a problem with its input.
> [snip.]

I think he understands most of that. His question was whether or not the compiler
can use enforcements to optimize code. Eg: (silly example)

int foo(int x){
   enforce(x==0);
   return x*extremely_costly_pure_function_call();
}

A smart compiler could optimize the function to

int foo(int x){
   enforce(x==0);
   return 0;
}

The answer is yes, theoretically it could. (It would either have to have some very
advanced code analysis caps, or would just have to treat enforce specially.)
DMD does not currently use either enforcements or assertions to generate optimized
code, so using an enforcement is sufficient.

Timon
May 30, 2011
Re: Helping the compiler with assumptions while using exceptions
On 2011-05-30 14:39, Timon Gehr wrote:
> Jonathan M Davis wrote:
> > On 2011-05-30 12:49, simendsjo wrote:
> >> I'm having some problems trying to get the best of both worlds here.
> >> 
> >> void f(Class c) {
> >> 
> >>    assert(c != null);
> >>    // use c
> >> 
> >> }
> >> 
> >> In this example, we tell the compiler that c is never able to be null.
> >> The compiler can use assertions like this for optimizations (not sure if
> >> dmd does this though).
> >> 
> >> But assert is only a debugging tool.
> >> Say we wanted to have this check at runtime too - just in case - so we
> >> can fail where the problem is.
> >> 
> >> So we do this:
> >> 
> >> void f(Class c) {
> >> 
> >>    enforce(c != null);
> >>    // use c
> >> 
> >> }
> >> 
> >> But now the compiler has no idea c will never be null later on (or does
> >> it...?).
> >> 
> >> We could always do this:
> >> 
> >> void f(Class c) {
> >> 
> >>    assert(c != null);
> >>    enforce(c != null);
> >>    // use c
> >> 
> >> }
> >> 
> >> But this is overly verbose.
> 
> Actually, what you have written does not make sense. Correct would be:
> void f(Class c) {
>    enforce(c != null);
>    assert(c != null); //if c is null, control can never reach this
> instruction. // use c
> }
> 
> You cannot move assert into the contract in your use case, because it is
> not a contract. It will be valid to call your function with a null
> reference, it will just throw an exception.
> 
> >> Or is this not a problem at all? E.g. Use enforce for runtime checks -
> >> the compiler understands them/won't use asserts for optimizations
> >> anyway?
> 
> Currently, DMD does not use assertions or exceptions for optimization of
> code, but it could.
> 
> > assert and enforce serve two very different purposes. assert is used for
> > code verification. You can't assume that it's always be there, because
> > it might be compiled out. As such, the compiler is not going to use it
> > for any optimizations. If an assertion fails, there's a bug in your
> > program. It's used to detect bugs.
> 
> And to give hints to the compiler. This is what his question was about.
> 
> > enforce and exceptions, on the other hand, are _not_ intended for bug
> > detection. They are for error handling. They're generally used in cases
> > where you want to ensure that something is true but where you can't
> > guarantee it (such as when dealing with user input or the file system).
> > You throw an exception if the condition isn't true, and then you deal
> > with it however is appropriate. Generally-speaking, an exception does
> > _not_ indicate a bug. It indicates an error, but that doesn't mean that
> > there's anything wrong with your program. Rather, there was a problem
> > with its input.
> > [snip.]
> 
> I think he understands most of that.

The fact that he was talking about essentially swapping out enforce for assert 
implied that he doesn't.

> His question was whether or not the
> compiler can use enforcements to optimize code. Eg: (silly example)
> 
> int foo(int x){
>     enforce(x==0);
>     return x*extremely_costly_pure_function_call();
> }
> 
> A smart compiler could optimize the function to
> 
> int foo(int x){
>     enforce(x==0);
>     return 0;
> }
> 
> The answer is yes, theoretically it could. (It would either have to have
> some very advanced code analysis caps, or would just have to treat enforce
> specially.) DMD does not currently use either enforcements or assertions
> to generate optimized code, so using an enforcement is sufficient.

I'd be very surprised to see the compiler ever optimize code based on assert 
or enforce statement. It's unlikely to do so based on assert simply because 
the assertion is going to be compiled out. I think that there's a high chance 
that optimizing based on an assertion that is removed in release mode would 
actually be violating the guarantees of assert, since then the code without it 
wouldn't act the same. But you'd have to ask Walter on that one. As for 
enforce, it'll never happen because enforce is a library function, and the 
compiler doesn't know anything about it. So, I wouldn't expect to ever seen 
any compiler optimzations based on assert or enforce.

- Jonathan M Davis
May 30, 2011
Re: Helping the compiler with assumptions while using exceptions
Timon Gehr:

> The answer is yes, theoretically it could. (It would either have to have some very
> advanced code analysis caps, or would just have to treat enforce specially.)

Id's not so advanced stuff.

Bye,
bearophile
May 30, 2011
Re: Helping the compiler with assumptions while using exceptions
On 5/30/2011 2:55 PM, Jonathan M Davis wrote:

> I'd be very surprised to see the compiler ever optimize code based on assert 
> or enforce statement. It's unlikely to do so based on assert simply because 
> the assertion is going to be compiled out. I think that there's a high chance 
> that optimizing based on an assertion that is removed in release mode would 
> actually be violating the guarantees of assert, since then the code without it 
> wouldn't act the same. But you'd have to ask Walter on that one. As for 
> enforce, it'll never happen because enforce is a library function, and the 
> compiler doesn't know anything about it. So, I wouldn't expect to ever seen 
> any compiler optimzations based on assert or enforce.
> 
> - Jonathan M Davis

I think you're underselling the possibilities.  The only thing preventing enforce from helping optimize in DMD is the
lack of inlining of it.  That will be improved at some point.  Once it's inlined, the rest of the optimizations can
easily take advantage of it.  There's nothing particularly magic about either assert or enforce.  They're just if's and
flow control which are an optimizer's bread and butter.

Later,
Brad
May 30, 2011
Re: Helping the compiler with assumptions while using exceptions
On 2011-05-30 15:03, Brad Roberts wrote:
> On 5/30/2011 2:55 PM, Jonathan M Davis wrote:
> > I'd be very surprised to see the compiler ever optimize code based on
> > assert or enforce statement. It's unlikely to do so based on assert
> > simply because the assertion is going to be compiled out. I think that
> > there's a high chance that optimizing based on an assertion that is
> > removed in release mode would actually be violating the guarantees of
> > assert, since then the code without it wouldn't act the same. But you'd
> > have to ask Walter on that one. As for enforce, it'll never happen
> > because enforce is a library function, and the compiler doesn't know
> > anything about it. So, I wouldn't expect to ever seen any compiler
> > optimzations based on assert or enforce.
> > 
> > - Jonathan M Davis
> 
> I think you're underselling the possibilities.  The only thing preventing
> enforce from helping optimize in DMD is the lack of inlining of it.  That
> will be improved at some point.  Once it's inlined, the rest of the
> optimizations can easily take advantage of it.  There's nothing
> particularly magic about either assert or enforce.  They're just if's and
> flow control which are an optimizer's bread and butter.

A valid point. But as long as enforce isn't inlineable, it can't be optimized. 
And with assert compiled out, I wouldn't expect it to have any affect on 
optimizations. If it did, depending on what was optimized, it could make bugs 
caused by values that would have made the assertion fail were it compiled in 
that much worse. Generally, assert is supposed to have no effect beyond 
throwing if it's compiled in and fails. Now, maybe Walter has a somewhat 
different take on that and maybe assertions could be used for optimizations, 
but the fact that they can be compiled out makes it a bit iffy.

But sure, as long as it all gets reduced down to if's and flow control, then 
the optimizer can have a field day. But at present, that can't happen with 
enforce (if it ever will), and I question that it can ever be done with assert 
thanks to the guarantees that are associated with it.

- Jonathan M Davis
May 30, 2011
Re: Helping the compiler with assumptions while using exceptions
> Timon Gehr:
>
>> The answer is yes, theoretically it could. (It would either have to have some very
>> advanced code analysis caps, or would just have to treat enforce specially.)
>
> Id's not so advanced stuff.
>
> Bye,
> bearophile

You are saying that analyzing a function for thrown exceptions and using the
conditions on which these exceptions are thrown in a meaningful way across
function calls is not advanced? Why don't we have it in that case?

Jonathan M Davis wrote:
> A valid point. But as long as enforce isn't inlineable, it can't be optimized.
> And with assert compiled out, I wouldn't expect it to have any affect on
> optimizations. If it did, depending on what was optimized, it could make bugs
> caused by values that would have made the assertion fail were it compiled in
> that much worse. Generally, assert is supposed to have no effect beyond
> throwing if it's compiled in and fails. Now, maybe Walter has a somewhat
> different take on that and maybe assertions could be used for optimizations,
> but the fact that they can be compiled out makes it a bit iffy.
>
> But sure, as long as it all gets reduced down to if's and flow control, then
> the optimizer can have a field day. But at present, that can't happen with
> enforce (if it ever will), and I question that it can ever be done with assert
> thanks to the guarantees that are associated with it.
>

assert does not give any guarantees whatsoever. That would be reverse. It takes
guarantees from the user.
If an assertion fails, the program is buggy no matter if it has been compiled out
or not. The behavior of the program is undefined if a compiled out assertion would
fail. It is like array bounds. If the check is disabled, the compiler can still
assume that all accesses are valid. (a smart compiler could even optimize based on
that fact ;))

@"It could make bugs so much worse." A bug is a bug, no matter what. Actually, the
optimizations based on assert could also make the bug less bad, that depends on
the case. And if the bug slipped through when testing with assertions enabled, I'd
usually like it to be bad enough to crash the program when it occurs.

Timon
Top | Discussion index | About this forum | D home