I'm having some problems trying to get the best of both worlds here.

void f(Class c) {
  assert(c != null);
  // use c
}

In this example, we tell the compiler that c is never able to be null. The compiler can use assertions like this for optimizations (not sure if dmd does this though).

But assert is only a debugging tool.
Say we wanted to have this check at runtime too - just in case - so we can fail where the problem is.

So we do this:

void f(Class c) {
  enforce(c != null);
  // use c
}

But now the compiler has no idea c will never be null later on (or does it...?).

We could always do this:

void f(Class c) {
  assert(c != null);
  enforce(c != null);
  // use c
}

But this is overly verbose.

Or is this not a problem at all? E.g. Use enforce for runtime checks - the compiler understands them/won't use asserts for optimizations anyway?

simendsjo:

> void f(Class c) {
>    assert(c != null);
>    // use c
> }
> 
> In this example, we tell the compiler that c is never able to be null. The compiler can use assertions like this for optimizations (not sure if dmd does this though).

I think currently DMD is not using this information, and probably it will keep not using it for some more time. Note: generally asserts go into Contracts. Here it goes in the precondition of f:

void f(Class c)
in {
    assert(c != null);
}
body {
    // use c
}


> void f(Class c) {
>    assert(c != null);
>    enforce(c != null);
>    // use c
> }

This is not meaningful. assert and enforce have different purposes, you don't use both of them at the same time. If f is a inner function/method of your system, then you add asserts inside Contracts to assert your code contains no bugs. If your f is a function/method that's on an "interface" border, so it's used by other users, then you may want to use enforce instead.

Bye,
bearophile

On 2011-05-30 12:49, simendsjo wrote:
> I'm having some problems trying to get the best of both worlds here.
> 
> void f(Class c) {
>    assert(c != null);
>    // use c
> }
> 
> In this example, we tell the compiler that c is never able to be null. The compiler can use assertions like this for optimizations (not sure if dmd does this though).
> 
> But assert is only a debugging tool.
> Say we wanted to have this check at runtime too - just in case - so we
> can fail where the problem is.
> 
> So we do this:
> 
> void f(Class c) {
>    enforce(c != null);
>    // use c
> }
> 
> But now the compiler has no idea c will never be null later on (or does
> it...?).
> 
> We could always do this:
> 
> void f(Class c) {
>    assert(c != null);
>    enforce(c != null);
>    // use c
> }
> 
> But this is overly verbose.
> 
> Or is this not a problem at all? E.g. Use enforce for runtime checks - the compiler understands them/won't use asserts for optimizations anyway?

assert and enforce serve two very different purposes. assert is used for code verification. You can't assume that it's always be there, because it might be compiled out. As such, the compiler is not going to use it for any optimizations. If an assertion fails, there's a bug in your program. It's used to detect bugs.

enforce and exceptions, on the other hand, are _not_ intended for bug detection. They are for error handling. They're generally used in cases where you want to ensure that something is true but where you can't guarantee it (such as when dealing with user input or the file system). You throw an exception if the condition isn't true, and then you deal with it however is appropriate. Generally-speaking, an exception does _not_ indicate a bug. It indicates an error, but that doesn't mean that there's anything wrong with your program. Rather, there was a problem with its input.

As such, if you were to use assert in the above code, then you're saying that it's a bug in your program if c is null and having that check go away in release mode is fine, because you're done looking for bugs. It's assumed that the code is correct enough that c will never be null. If, you use enforce, on the other hand, you're saying that it could be perfectly valid for c to be null, but you don't want your f function to ever execute with a null c, so you throw if c is null. Then whatever calls f can handle it in whatever way is appropriate, possibly resulting in an error being printed to the user or another attempt to call f with another value for c which isn't null or whatever is appropriate for your program. By using enforce, you're saying that it's not a bug for c to be null but that it _is_ an error (likely caused by input in some manner) if it occurs.

What assert and enforce are doing here are completely different. You shouldn't be choosing between assert and enforce based on whether they'll be around in release mode or not. They're two completely different types of error handling.

Now, if it's a bug in your code if c is null, and you want to be absolutely certain that c is never null - even in release mode - then you can use assert(0). e.g.

if(c is null)
	assert(0, "c cannot be null!");

assert(0) is translated to a HLT instruction in release mode. So, it'll kill your program if it's hit. It's intended for cases which should _never_ happen and you want to be absolutely sure that they never do (such as a default case statement or else clause which should never be hit as long as your logic is correct). It's still for bug detection like normal assert, but it's going the extra step of saying that if this line of code is ever reached in release mode, kill the program. It's still completely different from enforce and exceptions which are runtime errors which are potentially handleable.

- Jonathan M Davis

Jonathan M Davis wrote:
> On 2011-05-30 12:49, simendsjo wrote:
>> I'm having some problems trying to get the best of both worlds here.
>>
>> void f(Class c) {
>>    assert(c != null);
>>    // use c
>> }
>>
>> In this example, we tell the compiler that c is never able to be null. The compiler can use assertions like this for optimizations (not sure if dmd does this though).
>>
>> But assert is only a debugging tool.
>> Say we wanted to have this check at runtime too - just in case - so we
>> can fail where the problem is.
>>
>> So we do this:
>>
>> void f(Class c) {
>>    enforce(c != null);
>>    // use c
>> }
>>
>> But now the compiler has no idea c will never be null later on (or does
>> it...?).
>>
>> We could always do this:
>>
>> void f(Class c) {
>>    assert(c != null);
>>    enforce(c != null);
>>    // use c
>> }
>>
>> But this is overly verbose.

Actually, what you have written does not make sense. Correct would be:
void f(Class c) {
   enforce(c != null);
   assert(c != null); //if c is null, control can never reach this instruction.
   // use c
}

You cannot move assert into the contract in your use case, because it is not a contract. It will be valid to call your function with a null reference, it will just throw an exception.

>>
>> Or is this not a problem at all? E.g. Use enforce for runtime checks - the compiler understands them/won't use asserts for optimizations anyway?

Currently, DMD does not use assertions or exceptions for optimization of code, but it could.

>
> assert and enforce serve two very different purposes. assert is used for code verification. You can't assume that it's always be there, because it might be compiled out. As such, the compiler is not going to use it for any optimizations. If an assertion fails, there's a bug in your program. It's used to detect bugs.

And to give hints to the compiler. This is what his question was about.

>
> enforce and exceptions, on the other hand, are _not_ intended for bug
> detection. They are for error handling. They're generally used in cases where
> you want to ensure that something is true but where you can't guarantee it
> (such as when dealing with user input or the file system). You throw an
> exception if the condition isn't true, and then you deal with it however is
> appropriate. Generally-speaking, an exception does _not_ indicate a bug. It
> indicates an error, but that doesn't mean that there's anything wrong with
> your program. Rather, there was a problem with its input.
> [snip.]

I think he understands most of that. His question was whether or not the compiler can use enforcements to optimize code. Eg: (silly example)

int foo(int x){
    enforce(x==0);
    return x*extremely_costly_pure_function_call();
}

A smart compiler could optimize the function to

int foo(int x){
    enforce(x==0);
    return 0;
}

The answer is yes, theoretically it could. (It would either have to have some very
advanced code analysis caps, or would just have to treat enforce specially.)
DMD does not currently use either enforcements or assertions to generate optimized
code, so using an enforcement is sufficient.

Timon

On 2011-05-30 14:39, Timon Gehr wrote:
> Jonathan M Davis wrote:
> > On 2011-05-30 12:49, simendsjo wrote:
> >> I'm having some problems trying to get the best of both worlds here.
> >> 
> >> void f(Class c) {
> >> 
> >>    assert(c != null);
> >>    // use c
> >> 
> >> }
> >> 
> >> In this example, we tell the compiler that c is never able to be null. The compiler can use assertions like this for optimizations (not sure if dmd does this though).
> >> 
> >> But assert is only a debugging tool.
> >> Say we wanted to have this check at runtime too - just in case - so we
> >> can fail where the problem is.
> >> 
> >> So we do this:
> >> 
> >> void f(Class c) {
> >> 
> >>    enforce(c != null);
> >>    // use c
> >> 
> >> }
> >> 
> >> But now the compiler has no idea c will never be null later on (or does
> >> it...?).
> >> 
> >> We could always do this:
> >> 
> >> void f(Class c) {
> >> 
> >>    assert(c != null);
> >>    enforce(c != null);
> >>    // use c
> >> 
> >> }
> >> 
> >> But this is overly verbose.
> 
> Actually, what you have written does not make sense. Correct would be:
> void f(Class c) {
>    enforce(c != null);
>    assert(c != null); //if c is null, control can never reach this
> instruction. // use c
> }
> 
> You cannot move assert into the contract in your use case, because it is not a contract. It will be valid to call your function with a null reference, it will just throw an exception.
> 
> >> Or is this not a problem at all? E.g. Use enforce for runtime checks - the compiler understands them/won't use asserts for optimizations anyway?
> 
> Currently, DMD does not use assertions or exceptions for optimization of code, but it could.
> 
> > assert and enforce serve two very different purposes. assert is used for code verification. You can't assume that it's always be there, because it might be compiled out. As such, the compiler is not going to use it for any optimizations. If an assertion fails, there's a bug in your program. It's used to detect bugs.
> 
> And to give hints to the compiler. This is what his question was about.
> 
> > enforce and exceptions, on the other hand, are _not_ intended for bug
> > detection. They are for error handling. They're generally used in cases
> > where you want to ensure that something is true but where you can't
> > guarantee it (such as when dealing with user input or the file system).
> > You throw an exception if the condition isn't true, and then you deal
> > with it however is appropriate. Generally-speaking, an exception does
> > _not_ indicate a bug. It indicates an error, but that doesn't mean that
> > there's anything wrong with your program. Rather, there was a problem
> > with its input.
> > [snip.]
> 
> I think he understands most of that.

The fact that he was talking about essentially swapping out enforce for assert implied that he doesn't.

> His question was whether or not the
> compiler can use enforcements to optimize code. Eg: (silly example)
> 
> int foo(int x){
>     enforce(x==0);
>     return x*extremely_costly_pure_function_call();
> }
> 
> A smart compiler could optimize the function to
> 
> int foo(int x){
>     enforce(x==0);
>     return 0;
> }
> 
> The answer is yes, theoretically it could. (It would either have to have some very advanced code analysis caps, or would just have to treat enforce specially.) DMD does not currently use either enforcements or assertions to generate optimized code, so using an enforcement is sufficient.

I'd be very surprised to see the compiler ever optimize code based on assert or enforce statement. It's unlikely to do so based on assert simply because the assertion is going to be compiled out. I think that there's a high chance that optimizing based on an assertion that is removed in release mode would actually be violating the guarantees of assert, since then the code without it wouldn't act the same. But you'd have to ask Walter on that one. As for enforce, it'll never happen because enforce is a library function, and the compiler doesn't know anything about it. So, I wouldn't expect to ever seen any compiler optimzations based on assert or enforce.

- Jonathan M Davis

Timon Gehr:

> The answer is yes, theoretically it could. (It would either have to have some very advanced code analysis caps, or would just have to treat enforce specially.)

Id's not so advanced stuff.

Bye,
bearophile

On 5/30/2011 2:55 PM, Jonathan M Davis wrote:

> I'd be very surprised to see the compiler ever optimize code based on assert or enforce statement. It's unlikely to do so based on assert simply because the assertion is going to be compiled out. I think that there's a high chance that optimizing based on an assertion that is removed in release mode would actually be violating the guarantees of assert, since then the code without it wouldn't act the same. But you'd have to ask Walter on that one. As for enforce, it'll never happen because enforce is a library function, and the compiler doesn't know anything about it. So, I wouldn't expect to ever seen any compiler optimzations based on assert or enforce.
> 
> - Jonathan M Davis

I think you're underselling the possibilities.  The only thing preventing enforce from helping optimize in DMD is the lack of inlining of it.  That will be improved at some point.  Once it's inlined, the rest of the optimizations can easily take advantage of it.  There's nothing particularly magic about either assert or enforce.  They're just if's and flow control which are an optimizer's bread and butter.

Later,
Brad

On 2011-05-30 15:03, Brad Roberts wrote:
> On 5/30/2011 2:55 PM, Jonathan M Davis wrote:
> > I'd be very surprised to see the compiler ever optimize code based on assert or enforce statement. It's unlikely to do so based on assert simply because the assertion is going to be compiled out. I think that there's a high chance that optimizing based on an assertion that is removed in release mode would actually be violating the guarantees of assert, since then the code without it wouldn't act the same. But you'd have to ask Walter on that one. As for enforce, it'll never happen because enforce is a library function, and the compiler doesn't know anything about it. So, I wouldn't expect to ever seen any compiler optimzations based on assert or enforce.
> > 
> > - Jonathan M Davis
> 
> I think you're underselling the possibilities.  The only thing preventing enforce from helping optimize in DMD is the lack of inlining of it.  That will be improved at some point.  Once it's inlined, the rest of the optimizations can easily take advantage of it.  There's nothing particularly magic about either assert or enforce.  They're just if's and flow control which are an optimizer's bread and butter.

A valid point. But as long as enforce isn't inlineable, it can't be optimized. And with assert compiled out, I wouldn't expect it to have any affect on optimizations. If it did, depending on what was optimized, it could make bugs caused by values that would have made the assertion fail were it compiled in that much worse. Generally, assert is supposed to have no effect beyond throwing if it's compiled in and fails. Now, maybe Walter has a somewhat different take on that and maybe assertions could be used for optimizations, but the fact that they can be compiled out makes it a bit iffy.

But sure, as long as it all gets reduced down to if's and flow control, then the optimizer can have a field day. But at present, that can't happen with enforce (if it ever will), and I question that it can ever be done with assert thanks to the guarantees that are associated with it.

- Jonathan M Davis

> Timon Gehr:
>
>> The answer is yes, theoretically it could. (It would either have to have some very advanced code analysis caps, or would just have to treat enforce specially.)
>
> Id's not so advanced stuff.
>
> Bye,
> bearophile

You are saying that analyzing a function for thrown exceptions and using the conditions on which these exceptions are thrown in a meaningful way across function calls is not advanced? Why don't we have it in that case?

Jonathan M Davis wrote:
> A valid point. But as long as enforce isn't inlineable, it can't be optimized. And with assert compiled out, I wouldn't expect it to have any affect on optimizations. If it did, depending on what was optimized, it could make bugs caused by values that would have made the assertion fail were it compiled in that much worse. Generally, assert is supposed to have no effect beyond throwing if it's compiled in and fails. Now, maybe Walter has a somewhat different take on that and maybe assertions could be used for optimizations, but the fact that they can be compiled out makes it a bit iffy.
>
> But sure, as long as it all gets reduced down to if's and flow control, then the optimizer can have a field day. But at present, that can't happen with enforce (if it ever will), and I question that it can ever be done with assert thanks to the guarantees that are associated with it.
>

assert does not give any guarantees whatsoever. That would be reverse. It takes
guarantees from the user.
If an assertion fails, the program is buggy no matter if it has been compiled out
or not. The behavior of the program is undefined if a compiled out assertion would
fail. It is like array bounds. If the check is disabled, the compiler can still
assume that all accesses are valid. (a smart compiler could even optimize based on
that fact ;))

@"It could make bugs so much worse." A bug is a bug, no matter what. Actually, the optimizations based on assert could also make the bug less bad, that depends on the case. And if the bug slipped through when testing with assertions enabled, I'd usually like it to be bad enough to crash the program when it occurs.

Timon