View mode: basic / threaded / horizontal-split · Log in · Help
March 09, 2012
Re: Assert and the optional Message
On 03/09/2012 03:02 AM, Jonathan M Davis wrote:

> contorting your program to try and print assert
> messages to a file just because VisualD has problems with closing the 
Window on
> you is a _bad_ idea IMHO.

Absolutely!

Inserting lines at the end of programs is not the correct solution to 
the problem at hand. Those lines have nothing to do with the program 
that is being written.

> If it really has that problem, and you really need
> to see the assertion without running the debugger on it, I'd advise just
> running the program on the command line.

+1. That is a correct solution because it does not maim the very program 
that is being developed.

>
> - Jonathan M Davis

Ali
March 10, 2012
Re: Assert and the optional Message
On Friday, March 09, 2012 22:48:25 Timon Gehr wrote:
> On 03/09/2012 10:43 PM, Jonathan M Davis wrote:
> ...
> 
> >> Jonathan is just speculating. And I think he is wrong.
> > 
> > Speculating about what?
> > 
> > - Jonathan M Davis
> 
> About how assertion failures affect struct destructors, finally
> statements and scope statements. If they wouldn't be executed upon
> thrown AssertError, then a _passing_ in contract could screw up the
> program state. Furthermore, the implementation does execute them.

It's not speculation. There's no guessing involved. It's what I've understood 
the rules of the language to be.

But I asked on dmd-internals to verify it, and Walter has stated that it just 
so happens that  the destructors, scope statements, and finally blocks do 
currently get run for non-Exception exceptions, but it's not guaranteed:

http://forum.dlang.org/post/4F5AE8EC.4040408@digitalmars.com

- Jonathan M Davis
March 10, 2012
Re: Assert and the optional Message
On 2012-03-09 17:56, Jonathan M Davis wrote:

> It was never intended that AssertError or any other Error be particularly
> catchable, but it's also true that D is a systems programming language, so
> it'll let you do stuff which is unsafe, so if you know what you're doing, then
> there are circumstances where you can get away with (unit testing is one
> example of where catching AssertErrors might make sense). But you have to know
> what you're doing, since it's _not_ safe.
>
> - Jonathan M Davis

"might make sense". It makes perfectly sense to catch AssertErros in a 
unit testing framework.

-- 
/Jacob Carlborg
March 10, 2012
Re: Assert and the optional Message
On 2012-03-09 17:56, Jonathan M Davis wrote:

> The current implementation may not skip them, but if so, that might actually
> be a bug. Errors are not intended to be recoverable, so you can't rely on them
> hitting finally blocks, scope statements, or destructors. They may very well do
> so at present. While it's not guaranteed that they will, I'm not sure that
> it's guaranteed that they _won't_. So, it may or may not be a bug if they do.
>
> It was never intended that AssertError or any other Error be particularly
> catchable, but it's also true that D is a systems programming language, so
> it'll let you do stuff which is unsafe, so if you know what you're doing, then
> there are circumstances where you can get away with (unit testing is one
> example of where catching AssertErrors might make sense). But you have to know
> what you're doing, since it's _not_ safe.
>
> - Jonathan M Davis

If it's not safe to catch AssertErrors, how is the executable supposed 
to be able to continue a unit test run when an AssertError has been thrown?

I'm referring to the suggested changes that a unit test run should be 
able to continue in other modules even if an AssertError has been 
thrown. It seems to be issue 5283: 
http://d.puremagic.com/issues/show_bug.cgi?id=5283


-- 
/Jacob Carlborg
March 10, 2012
Re: Assert and the optional Message
On 2012-03-09 18:59, H. S. Teoh wrote:

> This opens up the question of, what's the *recommended* way of writing
> unittests that check for these sorts of stuff?
>
> For example, I believe in being thorough in unit tests, so I like to use
> them to verify that the complicated in-contract I just wrote actually
> prevents the erroneous calls that I *think* it prevents. But if catching
> AssertError's may leave the program in an undefined state, then that
> pretty much invalidates any further testing past that point (the program
> may appear to work when compiled with -funittest but actually fail in
> release mode).
>
> I agree, though, that catching Errors outside of unittests is a very,
> very bad idea in general.

I don't see what's so bad in making AssertError an exception instead of 
an error.

-- 
/Jacob Carlborg
March 10, 2012
Re: Assert and the optional Message
On 2012-03-09 20:06, Jonathan M Davis wrote:

> In what way? Yes, they're _catchable_, but everything that was on the unwound
> portion of the stack is now in an undefined state. So, recovering from the
> AssertError and continuing execution doesn't work. So, a handler can catch the
> AssertError and do something before letting it kill the program, but portions
> of the program are definitely in an undefined state when an AssertError is
> caught. The closer the catch point is to the throw point, the less of an issue
> that is, but unless you can guarantee that no destructors, scope statements,
> or finally blocks were between the throw point and the catch point, then some
> portion of the program is in an undefined state, and continuing program
> execution is a bad idea. It _is_ possible to catch an AssertError, but you
> have to be very careful about what you do after that.
>
> I'm not aware of anything in contracts which would involve catching an
> AssertError and then continuing execution. When a contract fails, it's going
> to kill the program just like any other assertion failure does. So, I'm not
> quite sure what you're referring to.
>
> - Jonathan M Davis

This might be how it works today, but I think it's a broken design. 
AssertError MUST be SAFELY catchable to be able to write unit testing 
frameworks.

-- 
/Jacob Carlborg
March 10, 2012
Re: Assert and the optional Message
On Saturday, March 10, 2012 16:53:42 Jacob Carlborg wrote:
> On 2012-03-09 18:59, H. S. Teoh wrote:
> > This opens up the question of, what's the *recommended* way of writing
> > unittests that check for these sorts of stuff?
> > 
> > For example, I believe in being thorough in unit tests, so I like to use
> > them to verify that the complicated in-contract I just wrote actually
> > prevents the erroneous calls that I *think* it prevents. But if catching
> > AssertError's may leave the program in an undefined state, then that
> > pretty much invalidates any further testing past that point (the program
> > may appear to work when compiled with -funittest but actually fail in
> > release mode).
> > 
> > I agree, though, that catching Errors outside of unittests is a very,
> > very bad idea in general.
> 
> I don't see what's so bad in making AssertError an exception instead of
> an error.

Then

catch(Excetion e) {}

would catch it. This would be a huge problem in normal code. Assertions are 
supposed to kill the program. This is specifically mentioned in TDPL.

- Jonathan M Davis
March 10, 2012
Re: Assert and the optional Message
On Saturday, March 10, 2012 16:52:43 Jacob Carlborg wrote:
> On 2012-03-09 17:56, Jonathan M Davis wrote:
> > The current implementation may not skip them, but if so, that might
> > actually be a bug. Errors are not intended to be recoverable, so you
> > can't rely on them hitting finally blocks, scope statements, or
> > destructors. They may very well do so at present. While it's not
> > guaranteed that they will, I'm not sure that it's guaranteed that they
> > _won't_. So, it may or may not be a bug if they do.
> > 
> > It was never intended that AssertError or any other Error be particularly
> > catchable, but it's also true that D is a systems programming language, so
> > it'll let you do stuff which is unsafe, so if you know what you're doing,
> > then there are circumstances where you can get away with (unit testing is
> > one example of where catching AssertErrors might make sense). But you
> > have to know what you're doing, since it's _not_ safe.
> > 
> > - Jonathan M Davis
> 
> If it's not safe to catch AssertErrors, how is the executable supposed
> to be able to continue a unit test run when an AssertError has been thrown?
> 
> I'm referring to the suggested changes that a unit test run should be
> able to continue in other modules even if an AssertError has been
> thrown. It seems to be issue 5283:
> http://d.puremagic.com/issues/show_bug.cgi?id=5283

1. Assertions are treated a bit differently in unit tests. I don't remember 
what the exact differences are at the moment (and I think that they've changed 
over time), but they're not completely normal from what I recall. So, the 
situation may or may not be somewhat different in unit tests. At minimum, they 
get left in when -unittest is used, even if -release is used, and I believe 
that they have a somewhat different handler. They probably function the same 
with regards to destructors, scope statements, and finally though.

2. As long as unit tests are properly isolated, then it's unlikely to be a big 
issue, and even if it _is_ problem, it's only a problem as long as tests are 
failing. So, while this may cause problems in some cases, it's still arguably 
worth it, since the _first_ test failure is still completely valid (as it is 
now), and the rest are _likely_ to be valid, so you'd generally be getting 
more information than before.

If it's a big enough problem, it could probably be made so that AssertErrors 
are treated differently in unit tests such that they _are_ guaranteed to hit 
destructors, scope statements, and finally. But the basic design of Errors is 
that they're supposed to be unrecoverable, so skipping all of those isn't 
generally an issue. And given that can get Errors thrown from nothrow 
functions, running them might actually do funny things in some cases, because 
the assumptions surrounding nothrow have been effectively violated.

- Jonathan M Davis
March 10, 2012
Re: Assert and the optional Message
On Saturday, March 10, 2012 17:03:43 Jacob Carlborg wrote:
> On 2012-03-09 20:06, Jonathan M Davis wrote:
> > In what way? Yes, they're _catchable_, but everything that was on the
> > unwound portion of the stack is now in an undefined state. So, recovering
> > from the AssertError and continuing execution doesn't work. So, a handler
> > can catch the AssertError and do something before letting it kill the
> > program, but portions of the program are definitely in an undefined state
> > when an AssertError is caught. The closer the catch point is to the throw
> > point, the less of an issue that is, but unless you can guarantee that no
> > destructors, scope statements, or finally blocks were between the throw
> > point and the catch point, then some portion of the program is in an
> > undefined state, and continuing program execution is a bad idea. It _is_
> > possible to catch an AssertError, but you have to be very careful about
> > what you do after that.
> > 
> > I'm not aware of anything in contracts which would involve catching an
> > AssertError and then continuing execution. When a contract fails, it's
> > going to kill the program just like any other assertion failure does. So,
> > I'm not quite sure what you're referring to.
> > 
> > - Jonathan M Davis
> 
> This might be how it works today, but I think it's a broken design.
> AssertError MUST be SAFELY catchable to be able to write unit testing
> frameworks.

I honestly don't think that Walter even considered unit testing frameworks 
when he came up with the design. He built unit tests into the language. So, 
from his point of view, why would you need a unit testing framework? He 
probably didn't even consider it.

And AssertErrors _are_ catchable. You just have to deal with the fact that 
destructors, scope statements, and finally blocks all get skipped by them. So, 
cleanup doesn't tend to happen in their presence. That's more of an issue with 
a general unit testing framework than it would be if you were doing it closer 
to the throw point, but it doesn't necessarily make it so that the unit test 
framework doesn't work. It _does_ risk making it flakier though. What would be 
particularly risky is trying to continue a particular unit test block after a 
failed assertion, since the failures wouldn't be isolated (though personally, 
I think that that would be a bad idea even AssertErrors _didn't_ skip 
anything, simply because the failures aren't isolated, and you get a bunch of 
failures simply because you already had one). Running subsequent unittest 
blocks should be less risky (though not necessarily without risk), because 
unittest blocks are normally isolated.

Regardless, if you don't like the fact that there's no guarantee that 
AssertErrors execute finally blocks, scope statements, and destructors, take it 
up with Walter. I'm just pointing out what the design is. I didn't make the 
decision - though in general, I do think that it was the correct one. It's 
just that there are a few cases (e.g. some unit testing stuff) where it can 
become an issue.

And if you really want to do your own unit testing framework, maybe you should 
just create your own custom assertion functions which throw something other 
than AssertError - though if they don't throw Errors, then any unittest blocks 
with a catch(Exception() {} in them may not work properly, so that's not 
necessarily all that great a solution either, depending.

- Jonathan M Davis
March 11, 2012
Re: Assert and the optional Message
On 2012-03-10 17:48, Jonathan M Davis wrote:
> On Saturday, March 10, 2012 16:53:42 Jacob Carlborg wrote:
>> On 2012-03-09 18:59, H. S. Teoh wrote:
>>> This opens up the question of, what's the *recommended* way of writing
>>> unittests that check for these sorts of stuff?
>>>
>>> For example, I believe in being thorough in unit tests, so I like to use
>>> them to verify that the complicated in-contract I just wrote actually
>>> prevents the erroneous calls that I *think* it prevents. But if catching
>>> AssertError's may leave the program in an undefined state, then that
>>> pretty much invalidates any further testing past that point (the program
>>> may appear to work when compiled with -funittest but actually fail in
>>> release mode).
>>>
>>> I agree, though, that catching Errors outside of unittests is a very,
>>> very bad idea in general.
>>
>> I don't see what's so bad in making AssertError an exception instead of
>> an error.
>
> Then
>
> catch(Excetion e) {}
>
> would catch it. This would be a huge problem in normal code. Assertions are
> supposed to kill the program. This is specifically mentioned in TDPL.
>
> - Jonathan M Davis

Maybe so. But the current design is broken and need to be fixed somehow.

-- 
/Jacob Carlborg
1 2 3 4 5
Top | Discussion index | About this forum | D home