March 09, 2012
On 03/09/2012 03:02 AM, Jonathan M Davis wrote:

> contorting your program to try and print assert
> messages to a file just because VisualD has problems with closing the Window on
> you is a _bad_ idea IMHO.

Absolutely!

Inserting lines at the end of programs is not the correct solution to the problem at hand. Those lines have nothing to do with the program that is being written.

> If it really has that problem, and you really need
> to see the assertion without running the debugger on it, I'd advise just
> running the program on the command line.

+1. That is a correct solution because it does not maim the very program that is being developed.

>
> - Jonathan M Davis

Ali

March 10, 2012
On Friday, March 09, 2012 22:48:25 Timon Gehr wrote:
> On 03/09/2012 10:43 PM, Jonathan M Davis wrote:
> ...
> 
> >> Jonathan is just speculating. And I think he is wrong.
> > 
> > Speculating about what?
> > 
> > - Jonathan M Davis
> 
> About how assertion failures affect struct destructors, finally statements and scope statements. If they wouldn't be executed upon thrown AssertError, then a _passing_ in contract could screw up the program state. Furthermore, the implementation does execute them.

It's not speculation. There's no guessing involved. It's what I've understood the rules of the language to be.

But I asked on dmd-internals to verify it, and Walter has stated that it just so happens that  the destructors, scope statements, and finally blocks do currently get run for non-Exception exceptions, but it's not guaranteed:

http://forum.dlang.org/post/4F5AE8EC.4040408@digitalmars.com

- Jonathan M Davis
March 10, 2012
On 2012-03-09 17:56, Jonathan M Davis wrote:

> It was never intended that AssertError or any other Error be particularly
> catchable, but it's also true that D is a systems programming language, so
> it'll let you do stuff which is unsafe, so if you know what you're doing, then
> there are circumstances where you can get away with (unit testing is one
> example of where catching AssertErrors might make sense). But you have to know
> what you're doing, since it's _not_ safe.
>
> - Jonathan M Davis

"might make sense". It makes perfectly sense to catch AssertErros in a unit testing framework.

-- 
/Jacob Carlborg
March 10, 2012
On 2012-03-09 17:56, Jonathan M Davis wrote:

> The current implementation may not skip them, but if so, that might actually
> be a bug. Errors are not intended to be recoverable, so you can't rely on them
> hitting finally blocks, scope statements, or destructors. They may very well do
> so at present. While it's not guaranteed that they will, I'm not sure that
> it's guaranteed that they _won't_. So, it may or may not be a bug if they do.
>
> It was never intended that AssertError or any other Error be particularly
> catchable, but it's also true that D is a systems programming language, so
> it'll let you do stuff which is unsafe, so if you know what you're doing, then
> there are circumstances where you can get away with (unit testing is one
> example of where catching AssertErrors might make sense). But you have to know
> what you're doing, since it's _not_ safe.
>
> - Jonathan M Davis

If it's not safe to catch AssertErrors, how is the executable supposed to be able to continue a unit test run when an AssertError has been thrown?

I'm referring to the suggested changes that a unit test run should be able to continue in other modules even if an AssertError has been thrown. It seems to be issue 5283: http://d.puremagic.com/issues/show_bug.cgi?id=5283


-- 
/Jacob Carlborg
March 10, 2012
On 2012-03-09 18:59, H. S. Teoh wrote:

> This opens up the question of, what's the *recommended* way of writing
> unittests that check for these sorts of stuff?
>
> For example, I believe in being thorough in unit tests, so I like to use
> them to verify that the complicated in-contract I just wrote actually
> prevents the erroneous calls that I *think* it prevents. But if catching
> AssertError's may leave the program in an undefined state, then that
> pretty much invalidates any further testing past that point (the program
> may appear to work when compiled with -funittest but actually fail in
> release mode).
>
> I agree, though, that catching Errors outside of unittests is a very,
> very bad idea in general.

I don't see what's so bad in making AssertError an exception instead of an error.

-- 
/Jacob Carlborg
March 10, 2012
On 2012-03-09 20:06, Jonathan M Davis wrote:

> In what way? Yes, they're _catchable_, but everything that was on the unwound
> portion of the stack is now in an undefined state. So, recovering from the
> AssertError and continuing execution doesn't work. So, a handler can catch the
> AssertError and do something before letting it kill the program, but portions
> of the program are definitely in an undefined state when an AssertError is
> caught. The closer the catch point is to the throw point, the less of an issue
> that is, but unless you can guarantee that no destructors, scope statements,
> or finally blocks were between the throw point and the catch point, then some
> portion of the program is in an undefined state, and continuing program
> execution is a bad idea. It _is_ possible to catch an AssertError, but you
> have to be very careful about what you do after that.
>
> I'm not aware of anything in contracts which would involve catching an
> AssertError and then continuing execution. When a contract fails, it's going
> to kill the program just like any other assertion failure does. So, I'm not
> quite sure what you're referring to.
>
> - Jonathan M Davis

This might be how it works today, but I think it's a broken design. AssertError MUST be SAFELY catchable to be able to write unit testing frameworks.

-- 
/Jacob Carlborg
March 10, 2012
On Saturday, March 10, 2012 16:53:42 Jacob Carlborg wrote:
> On 2012-03-09 18:59, H. S. Teoh wrote:
> > This opens up the question of, what's the *recommended* way of writing unittests that check for these sorts of stuff?
> > 
> > For example, I believe in being thorough in unit tests, so I like to use them to verify that the complicated in-contract I just wrote actually prevents the erroneous calls that I *think* it prevents. But if catching AssertError's may leave the program in an undefined state, then that pretty much invalidates any further testing past that point (the program may appear to work when compiled with -funittest but actually fail in release mode).
> > 
> > I agree, though, that catching Errors outside of unittests is a very, very bad idea in general.
> 
> I don't see what's so bad in making AssertError an exception instead of an error.

Then

catch(Excetion e) {}

would catch it. This would be a huge problem in normal code. Assertions are supposed to kill the program. This is specifically mentioned in TDPL.

- Jonathan M Davis
March 10, 2012
On Saturday, March 10, 2012 16:52:43 Jacob Carlborg wrote:
> On 2012-03-09 17:56, Jonathan M Davis wrote:
> > The current implementation may not skip them, but if so, that might actually be a bug. Errors are not intended to be recoverable, so you can't rely on them hitting finally blocks, scope statements, or destructors. They may very well do so at present. While it's not guaranteed that they will, I'm not sure that it's guaranteed that they _won't_. So, it may or may not be a bug if they do.
> > 
> > It was never intended that AssertError or any other Error be particularly catchable, but it's also true that D is a systems programming language, so it'll let you do stuff which is unsafe, so if you know what you're doing, then there are circumstances where you can get away with (unit testing is one example of where catching AssertErrors might make sense). But you have to know what you're doing, since it's _not_ safe.
> > 
> > - Jonathan M Davis
> 
> If it's not safe to catch AssertErrors, how is the executable supposed
> to be able to continue a unit test run when an AssertError has been thrown?
> 
> I'm referring to the suggested changes that a unit test run should be able to continue in other modules even if an AssertError has been thrown. It seems to be issue 5283: http://d.puremagic.com/issues/show_bug.cgi?id=5283

1. Assertions are treated a bit differently in unit tests. I don't remember what the exact differences are at the moment (and I think that they've changed over time), but they're not completely normal from what I recall. So, the situation may or may not be somewhat different in unit tests. At minimum, they get left in when -unittest is used, even if -release is used, and I believe that they have a somewhat different handler. They probably function the same with regards to destructors, scope statements, and finally though.

2. As long as unit tests are properly isolated, then it's unlikely to be a big issue, and even if it _is_ problem, it's only a problem as long as tests are failing. So, while this may cause problems in some cases, it's still arguably worth it, since the _first_ test failure is still completely valid (as it is now), and the rest are _likely_ to be valid, so you'd generally be getting more information than before.

If it's a big enough problem, it could probably be made so that AssertErrors are treated differently in unit tests such that they _are_ guaranteed to hit destructors, scope statements, and finally. But the basic design of Errors is that they're supposed to be unrecoverable, so skipping all of those isn't generally an issue. And given that can get Errors thrown from nothrow functions, running them might actually do funny things in some cases, because the assumptions surrounding nothrow have been effectively violated.

- Jonathan M Davis
March 10, 2012
On Saturday, March 10, 2012 17:03:43 Jacob Carlborg wrote:
> On 2012-03-09 20:06, Jonathan M Davis wrote:
> > In what way? Yes, they're _catchable_, but everything that was on the unwound portion of the stack is now in an undefined state. So, recovering from the AssertError and continuing execution doesn't work. So, a handler can catch the AssertError and do something before letting it kill the program, but portions of the program are definitely in an undefined state when an AssertError is caught. The closer the catch point is to the throw point, the less of an issue that is, but unless you can guarantee that no destructors, scope statements, or finally blocks were between the throw point and the catch point, then some portion of the program is in an undefined state, and continuing program execution is a bad idea. It _is_ possible to catch an AssertError, but you have to be very careful about what you do after that.
> > 
> > I'm not aware of anything in contracts which would involve catching an AssertError and then continuing execution. When a contract fails, it's going to kill the program just like any other assertion failure does. So, I'm not quite sure what you're referring to.
> > 
> > - Jonathan M Davis
> 
> This might be how it works today, but I think it's a broken design. AssertError MUST be SAFELY catchable to be able to write unit testing frameworks.

I honestly don't think that Walter even considered unit testing frameworks when he came up with the design. He built unit tests into the language. So, from his point of view, why would you need a unit testing framework? He probably didn't even consider it.

And AssertErrors _are_ catchable. You just have to deal with the fact that destructors, scope statements, and finally blocks all get skipped by them. So, cleanup doesn't tend to happen in their presence. That's more of an issue with a general unit testing framework than it would be if you were doing it closer to the throw point, but it doesn't necessarily make it so that the unit test framework doesn't work. It _does_ risk making it flakier though. What would be particularly risky is trying to continue a particular unit test block after a failed assertion, since the failures wouldn't be isolated (though personally, I think that that would be a bad idea even AssertErrors _didn't_ skip anything, simply because the failures aren't isolated, and you get a bunch of failures simply because you already had one). Running subsequent unittest blocks should be less risky (though not necessarily without risk), because unittest blocks are normally isolated.

Regardless, if you don't like the fact that there's no guarantee that AssertErrors execute finally blocks, scope statements, and destructors, take it up with Walter. I'm just pointing out what the design is. I didn't make the decision - though in general, I do think that it was the correct one. It's just that there are a few cases (e.g. some unit testing stuff) where it can become an issue.

And if you really want to do your own unit testing framework, maybe you should just create your own custom assertion functions which throw something other than AssertError - though if they don't throw Errors, then any unittest blocks with a catch(Exception() {} in them may not work properly, so that's not necessarily all that great a solution either, depending.

- Jonathan M Davis
March 11, 2012
On 2012-03-10 17:48, Jonathan M Davis wrote:
> On Saturday, March 10, 2012 16:53:42 Jacob Carlborg wrote:
>> On 2012-03-09 18:59, H. S. Teoh wrote:
>>> This opens up the question of, what's the *recommended* way of writing
>>> unittests that check for these sorts of stuff?
>>>
>>> For example, I believe in being thorough in unit tests, so I like to use
>>> them to verify that the complicated in-contract I just wrote actually
>>> prevents the erroneous calls that I *think* it prevents. But if catching
>>> AssertError's may leave the program in an undefined state, then that
>>> pretty much invalidates any further testing past that point (the program
>>> may appear to work when compiled with -funittest but actually fail in
>>> release mode).
>>>
>>> I agree, though, that catching Errors outside of unittests is a very,
>>> very bad idea in general.
>>
>> I don't see what's so bad in making AssertError an exception instead of
>> an error.
>
> Then
>
> catch(Excetion e) {}
>
> would catch it. This would be a huge problem in normal code. Assertions are
> supposed to kill the program. This is specifically mentioned in TDPL.
>
> - Jonathan M Davis

Maybe so. But the current design is broken and need to be fixed somehow.

-- 
/Jacob Carlborg