On 3/15/2012 2:13 PM, Leandro Lucarella wrote:
> Walter Bright, el 14 de marzo a las 16:42 me escribiste:
>> On 3/12/2012 7:07 PM, Sean Kelly wrote:
>>> On Mar 12, 2012, at 5:35 PM, Walter Bright<walter@digitalmars.com>   wrote:
>>>
>>>> On 3/12/2012 2:39 PM, Sean Kelly wrote:
>>>>> On Mar 12, 2012, at 2:30 PM, Walter Bright wrote:
>>>>>> On 3/12/2012 12:34 PM, Sean Kelly wrote:
>>>>>>> I'm on the fence about whether attempting cleanup when an Error is thrown is desired behavior.  If there is no cleanup, why allow Errors to be caught at all?  We may as well simply call abort() at the point they're thrown.
>>>>>>>
>>>>>> So that an informative message can be printed, the backup engaged, attempt to shut down gracefully, log the failure details to a file, etc.
>>>>> … none of which may work if scope(exit) calls weren't run when the stack was unwound, since acquired mutexes would still be locked, etc.  I'd feel a lot less safe with having effectively done a longjmp across code that normally assumes finalization than with whatever the cause of the assertion did in the first place.
>>>>>
>>>> It's understood it may not work.
>>> So what's the reason to not call finalizers?
>> The program is corrupted at that point. The less code one attempts
>> to run, the better.
> Why? That's not always the case.

How do you know it's not the case for a particular instance? That's the problem.


>   And what could happen if you run
> cleanup code in a program that you say is completely invalid? So why
> would you care if the program gets more corrupted?

Because it can corrupt whatever external thing it is supposed to be doing.


> This is specially bad if a memory allocation fail is an Error. It
> basically forces you to check every allocation for a failure and
> translate it yourself to some kind of Exception if you are being careful
> to write some code that can survive to a memory allocation failure.

That sounds good in theory, but in practice, almost no program can recover from out of memory failure. Even those that pretend they can usually cannot because such was never tested.

> Why on earth do you want to make life miserable to people that have some
> valid use case for this, just to avoid corrupting a little more
> a program that's already corrupted.

I believe it is a *serious* mistake to believe one can recover a program that has become corrupted.

> Also adding special cases is ALWAYS confusing and error prone. Please,
> please, please don't make errors a broken special case, unless you have
> a very strong reason.
>

I do have a good reason.

_______________________________________________
dmd-internals mailing list
dmd-internals@puremagic.com
http://lists.puremagic.com/mailman/listinfo/dmd-internals

On Thursday, March 15, 2012 14:22:49 Walter Bright wrote:
> On 3/15/2012 9:29 AM, Sean Kelly wrote:
> > Is the program really corrupted though?
> 
> You cannot know that.
> 
> I really think that going down the path of thinking one can recover from programming bugs is the route to disaster. It goes against everything I know about how to make safe, reliable systems.

There's a difference between saying that Errors are not recoverable and making it so that cleanup code runs when Errors are thrown. Catching an Error and continuing the execution of your program is almost never a good idea, but catching it, doing some cleanup and rethrowing it (as finally, scope(failure), and scope(exit) would do) is a different matter. No, it may not work, but it can also allow you to do things that need to be done at cleanup. And _not_ running scope statements, destructors, and finally blocks makes catching Errors even _less_ safe even when it _does_ make sense. And upon occasion, catching Errors _does_ make sense even for cases other than cleanup.

There is a difference between recovering from an Error and a recovering from a programming bug. Not all Errors necessarily indicate that there's a bug in the program. They just indicate that they've hit what is almost certainly a fatal condition. For instance, OutOfMemoryError is generally fatal, but it doesn't necessarily indicate a bug in the program. If caught close to the throw point, it might be possible to recover in some cases. It's just that the current state _is_ fatal unless you can do something to fix it. In most cases you can't, but in a few, rare ones, you can.

Catching an AssertError in normal code is a hugely bad idea - they _do_ indicate program bugs - but it could still make sense in unit testing code - especially if you're trying to build a fancier unit testing framework on top of the built-in one (to improve output or whatever).

So no, trying to recover from programming bugs is a _not_ a good idea, but there _are_ rare cases where catching Errors makes sense, and making it so that they skip scope statements, destructors, and finally blocks makes them even _less_ safe to catch and makes it so that you often can't recover from an Error even in the rare cases where you should be able to.

The main question that I see is whether trying to run cleanup code when an Error is thrown is likely to make things worse or not. If that's a really high risk, then it may be better to have scope statements, finally blocks, and destructors skipped. But if it's at all debatable as to whether it's better or worse to run cleanup code when an Error is thrown, then I'd argue that code will be _less_ buggy if they're run when an Error is thrown. That doesn't make it so that the programmer should be catching Errors much, but it _does_ make it so that it's reasonably safe in the few cases where it _does_ make sense to catch them.

- Jonathan M Davis
_______________________________________________
dmd-internals mailing list
dmd-internals@puremagic.com
http://lists.puremagic.com/mailman/listinfo/dmd-internals

On 3/15/2012 2:39 PM, Jonathan M Davis wrote:
> On Thursday, March 15, 2012 14:22:49 Walter Bright wrote:
>> On 3/15/2012 9:29 AM, Sean Kelly wrote:
>>> Is the program really corrupted though?
>> You cannot know that.
>>
>> I really think that going down the path of thinking one can recover from
>> programming bugs is the route to disaster. It goes against everything I know
>> about how to make safe, reliable systems.
> There's a difference between saying that Errors are not recoverable and making
> it so that cleanup code runs when Errors are thrown. Catching an Error and
> continuing the execution of your program is almost never a good idea, but
> catching it, doing some cleanup and rethrowing it (as finally, scope(failure),
> and scope(exit) would do) is a different matter. No, it may not work, but it
> can also allow you to do things that need to be done at cleanup. And _not_
> running scope statements, destructors, and finally blocks makes catching Errors
> even _less_ safe even when it _does_ make sense. And upon occasion, catching
> Errors _does_ make sense even for cases other than cleanup.

I don't agree. The more code you run after a program bug, the more likely you aren't going to be able to run your shutdown/logging code.

If any of that cleanup code actually needs to run on program bug shutdown, it still can be done with catch/throw. The other stuff assumes normal operation and is not needed to be run for shutdown/logging.

> There is a difference between recovering from an Error and a recovering from a
> programming bug. Not all Errors necessarily indicate that there's a bug in the
> program. They just indicate that they've hit what is almost certainly a fatal
> condition. For instance, OutOfMemoryError is generally fatal, but it doesn't
> necessarily indicate a bug in the program. If caught close to the throw point,
> it might be possible to recover in some cases. It's just that the current
> state _is_ fatal unless you can do something to fix it. In most cases you
> can't, but in a few, rare ones, you can.

The whole point of Errors is that they are non-recoverable. Yes, we did make a decision that out of memory is a non-recoverable error. If someone wants to recover from it, they should use malloc or some other API, not the GC.


> So no, trying to recover from programming bugs is a _not_ a good idea, but
> there _are_ rare cases where catching Errors makes sense, and making it so
> that they skip scope statements, destructors, and finally blocks makes them
> even _less_ safe to catch and makes it so that you often can't recover from an
> Error even in the rare cases where you should be able to.

If people want to recover from Errors, they should be using Exceptions instead. D should not be wishy washy about this.

> The main question that I see is whether trying to run cleanup code when an
> Error is thrown is likely to make things worse or not. If that's a really high
> risk, then it may be better to have scope statements, finally blocks, and
> destructors skipped. But if it's at all debatable as to whether it's better or
> worse to run cleanup code when an Error is thrown, then I'd argue that code
> will be _less_ buggy if they're run when an Error is thrown. That doesn't make
> it so that the programmer should be catching Errors much, but it _does_ make
> it so that it's reasonably safe in the few cases where it _does_ make sense to
> catch them.
>

I've been arguing with engineers for 25 years who insist that they should be able to recover from detected programming bugs. It's wrong, wrong, wrong, wrong.

If you want to recover, use an Exception.
_______________________________________________
dmd-internals mailing list
dmd-internals@puremagic.com
http://lists.puremagic.com/mailman/listinfo/dmd-internals

On Thursday, March 15, 2012 15:25:32 Walter Bright wrote:
> I've been arguing with engineers for 25 years who insist that they should be able to recover from detected programming bugs. It's wrong, wrong, wrong, wrong.
> 
> If you want to recover, use an Exception.

In general, I agree. It's just that there are a few cases where it makes sense. The main one has to do with unit tests. In some cases, it makes sense to catch an AssertError within a unit test, because assert is what's used in unit tests. For instance, some programmers want to test their in contracts. The biggest case though is that some programmers want to build more extensive unit testing frameworks on top of the built-in one. And to do that, you have to catch AssertErrors. Personally, I see no need for such frameworks, but some people (e.g. Jacob Carlborg) definitely want them, and if AssertErrors skip scope statements, finally blocks, or destructors, that's going to cause major problems for their frameworks.

- Jonathan M Davis
_______________________________________________
dmd-internals mailing list
dmd-internals@puremagic.com
http://lists.puremagic.com/mailman/listinfo/dmd-internals

On Mar 15, 2012, at 3:25 PM, Walter Bright wrote:
> 
> On 3/15/2012 2:39 PM, Jonathan M Davis wrote:
>> On Thursday, March 15, 2012 14:22:49 Walter Bright wrote:
>>> On 3/15/2012 9:29 AM, Sean Kelly wrote:
>>>> Is the program really corrupted though?
>>> You cannot know that.
>>> 
>>> I really think that going down the path of thinking one can recover from programming bugs is the route to disaster. It goes against everything I know about how to make safe, reliable systems.
>> There's a difference between saying that Errors are not recoverable and making it so that cleanup code runs when Errors are thrown. Catching an Error and continuing the execution of your program is almost never a good idea, but catching it, doing some cleanup and rethrowing it (as finally, scope(failure), and scope(exit) would do) is a different matter. No, it may not work, but it can also allow you to do things that need to be done at cleanup. And _not_ running scope statements, destructors, and finally blocks makes catching Errors even _less_ safe even when it _does_ make sense. And upon occasion, catching Errors _does_ make sense even for cases other than cleanup.
> 
> I don't agree. The more code you run after a program bug, the more likely you aren't going to be able to run your shutdown/logging code.
> 
> If any of that cleanup code actually needs to run on program bug shutdown, it still can be done with catch/throw. The other stuff assumes normal operation and is not needed to be run for shutdown/logging.

So I'm guessing that scope(failure) clauses will be executed when an Error is thrown because that's the point in having them.  But I imagine scope(exit) clauses will be run for Exceptions but not for Errors, despite the fact that the scope is clearly exiting via stack unwinding?  What's the difference in saying "I want this code to run when scope exits via a throw" vs. "I always want this to run regardless of how scope exits?"  I see the addition of special cases causing confusion here.

What if we kept the existing throw behavior and added a runtime callback, similar to onAssertError, that is called whenever any Error is about to be thrown?  At this point the user has the option interrupting the usual stack unwinding logic in favor of something more stringent, say logging an error and immediately calling abort()? This would allow those who don't want to take any chances with code being run between the throw point and their shutdown to do so, while leaving a consistent execution model for the average user?  Or flip the situation and make immediate abort the default behavior but let the user override to throw instead?  If the Error instance is passed to the callback the behavior could even vary based on the actual type of the Error generated.
_______________________________________________
dmd-internals mailing list
dmd-internals@puremagic.com
http://lists.puremagic.com/mailman/listinfo/dmd-internals

On 3/15/2012 2:22 PM, Leandro Lucarella wrote:
> Walter Bright, el 14 de marzo a las 16:45 me escribiste:
>> OutOfMemory is deliberately made not recoverable, as I've almost
>> never seen any application that can successfully recover from it.
>> Making it non-recoverable also means that pure functions can
>> allocate memory.
> Really? It seems that you never worked in a memory constrained system.
> There are applications that *really* need to take care about this.
>
> Even you say *almost*, so I guess you actually acknowledge their
> existence. If you go this route, please know that you'll make
> D a language not suitable for this king of applications.
>

If the memory constrained systems are that tight, they won't be able to use the GC anyway - they'll have to use malloc(), and what to do on malloc() failures is entirely up to the programmer.
_______________________________________________
dmd-internals mailing list
dmd-internals@puremagic.com
http://lists.puremagic.com/mailman/listinfo/dmd-internals

I'm a little confused, Errors come from program bugs, they shouldn't be recovered, but you are saying that cleanup can still be had through catch/throw? You mean that the cleanup code used for making Exception safe code would need to be duplicated in the try/catch of Errors too?

You say Exceptions should be used for recovery, but don't provide what you should use when you aren't trying to recover but want to cleanup on program bugs (I don't see cleanup as recovery). And yes I would consider an escaping exception to usually indicate a program bug too.

On Thu, Mar 15, 2012 at 3:25 PM, Walter Bright <walter@digitalmars.com> wrote:

> If any of that cleanup code actually needs to run on program bug shutdown, it still can be done with catch/throw. The other stuff assumes normal operation and is not needed to be run for shutdown/logging.
_______________________________________________
dmd-internals mailing list
dmd-internals@puremagic.com
http://lists.puremagic.com/mailman/listinfo/dmd-internals

Let me just reply to this thread with a perspective that people seem to not have regarded.

D is a systems language. That means it will be used to implement things such as operating systems, drivers, server software, compilers, debuggers, profilers, interpreters, JIT-backed virtual machines, and so on.

By making OutOfMemoryError uncatchable, you're trying to shield programmers from writing incorrect recovery code. I mean, excuse me, but really? This is a *systems* language. Don't get me wrong. Safety and expressiveness are the things I absolutely freaking love about D. But this is not safety - it is a limitation. In C, I can recover from lack of memory if I know what I'm doing; in D I can't (according to this rule). This is a serious mistake IMO.

(On a somewhat related note, if OutOfMemoryError really is uncatchable, why does the compiler not warn when you try to do so?)

But that's not all. Let's return to what I said before: Virtual machines. Almost /all/ virtual machines use garbage collection; in fact, most *enforce* it. In MCI, the D GC is provided as a GC implementation that virtualized applications can use. In other words, this means that the D GC is being used as *library*, not as a core component of a language. Now, when applications running under MCI run out of memory, they expect to see a null pointer returned, and to be able to act upon that. This can't be accomplished if OutOfMemoryError is made uncatchable. In other words, this forces me to remove the D GC as an option over something as *absolutely ridiculously stupid* as OutOfMemoryError being uncatchable. Really?!

What happens here is that a library component, something that is supposedly optional to use (!), is being coupled tightly to language semantics.

This needs to stop.

Regards,
Alex

On Sat, Mar 10, 2012 at 3:29 AM, Jonathan M Davis <jmdavisProg@gmx.com> wrote:
> The issue of catching Errors came up in D.learn today, and I need some clarification. It has been my understanding that anything which is thrown which is not derived from Exception skips all destructors, scope statements, and finally blocks, meaning that it's generally unsafe to catch them, because your program is potentially in an invalid state. However, it was brought to my attention that the compiler does not currently follow this behavior - all 3 of those get run for Errors at present. So, the question is whether I'm just completely misunderstanding something or whether something has changed.
>
> Is it guaranteed that all thrown Throwables will result in all destructors, scope statements (exit and failure at least), and finally blocks that they pass being executed? Or is it only guaranteed for Exception and its derived types and just so happens to work for other exception types right now?
>
> I've been completely certain that such was _not_ guaranteed unless the exception type is Exception or derived from Exception, but I can't find any support for that searching the docs or newsgroup save for what I've said about it myself. So, I'd like official clarification on the matter.
>
> - Jonathan M Davs
> _______________________________________________
> dmd-internals mailing list
> dmd-internals@puremagic.com
> http://lists.puremagic.com/mailman/listinfo/dmd-internals
_______________________________________________
dmd-internals mailing list
dmd-internals@puremagic.com
http://lists.puremagic.com/mailman/listinfo/dmd-internals

Walter Bright, el 15 de marzo a las 14:27 me escribiste:
> >This is specially bad if a memory allocation fail is an Error. It basically forces you to check every allocation for a failure and translate it yourself to some kind of Exception if you are being careful to write some code that can survive to a memory allocation failure.
> 
> That sounds good in theory, but in practice, almost no program can recover from out of memory failure. Even those that pretend they can usually cannot because such was never tested.

Do you know failure injection at all? Come on! And you keep saying "almost", so guess you realize there are people that are actually capable to write code that can survive to an out of memory error. Kernel developers for example.

Doesn't make any sense to keep discussing when you obviously have your
position taken here. But please be aware that you are making
D unsuitable for this kind of programs.

> >Why on earth do you want to make life miserable to people that have some
> >valid use case for this, just to avoid corrupting a little more
> >a program that's already corrupted.
> 
> I believe it is a *serious* mistake to believe one can recover a program that has become corrupted.

And I believe that thinking that out of memory is corruption is a *serious* mistake.

-- 
Leandro Lucarella (AKA luca)                     http://llucax.com.ar/
----------------------------------------------------------------------
GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145  104C 949E BFB6 5F5A 8D05)
----------------------------------------------------------------------
No debemos temer a la muerte, porque es la mejor recompensa de la vida.
	-- Ren & Stimpy
_______________________________________________
dmd-internals mailing list
dmd-internals@puremagic.com
http://lists.puremagic.com/mailman/listinfo/dmd-internals

Walter Bright, el 15 de marzo a las 16:57 me escribiste:
> On 3/15/2012 2:22 PM, Leandro Lucarella wrote:
> >Walter Bright, el 14 de marzo a las 16:45 me escribiste:
> >>OutOfMemory is deliberately made not recoverable, as I've almost never seen any application that can successfully recover from it. Making it non-recoverable also means that pure functions can allocate memory.
> >Really? It seems that you never worked in a memory constrained system. There are applications that *really* need to take care about this.
> >
> >Even you say *almost*, so I guess you actually acknowledge their
> >existence. If you go this route, please know that you'll make
> >D a language not suitable for this king of applications.
> 
> If the memory constrained systems are that tight, they won't be able
> to use the GC anyway - they'll have to use malloc(), and what to do
> on malloc() failures is entirely up to the programmer.

Well, you know that avoiding the GC in D is not particularly easy, as there are so many hidden allocations...

-- 
Leandro Lucarella (AKA luca)                     http://llucax.com.ar/
----------------------------------------------------------------------
GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145  104C 949E BFB6 5F5A 8D05)
----------------------------------------------------------------------
Bald men with no jobs and no money who live with their parents don't
approach strange women.
	-- George Constanza
_______________________________________________
dmd-internals mailing list
dmd-internals@puremagic.com
http://lists.puremagic.com/mailman/listinfo/dmd-internals