On Sun, Feb 19, 2012 at 09:17:23PM -0600, Andrei Alexandrescu wrote:
> On 2/19/12 9:06 PM, H. S. Teoh wrote:
[...]
> >Do you have a concrete scenario in mind where such a decision would actually be useful? Otherwise we'll just end up with boilerplate code copy-n-pasted everywhere of the form:
> >
> >	auto retries = SomeArbitraryNumber;
> >	do {
> >		try {
> >			...
> >		} catch(Exception e) {
> >			if (e.is_transient&&  retries-->  0)
> >				continue;
> >			throw e;
> >		}
> >	} while(false);
> >
> >But since this block is completely independent of what's inside the try block, why not just put it where the exception is generated in the first place?
> 
> I explained this. The raise locus does not have access to the high-level context.
[...]

This is why I'm liking the Lisp model more and more. The lower level code knows best what recovery strategies are available. But it can't make that decision at that level. So you need the higher level code to make this decision. But if you simply unwind the stack all the way back up to the higher level code, then you've lost the opportunity of reusing the execution context of the lower level code to perform the recovery. You have to start from scratch. You also have no other recovery strategies available, since the original execution context is gone. The only options left are retry or abort.

By having the high-level code register a delegate which can make this sorts of decisions, when the low-level code *does* encounter a problem it will be able to perform the chosen recovery procedure immediately, rather than throwing away the entire execution context and perhaps having to reestablish all of it again later.

This doesn't mean the low-level network socket that encounters the problem will be able to run the recovery right there; you do have to unwind the stack to some point where intelligent recovery is possible. But the point is that this is often lower down in the call stack than the code which is qualified to make decisions about which recovery strategy should be used. That's why you want to encapsulate the code that makes this decision in a delegate which is run at the lower-level code where intelligent recovery takes place.


T

-- 
Life is too short to run proprietary software. -- Bdale Garbee

On Mon, Feb 20, 2012 at 03:57:50PM +1300, James Miller wrote:
> I agree that the "Lispian" model works well, though I had issues trying to get my head around it when I encountered it.
> 
> I don't know how you'd make a simpler version for D (D lacking Lisps ridiculous macros) but maybe something that essentially "returns" a list of recovery codes (which would unfortunately have to be documented) that can be called depending on the context of the error.
[...]

The concept doesn't require Lisp macros to work. D already has the necessary machinery to implement it. All that's needed is some syntactic sugar to make it very easy to use. Personally I'd like language-level support for it, but I suspect judicious use of templates may alleviate even that.


> But error-handling is hard, programmers are naturally lazy, and checking errors is not something exiting. Exceptions are always going to be a source of contention amongst people. I know people (mostly C programmers) that hate them, and other people swear by them. I agree that incorrect parameters should not be Exceptions, and are contract-level issues, check your parameters before passing them if there are conditions on them!

D already supports contracts. The current convention throws AssertError on contract failure, but you could just as easily terminate the program right there.


> its a difficult topic, hence the ridiculously long thread we have going here. Error codes are not really the way to go, I prefer more of an Objective-C/Smalltalk null-pattern style, where null can be a valid argument almost anywhere the type system allows and code handles it properly, normally by returning null.
[...]

Error codes are passé. The only reason we still have them is because of OS's inheriting the legacy APIs and conventions from 20 years ago.


T

-- 
I am Ohm of Borg. Resistance is voltage over current.

On Sun, Feb 19, 2012 at 09:12:25PM -0600, Andrei Alexandrescu wrote:
> On 2/19/12 8:52 PM, H. S. Teoh wrote:
[...]
> >Correct, so that would be a recovery strategy at the operation level, say at sendHttpRequest or something like that. There is not enough information available to sendHttpRequest to know whether or not the caller wants the request to be retried if it fails.
> >
> >But if the higher-level code could indicate this by way of a recovery policy delegate, then this retry can be done at the sendHttpRequest level, instead of percolating up the call stack all the way to submitHttpForm, which then has to reparse user data, convert into JSON, say, and then retry the entire operation all over again.
> >
> >I'm really liking the Lisp approach.
> 
> Now we're talking. Ideas. Outside the box.
[...]

Alright. This thread has gone on for too long with lots of talk but no down-to-earth, real code. So I decided to code up a quick-n-dirty proof-of-concept implementation of a Lispian scheme of exception handling. The source files are attached.

This implementation is more to demonstrate the *semantics* of such a system, rather than a sample code of the real thing. Before anyone jumps on me for writing very bad code. ;-)

In a real implementation, I probably wouldn't inherit Condition from Exception, for example. A real implementation would have language-level support for declaring recovery strategies within the function that handles them, rather than the kludge of declaring them separately and abusing the exception system to handle recovery actions.

Also, registered handlers would need to be cleaned up upon scope exit, so that they don't cause crazy side-effects in unrelated code later on in the program.

But anyway. The point of this code is to demonstrate:

(1) How an exception generated in low-level code is recovered within the low-level code without unwinding the stack all the way back to the top-level caller;

(2) How the top-level caller can provide a delegate that makes decisions that only the top-level code is qualified to make, yet does so in the context of low-level code so that recovery can proceed without unwinding the stack any more than necessary.

I deliberately made the delegate prompt the user for a fixed filename, even allowing the option of aborting if the user wants to give up. This is to show how the system handles different recovery actions. This, of course, isn't the only way to do things; you can programmatically decide on a recovery strategy instead of prompting the user, for example.

What do y'all think?


T

-- 
Nobody is perfect.  I am Nobody. -- pepoluan, GKC forum

On Sunday, 19 February 2012 at 20:57:09 UTC, Andrei Alexandrescu wrote:
> On 2/19/12 1:19 PM, Nick Sabalausky wrote:
>> That wouldn't be as useful. What the catcher is typically interested in is
>> *what* happened, not *where* it happened.
>
> But module organization is partitioned by functional areas.
>
>> For example, if I want to do something upon a network error, minimum 99
>> times out of 100 I don't give a shit if it came from libNetworkFoo or
>> libNetworkBar, and I don't *want* to care. What I care is whether or not
>> there was a "network" error and possibly what *conceptual* type of network
>> error.
>
> Then it wouldn't help if each defined its own hierarchy.
>
> The way I see it is, a "well-designed" package and module hierarchy would naturally engender a "well-designed" exception hierarchy. This is because packages and modules are organized on functional areas, so e.g. there is an "std.net" package that has its own exception types etc. There would be some special cases indeed (e.g. a module initiating an exception defined in another), so it's good those are possible too. I want to automate the common case.
>
>> Furthurmore, what if I change some implementation detail to use a different
>> module? Then I have to go changing all my catch blocks even though it's
>> conceptually the same fucking error handled the same way.
>
> That is an issue regardless. Occasional exception translation is a fact of life.
>
>> However, I wouldn't object to the idea of an "originatingModule" member
>> being added to Exception that's automatically filled by the runtime (perhaps
>> lazily). Although really, I think what would be more useful that that would
>> be "Does xxx module/package exist in the portion of the callstack that's
>> been unwound?"
>
> That's why PackageException!"tango.io" inherits PackageException!"tango". That's all automatic. Essentially there's 1:1 correspondence between package/module hierarchy and exception hierarchy.
>
>> As far as "when to add or not add an exception class", it's perfectly
>> reasonable to err on the side of too many: If there's an unnecessary class,
>> you can just ignore it. Problem solved. If there's a missing exception
>> class, you're shit out of luck. Case closed.
>
> I disagree that having too many exception types comes at no cost.
>
>> I can't shake the feeling that we're desperately trying to reinvent the
>> wheel here. The round wheel is solid technology with a proven track record,
>> we don't need to waste time evaluating all these square and oval wheels just
>> for the fuck of it.
>
> The wheel is not round. We just got used to thinking it is. Exceptions are wanting and it's possible and desirable to improve them.
>
>
> Andrei

I just died a little reading this.
Are you suggesting that in order to handle IO exceptions I need to:
try {
 ...whatever...
} catch (PackageException!"std.io") {...}
} catch (PackageException!"tango.io") {...}
} catch (PackageException!"otherLib.io") {...}
...

What the hell is wrong with just using an IOException?
AS Nick wrote, it seems you have a complete lack of understanding of how exceptions work which is unsurprising coming from a c++ expert.

Also, this entire discussion you started about "improving" exceptions looks to me like a combination of NIH syndrome sparkled with heavy doses of premature optimization.

On 2/20/12 12:44 AM, foobar wrote:
> I just died a little reading this. Are you suggesting that in order
> to handle IO exceptions I need to: try { ...whatever... } catch
> (PackageException!"std.io") {...} } catch
> (PackageException!"tango.io") {...} } catch
> (PackageException!"otherLib.io") {...} ...
>
> What the hell is wrong with just using an IOException?

There's nothing wrong, but there's a possible misunderstanding. If
tango.io and otherLib.io cooperate with std, then they'd originate
exceptions in std.io (as opposed to their own). Do note that the issue
is exactly the same if libraries use IOException - they all must agree
on using the same nomenclature, whether it's called
PackageException!"std.io" or IOException.

ModuleException and PackageException have one important thing going for
them: they automate away a good amount of boilerplate, which makes them
interesting for me to look at, and worth sharing as long as we're
brainstorming. The associated issues as clear as the advantages.
Probably ModuleException is too specific to be interesting, but
PackageException seems useful.

> AS Nick wrote, it seems you have a complete lack of understanding of
>  how exceptions work which is unsurprising coming from a c++ expert.

Always eager to learn (so please come with all you've got), but quite
honest I hope in a way you're exaggerating, seeing as a lot of the stuff
I authored for D (the scope statement, exception chaining, a full
chapter in TDPL) would be completely broken.

> Also, this entire discussion you started about "improving" exceptions
> looks to me like a combination of NIH syndrome sparkled with heavy
> doses of premature optimization.

What is being optimized here?


Thanks,

Andrei

On Monday, February 20, 2012 01:10:39 Andrei Alexandrescu wrote:
> On 2/20/12 12:44 AM, foobar wrote:
> > I just died a little reading this. Are you suggesting that in order
> > to handle IO exceptions I need to: try { ...whatever... } catch
> > (PackageException!"std.io") {...} } catch
> > (PackageException!"tango.io") {...} } catch
> > (PackageException!"otherLib.io") {...} ...
> > 
> > What the hell is wrong with just using an IOException?
> 
> There's nothing wrong, but there's a possible misunderstanding. If tango.io and otherLib.io cooperate with std, then they'd originate exceptions in std.io (as opposed to their own). Do note that the issue is exactly the same if libraries use IOException - they all must agree on using the same nomenclature, whether it's called PackageException!"std.io" or IOException.
> 
> ModuleException and PackageException have one important thing going for them: they automate away a good amount of boilerplate, which makes them interesting for me to look at, and worth sharing as long as we're brainstorming. The associated issues as clear as the advantages. Probably ModuleException is too specific to be interesting, but PackageException seems useful.

It saves no boilerplate at the catch point, because you have to know what you're catching to do anything useful with it. It could be that you choose to catch a common exception rather than a specific one (e.g. IOException instead of FileException), but regardless of whether you use templates or mixins or whatever to generate the exception type's source code, you still need to write the handling code by hand. Handling code is _not_ the sort of thing that can be automated.

And as for saving boilerplate in defining exceptions, well some of that could be done via mixins, but since useful exceptions often have additional member variables, you're going to have to write many of them by hand anyway. _Some_ exceptions _could_ be done via mixins and the like when it's their type that matters and that's it, but I question that that's enough boilerplate to be a big deal - especially since it's not likely we should be adding exception types all the time.

I think that you're overestimating the amount of boilerplate code involved, and there are easy ways to reduce it without making exceptions be tied to modules or packages.

And as has been said before, ultimately the module that an exception comes from doesn't mean much. It's what went wrong that matters. And ideally, the standard library would have exception types that user code would throw or derive its own exception types from, in which case the exception types get even more divorced from the modules that they're declared in. So, ultimately, tying exceptions to modules or packages is _not_ a good idea in the general case. Sometimes it makes sense - particularly if you're talking about making a base exception type which a project as a whole uses (e.g. a PostgresException for a postgres library) - but in general, it's a bad approach, and it's one that we should be moving away from.

I say that we organize Phobos' exceptions into a proper hierarchy and then let other D projects derive from them or not as they see appropriate. But a standard set of exceptions should help standardize exceptions in D in general, even if not all libraries and projects choose to derive their exception types from the standard library's exception types.

We don't need to do anything elaborate with how exceptions are defined. We just need to organize them into a hierarchy. And it's not like we even have all that many exceptions to organize at the moment.

This thread was started due to a legitimate problem - the fact that our exception types are not always well chosen and definitely aren't always well organized. But that doesn't mean that we need to do anything new or fancy to fix the problem either.

- Jonathan M Davis

On 02/20/2012 02:53 AM, H. S. Teoh wrote:
> Perhaps the "ideal exception handling facility" that Andrei is looking
> for is a Lispian model,

+1.

On 2012-02-19 23:44, Jonathan M Davis wrote:
> On Sunday, February 19, 2012 16:07:27 Jacob Carlborg wrote:
>> On 2012-02-19 10:26, Jonathan M Davis wrote:
>>> On Sunday, February 19, 2012 19:00:20 Daniel Murphy wrote:
>>>> I wasn't really serious about implicit fallthrough.
>>>
>>> Lately, it seems like I can never tell whether anyone's being serious or
>>> not online. :)
>>>
>>>> Out of the syntaxes I could come up with:
>>>> catch(Ex1, Ex2 e)
>>>> catch(e : Ex1, Ex2)
>>>> catch(Ex1 | Ex2 e) // java 7 syntax, horrible
>>>>
>>>> I like (e : list) the best.  Naturally it would also accept a type tuple
>>>> of
>>>> exceptions.
>>>>
>>>> http://d.puremagic.com/issues/show_bug.cgi?id=7540
>>>
>>> LOL. Personally, I actually think that the Java 7 syntax looks great (I'd
>>> never seen it before), but catch(e : Ex1, Ex2) is just as good and more
>>> consistent with the language as a whole, since it doesn't try to give any
>>> operators a new meaning (as Java's does).
>>>
>>> - Jonathan M Davis
>>
>> How is "catch(e : Ex1, Ex2)" consistent with the language? It's
>> completely backwards. catch-block are written as follows:
>>
>> catch (Exception e) {}
>>
>> Not
>>
>> catch (e : Exception) {}
>
> I meant the meaning of  the : operator vs the meaning of the | operator. : has
> to do with derived types already, whereas | is for bitwise operations. Doing
> something like

Oh, as in template constrains, I see. Forgot about that.

> catch(Ex1, Ex2 : Ex0 e)
>
> would be even more consistent though for the reasons that you point out.
>
> - Jonathan m Davs

-- 
/Jacob Carlborg

On Monday, 20 February 2012 at 07:10:39 UTC, Andrei Alexandrescu wrote:
> On 2/20/12 12:44 AM, foobar wrote:
>> I just died a little reading this. Are you suggesting that in order
>> to handle IO exceptions I need to: try { ...whatever... } catch
>> (PackageException!"std.io") {...} } catch
>> (PackageException!"tango.io") {...} } catch
>> (PackageException!"otherLib.io") {...} ...
>>
>> What the hell is wrong with just using an IOException?
>
> There's nothing wrong, but there's a possible misunderstanding. If
> tango.io and otherLib.io cooperate with std, then they'd originate
> exceptions in std.io (as opposed to their own). Do note that the issue
> is exactly the same if libraries use IOException - they all must agree
> on using the same nomenclature, whether it's called
> PackageException!"std.io" or IOException.
>

The above is patently wrong.
Are you suggesting that tango.io and otherLib.io need to depend on Phobos IO?? If so, that removes the benefits of using 3rd party libraries. If that's not your intention (and I really hope it isn't!) than IOException must be defined in a *separate* module that tango can depend on.

> ModuleException and PackageException have one important thing going for
> them: they automate away a good amount of boilerplate, which makes them
> interesting for me to look at, and worth sharing as long as we're
> brainstorming. The associated issues as clear as the advantages.
> Probably ModuleException is too specific to be interesting, but
> PackageException seems useful.
>

Per above, your suggestion actually *adds* boilerplate since you now want me to use PackageException!"std.io" instead of just IOException.

>> AS Nick wrote, it seems you have a complete lack of understanding of
>> how exceptions work which is unsurprising coming from a c++ expert.
>
> Always eager to learn (so please come with all you've got), but quite
> honest I hope in a way you're exaggerating, seeing as a lot of the stuff
> I authored for D (the scope statement, exception chaining, a full
> chapter in TDPL) would be completely broken.
>

"even a broken clock shows the right time twice a day"

>> Also, this entire discussion you started about "improving" exceptions
>> looks to me like a combination of NIH syndrome sparkled with heavy
>> doses of premature optimization.
>
> What is being optimized here?
>
>
> Thanks,
>
> Andrei

It's clear that you are trying to generify exceptions. This contradicts the very notion of what exceptions are. You also seem to try to optimize the amount of exception classes. Making user code convoluted for the sake of some premature optimization which most likely has negligible affect is completely unacceptable. I get that you are a templates master, that does *NOT* mean everything must be made generic.
You seem to prove the old saying that when all you have is a hammer everything looks like a nail.

[Meta] side-note:
It's extremely irritating when you demand utmost pedantic reasoning from others while you often answer without providing such pedantic reasoning yourself or worse answer with a single word posts. That shows a complete lack of respect for others. You seem to be of high regard for yourself which is not justified at all given this attitude.

On 2012-02-20 02:03, H. S. Teoh wrote:
> On Sat, Feb 18, 2012 at 11:09:23PM -0500, bearophile wrote:
>> Sean Cavanaug:
>>
>>> In the Von Neumann model this has been made difficult by the stack
>>> itself.  Thinking of exceptions as they are currently implemented in
>>> Java, C++, D, etc is automatically artificially constraining how
>>> they need to work.
>>
>> It's interesting to take a look at how "exceptions" are designed in
>> Lisp:
>> http://www.gigamonkeys.com/book/beyond-exception-handling-conditions-and-restarts.html
> [...]
>
> I'm surprised nobody responded to this. I read through the article a
> bit, and it does present some interesting concepts that we may be able
> to make use of in D. Here's a brief (possibly incomplete) summary:
>
> One problem with the try-throw-catch paradigm is that whenever an
> exception is raised, the stack unwinds up some number of levels in the
> call stack. By the time it gets to the catch{} block, the context in
> which the problem happened is already long-gone, and there is no other
> recourse but to abort the operation, or try it again from scratch. There
> is no way to recover from the problem by, say, trying to fix it *in the
> context in which it happened* and then continuing with the operation.
>
> Say P calls Q, Q calls R, and R calls S. S finds a problem that prevents
> it from doing what R expects it to do, so it throws an exception. R
> doesn't know what to do, so it propagates the exception to Q. Q doesn't
> know what to do either, so it propagates the exception to P. By the time
> P gets to know about the problem, the execution context of S is long
> gone; the operation that Q was trying to perform has already been
> aborted. There's no way to recover except to repeat a potentially very
> expensive operation.
>
> The way Lisp handles this is by something called "conditions". I won't
> get into the definitions and stuff (just read the article), but the idea
> is this:
>
> - When D encounters a problem, it signals a "condition".
>
>     - Along with the condition, it may register 0 or more "restarts",
>       basically predefined methods of recovering from the condition.
>
> - The runtime then tries to recover from the condition by:
>
>     - Checking to see if there's a handler registered for this condition.
>       If there is, invoke the most recently registered one *in the
>       context of the function that triggered the condition*.
>
>     - If there's no handler, unwind the stack and propagate the condition
>       to the caller.
>
> - There are two kinds of handlers:
>
>     - The equivalent of a "catch": matches some subset of conditions that
>       propagated to that point in the code. Some stack unwinding may
>       already have taken place, so these are equivalent to catch block in
>       D.
>
>     - Pre-bound handlers: these are registered with the runtime condition
>       handler before the condition is triggered (possibly very high up
>       the call stack). They are invoked *in the context of the code that
>       triggered the condition*. Their primary use is to decide which of
>       the restarts associated with the condition should be used to
>       recover from it.
>
> The pre-bound handlers are very interesting. They allow in-place
> recovery by having high-level callers to decide what to do, *without
> unwinding the stack*. Here's an example:
>
> LoadConfig() is a function that loads an application's configuration
> files, parses them, and sets up some runtime objects based on
> configuration file settings. LoadConfig calls a bunch of functions to
> accomplish what it does, among which is ParseConfig(). ParseConfig() in
> turn calls ParseConfigItem() for each configuration item in the config
> file, to set up the runtime objects associated with that item.
> ParseConfigItem() calls DecodeUTF() to convert the configuration file's
> text representation from, say, UTF-8 to dchar. So the call stack looks
> like this:
>
> LoadConfig
> 	ParseConfig
> 		ParseConfigItem
> 			DecodeUTF
>
> Now suppose the config file has some UTF encoding errors. This causes
> DecodeUTF to throw a DecodingError. ParseConfigItem can't go on, since
> that configuration item is mangled. So it propagates DecodingError to
> ParseConfig.
>
> Now, ParseConfig could simply abort, but using the idea of prebound
> handlers, it can actually offer two ways of recovering: (1)
> SkipConfigItem, to simply skip the mangled config item and process the
> rest of the config file as usual, or (2) ReparseConfigItem, to allow
> custom code to manually fix a bad config item and reprocess it.
>
> The problem is, ParseConfig doesn't know which action to take. It's too
> low-level to make that sort of decision. You need higher-level code,
> that knows what the application needs to do, to decide that. But
> ParseConfig can't just propagate the exception to said high-level code,
> because if it does, parsing of the entire config file is aborted and
> will have to be restarted from scratch.
>
> The solution is to have the higher-level code register a delegate with
> the exception system. Something like this:
>
> 	// NOTE: not real D code
> 	void main() {
> 		registerHandler(auto delegate(ParseError e) {
> 			if (can_repair_item(e.item)) {
> 				return e.ReparseConfigItem(
> 					repairConfigItem(e.item));
> 			} else {
> 				return e.SkipConfigItem();
> 			}
> 		});
>
> 		ParseConfig(configfile);
> 	}
>
> Now when ParseConfig encounters a problem, it signals a ParseError
> object with two options for recovery: ReparseConfigItem and
> SkipConfigItem. It doesn't try to fix the problem on its own, but it
> lets the delegate from main() make that decision. The runtime exception
> system then sees if there's a matching handler, and calls the handler
> with the ParseError to determine which course of action to take. If no
> handler is found, or the handler decides to abort, then ParseError is
> propagated to the caller with stack unwinding.
>
> So ParseConfig might look something like this:
>
> // NOTE: not real D code
> auto ParseConfig(...) {
> 	foreach (item; config_items) {
> 		try {
> 			// Note: not real proposed syntax, this is just
> 			// to show the semantics of the mechanism:
> 			restart:
> 			auto objs = ParseConfigItem(item);
> 			SetupConfigObjects(objs);
> 		} catch(ParseConfigItemError) {
> 			// Note: not real proposed syntax, this is just
> 			// to show the semantics of the mechanism:
> 			ConfigError e;
> 			e.ReparseConfigItem = void delegate(ConfigItem
> 				fixedItem)
> 			{
> 				goto restart;
> 			};
> 			e.SkipConfigItem = void delegate() {
> 				continue;
> 			}
>
> 			// This will unwind stack if no handler is
> 			// found, or handler decides to propagate
> 			// exception.
> 			handleError(e);
> 		}
> 	}
> }
>
> OK, so it looks real ugly right now. But if this mechanism is built into
> the language, we could have much better syntax, something like this:
>
> auto ParseConfig(...) {
> 	foreach (item; config_items) {
> 		try {
> 			auto objs = ParseConfigItem(item);
> 			SetupConfigObjects(objs);
> 		} recoverBy ReparseConfigItem(fixedItem) {
> 			item = fixedItem;
> 			restart;	// restarts try{} block
> 		} recoverBy SkipConfigItem() {
> 			setDefaultConfigObjs();
> 			continue;	// continues foreach loop
> 		}
> 	}
> }
>
> This is just a rough sketch syntax, just to show the idea. It can of
> course be improved upon.
>
>
> T

I was actually thinking something similar, the part about registering exception handlers, i.e. using "registerHandler".

-- 
/Jacob Carlborg