Dear d-lang experts,

lets say in general I am quite happy with exceptions.
Recently though I stumbled upon two problems with them:
1. Its quite simple to loose valuable information
2. Its hard to present the exception messages to end users of your
   program

Let me elaborate on those:
Lets take a simple config parser example (pseudocode, the real code is in the end):
```d
auto parseConfig(string filename)
{
    return s
        .readText
        .parseJSON;
}
..
void main()
{
    ...
    auto config = parseConfig("config.json");
    run(config);
    ...
}
```
Lets look at what we would get in terms of error messages for the user of this program (besides that a full stacktrace is printed, which is nice for the developer of the program, but perhaps not so nice for the end user)
- file is not there: config.json: No such file or directory
  nice ... its almost human readable and one could guess that something
  with the config file is amiss
- file is not readable for the user: config.json: Permission denied
  nice ... quite a good explanation as well
- file with invalid UTF-8: Invalid UTF-8 sequence (at index 1)
  not so nice ... something is wrong with some UTF-8 in the program,
  but where and in which area of the program
- file with broken json: Illegal control character. (Line 4:0)
  not nice ... line of which file, illegal control character does
  not even sound like json processing.

Arguably readText could behave a little better and not throw away
information about the file its working with, but for parseJSON the
problem is "bigger" as it is only working with a string in memory not
with a file anymore, so it really needs some help of the application
developer I would say. When browsing through phobos I stumbled upon the
genius std.exception.ifThrown function, that allows for very nice
fallbacks in case of recoverable exceptions. Building on that I came up
with the idea (parhaps similar to Rusts error contexts) to use this
mechanism to wrap the exceptions in better explaining exceptions. This
would allow to provide exceptions with the information that might
otherwise be lost aswell as lift the error messages onto a end user consumable level (at least if only the topmost exception message is looked at).


```d
#!/usr/bin/env rdmd
import std;

auto contextWithString(T)(lazy scope T expression, string s)
{
    try
    {
        return expression();
    }
    catch (Exception e)
    {
        throw new Exception("%s\n%s".format(s, e.msg));
    }
    assert(false);
}

auto contextWithException(T)(lazy scope T expression, Exception delegate(Exception) handler)
{
    Exception newException;
    try
    {
        return expression();
    }
    catch (Exception e)
    {
        newException = handler(e);
    }
    throw newException;
}

// plain version, no special error handling
JSONValue readConfig1(string s)
{
    // dfmt off
    return s
        .readText
        .parseJSON;
    // dfmt.on
}

// wraps all exceptions with a description whats going on
JSONValue readConfig2(string s)
{
    // dfmt off
    return s
        .readText
        .parseJSON
        .contextWithString("Cannot process config file %s".format(s));
    // dfmt on
}

// tries to deduplicate the filename from the exception messages
// but misses on utf8 errors
JSONValue readConfig3(string s)
{
    // dfmt off
    auto t = s
        .readText;
    return t
        .parseJSON
        .contextWithString("Cannot process config file %s".format(s));
    // dfmt on
}

// same as 3 just different api
JSONValue readConfig4(string s)
{
    // dfmt off
    auto t = s
        .readText;
    return t
        .parseJSON
        .contextWithException((Exception e) {
            return new Exception("Cannot process config file%s\n %s".format(s, e.msg));
        });
    // dfmt on
}

void main()
{
    foreach (file; [
        "normal.txt",
        "missing.txt",
        "broken_json.txt",
        "not_readable.txt",
        "invalid_utf8.txt",
    ])
    {

writeln("=========================================================================");
        size_t idx = 0;
        foreach (kv; [
            tuple("readConfig1", &readConfig1),
	    tuple("readConfig2", &readConfig2),
	    tuple("readConfig3", &readConfig3),
            tuple("readConfig4", &readConfig4),
        ])
        {
            auto f = kv[1];
            try
            {
                if (idx++ > 0) writeln("-------------------------------------------------------------------------");
                writeln("Working on ", file, " with ", kv[0]);
                f("testfiles/%s".format(file));
            }
            catch (Exception e)
            {
                writeln(e.msg);
            }
        }
    }
}
```

What do you guys think about that?
Full dub project available at git@github.com:gizmomogwai/d-exceptions.git ... If you want to play with it, please run ./setup-files.sh first (which will create a file that is not readable).

One thing I am especially interested in would be how to annotate only
one part of a callchain with additional error information.
e.g. given the chain
```d
    someObject
         .method1
         .method2
         .method3
```
I would like to have the options to only add additional information to `method2` calls, without having to introduce intermediate variables (and
breaking the chain with it).

Also is there a way to do this kind of error handling without exceptions
(I think its hard to do this with optionals as they get verbose without
a special quick return feature).


Thanks in advance,
Christian

You might find my recent blog post interesting too:

http://dpldocs.info/this-week-in-d/Blog.Posted_2022_08_01.html#exception-template-concept

and a draft of some more concepts:
http://arsd-official.dpldocs.info/source/arsd.exception.d.html


I also find the lack of information disturbing, but I also hate formatting it all into strings, so I wanted to experiment with other options there that avoid the strings.

On 12.08.22 01:06, Adam D Ruppe wrote:
> You might find my recent blog post interesting too:
> 
> http://dpldocs.info/this-week-in-d/Blog.Posted_2022_08_01.html#exception-template-concept
> 
> and a draft of some more concepts:
> http://arsd-official.dpldocs.info/source/arsd.exception.d.html
> 
> 
> I also find the lack of information disturbing, but I also hate formatting it all into strings, so I wanted to experiment with other options there that avoid the strings.
Yes ... I already read your blog post. In general I am with you there as
it makes sense to have either dedicated exception classes or something more along the lines of what you describe in your post.

On Thu, Aug 11, 2022 at 11:06:45PM +0000, Adam D Ruppe via Digitalmars-d-learn wrote:
> You might find my recent blog post interesting too:
> 
> http://dpldocs.info/this-week-in-d/Blog.Posted_2022_08_01.html#exception-template-concept
> 
> and a draft of some more concepts: http://arsd-official.dpldocs.info/source/arsd.exception.d.html
> 
> 
> I also find the lack of information disturbing, but I also hate formatting it all into strings, so I wanted to experiment with other options there that avoid the strings.

I think the OP's idea is somewhat different: adding contextual information to a propagating exception that the throwing code may not have access to.

I've often encountered this situation: for example, I have a program that's driven by script, and contains a bunch of code that does different computations which are ultimately called by the script parser. There may be an error deep inside a computation, e.g., a vector was zero where it shouldn't be, for example, and this may be nested inside some deep expression tree walker. So it throws an exception.  But the resulting message is unhelpful: it merely says some obscure vector is unexpectedly zero, but not where the error occurred, because the expression walk code doesn't know where in the script it is. And it *shouldn't* know -- the filename/line of a script is completely orthogonal to evaluating expressions; for all it knows, it could be invoked from somewhere else *not* from the input script, in which case it would be meaningless to try to associate a filename/line with the exception.  Expression evaluation code should be decoupled from parser code; it shouldn't need to know about things only the parser knows.

And besides, if the expression code wasn't invoked from the parser, the exception shouldn't include filename/line information where there isn't any. It should be the parser that tacks on this information when the exception propagates up the call stack from the lower-level code. In fact, the exception should acquire *different* contextual information on its way up the call stack, depending on what triggered the upper-level call. If the expression evaluation was invoked, say, by network code, then the exception when it arrives at the catch block ought to carry network source IP information, for example.  If it was invoked by simulation code, then it should carry information about the current state of the simulation.

In the other direction, if the expression evaluator code calls, say, std.conv.to at some point, and .to throws a conversion error, then the outgoing exception should carry some information about where in the exception the problem happened. There is no way std.conv could know about this information (and it shouldn't know anyway); the expression code should be the one tacking this information on.  And as the exception propagates upwards, it should gather more higher-level contextual information, each coming from its respective level of abstraction.

The OP's idea of wrapping throwing code with a function that tacks on extra information is a good idea.  Perhaps the use of strings isn't ideal, but in principle I like his idea of exceptions acquiring higher-level information as it propagates up the call stack.


T

-- 
PNP = Plug 'N' Pray

On Thursday, 11 August 2022 at 23:50:58 UTC, H. S. Teoh wrote:
> I think the OP's idea is somewhat different: adding contextual information to a propagating exception that the throwing code may not have access to.

Yeah, but you can use the mechanism again: you'd catch the one then throw a new one with the old one tacked on the back.

> The OP's idea of wrapping throwing code with a function that tacks on extra information is a good idea.

Yeah, that is good. I also kinda wish that scope(failure) could do it so you could tack on info with a more convenient syntax... i have some more wild ideas brewing now lol

On Fri, Aug 12, 2022 at 12:12:13AM +0000, Adam D Ruppe via Digitalmars-d-learn wrote:
> On Thursday, 11 August 2022 at 23:50:58 UTC, H. S. Teoh wrote:
> > I think the OP's idea is somewhat different: adding contextual information to a propagating exception that the throwing code may not have access to.
> 
> Yeah, but you can use the mechanism again: you'd catch the one then throw a new one with the old one tacked on the back.

True.  But having to insert try/catch syntax wrapping around every potential abstraction level that might want to add contextual information is a pain.  If we could use a wrapper that can be inserted at strategic entry points, that'd be much better.


> > The OP's idea of wrapping throwing code with a function that tacks on extra information is a good idea.
> 
> Yeah, that is good. I also kinda wish that scope(failure) could do it so you could tack on info with a more convenient syntax... i have some more wild ideas brewing now lol

Hmm! That gets me thinking.  Maybe something like this?

	// Totally fantastical, hypothetical syntax :P
	auto myFunc(Args...)(Args args) {
		int additionalInfo = 123;
		scope(failure, additionalInfo);

		return runUnreliableOperation(args);
	}


T

-- 
"No, John.  I want formats that are actually useful, rather than over-featured megaliths that address all questions by piling on ridiculous internal links in forms which are hideously over-complex." -- Simon St. Laurent on xml-dev

On Friday, 12 August 2022 at 00:40:48 UTC, H. S. Teoh wrote:
> Hmm! That gets me thinking.  Maybe something like this?

aye. and scopes share dtors which means we can do it in the lib today:

---
struct AdditionalInfo {
        static string[] info;

        this(string info) {
                AdditionalInfo.info ~= info;
        }

        ~this() {
                AdditionalInfo.info = AdditionalInfo.info[0 .. $ - 1];

        }

        @disable this(this);
}

class AdditionalInfoException : Exception {
        this(string t) {
                import std.string;
                super(t ~ "\n" ~ AdditionalInfo.info.join(" "));
        }
}

void bar() {
        with(AdditionalInfo("zone 1")) {
                with(AdditionalInfo("zone 2")) {
                }

                throw new AdditionalInfoException("info");
        }
}

void main() {
        bar();
}
---

the throw site needs to cooperate with this but you could still try/catch an operation as a whole too and attach the original exception in a new one.

needs a bit more thought but this might work. biggest problem is still being stringly typed, ugh.

with compiler help tho we could possibly attach info on function levels right in the EH metadata, so it looks it up as it does the stack trace generation. but that would be limited to per-function i believe... but eh there's nested functions.

tbh i think op's delegate is a better plan at this point but still my brain is running some concepts.

On 12.08.22 01:50, H. S. Teoh wrote:
> ...
>
> The OP's idea of wrapping throwing code with a function that tacks on
> extra information is a good idea.  Perhaps the use of strings isn't
> ideal, but in principle I like his idea of exceptions acquiring
> higher-level information as it propagates up the call stack.

Thanks for the kind words ... I am still thinking how to put this
into a nicer API ... as it is in my current demo, its not much to type
out so I am happy with that. Actually having read the source of
ifThrown, its amazing how lazy makes wrapping easy!

On 12.08.22 23:05, Christian Köstlin wrote:
> On 12.08.22 01:50, H. S. Teoh wrote:
>> ...
>  >
>> The OP's idea of wrapping throwing code with a function that tacks on
>> extra information is a good idea.  Perhaps the use of strings isn't
>> ideal, but in principle I like his idea of exceptions acquiring
>> higher-level information as it propagates up the call stack.
> 
> Thanks for the kind words ... I am still thinking how to put this
> into a nicer API ... as it is in my current demo, its not much to type
> out so I am happy with that. Actually having read the source of
> ifThrown, its amazing how lazy makes wrapping easy!
One thing that can be done is to templateize the exception handler so
that only exceptions of a certain type are handled.
```d
auto contextWithException(T, E)(lazy scope T expression, Exception delegate(E) handler)
{
    Exception newException;
    try
    {
        return expression();
    }
    catch (E e)
    {
        newException = handler(e);
    }
    throw newException;
}
```

which would enable something like

```d
    return  s
        .readText
        .parseJSON
        .contextWithException((UTFException e) {
            return new Exception("Cannot process UTF-8 in config file%s\n  %s".format(s, e.msg), e);
        })
        .contextWithException((FileException e) {
            return new Exception("Cannot process config file%s\n %s".format(s, e.msg), e);
        });
```

Not sure if that makes it any better though, as I have the feeling that
most exceptions need to be wrapped at least once on their way to the end
user.

kind regards,
Christian

On Friday, 12 August 2022 at 21:41:25 UTC, Christian Köstlin wrote:

> which would enable something like
>
> ```d
>     return  s
>         .readText
>         .parseJSON
>         .contextWithException((UTFException e) {
>             return new Exception("Cannot process UTF-8 in config file%s\n  %s".format(s, e.msg), e);
>         })
>         .contextWithException((FileException e) {
>             return new Exception("Cannot process config file%s\n %s".format(s, e.msg), e);
>         });
> ```

This is not as DRY as it could be. Furthermore I would try implement the error handling completely outside the main execution path, ideally in a wrapper around a the old main function (renamed to main_). This approach becomes problematic if exceptions of the same class can be thrown from two functions of the chain.

Your code is printing e.msg. How to you localize that string?