June 04, 2022

On Saturday, 4 June 2022 at 16:55:50 UTC, Sebastiaan Koppe wrote:

>

The reasoning is simple: Error + nothrow will sidestep any RAII you may have. Since you cannot know what potentially wasn't destructed, the only safe course of action is to abandon ship.

Why can't Error unwind the stack properly?

>

Yes, in plenty of cases that is completely overkill.

Then again, programs should be written to not assert in the first place.

In a not-miniscule service you can be pretty certain that some ±1 bugs will be there, especially in a service that is receiving new features on a regular basis. So, if you get an index/key error/null-dereferencing that wasn't checked for, unwinding that actor/task/handler makes sense, shutting down the service doesn't make sense.

If you allow the whole service to go down then you have opened a Denial-of-Service vector, which is a problem if the service is attracting attention from teens/immature adults. (E.g. games, social apps, political sites, educational sites etc).

>

Considering most asserts I have seen are either due to a bad api or just laziness - and shouldn't have to exist in the first place - maybe it's not that bad.

Well, problem is if a usually reliable subsystem is intermittently flaky, and you get this behaviour, then that isn't something you can assume will be caught in tests (you cannot test for all scenarios, only the likely ones).

I am not a fan of Go, but it is difficult to find a more balanced solution, and Go 1.18 has generics, so it is becoming more competitive!

At the end of the day you don't have to love a language to choose it… and for a service, runtime behaviour is more important than other issues.

June 04, 2022

On Saturday, 4 June 2022 at 17:17:13 UTC, Ola Fosheim Grøstad wrote:

>

Why can't Error unwind the stack properly?

It does normally, but it doesn't destruct objects when those are in nothrow functions.

Nothrow functions don't throw, so have no cleanup.

You could argue it is strange that assert throws...

>

In a not-miniscule service you can be pretty certain that some ±1 bugs will be there, especially in a service that is receiving new features on a regular basis.

Most wont throw a Error though. And typical services have canary releases and rollback.

So you just fix it, which you have to do anyway.

Not saying its perfect, but if you only use asserts when you have to, and handle other things using the type system, it doesn't actually happen all that often.

June 04, 2022

On Saturday, 4 June 2022 at 18:32:48 UTC, Sebastiaan Koppe wrote:

>

Most wont throw a Error though. And typical services have canary releases and rollback.

So you just fix it, which you have to do anyway.

I take it you mean manual rollback, but the key issue is that you want to retry on failure. Not infrequently the source for the failure will be in the environment, the code just didn't handle the failure correctly.

On a service with SLA of 99.999% the probable "failure time" would be 6 seconds per week, so if you can retry you may still run fine even if you failed to check correctly for an error on that specific subsystem. That makes the system more resilient/robust.

June 04, 2022

On 6/4/22 2:46 PM, Ola Fosheim Grøstad wrote:

>

On Saturday, 4 June 2022 at 18:32:48 UTC, Sebastiaan Koppe wrote:

>

Most wont throw a Error though. And typical services have canary releases and rollback.

So you just fix it, which you have to do anyway.

I take it you mean manual rollback, but the key issue is that you want to retry on failure. Not infrequently the source for the failure will be in the environment, the code just didn't handle the failure correctly.

You shouldn't retry on Error, and you shouldn't actually have any Errors thrown.

I'll draw a line in the sand here -- OutOfMemoryError shouldn't be an Error, but an Exception. Because there's no way you can check if an allocation will succeed before doing it, and arguably, there are ways to deal with out of memory problems without shutting down the process.

>

On a service with SLA of 99.999% the probable "failure time" would be 6 seconds per week, so if you can retry you may still run fine even if you failed to check correctly for an error on that specific subsystem. That makes the system more resilient/robust.

Exceptions are perfectly fine to catch and retry. Anticipating the failing condition, and throwing an exception instead is a viable solution.

-Steve

June 04, 2022

On Saturday, 4 June 2022 at 14:05:14 UTC, Paul Backus wrote:
[...]

> >

What does that mean? Am I Error blind?

Generally you do not need to subclass Error yourself. The most common way of throwing an Error in user code is to use assert, which (with default compiler flags) throws an AssertError on failure. Function contracts and struct/class invariants work the same way.

git grep -Enw 'assert|unittest' reveals that my code contains assert statements only in unittests. Someone (was it Walter?) once pointed out that asserts are ignored in relase mode (like C assert) and that for my purposes enforce (i.e. throw an Exception) is best suited.

> >
  1. Can you provide some piece of code which must throw Error and cannot
    throw an appropriate Exception?

This is entirely a question of API design. If it should be the caller's responsibility to check for some condition before calling the function, then you can throw an Error when that condition does not hold (or more likely, use an assert or an in contract to check for it).

Provided one does not catch Errors this means one has to isolate such an API design by using a subprocess. This is what one usually tries to avoid.

>

If it should be the callee's responsibility to check, you should throw an Exception (or use enforce).

If the library always throws exceptions it can be used in both API "designs". In the case that the implementor of the caller expects Errors instead of Exceptions she could use a small wrapper which catches the Exceptions and rethrows them as Errors. Likewise for error codes.

Using contracts and invariants impedes this approach.

June 04, 2022

On Saturday, 4 June 2022 at 22:01:57 UTC, Steven Schveighoffer wrote:

>

You shouldn't retry on Error, and you shouldn't actually have any Errors thrown.

So what do you have to do to avoid having Errors thrown? How do you make your task/handler fault tolerant in 100% @safe code?

June 04, 2022

On Saturday, 4 June 2022 at 16:55:31 UTC, Steven Schveighoffer wrote:
[...]

>

The point of an Error is that your code can assume it cannot happen. If it does happen, the code is invalid.

According to my favorite dictionary "assume" means "take for granted" [1]. If Errors may happen how can code (or its author) "assume" that Errors cannot happen?

That makes absolutely no sense to me.

>

This is reflected in the fact that the compiler will omit cleanup code if an Error is thrown (it can assume that it will never happen).

But instead the compiler should emit the cleanup code and we would not have to discuss here carefully avoiding to name the root cause of all this entanglements.

>

The point of using Error is for a last resort check for program correctness (because you failed to validate the input before getting to that point).

If the code threw an Exception instead of an Error everything would be fine.

>

[...] I actually replaced some arrays with an
Exception throwing wrapper because I didn't want to crash the whole server for certain cases, and I didn't want to continuously validate array indexes.

+1

>

[...]
A great example are range functions. Often times you see at the beginning of any popFront method the statement assert(!empty);. This is universally accepted, as you shouldn't be calling popFront if you haven't checked for empty.

Yep.

core.exception.AssertError@[...]linux/bin64/../../src/phobos/std/range/primitives.d(2280): Attempting to popFront() past the end of an array of int

I see no difference to the potentially invalid array index case. It would ease the use of the range if it threw an Exception.

[1] https://www.thefreedictionary.com/assume

June 05, 2022

On Saturday, 4 June 2022 at 22:31:38 UTC, Ola Fosheim Grøstad wrote:

>

So what do you have to do to avoid having Errors thrown? How do you make your task/handler fault tolerant in 100% @safe code?

Run it in a separate process with minimum shared memory.

June 04, 2022
On 6/4/22 10:17, Ola Fosheim Grøstad wrote:

> Why can't Error unwind the stack properly?

Errors are thrown when the program is discovered to be in an invalid state. We don't know what happened and when. For example, we don't know whether the memory has been overwritten by some rogue code. Or perhaps a bit got flipped in memory.

When the state of the program is discovered to be outside of what we think is normal, we cannot execute further code. It would be madness to wish that some cleanup code would do the right thing when e.g. we were sure that an array would never be empty but we found it to be empty.

What happened? What can we assume. We don't know and we cannot assume any state.

(As has been stated many times in this thread and elsewhere, Exceptions are different. They don't have anything to do with invariants.)

> In a not-miniscule service you can be pretty certain that some ±1 bugs
> will be there, especially in a service that is receiving new features on
> a regular basis. So, if you get an index/key error/null-dereferencing
> that wasn't checked for, unwinding that actor/task/handler makes sense,
> shutting down the service doesn't make sense.

Is the service in a usable state?

> If you allow the whole service to go down then you have opened a
> Denial-of-Service vector

Possibly. Not shutting down might produce incorrect results. Do we prefer up but incorrect or dead?

> I am not a fan of Go, but it is difficult to find a more balanced
> solution, and Go 1.18 has generics, so it is becoming more competitive!

I hope there is a way of aborting the program when there are invariant violations discovered.

Ali

June 04, 2022

On 6/4/22 6:56 PM, kdevel wrote:

>

On Saturday, 4 June 2022 at 16:55:31 UTC, Steven Schveighoffer wrote:
[...]

>

The point of an Error is that your code can assume it cannot happen. If it does happen, the code is invalid.

According to my favorite dictionary "assume" means "take for granted" [1]. If Errors may happen how can code (or its author) "assume" that Errors cannot happen?

You don't assume it, you guarantee it. You are expected to provide a guarantee to the compiler that your code won't throw these errors.

But you aren't perfect, and so maybe you make a mistake, and trigger an Error. The compiler handles this unexpected condition by unwinding the stack back to the main function, printing the error and exiting, so you can go fix whatever mistake you made.

It's kind of like a segfault. There's no valid reason to read memory you don't own (yes, I know you can use segfaults to trigger loading of memory, I'm not talking about that kind of segfault). So what do you do when an unexpected segfault happens? You crash the program, and exit. In this case, the language is giving you by default a hint about where it occurred, and if you desire, you can get more information by catching the error where you want and doing more checks, etc.

> >

This is reflected in the fact that the compiler will omit cleanup code if an Error is thrown (it can assume that it will never happen).

But instead the compiler should emit the cleanup code and we would not have to discuss here carefully avoiding to name the root cause of all this entanglements.

A compiler could do this, and in fact, the compiler used to do this. But I think you are still not supposed to continue execution. I'm not sure what a compiler might assume at this point, and I unfortunately can't find in the language specification where it states this. It might not be in there at all, the spec is sometimes lacking compared to the implementation.

However, I did ask Walter about this last beerconf, and he said to treat a throw/catch of an error like a goto, anything can happen.

> >

The point of using Error is for a last resort check for program correctness (because you failed to validate the input before getting to that point).

If the code threw an Exception instead of an Error everything would be fine.

I think the point of Errors is that you can remove them for efficiency. In other words, just like asserts or bounds checks, they are not expected to be part of the normal working program. Exceptions are part of the program, and provide a different mechanism of handling error conditions.

> >

[...]
A great example are range functions. Often times you see at the beginning of any popFront method the statement assert(!empty);. This is universally accepted, as you shouldn't be calling popFront if you haven't checked for empty.

Yep.

core.exception.AssertError@[...]linux/bin64/../../src/phobos/std/range/primitives.d(2280): Attempting to popFront() past the end of an array of int

I see no difference to the potentially invalid array index case. It would ease the use of the range if it threw an Exception.

But it's possible to turn off asserts and make the code run faster. I personally never turn them off on certain programs (web server) because the penalty is not noticeable enough. But if these were Exceptions, they could not be turned off.

Consider the normal flow of a range in a foreach loop, it's:

// foreach(elem; range)
for(auto r = range; !r.empty; r.popFront) {
    auto elem = r.front;
}

If both popFront and front also always call empty you are calling empty 3 times per loop, with an identical value for the 2nd and 3rd calls. Having the assert allows diagnosing invalid programs without crashing your program, but also allowing full performance when you want it.

Phobos' RedBlackTree has an invariant which walks the entire RBT and validates the red-black property holds before and after every method call. This is not what you would want for performant code as it completely destroys the complexity guarantees. Yet it's there to help diagnose problems with RBT if you are working on modifying it. These kinds of checks are to help the developer prove their code is correct without having to continually prove it's correct for normal use.

-Steve