June 05, 2022
On Sunday, 5 June 2022 at 00:40:26 UTC, Ali Çehreli wrote:
[...]
> Errors are thrown when the program is discovered to be in an invalid state.

The following program throws an `Error` in popFront:

   import std.range;

   void main ()
   {
      int [1] a;
      auto q = a[1..$]; // line 6
      q.popFront;       // line 7, throws core.exception.AssertError
   }

When the program counter (PC) is at line 6 the program is in a valid state.

At no time the program is in an invalid state and it would not pass into an invalid state if popFront threw an `Exception` instead of an `Error`.
June 05, 2022

On Sunday, 5 June 2022 at 01:43:06 UTC, Steven Schveighoffer wrote:

[...]

>

But you aren't perfect, and so maybe you make a mistake, and trigger an Error. The compiler handles this unexpected condition by unwinding the stack back to the main function, printing the error and exiting, so you can go fix whatever mistake you made.

For this purpose nobody needs a separate subclass named Error. That works with Exceptions.

[...]

> >

If the code threw an Exception instead of an Error everything would be fine.

I think the point of Errors is that you can remove them for efficiency.

elephant/room.

>

In other words, just like asserts or bounds checks, they are not expected to be part of the normal working program.

"removing [Errors, asserts, bounds checks] is a bit like wearing a life-jacket to practice in the harbour, but then leaving the life-jackets behind when your ship leaves for open ocean" [1]

[...]

>

Consider the normal flow of a range in a foreach loop, it's:

// foreach(elem; range)
for(auto r = range; !r.empty; r.popFront) {
    auto elem = r.front;
}

If both popFront and front also always call empty you are calling empty 3 times per loop, with an identical value for the 2nd and 3rd calls.

Solution: Implement explicitly unchecked popFront() and front() versions.

>

Having the assert allows diagnosing invalid programs without crashing your program,

That depends on the understanding of "crashing a program". If library code throws an Error instead of an Exception I have to isolate that code in a subprocess in order to make my program gracefully handle the error condition.

Think of CGI processes which provide output direct to a customer. If there is an assert the customer will see the famous Internal Server Error message (in case of apache httpd).

[...]

[1] http://wiki.c2.com/?AssertionsAsDefensiveProgramming

June 05, 2022

On Sunday, 5 June 2022 at 07:21:18 UTC, Ola Fosheim Grøstad wrote:
[...]

>

The reality is that software is layered. Faults at different layers should have different consequences at the discretion of a capable programmer.

+1

June 05, 2022
On 6/4/22 23:31, Ola Fosheim Grøstad wrote:
> On Sunday, 5 June 2022 at 00:40:26 UTC, Ali Çehreli wrote:
>> Errors are thrown when the program is discovered to be in an invalid
>> state. We don't know what happened and when. For example, we don't
>> know whether the memory has been overwritten by some rogue code.
>
> That is not very probable in 100% @safe code. You are basically saying
> that D cannot compete with Go and other «safe» languages.

I did not mean that. I think we have a misunderstanding at a fundamental level.

> Dereferencing
> a null pointer usually means that some code failed to create an instance
> and check for it.

Dereferencing a null pointer does not throw Error but fine. I agree.

>> What happened? What can we assume. We don't know and we cannot assume
>> any state.
>
> So D will never be able to provide actors and provide fault tolerance.

Let me show an example. Here is a piece of code that could be running in an actor:

struct S {
  int[] a;
  int[] b;

  void add(int i) {    // <-- Both arrays always same size
    a ~= i;
    b ~= i * 10;
  }

  void foo() {
    assert(a.length == b.length);  // <-- Invariant check
    // ...
  }
}

void main() {
  auto s = S();
  s.add(42);
  s.foo();
}

The code is written in a way that both arrays will *always* have equal number of elements. And there is a "silly" check for that invariant. So far so good. The issue is what to do *when* that assert fails.

Are you sure that it was a silly programmer mistake? I am not sure at all.

>> Is the service in a usable state?
>
> Yes, the actor code failed. The actor code change frequently, not the
> runtime kernel.

Is the only other culprit the runtime kernel? I really don't know who else may be involved.

>> Possibly. Not shutting down might produce incorrect results. Do we
>> prefer up but incorrect or dead?
>
> I prefer that service keeps running: chat service, game service, data
> delivered with hashed «checksum». Not all software are database engines
> where you have to pessimize about bugs in the runtime kernel.

There are also bugs in unrelated actor code writing over each others' memory.

It is possible that the service will do unexpected or very wrong things. But you answer my question: Your game server can do weird things. Hopefully all acceptable by paying customers.

> If the data delivered is good enough for the client and better than
> nothing then the service should keep running!!!

Yes! How can you be sure data is good when the silly assertion above failed. How could that happen? Is there any logical way to describe it was actor code's mistake? I don't think so. Let's assume the commented-out parts do not touch the arrays at all.

You are free to choose to catch Errors and continue under the assumption that it is safe to do so. The advice in the article still holds for me. I think the main difference is in the assumptions we make about an Errors: Is it a logic error in actor code or some crazy state that will cause weirder results if we continue. We can't know for sure.

>> I hope there is a way of aborting the program when there are invariant
>
> Invariants are USUALLY local. I dont write global spaghetti code. As a
> programmer you should be able to distinguish between local and global
> failure.
>
> You are assuming that the programmer is incapable of making judgements.
> That is assuming way too much.

I resent causing that misunderstanding. I apologize.

The only assumption I make about the programmer is that they do not mix Error and Exception so that in the end an Error points at a situation that warrants aborting the mission. Hm... Thinking more about it, assuming that an Error is due to a local programmer error would be a judgment.

Ali

June 05, 2022
On 6/5/22 04:43, kdevel wrote:
> On Sunday, 5 June 2022 at 00:40:26 UTC, Ali Çehreli wrote:
> [...]
>> Errors are thrown when the program is discovered to be in an invalid
>> state.
>
> The following program throws an `Error` in popFront:
>
>     import std.range;
>
>     void main ()
>     {
>        int [1] a;
>        auto q = a[1..$]; // line 6
>        q.popFront;       // line 7, throws core.exception.AssertError
>     }
>
> When the program counter (PC) is at line 6 the program is in a valid state.
>
> At no time the program is in an invalid state and it would not pass into
> an invalid state if popFront threw an `Exception` instead of an `Error`.

That looks like an argument for changing the behavior of D runtime when an out-of-bounds access occurs. Of course, it is too late to change that at this time but perhaps there can be a compiler switch like -boundstype=[error|exception].

The programmer has many options to prove that nothing crazy is happening. A common example suitable for many cases:

  enforce(!q.empty); // (Throws Exception)
  q.popFront;

Ali

June 05, 2022
On Sunday, 5 June 2022 at 14:24:39 UTC, Ali Çehreli wrote:
[...]
> struct S {
>   int[] a;
>   int[] b;
>
>   void add(int i) {    // <-- Both arrays always same size
>     a ~= i;
>     b ~= i * 10;
>   }
>
>   void foo() {
>     assert(a.length == b.length);  // <-- Invariant check
>     // ...
>   }
> }
>
> void main() {
>   auto s = S();
>   s.add(42);
>   s.foo();
> }
>
> The code is written in a way that both arrays will *always* have equal number of elements.

I think this is what Sean Parent called "incidental data structure" [1]. I would refactor the code:

struct T {
   int a;
   int b;
}

struct S {
   T [] t;

  void add(int i) {
    t ~= T (i, i * 10);
  }

  void foo() {
                // Look Ma no assert!
    // ...

  }
}

void main() {
  auto s = S();
  s.add(42);
  s.foo();
  s.a ~= 1; // does not compile
  s.foo();
}
```
[1] <https://www.google.de/search?q=incidental+data+structures>
June 05, 2022
On Sunday, 5 June 2022 at 15:07:13 UTC, kdevel wrote:
> ... I would refactor the code:

I really liked this one. The way it solves and at same time restrict the "external access" with that struct of (a,b) makes the code easier to maintain too.

Glad I keep lurking around this forum.

Matheus.
June 05, 2022

Ok, so I am a bit confused about what is Error and what is not… According to core.exception there is wide array of runtime Errors:

RangeError
ArrayIndexError
ArraySliceError
AssertError
FinalizeError
OutOfMemoryError
InvalidMemoryOperationError
ForkError
SwitchError

I am not sure that any of these should terminate anything outside the offending actor, but I could be wrong as it is hard to tell exactly when some of those are thrown.

InvalidMemoryOperationError sound bad, of course, but the docs says «An invalid memory operation error occurs in circumstances when the garbage collector has detected an operation it cannot reliably handle. The default D GC is not re-entrant, so this can happen due to allocations done from within finalizers called during a garbage collection cycle.»

This sounds more like an issue that needs fixing…

On Sunday, 5 June 2022 at 14:24:39 UTC, Ali Çehreli wrote:

>

The code is written in a way that both arrays will always have equal number of elements. And there is a "silly" check for that invariant. So far so good. The issue is what to do when that assert fails.

Are you sure that it was a silly programmer mistake? I am not sure at all.

Ok, so this is a question of the layers:

-------- top layer ------
D          | E
           |
----- middle layer ------
B          | C
           |
           |
---- bottom layer -------
A

If the failed assert is happening in a lower layer A then code on the outer layer should fault (or roll back a transaction). Whether it is reasonable to capture that error and suppress it depends on how independent you want those layers to be in your architecture. It also depends on the nature of layer A.

If the failed assert happens in middle layer section B, then the D would be affected, but not A, C or E.

The key issue is that the nature of layers is informal in the language (in fact, in most languages, a weakness), so only the programmer can tell what is reasonable or not.

In fact, when we think about it; most aspects about what is expected from a program is informal… so it is very difficult to make judgment at the compiler level.

>

Is the only other culprit the runtime kernel? I really don't know who else may be involved.

I mean the application's «custom actor kernel», a hardened piece of code that is not modified frequently and heavily tested. The goal is to keep uncertainty local to an actor so that you can toss out misbehaving actors and keep the rest of the system working smoothly (99.5% of the time, 50 minutes downtime per week).

Actors are expected to contain bugs because the game system is continuously modified (to balance the game play, to get new content, more excitement, whatever…). This is why we want 100% @safe code as a feature.

>

There are also bugs in unrelated actor code writing over each others' memory.

But that cannot happen if I decide to make actors 100% @safe and only let them interact with each other through my «custom actor kernel»?

>

You are free to choose to catch Errors and continue under the assumption that it is safe to do so.

Now I am confused!! Steven wrote «I've thought in the past that throwing an error really should not throw, but log the error (including the call stack), and then exit without even attempting to unwind the stack.»

Surely, the perspective being promoted is to make sure that Errors cannot be stopped from propagating? That is why this becomes a discussion?

If an Error can propagate through "nothrow" then the compiler should emit handler code for it and issue a warning. If you don't want that then the programmer should safe guard against it, meaning: manually catch and abort or do manual checks in all locations above it where Errors can arise. The compiler knows where.

Not really sure why D has "nothrow", it doesn't really fit with the rest of the language? To interface with C++ perhaps?? If that is the case, just adopt C++ "noexcept" semantics, use assert() for debugging only, in "nothrow" code. And discourage the use of "nothrow". Heck, throw in a compiler switch to turn off "nothrow" if that is safe.

>

The advice in the article still holds for me. I think the main difference is in the assumptions we make about an Errors: Is it a logic error in actor code or some crazy state that will cause weirder results if we continue. We can't know for sure.

And this is no different from other languages with a runtime. You cannot be sure, but it probably isn't a runtime issue, and even if it was… players will be more upset by not being able to play than to have some weird effects happening. Same for chat service. Same for being able to read Wikipedia-caches (worse to have no access than to have 1% of pages garbled on display until the server is updated).

Different settings need different solutions. So, maybe interfacing with C++ requires "nothrow", but if that is the only reason… why let everybody pay a price for it?

(I use "noexcept" in my C++ code, but that is more an act of documentation, and that clearly falls apart in D with Error.)

June 05, 2022
On 6/5/22 08:07, kdevel wrote:
> On Sunday, 5 June 2022 at 14:24:39 UTC, Ali Çehreli wrote:
> [...]
>> struct S {
>>   int[] a;
>>   int[] b;
>>
>>   void add(int i) {    // <-- Both arrays always same size
>>     a ~= i;
>>     b ~= i * 10;
>>   }
>>
>>   void foo() {
>>     assert(a.length == b.length);  // <-- Invariant check
>>     // ...
>>   }
>> }
>>
>> void main() {
>>   auto s = S();
>>   s.add(42);
>>   s.foo();
>> }
>>
>> The code is written in a way that both arrays will *always* have equal
>> number of elements.
>
> I think this is what Sean Parent called "incidental data structure" [1].

Like many other programmers who include me, Sean Parent may be right.[1]

Other than being a trivial example to make a point, the code I've shown may be taking advantage of the "structure of array" optimization. I am sure Sean Parent knows that as well.

> I would refactor the code:

Most likely me too.

> struct T {
>     int a;
>     int b;
> }
>
> struct S {
>     T [] t;
>
>    void add(int i) {
>      t ~= T (i, i * 10);
>    }
>
>    void foo() {
>                  // Look Ma no assert!

The assert may have been moved to another place:

struct T {
    int a;
    int b;

  invariant() {
    // Look Pa it's still here!
    assert(b == a * 10);
  }
}

Ali

[1] I stopped following Sean Parent when he dismissed D and me by waving his hand behind and walking away: "A language with reference types? No, thanks." That happened at the end of one of his presentations here at the Silicon Valley C++, which I happened to co-host. I am sure he is observant enough to one day realize that C++ has reference types by convention. (I recently posted links to C++ core guidelines proving that point of mine, one of which is something to the effect of "never pass polymorphic types by-value".)

June 05, 2022

On Sunday, 5 June 2022 at 14:24:39 UTC, Ali Çehreli wrote:

>

void add(int i) { // <-- Both arrays always same size
a ~= i;
b ~= i * 10;
}

void foo() {
assert(a.length == b.length); // <-- Invariant check
// ...
}

Maybe it would help if we can agree that this assert ought to be statically proven to hold and the assert would therefore never be evaluated in running code. Emitting asserts is just a sign of failed statical analysis (which is common, but that is the most sensible interpretation from a verification viewpoint).

The purpose of asserts is not to test the environment. The purpose is to "prove" that the specified invariant of the function holds for all legal input.

It follows that the goal of an assert is not to test if the program is in a legal state!

I understand why you say this, but if this was the case then we could not remove any asserts by statical analysis. :-/