November 25, 2022

On Friday, 25 November 2022 at 08:12:41 UTC, Dukc wrote:

>

What does "best-effort" mean?

Generally, it means that a procedure is not a complete or ideal solution and won’t always give you what you need (even if it theoretically could), but still covers common and simple cases and maybe some of the uncommon or complicated ones. Usually, best-effort solutions are subject to incremental improvements.

A simple example for best effort is compiler error messages: Usually, they’re pointing you to the spot where you change something and it works. Sometimes, the error is nowhere the error message indicates.

Another example well-known to many D programmers that is both, best-effort and ideal, is @safe. It intends to be a complete (or ideal) solution in the sense that when checking for @safe succeeds, the code has no UB (give or take @trusted, of course). A compiler that accepts UB in @safe code has a bug!
On the other hand, @safe is best-effort in the sense that it does not even intend to cover all non-UB code. There will always be (rather simple) code for which you can (rather easily) prove that it is indeed free of UB, but the compiler’s @safe checks reject it. The language development goes into the direction of recognizing more and more of non-UB code as @safe (DIP 1000 most notably), but it will never be able to cover the entirety of non-UB because that would equate to solving the halting problem, which is provably impossible. Because it ideally would be, but mathematically can’t, and even in cases where there is a proof showing no UB, it’s not intended the compiler finds it, because practically, it’s not worth it, it is a best-effort solution.

I don’t think “best-effort” is used here with any more special meaning.

November 25, 2022
On 11/25/22 00:12, Dukc wrote:

>> Discussion: As I commented on the bug report as well, D is already in
>> the best-effort business when it comes to Errors:
>
> What does "best-effort" mean?

(I am not sure whether I should have used the word "hopeful" here. I will continue with best-effort.)

According to common assert wisdom, the program must not continue once an assertion has failed because the program is in invalid state.

However, the main thread of a D program does execute code to display the backtrace to help the programmer discover why the assertion failed. By the wisdom above, this is best-effort because if the program is in an invalid state, the code that attempts to display the backtrace can e.g. delete files.

I had discovered that fact when I first started catching Errors in non-main threads to do exactly what the main thread does. I used to feel guilty about that but now I know it's all best-effort anyway: it may or may not work.

Add to the above another observation which I had after Patrick Schluter's message: Even the execution of the assert check is best-effort because if the check will fail, the program has been in invalid state for an unknow amount of time anyway.

That logic can be carried all the way down to hardware and in the end most computing is best-effort anyway.

Given all that, D's assert behavior can do anything it wants so I still agree with the code above but I think it can be improved by caching Throwable in message generation because the programmer would still be interested in the original failed assert:

  catch (Throwable exc)

Ali

November 25, 2022

On Friday, 25 November 2022 at 09:33:21 UTC, Quirin Schroll wrote:

>

Another example well-known to many D programmers that is both, best-effort and ideal, is @safe. It intends to be a complete (or ideal) solution in the sense that when checking for @safe succeeds, the code has no UB (give or take @trusted, of course). A compiler that accepts UB in @safe code has a bug!

A false positive.

>

On the other hand, @safe is best-effort in the sense that it does not even intend to cover all non-UB code. There will always be (rather simple) code for which you can (rather easily) prove that it is indeed free of UB, but the compiler’s @safe checks reject it.

A false negative.

>

The language development goes into the direction of recognizing more and more of non-UB code as @safe (DIP 1000 most notably), but it will never be able to cover the entirety of non-UB because that would equate to

Make @safe a no-op, then there will be no more false negatives. Mission accomplished. I mean: What is the objective of @safe anyhow? I stumble already at the first sentence of the manual:

"Safe functions are marked with the @safe attribute."

Is this an imperative or a descriptive sentence?

November 25, 2022
On Friday, 25 November 2022 at 12:29:26 UTC, Ali Çehreli wrote:

> According to common assert wisdom, the program must not continue once an assertion has failed because the program is in invalid state.

Whatever "invalid state" means. I could not find a definition in the manual. Usually a system is said to be in some state s of a set of possible states S. Now one can devise a proper subset I ⊊ S and say "the system is in an invalid state" iff s ∈ I. That is a precise definition but is still meaningless.

> However, the main thread of a D program does execute code to display the backtrace to help the programmer discover why the assertion failed. By the wisdom above, this is best-effort because if the program is in an invalid state, the code that attempts to display the backtrace can e.g. delete files.

Pardon! What?

> I had discovered that fact when I first started catching Errors in non-main threads to do exactly what the main thread does. I used to feel guilty about that but now I know it's all best-effort anyway: it may or may not work.
>
> Add to the above another observation which I had after Patrick Schluter's message: Even the execution of the assert check is best-effort because if the check will fail, the program has been in invalid state for an unknow amount of time anyway.

int main ()
{
   int a = 1;
   try
      assert (a == 0);
   catch (Throwable t)
      {}
   return 0;
}

Which line makes the program enter the "invalid state"? Additional question: What constitutes that "invalid state"?


November 25, 2022
On 11/25/22 05:32, kdevel wrote:
> On Friday, 25 November 2022 at 12:29:26 UTC, Ali Çehreli wrote:
>
>> According to common assert wisdom, the program must not continue once
>> an assertion has failed because the program is in invalid state.
>
> Whatever "invalid state" means. I could not find a definition in the
> manual.

The assertion check defines the legal state. Its failure is the invalid state:

    int a = 42;
    foo(&a);
    assert(a == 42);

The programmer defined the valid state of the program as being "when I pass 'a' by reference to 'foo', its value shall remain 42."

If that assertion fails, the program is in an invalid state.

>> However, the main thread of a D program does execute code to display
>> the backtrace to help the programmer discover why the assertion
>> failed. By the wisdom above, this is best-effort because if the
>> program is in an invalid state, the code that attempts to display the
>> backtrace can e.g. delete files.
>
> Pardon! What?

Assume the assertion failure was an indication of a catastrophic event, which overwrote a function pointer that will end up executing unrelated code that deletes files.

>> I had discovered that fact when I first started catching Errors in
>> non-main threads to do exactly what the main thread does. I used to
>> feel guilty about that but now I know it's all best-effort anyway: it
>> may or may not work.
>>
>> Add to the above another observation which I had after Patrick
>> Schluter's message: Even the execution of the assert check is
>> best-effort because if the check will fail, the program has been in
>> invalid state for an unknow amount of time anyway.
>
> int main ()
> {
>     int a = 1;
>     try
>        assert (a == 0);
>     catch (Throwable t)
>        {}
>     return 0;
> }
>
> Which line makes the program enter the "invalid state"?

Although that looks like a bug in the programmer's mind, I can still imagine the programmer expected some code perhaps running in a thread started e.g. in a 'static this' block was expected to alter the stack. It must be in an operating system that guarantees that other thread is executed so frequently that even after assigning 1 to a, it is expected to be observed as 0.

There: The programmer knew things we don't see.

If that was the entirety of the program, then I think the programmer is wrong.

> Additional
> question: What constitutes that "invalid state"?

I have to trust the assertion check, which may be buggy itself.

Ali

November 25, 2022
On Friday, 25 November 2022 at 14:38:57 UTC, Ali Çehreli wrote:
> On 11/25/22 05:32, kdevel wrote:

> If that assertion fails, the program is in an invalid state.

[reordering ...]

> > int main ()
> > {
> >     int a = 1;
> >     try
> >        assert (a == 0);
> >     catch (Throwable t)
> >        {}
> >     return 0;
> > }
> >
> > Which line makes the program enter the "invalid state"?

Is it in the "invalid state" right after the condition has been evaluated or not before the AssertError has been thrown? What if the program is compiled with -release? Is it still in "invalid state"?

> Although that looks like a bug in the programmer's mind, I can still imagine the programmer expected some code perhaps running in a thread started e.g. in a 'static this' block was expected to alter the stack.

The fabricated nine lines of code above is all I have. No static this block. It is meant as it is.

November 25, 2022
On 11/25/22 07:33, kdevel wrote:
> On Friday, 25 November 2022 at 14:38:57 UTC, Ali Çehreli wrote:
>> On 11/25/22 05:32, kdevel wrote:
>
>> If that assertion fails, the program is in an invalid state.
>
> [reordering ...]
>
>> > int main ()
>> > {
>> >     int a = 1;
>> >     try
>> >        assert (a == 0);
>> >     catch (Throwable t)
>> >        {}
>> >     return 0;
>> > }
>> >
>> > Which line makes the program enter the "invalid state"?

I am not an expert here; just philosophizing.

[reordering...] :)

>> Although that looks like a bug in the programmer's mind, I can still
>> imagine the programmer expected some code perhaps running in a thread
>> started e.g. in a 'static this' block was expected to alter the stack.
>
> The fabricated nine lines of code above is all I have. No static this
> block. It is meant as it is.
>

Your example proves that we all assume that the assertion condition is correct. I don't think the example above has a correct condition; so let's focus on correct ones.

> Is it in the "invalid state" right after the condition has been
> evaluated or not before the AssertError has been thrown? What if the
> program is compiled with -release? Is it still in "invalid state"?

I think assert is just a tool that exposes an invalid state. The invalid state may have been reached long ago. That's why I say the whole assert mechanism starting with executing the condition all the way down to printing backtrace or even not doing anything at all is best-effort.

I am fine with that because such is life. But in philosophy, it gives us liberty to do anything we want.

Ali

November 25, 2022
On Friday, 25 November 2022 at 15:48:49 UTC, Ali Çehreli wrote:
> On 11/25/22 07:33, kdevel wrote:
> > On Friday, 25 November 2022 at 14:38:57 UTC, Ali Çehreli
> wrote:
> >> On 11/25/22 05:32, kdevel wrote:
> >
> >> If that assertion fails, the program is in an invalid state.
> >
> > [reordering ...]
> >
> >> > int main ()
> >> > {
> >> >     int a = 1;
> >> >     try
> >> >        assert (a == 0);
> >> >     catch (Throwable t)
> >> >        {}
> >> >     return 0;
> >> > }
> >> >
> >> > Which line makes the program enter the "invalid state"?
>
> I am not an expert here; just philosophizing.

If I tell you "A program is running on a machine. There is an int variable a in memory at adress 0x47110816. The variable has the value -4." you probably can explain to me how and where the negative sign is represented in the machine.

I would like to know if there is a representation of that ominous "invalid state" in terms of bits and bytes. Well, it doesn't have to. But if we are talking about "invalid state" we should be able to explain what we mean when we use that group of words. I don't use the term because I do not intuitively know what it means and the D docs do not define the term either.

> [reordering...] :)
>
> >> Although that looks like a bug in the programmer's mind, I can still
> >> imagine the programmer expected some code perhaps running in a thread
> >> started e.g. in a 'static this' block was expected to alter the stack.
> >
> > The fabricated nine lines of code above is all I have. No static this
> > block. It is meant as it is.
> >
>
> Your example proves that we all assume that the assertion condition is correct.

If you allow I would like to discuss the interpretation of the code a bit more thoroughly.

> I don't think the example above has a correct condition; so let's focus on correct ones.

The condition is false. That is what I want to discuss. My question is: What does that mean? Does it mean more than writing a comment

   // I solemnly swear a is zero.

?

> > Is it in the "invalid state" right after the condition has been
> > evaluated or not before the AssertError has been thrown? What if the
> > program is compiled with -release? Is it still in "invalid state"?
>
> I think assert is just a tool that exposes an invalid state.

In your last post you wrote "If that assertion fails, the program is in an invalid state." The assertion in my program fails. According to the common inference rules the program must be in an invalid state and according to your last statement this invalid state is only exposed, i.e. the program was already in "invalid state" before the assert statment.

Thus my question: What constitutes the invalid state in my example program? The program is syntactically valid, no UB, nothing else.

> The invalid state may have been reached long ago.

In my fabricated example program: Where and by what event went the program into "invalid state"?


November 26, 2022
On 11/25/22 08:48, kdevel wrote:

> I would like to know if there is a representation of that ominous
> "invalid state" in terms of bits and bytes. Well, it doesn't have to.
> But if we are talking about "invalid state" we should be able to explain
> what we mean when we use that group of words. I don't use the term
> because I do not intuitively know what it means and the D docs do not
> define the term either.

I think it is definition and comes from convention:

- Errors represent situations where the program should not continue

- AssertError is an Error, which is thrown when an assert condition fails

For that reason, assert conditions do include checks that define the valid state of a program.

The same condition can be used for throwing an Error or an Exception:

  exists(configFile)

Only the programmer knows which one makes sense:

- If configFile was a string input by the user, then the following is appropriate:

  enforce(exists(configFile), /* ... */)

- If configFile was a string representing a file e.g. that has just been generated by the same program, then the following is appropriate:

  assert(exists(configFile), /* ... */)

Only the programmer knows.

> The condition is false. That is what I want to discuss. My question is:
> What does that mean? Does it mean more than writing a comment
>
>     // I solemnly swear a is zero.

Your example code is interesting because it expects 'a == 0' right after initializing it with 1. However, since that AssertError is caught (against convention), there is no observable remnant of that AssertError in the program. Since it did not represent a valid state anyway, no harm done there. Humans are fallible, so we expect cases where assert conditions do not represent a valid program state. We can discount those cases.

Having said that, that assert means this to me: "For this program to be able continue, the value of 'a' must be 0."

> In your last post you wrote "If that assertion fails, the program is in
> an invalid state." The assertion in my program fails. According to the
> common inference rules the program must be in an invalid state

I trust the programmer makes mistakes. :) I don't think that is a correct assert condition.

> and
> according to your last statement this invalid state is only exposed,
> i.e. the program was already in "invalid state" before the assert statment.

I strongly think so. An assertion failure is a late discovery of a problem.

> Thus my question: What constitutes the invalid state in my example
> program? The program is syntactically valid, no UB, nothing else.
>
>> The invalid state may have been reached long ago.
>
> In my fabricated example program: Where and by what event went the
> program into "invalid state"?

I am repeating myself but I think your program has a programmer error in that condition. Swallowing the thrown AssertError is interesting but I don't think that program has any invalid state.

The way I see it, valid state is defined by the collection of assertion checks, which may be incorrect themselves or conflict with each other. Hopefully, the programmer catches and corrects mistakes in the checks and approaches a more correct definition of the valid state of that program.

Ali

November 27, 2022
On Saturday, 26 November 2022 at 12:32:40 UTC, Ali Çehreli wrote:
> On 11/25/22 08:48, kdevel wrote:
>
>> I would like to know if there is a representation of that ominous
>> "invalid state" in terms of bits and bytes. Well, it doesn't have to.
>> But if we are talking about "invalid state" we should be able to explain
>> what we mean when we use that group of words. I don't use the term
>> because I do not intuitively know what it means and the D docs do not
>> define the term either.
>
> I think it is definition and comes from convention:
>
> - Errors represent situations where the program should not continue

If the programmer who implements a function/method is not the author of `main` he most certainly is not in charge of deciding on the fate of the running program.

[...]

> - If configFile was a string input by the user, then the following is appropriate:
>
>   enforce(exists(configFile), /* ... */)
> 
> - If configFile was a string representing a file e.g. that has just been generated by the same program, then the following is appropriate:
>
>   assert(exists(configFile), /* ... */)

That is a nice example! In both cases I would define an Exception-Class

   ConfigFileNotFound : Exception {
      ...
   }

and let enforce create and throw a corresponding object. For me, it makes no difference if the file name is user input, taken from the command line, or hardcoded into the program. Well, actually I would not code such a check at all. I would let the open function throw the exception (also to avoid TOCTTOU).

The decision if the program can continue without configuration file is probably taken in `main` or in the consumer of that configuration data: If there is a default configuration dataset the program might continue.

[...]

> Having said that, that assert means this to me: "For this program to be able continue, the value of 'a' must be 0."

In order not to convey that meaning in real programms I only use asserts in unittests.

[...]

> I am repeating myself but I think your program has a programmer error in that condition.

There really is none. The failure of the assert expression is on purpose.

> Swallowing the thrown AssertError is interesting but I don't think that program has any invalid state.

Actually you do not extract that information from the code itself.

> The way I see it, valid state is defined by the collection of assertion checks,

But sometimes you don't believe what the code says. If it does not match up with your presuppositions you testify validity though the assert checks fail. But then isn't that invalid state a non-local or even a subjective thing?