July 27

On Friday, 26 July 2024 at 16:11:28 UTC, Walter Bright wrote:

>

On 7/26/2024 3:21 AM, Richard (Rikki) Andrew Cattermole wrote:

>

If the MMU is throwing an exception something has gone terribly wrong elsewhere and having a programming language force you to check for null is a reasonable precaution against this.

assert(p != null); // throws an unrecoverable exception

Array bounds errors also throw an unrecoverable exception

From what I've seen, the best argument against adding something to the type system to assist the programmer with handling null correctly where it may occur is that new, expensive analysis and possibly new control-flow statements are needed so that in cases where something nullable has been definitely ruled out of being null, the compiler won't complain anymore about the programmer failing to handle null.

The difference between an array index and a possibly null pointer is that almost all the time almost all the possible values for the array index are invalid, so it's quite obvious that out of bounds access must be accounted for, but also static analysis can't really make sure an index is in bounds. For null, however, it's a single value that's problematic. That a variable has a value different from one known iffy one is rather easy to express in a type system. What helps is that some core language constructs (new) return non-null values, but as soon as the value is created, the type system forgets its non-nullness.

It's not about having defined semantics or undefined behavior, the semantics and behavior are defined. A type system that distinguishes nullable and non-nullable reference types helps programmers keeping track of when they have to handle null and when they don't, and whether a function expects null arguments or not. When all reference types can be null everywhere, but won't be in practice, it teaches programmers not to think of them. But the worst offender are ref parameters. They can be null, too, (or *null if you're pedantic), but practically no function taking a reference expects and handles the case where the reference is null — or even documents that assumption. Even seeing &var is null probably looks to most like an obvious case of a vacuously false expression. The reason why null references are allowed is because a reference can easily become null by accident in @safe code, which means they need to have defined behavior. But that's only because pointers aren't distinguished if they can be null or not. Most references are initialized by obvious non-null expressions, only a pointer dereference is the an exception.

July 26
You're right that ref's can be null, too. C++ says they can't be null, but it's trivial to make one in C++, so a fat lot of good that does.

Let's say we have a linked list. It will have a `next` pointer:

```
struct List {
    int payload;
    List* next;
}
```

A null value is perfectly valid for `next`, as that is how the end is found. At least in my coding, I use null values all the time to signify there is nothing there.

It it wasn't null, some other value would have to be there to signify "not a valid item". If there must be something there, then the value would have to be checked against the "not a valid item" value. What should I then do if it unexpectedly is the "not a valid item"? The only sensible thing is to abort the program, as it's a program bug.

But with null, I don't have to check, because the hardware does it for free.

The only time a null pointer dereference is an actual problem is when running on a machine that does not have memory protection, which are decades in obsolescence.

The real issue with null pointer seg faults is the screen dump of mysterious numbers and letters.

The real biggest mistake in C is the eager decay of arrays to pointers, and the tragedy of C is nobody has any interest in fixing it.
July 27
On Saturday, 27 July 2024 at 01:12:09 UTC, Walter Bright wrote:
> But with null, I don't have to check, because the hardware does it for free.
>
> The only time a null pointer dereference is an actual problem is when running on a machine that does not have memory protection, which are decades in obsolescence.

Seems not to be exactly true…

<https://neugierig.org/software/blog/2022/06/wasm-c++.html>: “In Wasm a null pointer just refers to memory[0] and it is a legal address to read and write from.”
July 27
On Saturday, 27 July 2024 at 01:12:09 UTC, Walter Bright wrote:
> The only time a null pointer dereference is an actual problem is when running on a machine that does not have memory protection, which are decades in obsolescence.

I fail to see how this reply relates to Quirin's post. It's just a fact that programmers accidentally forget to handle the null case *when there is a valid way to handle null*. That is obviously a problem because users don't want their programs to abort unnecessarily.

That valid way can just skipping the code that needs a dereference, or printing an error message, instead of trying to dereference null, depending on the situation. You know this. There is no single solution to always handle null, because it is not a valid address for data (for most programs). Aborting the program is fine, if the programmer was aware that a null dereference might occur. The problem is they often are not aware, even good programmers, because the type system always conflates nullability with pointers and programs are complex.

It would be acceptable if you just said that making D null-safe was too disruptive. It is not acceptable to deny the problem exists.
July 27
On 27/07/2024 1:12 PM, Walter Bright wrote:
> The real issue with null pointer seg faults is the screen dump of mysterious numbers and letters.

No. This isn't a problem at all.

What is a problem is the loss of money.

When you crash you take out many users interactions with your system. If somebody is mid-way through buying something that gives them a chance to decide not to.

It is effectively a Denial of Service (DoS) attack that can occur with the press of the F5 key. It can take out your entire multi-server money source without any defense.

In a lot of cases you do not own the thread, or the process. You can't just set the signal handler and do it at runtime. Audio plugins, game engines, interop with an application VM, kernel drivers, web services, COM (Office suite plugins, Windows services, explorer extensions).

All of these are where D makes its money. It is where the foundation gets its donations from.

If you are using an application VM language, an exception is thrown and you can handle it. It's solved. For a native language IFF you own the process AND thread you can use signal handlers safely to throw an exception (MOSTLY true, signals are very hairy and should be avoided if at all possible).

There are two solutions for a native language that isn't just ignoring it like we do today. Check it at CT and warn/error if you do not handle null, or inject read barriers like we do for bounds checks to cause the runtime exception.

Note: this is solved in C/C++ world. They have such analysis in their compilers: analyzer-null-dereference

https://gcc.gnu.org/onlinedocs/gcc-10.1.0/gcc/Static-Analyzer-Options.html

It only takes one major outage where a business loses money before they consider dumping a D companies solution. No client wants to hear: "We did this in a known unsafe language for this particular error, when a more main stream language has solutions to it and D doesn't."

July 27
On Saturday, 27 July 2024 at 01:12:09 UTC, Walter Bright wrote:
> You're right that ref's can be null, too. C++ says they can't be null, but it's trivial to make one in C++,

You can hardly produce one accidentally although it is trivial to
make one.

But such code is still not C++ as it wasn't in January 2023 [1].

[1] Re: Fixing C's Biggest Mistake
    https://forum.dlang.org/post/lzqfkfsuhoivsbbeuxmy@forum.dlang.org

July 27
On 7/26/2024 8:23 PM, Elias (0xEAB) wrote:
> <https://neugierig.org/software/blog/2022/06/wasm-c++.html>: “In Wasm a null pointer just refers to memory[0] and it is a legal address to read and write from.”

Wow. A big mistake by Wasm.
July 27
It's true that many algorithms depend on a null pointer being a "sentinel", and people sometimes forget to check for it. That means:

1. if they forgot to check for the null special case, then the seg fault tells them where the error is

2. if null was supposed not ever happen, then the seg fault tells where the error is

Dereferencing a null pointer is always a bug.
July 28
On 28/07/2024 6:17 AM, Walter Bright wrote:
> On 7/26/2024 8:23 PM, Elias (0xEAB) wrote:
>> <https://neugierig.org/software/blog/2022/06/wasm-c++.html>: “In Wasm a null pointer just refers to memory[0] and it is a legal address to read and write from.”
> 
> Wow. A big mistake by Wasm.

Turns out that the proposal to rectify this is WIP: https://github.com/WebAssembly/memory-control/blob/main/proposals/memory-control/Overview.md

Although nothing in there regarding the trapping itself, but does mention one use case with protecting null.
July 27
On 7/27/2024 2:30 AM, Richard (Rikki) Andrew Cattermole wrote:
> There are two solutions for a native language that isn't just ignoring it like we do today. Check it at CT and warn/error if you do not handle null, or inject read barriers like we do for bounds checks to cause the runtime exception.

How is `assert(p != null)` better?


> Note: this is solved in C/C++ world. They have such analysis in their compilers: analyzer-null-dereference
> 
> https://gcc.gnu.org/onlinedocs/gcc-10.1.0/gcc/Static-Analyzer-Options.html

-Wanalyzer-possible-null-argument

That's not solving it.

Many simple cases can indeed be found by DFA. But not all. There's also a lot of DFA trying to eliminate array bounds errors, but such errors remain the #1 security problem with C and C++.

D does insert checks for array bounds overflow, because the hardware is of no help.


> It only takes one major outage where a business loses money before they consider dumping a D companies solution. No client wants to hear: "We did this in a known unsafe language for this particular error, when a more main stream language has solutions to it and D doesn't."

How is this different from `assert(p != null)` ?