April 16

On Wednesday, 16 April 2025 at 19:44:09 UTC, Walter Bright wrote:

>

On 4/16/2025 11:43 AM, Derek Fawcus wrote:

>

However I do have an interest in being able to write code with distinct nullable and nonnull pointers.  That such that the compiler (or an SA tool) can complain when they're incorrectly confused.

That's what templates are for!

I think what people are complaining about isn’t that null pointers exist at all, it’s that every pointer and reference type has a null value instead of you opting into it. The type system fights you if you want to use it to prove or enforce certain values are nullable or not.

Something that people don’t bring up is that clang’s nullability extension allows you to change what the default is. You put a #pragma clang assume_nonnull begin at the top of your C/C++/Objective-C code and you have to annotate only the nullable pointers. Most pointers in a program should be non-null and the nullable ones should be the exception that you have to annotate.

April 16
The focus of safe code in D is preventing memory corruption, not null pointer dereference seg faults or other programming bugs.

A null pointer deference may be a symptom of memory corruption or some logic bug, but it does not cause memory corruption.

D has many aspects that reduce the likelihood of bugs (such as no variable shadowing), but that is not what safe is about.

(Yes I know that if there's a constant offset to a null pointer larger than the guard page, it can cause memory corruption.)
April 16
On 4/16/2025 10:08 AM, Richard (Rikki) Andrew Cattermole wrote:
> We've lately been discussing what to do with Error on Discord, and so far it seems like the discussion is going in the direction of it should either do what assert does and kill the process with a function pointer in the middle to allow configurability, or throw what I've dubbed a framework exception.

You can configure what assert() does via a command line switch.

> Manu wanted something like this recently for identical reasons that me and Adam do.

Adam and I discussed this extensively at the last Coffee Haus meeting. I haven't talked with Manu about it.

Anyone can configure assert() to do whatever they want, after all, D is a systems programming language. But if it does something other than exit the process, you're on your own.
April 16
On Wednesday, 16 April 2025 at 19:58:45 UTC, Walter Bright wrote:
> The focus of safe code in D is preventing memory corruption, not null pointer dereference seg faults or other programming bugs.
>
> A null pointer deference may be a symptom of memory corruption or some logic bug, but it does not cause memory corruption.
>
> D has many aspects that reduce the likelihood of bugs (such as no variable shadowing), but that is not what safe is about.
>
> (Yes I know that if there's a constant offset to a null pointer larger than the guard page, it can cause memory corruption.)

The 0 address on WASM is writable, which has burned me many times.
April 16
On 4/16/2025 1:16 PM, Dave P. wrote:
> The 0 address on WASM is writable, which has burned me many times.

How that oversight was made seems incredible.

Anyhow, what you can do is allocate some memory at location 0, and fill it with 0xDEAD_BEEF. Then, periodically, check to see if those values changed.
April 16
On 4/16/2025 12:57 PM, Dave P. wrote:
> You put a `#pragma clang assume_nonnull begin` at the top of your C/C++/Objective-C code and you have to annotate only the nullable pointers. Most pointers in a program should be non-null and the nullable ones should be the exception that you have to annotate.

Annotation means more than one pointer type.

Back in the old MSDOS days, there were 5 pointer types - near, far, stack, code and huge. Dealing with that is a gigantic mess - which pointer type does strlen() take? Or worse, strcpy()?

Microsoft's Managed C++ has two pointer types with different syntax, a GC pointer and a non-GC pointer. The same problem - what pointer type does strcpy() accept?

It's an ugly mess, and why I've avoided any such thing in D.

I'm curious - how does one traverse a binary tree with non-null pointers? How does one create a circular data structure with non-null pointers?
April 17
On 17/04/2025 9:12 AM, Walter Bright wrote:
> On 4/16/2025 12:57 PM, Dave P. wrote:
>> You put a `#pragma clang assume_nonnull begin` at the top of your C/C+ +/Objective-C code and you have to annotate only the nullable pointers. Most pointers in a program should be non-null and the nullable ones should be the exception that you have to annotate.
> 
> Annotation means more than one pointer type.

Annotating the type, and annotating the variable/expression are two different things.

In the DFA literature they have distinct properties and are applied differently.

I read Principles of Program Analysis today, it was very interesting and did have some details on the subject (but not much). It also confirmed some things that I had already come up with independently which was nice!

From what I've seen, application VM languages annotate the type, whereas C++ annotates the variable. As of DIP1000, we annotate the variable i.e. scope.

From a link made previously in this thread, the state of the art annotation of nullability in C++: https://clang.llvm.org/docs/analyzer/developer-docs/nullability.html

Very similar to what I'm wanting.

> Back in the old MSDOS days, there were 5 pointer types - near, far, stack, code and huge. Dealing with that is a gigantic mess - which pointer type does strlen() take? Or worse, strcpy()?
> 
> Microsoft's Managed C++ has two pointer types with different syntax, a GC pointer and a non-GC pointer. The same problem - what pointer type does strcpy() accept?

I genuinely would prefer throwing a full fledged CFG DFA at this kind of thing, and only annotate the variable, not the type.

Its a shame not everyone would accept that as a solution.
It is forcing me to verify that there are no alternative solutions for these people.

I know you and I Walter would be happy with a full CFG DFA as the resolution, but alas.

I remain heavily concerned at the idea of boxing types in D, in any scenario. It seems to spell an absolute mess in any attempts I have modeled mentally.

> It's an ugly mess, and why I've avoided any such thing in D.
> 
> I'm curious - how does one traverse a binary tree with non-null pointers? How does one create a circular data structure with non-null pointers?

Sentinels.

They are used pretty heavily in data structures, such as head and foot nodes.

My recommendation for data structure/algorithm book: https://www.amazon.com/Algorithms-Parts-1-4-Fundamentals-Structures/dp/0201314525

April 16

On Wednesday, 16 April 2025 at 21:12:08 UTC, Walter Bright wrote:

>

On 4/16/2025 12:57 PM, Dave P. wrote:

>

You put a #pragma clang assume_nonnull begin at the top of your C/C++/Objective-C code and you have to annotate only the nullable pointers. Most pointers in a program should be non-null and the nullable ones should be the exception that you have to annotate.

Annotation means more than one pointer type.

Back in the old MSDOS days, there were 5 pointer types - near, far, stack, code and huge. Dealing with that is a gigantic mess - which pointer type does strlen() take? Or worse, strcpy()?

Microsoft's Managed C++ has two pointer types with different syntax, a GC pointer and a non-GC pointer. The same problem - what pointer type does strcpy() accept?

It's an ugly mess, and why I've avoided any such thing in D.

I'm curious - how does one traverse a binary tree with non-null pointers? How does one create a circular data structure with non-null pointers?

There are three annotations: _Nullable, _Nonnull and _Null_unspecified. _Null_unspecified can freely convert between the other two types. If you don’t annotate a pointer (and don’t change the default), then it is a _Null_unspecified for compatibility with existing code. strlen and company thus take _Null_unspecified. They should take _Nonnull, but it’s old code.

For data structures, you mostly use _Null_unspecified as the nullability is not as simple as this pointer may be null or not, it is a potentially dynamic invariant of your data structure. In D, you would annotate it as an @system variable so only trusted code can modify it.

Where the annotations are really valuable is for function arguments and return. Can this argument be null or not? Now the compiler can help you instead of you having to memorize the documentation of every function you call.

I’ve consistently applied these annotations to my C projects and it works wonders. Many classes of mistakes are caught at compile time. Somethings are a dynamic property of your system, and for that there is a nullability sanitizer that can detect at runtime if a _Nonnull pointer gets a null value (usually from a data structure that was not properly initialized).

April 16
On Wednesday, 16 April 2025 at 21:30:58 UTC, Richard (Rikki) Andrew Cattermole wrote:
> From a link made previously in this thread, the state of the art annotation of nullability in C++: https://clang.llvm.org/docs/analyzer/developer-docs/nullability.html

An example using it:

```C
// Compile with -Wnullable-to-nonnull-conversion

#if defined(__clang__)
	#define ASSUME_NONNULL_BEGIN _Pragma("clang assume_nonnull begin")
	#define ASSUME_NONNULL_END _Pragma("clang assume_nonnull end")
#else
	#define ASSUME_NONNULL_BEGIN
	#define ASSUME_NONNULL_END
	#define __has_feature(x) (0)
#endif /* clang */

#if __has_feature (nullability)
	#define NONNULL _Nonnull
	#define NULLABLE _Nullable
	#define NULL_UNSPECIFIED _Null_unspecified
#else
	#define NONNULL
	#define NULLABLE
	#define NULL_UNSPECIFIED
#endif /* nullability */

ASSUME_NONNULL_BEGIN

int *someNonNull(int *);
int *someNullable(int * NULLABLE);

int foo(int * arg1, int * NULL_UNSPECIFIED arg2, int * NULLABLE arg3) {
	int a = 0;
	int * NONNULL ptr;

	a += *someNullable(arg1);
	a += *someNullable(arg2);
	a += *someNullable(arg3);

	ptr = arg1;
	ptr = arg2;
	ptr = arg3;

	a += *someNonNull(arg1);
	a += *someNonNull(arg2);
	a += *someNonNull(arg3);

	return a;
}

ASSUME_NONNULL_END
```

```
$ clang-14 -Wall -Wnullable-to-nonnull-conversion -c ttt.c
ttt.c:37:8: warning: implicit conversion from nullable pointer 'int * _Nullable' to non-nullable pointer type 'int * _Nonnull' [-Wnullable-to-nonnull-conversion]
        ptr = arg3;
              ^
ttt.c:41:20: warning: implicit conversion from nullable pointer 'int * _Nullable' to non-nullable pointer type 'int * _Nonnull' [-Wnullable-to-nonnull-conversion]
        a += *someNonNull(arg3);
                          ^
ttt.c:29:16: warning: variable 'ptr' set but not used [-Wunused-but-set-variable]
        int * NONNULL ptr;
                      ^
3 warnings generated.
```
April 16
On Wednesday, 16 April 2025 at 21:12:08 UTC, Walter Bright wrote:
> I'm curious - how does one traverse a binary tree with non-null pointers? How does one create a circular data structure with non-null pointers?

One has nullable and nonnull pointers.  One uses the former for the circular data structures.

Now the current ugly bit with clang is this:

```C
int foo(int * arg1, int * NULL_UNSPECIFIED arg2, int * NULLABLE arg3) {

        if (arg3 != 0)
                ptr = arg3;

        return a;
}
```

```
$ clang-14 -Wall -Wnullable-to-nonnull-conversion -c ttt.c
ttt.c:36:9: warning: implicit conversion from nullable pointer 'int * _Nullable' to non-nullable pointer type 'int * _Nonnull' [-Wnullable-to-nonnull-conversion]
                ptr = arg3;
                      ^
```

i.e. there is no DFA, and one still has to cast the value in the assignment.  So the clang support could stand some improvement.