On Monday, 23 November 2020 at 02:21:20 UTC, ag0aep6g wrote:
> What I meant is that DMD allows dereferencing null in @safe code. Since @safe code must not corrupt memory, it must then take the necessary steps to make that safe.

Yes, the implementation dereferences null and traps it, but the in terms of the language it prevents dereferencing of null and terminates (as the dereferencing is supposed to be without other side effects than termination).

> I'm pretty sure that DMD doesn't actually take the necessary steps (as you say, it ignores large objects). And I'm not sure if Walter has fully considered the implications, but he has made it clear that null is supposed to be a safe value. And that can be made to work fairly easily, at the cost of run-time checks. Maybe treating null as unsafe could also work, but that would need a lot more design work.

Yes, that does pose problems with objects of variable size. So then you either have to put a limit of how large objects nullable pointers are allowed to point to or add runtime checks.

This is a good reason to add nonnullable pointers to the language. That would reduce the number of places where you need dynamic checks.

> I think that's an implementation detail. An implementation must ensure that null is safe (at least in @safe code). We don't really care how it does that, but adding checks before every dereference is the obvious solution.

Indeed. Although specifications sometimes add implementation notes to clarify how things are supposed to work (so that an embedded compiler doesn't ignore runtime checks altogether).

In the case of D, it would be interesting and useful to have implementation notes referring to what the different D compilers do.

On Sunday, 22 November 2020 at 23:00:25 UTC, Ola Fosheim Grostad wrote:
> On Sunday, 22 November 2020 at 22:36:40 UTC, Paul Backus wrote:
>> Memory safety is concerned specifically with avoiding undefined behavior. Crashing the program isn't undefined behavior, so it's allowed in @safe code.
>
> I understand what you mean, but at high optimization levels dereferencing a null pointer can trigger undefined behaviour.

I'll reiterate what I've been saying many times already: in LDC, null dereference in Undefined Behavior. What follows is that code should _actively_ check for null to be safe. Checking for null is not included by the compiler in @safe code.

- Note that accessing memory location 0x0 will probably crash your program on most systems, but definitely not all. For example with WebAssembly, 0 is a validly accessibly memory location.
- "Dereferencing a null pointer" in D language context does _not_ mean accessing memory address 0 on system level.
-- Calling a class method a.foo() when a==null is technically dereferencing `a`, even if the `foo` call is devirtualized, does not access any member variables, etc.
-- Accessing struct member that is not at offset 0 will also lead to accesses to other locations than 0.


-Johan

On Monday, 23 November 2020 at 12:01:08 UTC, Johan Engelen wrote:
> On Sunday, 22 November 2020 at 23:00:25 UTC, Ola Fosheim Grostad wrote:
>> On Sunday, 22 November 2020 at 22:36:40 UTC, Paul Backus wrote:
>>> Memory safety is concerned specifically with avoiding undefined behavior. Crashing the program isn't undefined behavior, so it's allowed in @safe code.
>>
>> I understand what you mean, but at high optimization levels dereferencing a null pointer can trigger undefined behaviour.
>
> I'll reiterate what I've been saying many times already: in LDC, null dereference in Undefined Behavior. What follows is that code should _actively_ check for null to be safe. Checking for null is not included by the compiler in @safe code.

Then LDC allows memory corruption in @safe code, which is a bug.

On Monday, 23 November 2020 at 01:29:12 UTC, Ola Fosheim Grostad wrote:
> On Monday, 23 November 2020 at 01:04:38 UTC, Paul Backus wrote:
>> Please show me the definition of null that requires it to be invalid.
>
> null points to nothing, that is not a valid value for the referenced type.

No. null is not a trap representation as a C standard would call it. It is a valid value for a pointer. Dereferencing it is an entirely other thing.

>
> Trivially invalid.

Nope.

>
>> Please show me the relevant definitions of these terms and explain how they contradict.
>
> What do mean? A crash is by definition undefined behaviour.

Nope. In the case of D. The error generated by dereferencing a null pointer is a defined behaviour. As defined as is calling abort() in a C program.

Catching the error and continuing processing that would be undefined behaviour.

>
> The spec does not provide adequate definitions and requirements, which is what makes it unsound.
>
>> The implementation allows undefined behavior in @safe code. That means the implementation is incorrect, period. Neither of the possible interpretations of the spec allow this.
>
> The spec isn't well defined or consistent just because people claim things in the forum.

The issue with null pointer dereferencing has nothing to do with its definition but with its implementation as the defined behaviour of aborting the program is not guaranteed in all circumstances (null pointer + offset big enough to hit a real page).

On Monday, 23 November 2020 at 12:28:39 UTC, Patrick Schluter wrote:
> No. null is not a trap representation as a C standard would call it. It is a valid value for a pointer. Dereferencing it is an entirely other thing.

A trap is an interrupt at the hardware level. It has nothing to do with C.

>> Trivially invalid.
>
> Nope.

Yes. "nothing" in not a valid value for "int". That is trivially invalid.

Try to think of null as an empty set.

> Nope. In the case of D. The error generated by dereferencing a null pointer is a defined behaviour. As defined as is calling abort() in a C program.

You are speaking of DMD, not the spec?

> The issue with null pointer dereferencing has nothing to do with its definition but with its implementation as the defined behaviour of aborting the program is not guaranteed in all circumstances (null pointer + offset big enough to hit a real page).

My argument is based on the spec and what is required to get to something that is consistent/sound and portable.

Also, as it has been pointed out, it does not work this way in shipping compilers so the spec should provide implemtation notes on what actually happens in shipping compilers.

On Monday, 23 November 2020 at 12:03:49 UTC, Paul Backus wrote:
> On Monday, 23 November 2020 at 12:01:08 UTC, Johan Engelen wrote:
>>
>> I'll reiterate what I've been saying many times already: in LDC, null dereference in Undefined Behavior. What follows is that code should _actively_ check for null to be safe. Checking for null is not included by the compiler in @safe code.
>
> Then LDC allows memory corruption in @safe code,

Correct. And so do all other D compilers. Completely independent of optimization level.

> which is a bug.

Without explicit null pointer checking, this is effectively a "won't fix".

-Johan

On Monday, 23 November 2020 at 16:50:44 UTC, Johan Engelen wrote:
> On Monday, 23 November 2020 at 12:03:49 UTC, Paul Backus wrote:
>>
>> Then LDC allows memory corruption in @safe code,
>
> Correct. And so do all other D compilers. Completely independent of optimization level.
>
>> which is a bug.
>
> Without explicit null pointer checking, this is effectively a "won't fix".

Maybe it is for LDC. I suspect Walter puts a high enough priority on memory-safety that he would accept a fix for this issue into DMD.

On Monday, 23 November 2020 at 17:17:26 UTC, Paul Backus wrote:
> On Monday, 23 November 2020 at 16:50:44 UTC, Johan Engelen wrote:
>> On Monday, 23 November 2020 at 12:03:49 UTC, Paul Backus wrote:
>>>
>>> Then LDC allows memory corruption in @safe code,
>>
>> Correct. And so do all other D compilers. Completely independent of optimization level.
>>
>>> which is a bug.
>>
>> Without explicit null pointer checking, this is effectively a "won't fix".
>
> Maybe it is for LDC. I suspect Walter puts a high enough priority on memory-safety that he would accept a fix for this issue into DMD.

Hi,

I think it is not fixable without introducing severe penalty in performance I suspect. Some languages are adding features to help the compiler / user specicify where nulls can be expected. Maybe D needs this too.

My original issue was that it failed to check an obvious case.
It seems that DMD does detect that case when -O is used. But that is perhaps not great.

Other languages - even C or C+= these days, will immediately warn you about possible NULL de-reference when it is very obvious.

On Monday, 23 November 2020 at 12:39:05 UTC, Ola Fosheim Grøstad wrote:
> On Monday, 23 November 2020 at 12:28:39 UTC, Patrick Schluter wrote:
>> No. null is not a trap representation as a C standard would call it. It is a valid value for a pointer. Dereferencing it is an entirely other thing.
>
> A trap is an interrupt at the hardware level. It has nothing to do with C.

Read the C standard, they explain what a trap representation is. It has nothing to do with an interrupt. I refer to the C standard because they make the difference between allowed pointer values and unallowed values even if never dereferenced.

Read the selected answer on this stackoverflow [1] page, he explains it better than me.

[1]: https://stackoverflow.com/questions/6725809/trap-representation

On Monday, 23 November 2020 at 20:00:29 UTC, Patrick Schluter wrote:
> On Monday, 23 November 2020 at 12:39:05 UTC, Ola Fosheim Grøstad wrote:
>> A trap is an interrupt at the hardware level. It has nothing to do with C.
>
> Read the C standard, they explain what a trap representation is. It has nothing to do with an interrupt. I refer to the C standard because they make the difference between allowed pointer values and unallowed values even if never dereferenced.

When I intoduced the word "trap" in this discussion I used the traditional hardware terminology as used in Motorola 68K reference manuals. Has nothing to do with C whatsoever.