On 07/26/2017 06:16 PM, Steven Schveighoffer wrote:
> So I guess I should restate that we can assume no implementations exist that intentionally cause UB when stream is NULL (as in Timon's example). Either they check for null, and handle gracefully, or don't check and segfault.

No need to worry about that at all. If worse comes to worst - i.e. we do port to such an implementation - we can always provide a thin wrapper that checks for NULL then calls the native function. No need to change the signatures. -- Andrei

On 27.07.2017 01:56, Andrei Alexandrescu wrote:
> On 07/26/2017 06:16 PM, Steven Schveighoffer wrote:
>> So I guess I should restate that we can assume no implementations exist that intentionally cause UB when stream is NULL (as in Timon's example).

My argument was not that we need to fear implementations that take explicit measures to screw you, but UB is UB. Compilers can in principle turn segfaults into any other behaviour they want, and this behaviour can change between releases. I'd just rather not codify guarantees that do not exist into the type system, as it is not really feasible to check them, even if in practice you will in the overwhelming majority get the expected behaviour.

> Either they check for null, and handle gracefully, or don't 
>> check and segfault.
> 
> No need to worry about that at all. If worse comes to worst - i.e. we do port to such an implementation

How do you notice?

> - we can always provide a thin wrapper that checks for NULL then calls the native function. No need to change the signatures. -- Andrei

I don't see how that works, as you'd end up with two different implementations of the same C function. (I.e. you get a name clash in the object file.)

On 7/26/17 7:56 PM, Andrei Alexandrescu wrote:
> On 07/26/2017 06:16 PM, Steven Schveighoffer wrote:
>> So I guess I should restate that we can assume no implementations exist that intentionally cause UB when stream is NULL (as in Timon's example). Either they check for null, and handle gracefully, or don't check and segfault.
> 
> No need to worry about that at all. If worse comes to worst - i.e. we do port to such an implementation - we can always provide a thin wrapper that checks for NULL then calls the native function. No need to change the signatures. -- Andrei

Hm.. so you mean:

pragma(mangle, "fgetc")
private extern(C) int real_fgetc(FILE * stream)

extern(D) int fgetc(FILE *stream) @trusted
{
  if(stream == null) assert(0);
  return real_fgetc(stream);
}

Yeah, that should work well actually. Nice!

-Steve

On 7/26/17 8:09 PM, Timon Gehr wrote:
> On 27.07.2017 01:56, Andrei Alexandrescu wrote:
>> On 07/26/2017 06:16 PM, Steven Schveighoffer wrote:
>>> So I guess I should restate that we can assume no implementations exist that intentionally cause UB when stream is NULL (as in Timon's example).
> 
> My argument was not that we need to fear implementations that take explicit measures to screw you, but UB is UB. Compilers can in principle turn segfaults into any other behaviour they want, and this behaviour can change between releases. I'd just rather not codify guarantees that do not exist into the type system, as it is not really feasible to check them, even if in practice you will in the overwhelming majority get the expected behaviour.

I can't see how compilers can take advantage of this one. However, we can take advantage that this UB is almost universally implemented as a hardware segfault that ends the process.

-Steve

On 2017-07-27 03:14, Steven Schveighoffer wrote:

> I can't see how compilers can take advantage of this one. However, we can take advantage that this UB is almost universally implemented as a hardware segfault that ends the process.

Unfortunately it's not that easy with optimizing compilers for C and C++:

void contains_null_check(int* p)
{
    int dead = *p;

    if (p == 0)
        return;

    *p = 4;
}

If the compiler runs the "Dead Code Elimination" optimization before "Redundant Null Check Elimination" then the above code will turn into:


void contains_null_check(int* p)
{
    if (p == 0) // Null check not redundant, and is kept.
        return;

    *p = 4;
}

But if the compiler runs the optimizations in the opposite order we end up with this code:


void contains_null_check(int* p)
{
    int dead = *p;

    if (false) // "p" was dereferenced by this point, so it can't be null
        return;

    *p = 4;
}

And then the compiler runs the "Dead Code Elimination" pass and we're left with:

void contains_null_check(int* p)
{
    *p = 4;
}

This can change between releases of compilers and between different vendors. Introducing an inlining pass will make this even more complicated, because the above example might be spread a cross multiple functions that have now been inlined.

For reference: http://blog.llvm.org/2011/05/what-every-c-programmer-should-know_14.html

-- 
/Jacob Carlborg

On 27.07.2017 02:11, Steven Schveighoffer wrote:
> On 7/26/17 7:56 PM, Andrei Alexandrescu wrote:
>> On 07/26/2017 06:16 PM, Steven Schveighoffer wrote:
>>> So I guess I should restate that we can assume no implementations exist that intentionally cause UB when stream is NULL (as in Timon's example). Either they check for null, and handle gracefully, or don't check and segfault.
>>
>> No need to worry about that at all. If worse comes to worst - i.e. we do port to such an implementation - we can always provide a thin wrapper that checks for NULL then calls the native function. No need to change the signatures. -- Andrei
> 
> Hm.. so you mean:
> 
> pragma(mangle, "fgetc")
> private extern(C) int real_fgetc(FILE * stream)
> 
> extern(D) int fgetc(FILE *stream) @trusted
> {
>    if(stream == null) assert(0);
>    return real_fgetc(stream);
> }
> 
> Yeah, that should work well actually. Nice!
> 
> -Steve

That works but it changes the signature. (extern(D) vs. extern(C)).

On 7/27/17 2:48 AM, Jacob Carlborg wrote:
> And then the compiler runs the "Dead Code Elimination" pass and we're left with:
> 
> void contains_null_check(int* p)
> {
>      *p = 4;
> }

So the result is that it will segfault. I don't see a problem with this. It's what I would have expected.

-Steve

On 7/27/17 7:27 AM, Timon Gehr wrote:
> On 27.07.2017 02:11, Steven Schveighoffer wrote:
>> On 7/26/17 7:56 PM, Andrei Alexandrescu wrote:
>>> On 07/26/2017 06:16 PM, Steven Schveighoffer wrote:
>>>> So I guess I should restate that we can assume no implementations exist that intentionally cause UB when stream is NULL (as in Timon's example). Either they check for null, and handle gracefully, or don't check and segfault.
>>>
>>> No need to worry about that at all. If worse comes to worst - i.e. we do port to such an implementation - we can always provide a thin wrapper that checks for NULL then calls the native function. No need to change the signatures. -- Andrei
>>
>> Hm.. so you mean:
>>
>> pragma(mangle, "fgetc")
>> private extern(C) int real_fgetc(FILE * stream)
>>
>> extern(D) int fgetc(FILE *stream) @trusted
>> {
>>    if(stream == null) assert(0);
>>    return real_fgetc(stream);
>> }
>>
>> Yeah, that should work well actually. Nice!
>>
> 
> That works but it changes the signature. (extern(D) vs. extern(C)).

Hm... you could use pragma(mangle) to get the signature the same. I was just thinking since it's going to be a D wrapper, it could be extern(D).

But you are right, &fgetc would result in a different type, so we should use pragma(mangle) instead.

-Steve

On 07/27/2017 07:27 AM, Timon Gehr wrote:
> On 27.07.2017 02:11, Steven Schveighoffer wrote:
>> On 7/26/17 7:56 PM, Andrei Alexandrescu wrote:
>>> On 07/26/2017 06:16 PM, Steven Schveighoffer wrote:
>>>> So I guess I should restate that we can assume no implementations exist that intentionally cause UB when stream is NULL (as in Timon's example). Either they check for null, and handle gracefully, or don't check and segfault.
>>>
>>> No need to worry about that at all. If worse comes to worst - i.e. we do port to such an implementation - we can always provide a thin wrapper that checks for NULL then calls the native function. No need to change the signatures. -- Andrei
>>
>> Hm.. so you mean:
>>
>> pragma(mangle, "fgetc")
>> private extern(C) int real_fgetc(FILE * stream)
>>
>> extern(D) int fgetc(FILE *stream) @trusted
>> {
>>    if(stream == null) assert(0);
>>    return real_fgetc(stream);
>> }
>>
>> Yeah, that should work well actually. Nice!
>>
>> -Steve
> 
> That works but it changes the signature. (extern(D) vs. extern(C)).

There are a number of techniques allowing you to daisy chain C functions in libraries without changing names by using e.g. linking order or dynamic symbol loading. Sounds exactly like the kind of problem to tackle when you see it. We have much more pressing things to be on. -- Andrei

On Wednesday, 26 July 2017 at 01:09:50 UTC, Steven Schveighoffer wrote:
> On 7/25/17 8:45 PM, Timon Gehr wrote:
>> On 26.07.2017 02:35, Steven Schveighoffer wrote:
>>> On 7/25/17 5:23 PM, Moritz Maxeiner wrote:
>>>> On Tuesday, 25 July 2017 at 20:16:41 UTC, Steven Schveighoffer wrote:
>>>>> The behavior is defined. It will crash with a segfault.
>>>>
>>>> In C land that behaviour is a platform (hardware/OS/libc) specific implementation detail (it's what you generally expect to happen, but AFAIK it isn't defined in official ISO/IEC C).
>>>
>>> In cases where C does not crash when dereferencing null, then D would not crash when dereferencing null. D depends on the hardware doing this (Walter has said so many times), so if C doesn't do it, then D won't. So those systems would have to be treated specially, and you'd have to work out your own home-grown mechanism for memory safety.
>> 
>> What Moritz is saying is that the following implementation of fclose is correct according to the C standard:
>> 
>> int fclose(FILE *stream){
>>      if(stream == NULL){
>>          go_wild_and_corrupt_all_the_memory();
>>      }else{
>>          actually_close_the_file(stream);
>>      }
>> }
>
> I think we can correctly assume no fclose implementations exist that do anything but access data pointed at by stream. Which means a segfault on every platform we support.
>
> On platforms that may not segfault, you'd be on your own.
>
> In other words, I think we can assume for any C functions that are passed pointers that dereference those pointers, passing null is safely going to segfault.
>
> Likewise, because D depends on hardware flagging of dereferencing null as a segfault, any platforms that *don't* have that for C also won't have it for D. And then @safe doesn't even work in D code either.
>
> As we have good support for different prototypes for different platforms, we could potentially unmark those as @trusted in those cases.

--- null.d ---
version (linux):

import core.stdc.stdio : FILE;
import core.sys.linux.sys.mman;

extern (C) @safe int fgetc(FILE* stream);

void mmapNull()
{
	void* mmapNull = mmap(null, 4096, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS | MAP_FIXED | MAP_POPULATE, -1, 0);
	assert (mmapNull == null, "Do `echo 0 > /proc/sys/vm/mmap_min_addr` as root");
	*(cast (char*) null) = 'D';
}

void nullDeref() @safe
{
	fgetc(null);
}

void main(string[] args)
{
	mmapNull();
	nullDeref();
}
---

For some fun on Linux, try out
# echo 0 > /proc/sys/vm/mmap_min_addr
$ rdmd null.d

Consider `mmapNull` being run in some third party shared lib you don't control.