On 7/26/17 6:01 AM, Timon Gehr wrote:
> On 26.07.2017 03:09, Steven Schveighoffer wrote:
>> On 7/25/17 8:45 PM, Timon Gehr wrote:
>>> ...
>>> What Moritz is saying is that the following implementation of fclose is correct according to the C standard:
>>>
>>> int fclose(FILE *stream){
>>>      if(stream == NULL){
>>>          return go_wild_and_corrupt_all_the_memory();
>>>      }else{
>>>          return actually_close_the_file(stream);
>>>      }
>>> }
>>
>> I think we can correctly assume no fclose implementations exist that do anything but access data pointed at by stream. Which means a segfault on every platform we support.
>>
>> On platforms that may not segfault, you'd be on your own.
>>
>> In other words, I think we can assume for any C functions that are passed pointers that dereference those pointers, passing null is safely going to segfault.
> 
> I'm not going to assume that.

Tell you what, when you find a D platform that this doesn't happen, we can fix it with a version statement ;)

-Steve

On 26.07.2017 13:22, Steven Schveighoffer wrote:
> On 7/26/17 6:01 AM, Timon Gehr wrote:
>> On 26.07.2017 03:09, Steven Schveighoffer wrote:
>>> ...
>>> In other words, I think we can assume for any C functions that are passed pointers that dereference those pointers, passing null is safely going to segfault.
>>
>> I'm not going to assume that.
> 
> Tell you what, when you find a D platform that this doesn't happen, > we can fix it with a version statement ;)
> 
> -Steve

The burden of proof is on you, not me. You are advocating the C approach to memory safety.

On 7/26/17 7:55 AM, Timon Gehr wrote:
> On 26.07.2017 13:22, Steven Schveighoffer wrote:
>> On 7/26/17 6:01 AM, Timon Gehr wrote:
>>> On 26.07.2017 03:09, Steven Schveighoffer wrote:
>>>> ...
>>>> In other words, I think we can assume for any C functions that are passed pointers that dereference those pointers, passing null is safely going to segfault.
>>>
>>> I'm not going to assume that.
>>
>> Tell you what, when you find a D platform that this doesn't happen, > we can fix it with a version statement ;)
>>
> 
> The burden of proof is on you, not me. You are advocating the C approach to memory safety.

They leave NULL dereferencing undefined because in some quirky old useless no-longer-existing hardware, it doesn't segfault.

Note that this is more implementation defined than undefined (in fact, I couldn't find it listed in the UB section at all in the C11 spec).

Look at Walter's response. I think D can simply only work with C implementations on platforms where null dereferencing segfaults and ignore the rest.

Walter, can we update the @safe spec to say that reading/writing data from the null page in C is required to generate a program crash for @safe to be valid? This can be an exception to the UB rule.

I just don't see the point of adding extra checks for null when the hardware already does it.

-Steve

On Tuesday, 25 July 2017 at 18:36:35 UTC, Moritz Maxeiner wrote:
> fgetc cannot be @trusted the same way fclose cannot be @trusted.
> If you pass either of them `null` - which constitutes a legal @safe context - the behaviour is undefined, which contradicts @trusted definition:
> <Trusted functions are guaranteed by the programmer to not exhibit any undefined behavior if called by a safe function.>

There's a less questionable problem with it.

On Wednesday, 26 July 2017 at 02:54:34 UTC, Walter Bright wrote:
> * Operating system APIs grow like weeds. We'd set ourselves an impossible task.
>
> It is worthwhile, however, to augment the APIs with the appropriate attributes like @nogc, scope, nothrow, @safe (for the ones that are), etc.

Given that C and OS api have no notion of memory safety, they don't support it and don't maintain it, so if it once was safe, it can be refactored later and become unsafe relying on proper usage of the api. Then if it was marked safe, the qualifier must be removed, which will be a breaking change for D code, but not for C code. Should we still try to mark them safe at all?

On Wednesday, 26 July 2017 at 01:09:50 UTC, Steven Schveighoffer wrote:
> On 7/25/17 8:45 PM, Timon Gehr wrote:
>> On 26.07.2017 02:35, Steven Schveighoffer wrote:
>>> On 7/25/17 5:23 PM, Moritz Maxeiner wrote:
>>>> On Tuesday, 25 July 2017 at 20:16:41 UTC, Steven Schveighoffer wrote:
>>>>> The behavior is defined. It will crash with a segfault.
>>>>
>>>> In C land that behaviour is a platform (hardware/OS/libc) specific implementation detail (it's what you generally expect to happen, but AFAIK it isn't defined in official ISO/IEC C).
>>>
>>> In cases where C does not crash when dereferencing null, then D would not crash when dereferencing null. D depends on the hardware doing this (Walter has said so many times), so if C doesn't do it, then D won't. So those systems would have to be treated specially, and you'd have to work out your own home-grown mechanism for memory safety.
>> 
>> What Moritz is saying is that the following implementation of fclose is correct according to the C standard:
>> 
>> int fclose(FILE *stream){
>>      if(stream == NULL){
>>          go_wild_and_corrupt_all_the_memory();
>>      }else{
>>          actually_close_the_file(stream);
>>      }
>> }
>
> I think we can correctly assume no fclose implementations exist that do anything but access data pointed at by stream. Which means a segfault on every platform we support.

What a luck that Solaris/SPARC is not supported as on that platform fclose(NULL) and even close(-1) do not segfault. Had to learn it the hard way when we ported our project from Solaris/SPARC to Linux/x86_64. It was surprizing how often that (wrong) behavior happenned in our code base (100K line of C).

>
> On platforms that may not segfault, you'd be on your own.
>
> In other words, I think we can assume for any C functions that are passed pointers that dereference those pointers, passing null is safely going to segfault.
Dereferencing NULL pointer on Solaris/SPARC segfaults but fclose() does apparently not dereference blindly the passed pointer. I suspect that SUN intentionnally reduced the opportunities to segfault on a lot of system calls and libs. The port to Linux revealed several violations (stale pointer usage, double frees, buffer overflows) that never triggered on Solaris and the project is more than 20 year old.

>
> Likewise, because D depends on hardware flagging of dereferencing null as a segfault, any platforms that *don't* have that for C also won't have it for D. And then @safe doesn't even work in D code either.
>
> As we have good support for different prototypes for different platforms, we could potentially unmark those as @trusted in those cases.
>

On Wednesday, 26 July 2017 at 03:16:44 UTC, Walter Bright wrote:
> On 7/25/2017 6:09 PM, Steven Schveighoffer wrote:
>> Likewise, because D depends on hardware flagging of dereferencing null as a segfault, any platforms that *don't* have that for C also won't have it for D. And then @safe doesn't even work in D code either.
>
> I spent 10 years programming on DOS with zero memory protection, and people have forgotten how awful that was. You couldn't simply instrument the code with null pointer checks, either, because then the program would be too big to fit.
>
> The solution finally appeared with the 286 DOS Extenders, which ran in protected mode. I switched to doing all my development under them, and would port to DOS only after passing all the test suite.
>
> D is definitely predicated on having hardware memory protection.
>
> The C/C++ Standards are still hanging on to EBCDIC, 10 bit bytes, non-IEEE floating point, etc. It's time to let that crap go :-)
>
> One C++ programmer told me that C++ could handle any character set. I asked him how RADIX50 was supported. Segfault! (I learned to program on RADIX50 systems.)
>
> D made some fundamental decisions:
>
> * Unicode
> * 2's complement
> * 8 bit bytes
> * IEEE arithmetic
> * memory protection
> * fixed sizes for integral types
> * single pointer type
> * >= 32 bit processors
>
> that relegated a lot of junk to the dustbin of history. (It's awful pretending to support that stuff. C and C++ pretend do, but just about zero programs will actually work on such systems, because there aren't any to try the code out on.)

And alone for that list of decision do I love you. I can not hear anymore all the crap about "undefined behaviour", "nasal demons" and optimizer that think that they are entitled to sabotage programs because he is an over zealous language lawyer in the C world practicing POOP (premature optimisation oriented programming).

On 7/26/2017 3:14 AM, Timon Gehr wrote:
> On 26.07.2017 05:02, Walter Bright wrote:
>> The implementation checks for fp being NULL and returns EOF if it is.
> 
> The C mindset is that this check is a waste of precious processing resources and morally wrong, as only a fool would pass NULL anyway, and fools deserve to get UB.

I wrote that code 30+ years ago, and no longer remember why I put the null check in. It might have been because other C compiler libraries did it.

On 7/26/2017 6:29 AM, Kagamin wrote:
> Should we still try to mark them safe at all?

Marking ones that are safe with @safe is fine. OS APIs pretty much never change.

On 7/26/17 12:08 PM, Patrick Schluter wrote:
> On Wednesday, 26 July 2017 at 01:09:50 UTC, Steven Schveighoffer wrote:
>> On 7/25/17 8:45 PM, Timon Gehr wrote:
>>> On 26.07.2017 02:35, Steven Schveighoffer wrote:
>>>> On 7/25/17 5:23 PM, Moritz Maxeiner wrote:
>>>>> On Tuesday, 25 July 2017 at 20:16:41 UTC, Steven Schveighoffer wrote:
>>>>>> The behavior is defined. It will crash with a segfault.
>>>>>
>>>>> In C land that behaviour is a platform (hardware/OS/libc) specific implementation detail (it's what you generally expect to happen, but AFAIK it isn't defined in official ISO/IEC C).
>>>>
>>>> In cases where C does not crash when dereferencing null, then D would not crash when dereferencing null. D depends on the hardware doing this (Walter has said so many times), so if C doesn't do it, then D won't. So those systems would have to be treated specially, and you'd have to work out your own home-grown mechanism for memory safety.
>>>
>>> What Moritz is saying is that the following implementation of fclose is correct according to the C standard:
>>>
>>> int fclose(FILE *stream){
>>>      if(stream == NULL){
>>>          go_wild_and_corrupt_all_the_memory();
>>>      }else{
>>>          actually_close_the_file(stream);
>>>      }
>>> }
>>
>> I think we can correctly assume no fclose implementations exist that do anything but access data pointed at by stream. Which means a segfault on every platform we support.
> 
> What a luck that Solaris/SPARC is not supported as on that platform fclose(NULL) and even close(-1) do not segfault. Had to learn it the hard way when we ported our project from Solaris/SPARC to Linux/x86_64. It was surprizing how often that (wrong) behavior happenned in our code base (100K line of C).

I'm guessing though that it's an implementation detail (like Walter's DMC example). A segfault is fine, and returning an error is fine. Both will properly be handled, and do not cause UB.

So I guess I should restate that we can assume no implementations exist that intentionally cause UB when stream is NULL (as in Timon's example). Either they check for null, and handle gracefully, or don't check and segfault.

What I was talking about is platforms that don't segfault on reading/writing from the zero page. Those we couldn't support with @safe D anyway.

-Steve