July 25, 2017
On 7/25/2017 5:56 PM, Andrei Alexandrescu wrote:
> I'd think that would be the case, but failed to find a fgetc implementation that mentions it's undefined for a null FILE*. Is there a link? Thx. -- Andrei

The documentation for DMC++ fgetc() is:

  https://digitalmars.com/rtl/stdio.html#fgetc

and says:

  "Returns the character just read on success, or EOF if end-of-file or a read error is encountered."

The implementation checks for fp being NULL and returns EOF if it is.
July 25, 2017
On 7/25/2017 6:09 PM, Steven Schveighoffer wrote:
> Likewise, because D depends on hardware flagging of dereferencing null as a segfault, any platforms that *don't* have that for C also won't have it for D. And then @safe doesn't even work in D code either.

I spent 10 years programming on DOS with zero memory protection, and people have forgotten how awful that was. You couldn't simply instrument the code with null pointer checks, either, because then the program would be too big to fit.

The solution finally appeared with the 286 DOS Extenders, which ran in protected mode. I switched to doing all my development under them, and would port to DOS only after passing all the test suite.

D is definitely predicated on having hardware memory protection.

The C/C++ Standards are still hanging on to EBCDIC, 10 bit bytes, non-IEEE floating point, etc. It's time to let that crap go :-)

One C++ programmer told me that C++ could handle any character set. I asked him how RADIX50 was supported. Segfault! (I learned to program on RADIX50 systems.)

D made some fundamental decisions:

* Unicode
* 2's complement
* 8 bit bytes
* IEEE arithmetic
* memory protection
* fixed sizes for integral types
* single pointer type
* >= 32 bit processors

that relegated a lot of junk to the dustbin of history. (It's awful pretending to support that stuff. C and C++ pretend do, but just about zero programs will actually work on such systems, because there aren't any to try the code out on.)
July 26, 2017
On 25/07/17 18:29, Moritz Maxeiner wrote:
> On Tuesday, 25 July 2017 at 14:39:15 UTC, Shachar Shemesh wrote:
>> On 25/07/17 17:24, Moritz Maxeiner wrote:
>>> On Tuesday, 25 July 2017 at 13:50:16 UTC, Shachar Shemesh wrote:
>>>> The title really does says it all.
>>>
>>> Since you explicitly state *all* OS functions:
>>> nothrow: Should be OK (only callbacks could violate this and they should be nothrow, anyway).
>> Technically, any system call that is a pthreads cancellation point may throw a C++ exception.
> 
> Good to know, then since D is supposed to be able to catch C++ exceptions (and can on 64bit Linux [1]) none of those may be attributed as `nothrow`, because C++ exceptions don't derive from `Error` >
>>
>> If we go down that route, however, calling system calls from nothrow becomes completely impossible, which is another way of saying that decorating just about anything with nothrow becomes impossible.
> 
> No. `nothrow` functions can call throwing ones, as long as they catch any exceptions not derived from Error thrown by them.
> 

And right there and then you've introduced a serious problem. The purpose of the C++ exception thrown on cancellation point is to terminate the thread. It is designed to be uncatchable.

Had that been D, it might derive from Error, or even directly from Throwable. This is C++, however. It some weirdly named class.

I think labeling these "nothrow" is the correct course of action.

>> The decoration's situation with callbacks is pretty horrible throughout D.
> 
> Do you mean throughout druntime and phobos?

I'm rechecking what I mean. I may have misspoke.

> 
>> I'm not sure this is the most compelling argument, however. The function passed to pthread_create does not, logically, run in the pthread_create function. As such, I don't think this logic holds.
> 
> Then the @nogc definition would need to be updated from: "or indirectly through functions it may call" to reflect this, because that can be interpreted both ways.
> 
>>
>> As for pthread_join, I have no idea what you meant by it. Please elaborate why you think it is a problem.
> 
> Possible scenario on single core (no hyperthreading) system:
> - thread 1 spawns thread 2
> - thread 1 enters @nogc function `foo` and calls `pthread_join` on thread 2 before its own timeslice is over (and thus enters blocked state)
> - thread 2 does work allocating via the GC, then terminates
> - thread 1 wakes up and leaves @nogc function `foo`
> 
> Because @nogc (in contrast to nothrow) is explicitly designed as transitive, logically speaking, `foo` violated its @nogc constraint (it *caused* the GC allocations in thread 2).

Following that logic, ANY function that might affect another thread cannot be @nogc. I think this way madness lies. I don't think other threads action, even if linked in some weird semantic way to ours, make us accountable to their actions.

If you pass a callback that is not @nogc to pthread_create, then your other thread might allocate. This doesn't change the fact that pthread_create doesn't allocate.

At Weka, we use this understanding of the semantics all the time. Our main thread is as @nogc as we can possibly make it. Whenever we need anything that violates our usual restrictions, we send it either to other threads or other processes for execution, and use the results when they return. Defining the various attributes too strictly will simply mean we cannot use them anywhere (which is pretty much what happens today, but the very thing I'm trying to change here).

Shachar
July 26, 2017
On 25/07/17 18:26, Andrei Alexandrescu wrote:
> (btw void[] doesn't work)

Can you expand on this point?

Shachar
July 26, 2017
On Wednesday, 26 July 2017 at 00:35:13 UTC, Steven Schveighoffer wrote:
> On 7/25/17 5:23 PM, Moritz Maxeiner wrote:
>> On Tuesday, 25 July 2017 at 20:16:41 UTC, Steven Schveighoffer wrote:
>>> The behavior is defined. It will crash with a segfault.
>> 
>> In C land that behaviour is a platform (hardware/OS/libc) specific implementation detail (it's what you generally expect to happen, but AFAIK it isn't defined in official ISO/IEC C).
>
> In cases where C does not crash when dereferencing null, then D would not crash when dereferencing null. [...]

OK, my (wrong) assumption was that a D compiler would on those platforms be required to inject null checks+crash in order to satisfy the property that null dereferences crashes D programs rely on.
Since that seems to not be the case: Is this documented in the D spec somewhere (I couldn't find it)? If not, imho it should.
July 26, 2017
On Wednesday, 26 July 2017 at 06:44:51 UTC, Shachar Shemesh wrote:
> On 25/07/17 18:29, Moritz Maxeiner wrote:
>> On Tuesday, 25 July 2017 at 14:39:15 UTC, Shachar Shemesh wrote:
>>> On 25/07/17 17:24, Moritz Maxeiner wrote:
>>>> On Tuesday, 25 July 2017 at 13:50:16 UTC, Shachar Shemesh wrote:
>>>>> The title really does says it all.
>>>>
>>>> Since you explicitly state *all* OS functions:
>>>> nothrow: Should be OK (only callbacks could violate this and they should be nothrow, anyway).
>>> Technically, any system call that is a pthreads cancellation point may throw a C++ exception.
>> 
>> Good to know, then since D is supposed to be able to catch C++ exceptions (and can on 64bit Linux [1]) none of those may be attributed as `nothrow`, because C++ exceptions don't derive from `Error` >
>>>
>>> If we go down that route, however, calling system calls from nothrow becomes completely impossible, which is another way of saying that decorating just about anything with nothrow becomes impossible.
>> 
>> No. `nothrow` functions can call throwing ones, as long as they catch any exceptions not derived from Error thrown by them.
>> 
>
> And right there and then you've introduced a serious problem.
> The purpose of the C++ exception thrown on cancellation point is to terminate the thread. It is designed to be uncatchable.

The issue lies with the definition of `nothrow` considering only D Throwables;
it would have to be updated to apply to C++ exceptions that equate to D exceptions derived from Error.

>
> I think labeling these "nothrow" is the correct course of action.

Not with the `nothrow` spec as it is right now. After the spec having been updated to apply to C++ exception that may not be caught, sure.

>>> I'm not sure this is the most compelling argument, however. The function passed to pthread_create does not, logically, run in the pthread_create function. As such, I don't think this logic holds.
>> 
>> Then the @nogc definition would need to be updated from: "or indirectly through functions it may call" to reflect this, because that can be interpreted both ways.
>> 
>>>
>>> As for pthread_join, I have no idea what you meant by it. Please elaborate why you think it is a problem.
>> 
>> Possible scenario on single core (no hyperthreading) system:
>> - thread 1 spawns thread 2
>> - thread 1 enters @nogc function `foo` and calls `pthread_join` on thread 2 before its own timeslice is over (and thus enters blocked state)
>> - thread 2 does work allocating via the GC, then terminates
>> - thread 1 wakes up and leaves @nogc function `foo`
>> 
>> Because @nogc (in contrast to nothrow) is explicitly designed as transitive, logically speaking, `foo` violated its @nogc constraint (it *caused* the GC allocations in thread 2).
>
> Following that logic, ANY function that might affect another thread cannot be @nogc.

Not any function; as I interpret the spec only those who manually interleave another thread allocating via the GC such that it looks to a caller as if they had allocated using the GC.

> I think this way madness lies. I don't think other threads action, even if linked in some weird semantic way to ours, make us accountable to their actions.

I would say it depends on the exact semantics of each use case whether we are accountable.

>
> If you pass a callback that is not @nogc to pthread_create, then your other thread might allocate. This doesn't change the fact that pthread_create doesn't allocate.

The "indirectly through functions it may call" of the @nogc spec is ambiguous here because it doesn't actually require a direct call chain to the allocation. It would need to be updated.

> At Weka, we use this understanding of the semantics all the time. Our main thread is as @nogc as we can possibly make it. Whenever we need anything that violates our usual restrictions, we send it either to other threads or other processes for execution, and use the results when they return. Defining the various attributes too strictly will simply mean we cannot use them anywhere (which is pretty much what happens today, but the very thing I'm trying to change here).

There is a difference between what's sensible and what the current wording of the spec allows for and before it's OK to attribute functions where the ambiguity applies, the spec wording (for both @nogc and nothrow) has to be made unambiguous.

P.S.: In case it's not clear: I'm playing devil's advocate in this subthread.
July 26, 2017
On 26.07.2017 03:09, Steven Schveighoffer wrote:
> On 7/25/17 8:45 PM, Timon Gehr wrote:
>> ...
>> What Moritz is saying is that the following implementation of fclose is correct according to the C standard:
>>
>> int fclose(FILE *stream){
>>      if(stream == NULL){
>>          return go_wild_and_corrupt_all_the_memory();
>>      }else{
>>          return actually_close_the_file(stream);
>>      }
>> }
> 
> I think we can correctly assume no fclose implementations exist that do anything but access data pointed at by stream. Which means a segfault on every platform we support.
> 
> On platforms that may not segfault, you'd be on your own.
> 
> In other words, I think we can assume for any C functions that are passed pointers that dereference those pointers, passing null is safely going to segfault.

I'm not going to assume that.
July 26, 2017
On 26.07.2017 02:56, Andrei Alexandrescu wrote:
>>
>> What Moritz is saying is that the following implementation of fclose is correct according to the C standard:
>>
>> int fclose(FILE *stream){
>>      if(stream == NULL){
>>          return go_wild_and_corrupt_all_the_memory();
>>      }else{
>>          return actually_close_the_file(stream);
>>      }
>> }
> 
> I'd think that would be the case, but failed to find a fgetc implementation that mentions it's undefined for a null FILE*. Is there a link? Thx. -- Andrei

It's implicit. In C, whenever you pass something that is outside the interface specification, you get UB. Also, in C, there is no way to get a segmentation fault except for UB, and fgetc(NULL) segfaults with glibc.
July 26, 2017
On 26.07.2017 05:02, Walter Bright wrote:
> On 7/25/2017 5:56 PM, Andrei Alexandrescu wrote:
>> I'd think that would be the case, but failed to find a fgetc implementation that mentions it's undefined for a null FILE*. Is there a link? Thx. -- Andrei
> 
> The documentation for DMC++ fgetc() is:
> 
>    https://digitalmars.com/rtl/stdio.html#fgetc
> 
> and says:
> 
>    "Returns the character just read on success, or EOF if end-of-file or a read error is encountered."
> 
> The implementation checks for fp being NULL and returns EOF if it is.

The C mindset is that this check is a waste of precious processing resources and morally wrong, as only a fool would pass NULL anyway, and fools deserve to get UB.
July 26, 2017
On 7/26/17 3:05 AM, Shachar Shemesh wrote:
> On 25/07/17 18:26, Andrei Alexandrescu wrote:
>> (btw void[] doesn't work)
> 
> Can you expand on this point?
> 
> Shachar

Because anything casts to void[] implicitly.

e.g.:

void main() @safe
{
   int *[] arr = new int*[5];
   read(0, arr); // reading raw pointer data, shouldn't be allowed
}

-Steve