On 19.11.2017 01:07, Walter Bright wrote:
> On 11/18/2017 6:16 AM, Timon Gehr wrote:
>> On 18.11.2017 05:05, Walter Bright wrote:
>>> On 11/17/2017 6:05 AM, Timon Gehr wrote:
>>>> There are type systems that do that, which is what is being proposed for C#. It's pretty straightforward: If I have a variable of class reference type C, it actually contains a reference to a class instance of type C.
>>>
>>> One of the difficulties with this is you'll still need an "empty" instance of C for the non-null reference to point to.
>>
>> Why would you need an empty instance?
> 
> Consider the ClassDeclaration in the DMD source. Each ClassDeclaration has a pointer to its base class, `baseClass`. Except for `Object`, which doesn't have a base class. This is represented to assigning `null` to `baseClass`.
> ...

I.e., baseClass should have type Nullable!ClassDeclaration. This does not in any form imply that ClassDeclaration itself needs to have a null value.

> So I can run up the base class list by:
> 
>      for (b = c; b; b = b.baseClass) { ... }
>  > If it cannot be null, I just have to invent something else that does the
> same thing:
> 
>      for (b = c; b != nullInstanceOfClass; b = b.baseClass) { ... }
> 
> and nothing really has changed.
> ...

Nullable!ClassDeclaration can be null, so this is not relevant.

>> Just use a Nullable!C instead of a C if a special 'null' state is actually required.
> 
> What should the default initializer for a type do?
> ...

There should be none for non-nullable types.

> 
>>> Any attempts to use a method on the empty instance should throw. 
>>
>> The idea is that the type system makes potential such attempts explicit while verifying that they don't occur in most of the cases. Then you can grep for potential null dereferences.
> 
> There are cases where the actual path taken guarantees initialization, but the graph of all paths does have uninitialized edges. Figuring out which are paths never taken is the halting problem.

The same applies to all other errors prevented by a type system. It's just not a useful argument. The halting problem is undecidable mostly because it is possible to write ridiculous programs. The ones we write in practice are often easier to understand (especially when they come with some useful documentation), because they were /designed/ to serve a particular purpose. Note that the undecidability of the halting problem is not something that applies exclusively to programs, it also applies to programmers.

> I found this out when testing my DFA (data flow analysis) algorithms.
> 
>    void test(int i) {
>      int* p = null;
>      if (i) p = &i;
>      ...
>      if (i) *p = 3;
>      ...
>    }
> 
> Note that the code is correct, but DFA says the *p could be a null dereference. (Replace (i) with any complex condition that there's no way the DFA can prove always produces the same result the second time.)

Yes, there is a way. Put in an assertion. Of course, at that point you are giving up, but this is not the common case. Also, you can often just write the code in a way that the DFA will understand. We are doing this all the time in statically-typed programming languages.

Is it possible to somehow change the concept of uninitialized values to something like 'zeroOrOne' instead of 'null'?


  Instrument instrument1 = new Instrument();
  Instrument instrument2 = new Instrument();

  zeroOrOne!Instrument getInstrument() {
	  zeroOrOne!Instrument instrument;
	  if( instrument1.power > 10 ) instrument = instrument1;
	  else if( instrument2.power > 5 ) instrument = instrument2;
	  return instrument;
  }

  //zeroOrOne!Instrument instead of 'InstrumentOrNull'
  auto instrument = getInstrument();

  /*1*/ instrument.setPower( 20.4f );
  /*2*/ instrument.doit();

  /*3*/ instrument1.setPower = 20;

  instrument = getInstrument();
  /*4*/ instrument.doit();
  /*5*/ instrument.setPower( 20.4f );

  /*
  1: opDispatch: setPower
  2: opDispatch: doit

  3: Instrument.setPower: 20

  4: opDispatch: doit
  4: Instrument.doit;
  5: opDispatch: setPower
  5: Instrument.setPower: 20.4
  */

On 11/18/2017 6:25 PM, Timon Gehr wrote:
> I.e., baseClass should have type Nullable!ClassDeclaration. This does not in any form imply that ClassDeclaration itself needs to have a null value.

Converting back and forth between the two types doesn't sound appealing.


>> What should the default initializer for a type do?
> There should be none for non-nullable types.

I suspect you'd wind up needing to create an "empty" object just to satisfy that requirement. Such as for arrays of objects, or objects with a cyclic graph.

Interestingly, `int` isn't nullable, and we routinely use rather ugly hacks to fake it being nullable, like reserving a bit pattern like 0, -1 or 0xDEADBEEF and calling it INVALID_VALUE, or carrying around some other separate flag that says if it is valid or not. These are often rich sources of bugs.

As you can guess, I happen to like null, because there are no hidden bugs from pretending it is a valid value - you get an immediate program halt - rather than subtly corrupted results.

Yes, my own code has produced seg faults from erroneously assuming a value was not null. But it wouldn't have been better with non-nullable types, since the logic error would have been hidden and may have been much, much harder to recognize and track down. I wish there was a null for int types. At least we sort of have one for char types, 0xFF. And there's that lovely NaN for floating point! Too bad it's woefully underused.


>> I found this out when testing my DFA (data flow analysis) algorithms.
>>
>>    void test(int i) {
>>      int* p = null;
>>      if (i) p = &i;
>>      ...
>>      if (i) *p = 3;
>>      ...
>>    }
>>
>> Note that the code is correct, but DFA says the *p could be a null dereference. (Replace (i) with any complex condition that there's no way the DFA can prove always produces the same result the second time.)
> 
> Yes, there is a way. Put in an assertion. Of course, at that point you are giving up, but this is not the common case.

An assertion can work, but doesn't it seem odd to require adding a runtime check in order to get the code to compile?

(This is subtly different from the current use of assert(0) to flag unreachable code.)


> Also, you can often just write the code in a way that the DFA will understand. We are doing this all the time in statically-typed programming languages.

I didn't invent this case. I found it in real code; it happens often enough. The cases are usually much more complex, I just posted the simplest reduction. I was not in a position to tell the customer to restructure his code, though :-)

On Sunday, 19 November 2017 at 04:04:04 UTC, Walter Bright wrote:
> I wish there was a null for int types. At least we sort of have one for char types, 0xFF. And there's that lovely NaN for floating point! Too bad it's woefully underused.

"I wish there was a null for int types."

+1000

On Sunday, 19 November 2017 at 04:19:32 UTC, codephantom wrote:
> On Sunday, 19 November 2017 at 04:04:04 UTC, Walter Bright wrote:
>> I wish there was a null for int types. At least we sort of have one for char types, 0xFF. And there's that lovely NaN for floating point! Too bad it's woefully underused.
>
> "I wish there was a null for int types."
>
> +1000

Yes. The only value that can sometime be used as invalid value is int.min.

On 19.11.2017 05:04, Walter Bright wrote:
> On 11/18/2017 6:25 PM, Timon Gehr wrote:
>> I.e., baseClass should have type Nullable!ClassDeclaration. This does not in any form imply that ClassDeclaration itself needs to have a null value.
> 
> Converting back and forth between the two types doesn't sound appealing.
> ...

I can't see the problem. You go from nullable to non-nullable by checking for null, and the other direction happens implicitly.

> 
>>> What should the default initializer for a type do?
>> There should be none for non-nullable types.
> 
> I suspect you'd wind up needing to create an "empty" object just to satisfy that requirement. Such as for arrays of objects, or objects with a cyclic graph.
> ...

Again, just use a nullable reference if you need null. The C# language change makes the type system strictly more expressive. There is nothing that cannot be done after the change that was possible before, it's just that the language allows to document and verify intent better.

> Interestingly, `int` isn't nullable, and we routinely use rather ugly hacks to fake it being nullable, like reserving a bit pattern like 0, -1 or 0xDEADBEEF and calling it INVALID_VALUE, or carrying around some other separate flag that says if it is valid or not. These are often rich sources of bugs.
> 
> As you can guess, I happen to like null, because there are no hidden bugs from pretending it is a valid value - you get an immediate program halt - rather than subtly corrupted results.
> ...

Making null explicit in the type system is compatible with liking null. (In fact, it is an endorsement of null. There are other options to accommodate optional values in your language.)

> Yes, my own code has produced seg faults from erroneously assuming a value was not null. But it wouldn't have been better with non-nullable types, since the logic error would have been hidden

It was your own design decision to hide the error. This is not something that a null-aware type system promotes, and I doubt this is what you would be promoting if mainstream type systems had gone that route earlier.

> and may have been much, much harder to recognize and track down.

No, it would have been better because you would have been used to the more explicit system from the start and you would have just written essentially the same code with a few more compiler checks in those cases where they apply, and perhaps you would have suffered a handful fewer null dereferences. Being able to document intent across programmers in a compiler-checked way is also useful, even if one manages to remember all assumptions that are valid about one's own code. Note that the set of valid assumptions may change as the code base evolves.

The point of types is to classify values into categories such that types in the same category support the same operations. It is not very clean to have a special null value in all those types that does not support any of the operations that references are supposed to support. Decoupling the two concepts into references an optionality gets rid of this issue, cleaning up both concepts.

> I wish there was a null for int types.

AFAIU, C# will now have 'int?'.

> At least we sort of have one for char types, 0xFF. And there's that lovely NaN for floating point! Too bad it's woefully underused.
> ...

It can also be pretty annoying. It really depends on the use case. Also this is in direct contradiction with your earlier points. NaNs don't usually blow up.

> 
>>> I found this out when testing my DFA (data flow analysis) algorithms.
>>>
>>>    void test(int i) {
>>>      int* p = null;
>>>      if (i) p = &i;
>>>      ...
>>>      if (i) *p = 3;
>>>      ...
>>>    }
>>>
>>> Note that the code is correct, but DFA says the *p could be a null dereference. (Replace (i) with any complex condition that there's no way the DFA can prove always produces the same result the second time.)
>>
>> Yes, there is a way. Put in an assertion. Of course, at that point you are giving up, but this is not the common case.
> 
> An assertion can work, but doesn't it seem odd to require adding a runtime check in order to get the code to compile?
> ...

Not really. The runtime check is otherwise just implicit in every pointer dereference (though here there happens to be hardware support for that check).

> (This is subtly different from the current use of assert(0) to flag unreachable code.)
> ...

It's adding a runtime check in order to get the code to compile. ;)

> 
>> Also, you can often just write the code in a way that the DFA will understand. We are doing this all the time in statically-typed programming languages.
> 
> I didn't invent this case. I found it in real code; it happens often enough. The cases are usually much more complex, I just posted the simplest reduction. I was not in a position to tell the customer to restructure his code, though :-)
			
I don't doubt that this happens. I'm just saying that often enough it does not. (Especially if the check is in the compiler.)

I'm not fighting for explicit nullable in D by the way. I'm mostly trying to dispel wrong notions of what it is.

On 11/19/2017 11:36 AM, Timon Gehr wrote:
> On 19.11.2017 05:04, Walter Bright wrote:
>> On 11/18/2017 6:25 PM, Timon Gehr wrote:
>>> I.e., baseClass should have type Nullable!ClassDeclaration. This does not in any form imply that ClassDeclaration itself needs to have a null value.
>>
>> Converting back and forth between the two types doesn't sound appealing.
>> ...
> 
> I can't see the problem. You go from nullable to non-nullable by checking for null, and the other direction happens implicitly.

Implicit conversions have their problems with overloading, interactions with const, template argument deduction, surprising edge cases, probably breaking a lot of Phobos, etc. It's best not to layer on more of this stuff. Explicit casting is a problem, too.

There's also an issue of how to derive a class from a base class.


>>>> What should the default initializer for a type do?
>>> There should be none for non-nullable types.
>> I suspect you'd wind up needing to create an "empty" object just to satisfy that requirement. Such as for arrays of objects, or objects with a cyclic graph.
> Again, just use a nullable reference if you need null. The C# language change makes the type system strictly more expressive. There is nothing that cannot be done after the change that was possible before, it's just that the language allows to document and verify intent better.

This implies one must know all the use cases of a type before designing it.


>> Yes, my own code has produced seg faults from erroneously assuming a value was not null. But it wouldn't have been better with non-nullable types, since the logic error would have been hidden
> 
> It was your own design decision to hide the error.

No, it was a bug. Nobody makes design decisions to insert bugs :-) The issue is how easy the bug is to have, and how difficult it would be to discover it.


>> and may have been much, much harder to recognize and track down.
> No, it would have been better because you would have been used to the more explicit system from the start and you would have just written essentially the same code with a few more compiler checks in those cases where they apply, and perhaps you would have suffered a handful fewer null dereferences.

I'm just not convinced of that.


> The point of types is to classify values into categories such that types in the same category support the same operations. It is not very clean to have a special null value in all those types that does not support any of the operations that references are supposed to support. Decoupling the two concepts into references an optionality gets rid of this issue, cleaning up both concepts.

I do understand that point. But I'm not at all convinced that non-nullable types in aggregate results in cleaner, simpler code, for reasons already mentioned.

>> I wish there was a null for int types.
> AFAIU, C# will now have 'int?'.

Implemented as a pointer to int? That is indeed one way to do it, but rather costly.


> It can also be pretty annoying.

Yes, it can be annoying, so much better to have a number that looks like it might be right, but isn't, because 0.0 was used as a default initializer when it should have been 1.6. :-)


> It really depends on the use case. Also this is in direct contradiction with your earlier points. NaNs don't usually blow up.

"blow up" - as I've said many times, I find the visceral aversion to seg faults puzzling. Why is that worse than belatedly discovering a NaN in your output, which you now have to back search it to its source?

My attitude towards programming bugs is to have them immediately halt the program as soon as possible, so:

1. I know an error has occurred, i.e. I don't get corrupt results that I assumed were correct, leading to more adverse consequences
2. The detection of the error is as close as possible to where things went wrong

Having floats default initialize to 0.0 is completely anti-ethical to (1) and (2), and NaN at least addresses (1).

There have been many long threads on this topic in this forum. Yes, I understand that it's better for game programs to ignore bugs because gamers don't care about corrupt results, they only care that the program continues to run and do something. For the rest of us, are we ready to be done with malware inserted via exploitable bugs?

By the way, I was initially opposed to having seg faults produce stack traces, saying it was the debugger's job to do that. I've since changed my mind. I do like very much the convenience of the stack trace dump, and rely on it all the time. I even insert code to force a seg fault to get a stack trace. I was wrong about its utility.


> I'm not fighting for explicit nullable in D by the way.

Thanks for clarifying that.

Timon Gehr <timon.gehr@gmx.ch> wrote:
>> I wish there was a null for int types.
> 
> AFAIU, C# will now have 'int?'.

C# had 'int?' (nullable value types) for ages.
The new thing is explicitly nullable classes (reference types). I'm really
looking forward to use those.

On Monday, 20 November 2017 at 06:24:31 UTC, Tobias Müller wrote:
> Timon Gehr <timon.gehr@gmx.ch> wrote:
>>> I wish there was a null for int types.
>> 
>> AFAIU, C# will now have 'int?'.
>
> C# had 'int?' (nullable value types) for ages.
> The new thing is explicitly nullable classes (reference types). I'm really
> looking forward to use those.

int? is just syntactic sugar for Nullable<int>. It has been around since 2005. Nullable<T> is just a struct with an implementation similar to Nullable!T from D's std.typecons

This topic is about a new C# feature called "nullable reference types":

1. if you declare SomeClass x, x is assumed to *not hold null values*, that means that when you try "x = null" or x =" somepossiblenullvalue", this will result in a compiler warning: "Warning, x is supposed to hold a value" The warning can be avoided by using "x = null!" or "x = somepossiblenullvalue!"

2.if you declare SomeClass? x, x is allowed to *hold null values*, meaning that if you try "x.someFunction()", this will result in a compiler warning: "Warning, x can be null". The warning can be avoided in two ways:

2a. test for null: "if (x != null) { x.someFunction(); }"
2b. show the compiler that you know better:: x!.someFunction()


In fact, this is the introduction of a new operator "!", probably named "I know better" operator.

On Monday, 20 November 2017 at 08:49:41 UTC, rumbu wrote:
> In fact, this is the introduction of a new operator "!", probably named "I know better" operator.

It's called the "bang" operator, because of how things blow up when you're wrong.