It has been discussed before but I'd still love to see non-nullable references on D. Here's Anders Hejlsberg position on the subject. 

> 50% of the bugs that people run into today, coding with C# in our
> platform, and the same is true of Java for that matter, are probably
> null reference exceptions. If we had had a stronger type system
> that would allow you to say that 'this parameter may never be null,
> and you compiler please check that at every call, by doing static
> analysis of the code'. Then we could have stamped out classes
> of bugs.

He then goes on to kill our hope of having this checks on any future version of C#. Too bad.

This is a change that will break lots of code but I think it will benefit the language and quality of D applications. Is it at least been looked at for D 3.0?

Thanks

> > 50% of the bugs that people run into today, coding with C# in our platform, and the same is true of Java for that matter, are probably null reference exceptions.

Yes, but even if this is accurate, it's misleading.  Very seldom are null references difficult-to-find, time-consuming bugs.  Most other things in D and other languages that are designed to prevent specific classes of bugs are designed to prevent extremely aggravating, hard to track down bugs.

For example, GC was invented because tracking down memory leaks is a huge PITA. Bounds checking was invented because off-by-one errors can be subtle and hard to find.  Null reference errors, on the other hand, manifest themselves in a predictable way (segfaults), and if you're using the safe memory model, i.e. bounds checking, no pointers, that's about the only thing a segfault could mean. Furthermore, null checking could be added to an implementation kind of like bounds checking without changing the language spec.  I haven't made up my mind fully yet, but I question whether adding lots of complexity to the language to prevent a very easy to track down class of bugs, even if it's a common one, is justified.

dsimcha wrote:
>>> 50% of the bugs that people run into today, coding with C# in our
>>> platform, and the same is true of Java for that matter, are probably
>>> null reference exceptions.
> 
> Yes, but even if this is accurate, it's misleading.  Very seldom are null
> references difficult-to-find, time-consuming bugs.  Most other things in D and
> other languages that are designed to prevent specific classes of bugs are designed
> to prevent extremely aggravating, hard to track down bugs.
> 
> For example, GC was invented because tracking down memory leaks is a huge PITA.
> Bounds checking was invented because off-by-one errors can be subtle and hard to
> find.  Null reference errors, on the other hand, manifest themselves in a
> predictable way (segfaults), and if you're using the safe memory model, i.e.
> bounds checking, no pointers, that's about the only thing a segfault could mean.
> Furthermore, null checking could be added to an implementation kind of like bounds
> checking without changing the language spec.  I haven't made up my mind fully yet,
> but I question whether adding lots of complexity to the language to prevent a very
> easy to track down class of bugs, even if it's a common one, is justified.

I find this counterpoint to be a bit iffy.  The problem with null dereference problems isn't knowing that they're there: that's the easy part.  You helpfully get an exception to the face when that happens. The hard part is figuring out *where* the problem originally occurred. It's not when the exception is thrown that's the issue; it's the point at which you placed a null reference in a slot where you shouldn't have.

Yes, we have invariants and contracts; but inevitably you're going to forget one, and it's that one slip-up that's going to bite you in the rear.

One could use roughly the same argument for non-null references as for const: you could document it, but documentation is inevitably wrong or out of date.  :P

That said, I wonder what's stopping us from implementing them today. Last time I checked (admittedly, that was like a year ago) we didn't have the compile-time magic necessary to completely and transparently proxy any given object.

Ideally, we should be able to do this:

class Foo
{
  void bar() { writefln("OH HAI WURLD!"); }
}

NonNull!(Foo) foo = new Foo;
foo.bar();
Foo baz = foo;

Off the top of my head, NonNull would require an opAssign, opImplicitCast to the templated type, and a way to build a full proxy of the underlying type.

It also just occured to me that this would probably break scope.  Damn.

A nicer syntax would be appreciated, but probably isn't a requirement.  :)

  -- Daniel

On Wed, 31 Dec 2008 08:37:56 +0300, Daniel Keep <daniel.keep.lists@gmail.com> wrote:

>
>
> dsimcha wrote:
>>>> 50% of the bugs that people run into today, coding with C# in our
>>>> platform, and the same is true of Java for that matter, are probably
>>>> null reference exceptions.
>>  Yes, but even if this is accurate, it's misleading.  Very seldom are null
>> references difficult-to-find, time-consuming bugs.  Most other things in D and
>> other languages that are designed to prevent specific classes of bugs are designed
>> to prevent extremely aggravating, hard to track down bugs.
>>  For example, GC was invented because tracking down memory leaks is a huge PITA.
>> Bounds checking was invented because off-by-one errors can be subtle and hard to
>> find.  Null reference errors, on the other hand, manifest themselves in a
>> predictable way (segfaults), and if you're using the safe memory model, i.e.
>> bounds checking, no pointers, that's about the only thing a segfault could mean.
>> Furthermore, null checking could be added to an implementation kind of like bounds
>> checking without changing the language spec.  I haven't made up my mind fully yet,
>> but I question whether adding lots of complexity to the language to prevent a very
>> easy to track down class of bugs, even if it's a common one, is justified.
>
> I find this counterpoint to be a bit iffy.  The problem with null dereference problems isn't knowing that they're there: that's the easy part.  You helpfully get an exception to the face when that happens. The hard part is figuring out *where* the problem originally occurred. It's not when the exception is thrown that's the issue; it's the point at which you placed a null reference in a slot where you shouldn't have.
>
> Yes, we have invariants and contracts; but inevitably you're going to forget one, and it's that one slip-up that's going to bite you in the rear.
>
> One could use roughly the same argument for non-null references as for const: you could document it, but documentation is inevitably wrong or out of date.  :P
>
> That said, I wonder what's stopping us from implementing them today. Last time I checked (admittedly, that was like a year ago) we didn't have the compile-time magic necessary to completely and transparently proxy any given object.
>
> Ideally, we should be able to do this:
>
> class Foo
> {
>    void bar() { writefln("OH HAI WURLD!"); }
> }
>
> NonNull!(Foo) foo = new Foo;
> foo.bar();
> Foo baz = foo;
>
> Off the top of my head, NonNull would require an opAssign, opImplicitCast to the templated type, and a way to build a full proxy of the underlying type.
>

You took the idea slightly wrong. The default should be non-nullable,

Foo foo = null; // error
Nullable!(Foo) foo2 = null; // ok

No implicit conversion is possible from Nullable!(Foo) to Foo (unless one that throws an Exception on null-reference, but I believe it contradicts the very idea of non-null references).

> It also just occured to me that this would probably break scope.  Damn.
>

Scope is not broken. Quite the contrary, it goes on par with its meaning - scope variable can't be null-initialized (it makes no sense at least).

> A nicer syntax would be appreciated, but probably isn't a requirement.  :)
>
>    -- Daniel

A nicer syntax is suggested (the one used by C#):

Foo nonNull = new Foo();
Foo? possiblyNull = null;

Denis Koroskin wrote:
> On Wed, 31 Dec 2008 08:37:56 +0300, Daniel Keep <daniel.keep.lists@gmail.com> wrote:
> 
>>
>>
>> dsimcha wrote:
>>>>> 50% of the bugs that people run into today, coding with C# in our
>>>>> platform, and the same is true of Java for that matter, are probably
>>>>> null reference exceptions.
>>>  Yes, but even if this is accurate, it's misleading.  Very seldom are null
>>> references difficult-to-find, time-consuming bugs.  Most other things in D and
>>> other languages that are designed to prevent specific classes of bugs are designed
>>> to prevent extremely aggravating, hard to track down bugs.
>>>  For example, GC was invented because tracking down memory leaks is a huge PITA.
>>> Bounds checking was invented because off-by-one errors can be subtle and hard to
>>> find.  Null reference errors, on the other hand, manifest themselves in a
>>> predictable way (segfaults), and if you're using the safe memory model, i.e.
>>> bounds checking, no pointers, that's about the only thing a segfault could mean.
>>> Furthermore, null checking could be added to an implementation kind of like bounds
>>> checking without changing the language spec.  I haven't made up my mind fully yet,
>>> but I question whether adding lots of complexity to the language to prevent a very
>>> easy to track down class of bugs, even if it's a common one, is justified.
>>
>> I find this counterpoint to be a bit iffy.  The problem with null dereference problems isn't knowing that they're there: that's the easy part.  You helpfully get an exception to the face when that happens. The hard part is figuring out *where* the problem originally occurred. It's not when the exception is thrown that's the issue; it's the point at which you placed a null reference in a slot where you shouldn't have.
>>
>> Yes, we have invariants and contracts; but inevitably you're going to forget one, and it's that one slip-up that's going to bite you in the rear.
>>
>> One could use roughly the same argument for non-null references as for const: you could document it, but documentation is inevitably wrong or out of date.  :P
>>
>> That said, I wonder what's stopping us from implementing them today. Last time I checked (admittedly, that was like a year ago) we didn't have the compile-time magic necessary to completely and transparently proxy any given object.
>>
>> Ideally, we should be able to do this:
>>
>> class Foo
>> {
>>    void bar() { writefln("OH HAI WURLD!"); }
>> }
>>
>> NonNull!(Foo) foo = new Foo;
>> foo.bar();
>> Foo baz = foo;
>>
>> Off the top of my head, NonNull would require an opAssign, opImplicitCast to the templated type, and a way to build a full proxy of the underlying type.
>>
> 
> You took the idea slightly wrong. The default should be non-nullable,
> 
> Foo foo = null; // error
> Nullable!(Foo) foo2 = null; // ok
> 
> No implicit conversion is possible from Nullable!(Foo) to Foo (unless one that throws an Exception on null-reference, but I believe it contradicts the very idea of non-null references).
> 
>> It also just occured to me that this would probably break scope.  Damn.
>>
> 
> Scope is not broken. Quite the contrary, it goes on par with its meaning - scope variable can't be null-initialized (it makes no sense at least).
> 
>> A nicer syntax would be appreciated, but probably isn't a requirement.  :)
>>
>>    -- Daniel
> 
> A nicer syntax is suggested (the one used by C#):
> 
> Foo nonNull = new Foo();
> Foo? possiblyNull = null;
Wouldn't this cause ambiguity with the "?:" operator?

Denis Koroskin wrote:
> [snip]
> 
> You took the idea slightly wrong. The default should be non-nullable,
> 
> Foo foo = null; // error
> Nullable!(Foo) foo2 = null; // ok

I didn't suggest that because it would require semantically breaking all existing OOP code in D.  I went with a proxy struct idea since that could conceivably be implemented in a library, and provide a stepping stone to full non-nullable by default.

> [snip]
>
>> It also just occured to me that this would probably break scope.  Damn.
>>
> 
> Scope is not broken. Quite the contrary, it goes on par with its meaning - scope variable can't be null-initialized (it makes no sense at least).

I was referring to the case of using a proxy struct.

scope NonNull!(Foo) bar = new Foo;

AFAIK, this wouldn't have the desired behaviour (auto-collecting bar at the end of scope.)

  -- Daniel

Don wrote:
> Denis Koroskin wrote:
> [snip]
>>
>>> A nicer syntax would be appreciated, but probably isn't a requirement.  :)
>>>
>>>    -- Daniel
>>
>> A nicer syntax is suggested (the one used by C#):
>>
>> Foo nonNull = new Foo();
>> Foo? possiblyNull = null;
> Wouldn't this cause ambiguity with the "?:" operator?

That's why I didn't suggest it.  This wouldn't be the only context-sensitive bit of grammar C# has.

I think "@" and "%" are still up for grabs, but aren't particularly attractive...

  -- Daniel

Denis Koroskin:
> A nicer syntax is suggested (the one used by C#):
> Foo nonNull = new Foo();
> Foo? possiblyNull = null;

That syntax is now used by various other languages, it's slowly becoming a kind of standard in modern languages that support some form of OOP. Eventually D too will probably grow it (or something close, maybe with another symbol).

Bye,
bearophile

Don wrote:
> Denis Koroskin wrote:
>> Foo nonNull = new Foo();
>> Foo? possiblyNull = null;
>
> Wouldn't this cause ambiguity with the "?:" operator?

At first, thought you might be right, and that there would some ambiguity calling constructors of nullable classes (especially given optional parentheses).

But for the life of me, I couldn't come up with a truly ambiguous example, that couldn't be resolved with an extra token or two of lookahead.

The '?' nullable-type operator is only used  in type declarations, not in expressions, and the '?:' operator always consumes a few trailing expressions.

Also (at least in C#) the null-coalesce operator (which converts nullable objects to either a non-null instance or a default value) looks like this:

  MyClass? myNullableObj = getNullableFromSomewhere();
  MyClass myNonNullObj = myNullableObj ?? DEFAULT_VALUE;

Since the double-hook is a single token, it's also unambiguous to parse.

--benji

Benji Smith wrote:
> Don wrote:
>> Denis Koroskin wrote:
>>> Foo nonNull = new Foo();
>>> Foo? possiblyNull = null;
>  >
>> Wouldn't this cause ambiguity with the "?:" operator?
> 
> At first, thought you might be right, and that there would some ambiguity calling constructors of nullable classes (especially given optional parentheses).
> 
> But for the life of me, I couldn't come up with a truly ambiguous example, that couldn't be resolved with an extra token or two of lookahead.
> 
> The '?' nullable-type operator is only used  in type declarations, not in expressions, and the '?:' operator always consumes a few trailing expressions.
> 
> Also (at least in C#) the null-coalesce operator (which converts nullable objects to either a non-null instance or a default value) looks like this:
> 
>   MyClass? myNullableObj = getNullableFromSomewhere();
>   MyClass myNonNullObj = myNullableObj ?? DEFAULT_VALUE;
> 
> Since the double-hook is a single token, it's also unambiguous to parse.
> 
> --benji

Disclaimer: I'm not an expert on compilers.  Plus, I just got up.  :P

The key is that the parser has to know what "MyClass" means before it can figure out what the "?" is for; that's why it's context-dependant. D avoids this dependency between compilation stages, because it complicates the compiler.  When the parser sees "MyClass", it *doesn't know* that it's a type, so it can't distinguish between a nullable type and an invalid ?: expression.

At least, I think that's how it works; someone feel free to correct me if it's not.  :P

  -- Daniel