On Wednesday, 3 October 2012 at 16:58:52 UTC, Maxim Fomin wrote:
> How much code would be broken by moving nullable references from current state to "question mark notation"?
That's another question :]

> I expect that non-nullable class objects (called references here) addition (if there is no objections to idea in general) would not break much code and would not request vast syntax changes. And it likely can be done by still defaulting to nullable references. For example, it can be done with the help of @nonnullable (like immutable) type qualifier and semantic check of operations involving references with such qualifier.
Sounds like a deal for now, but @nonnullable will only work with class references and anything else will be an error. So directly attaching it to the type (like the questionmark) makes more sense.

> Anyway, my estimation of probability of accepting constructions like "type&", "type*?", etc and defaulting to non-null references is very low, at least for D2.
That's right, but let's use the youth of the language to change this. I guess many will hate me if we do so.

On Wednesday, 3 October 2012 at 16:58:52 UTC, Maxim Fomin wrote:
> How much code would be broken by moving nullable references from current state to "question mark notation"?
That's another question :]

> I expect that non-nullable class objects (called references here) addition (if there is no objections to idea in general) would not break much code and would not request vast syntax changes. And it likely can be done by still defaulting to nullable references. For example, it can be done with the help of @nonnullable (like immutable) type qualifier and semantic check of operations involving references with such qualifier.
Sounds like a deal for now, but @nonnullable will only work with class references and anything else will be an error. So directly attaching it to the type (like the questionmark) makes more sense.

> Anyway, my estimation of probability of accepting constructions like "type&", "type*?", etc and defaulting to non-null references is very low, at least for D2.
That's right, but let's use the youth of the language to change this. I guess many will hate me if we do so.

On Wednesday, 3 October 2012 at 16:58:52 UTC, Maxim Fomin wrote:
> How much code would be broken by moving nullable references from current state to "question mark notation"?
That's another question :]

> I expect that non-nullable class objects (called references here) addition (if there is no objections to idea in general) would not break much code and would not request vast syntax changes. And it likely can be done by still defaulting to nullable references. For example, it can be done with the help of @nonnullable (like immutable) type qualifier and semantic check of operations involving references with such qualifier.
Sounds like a deal for now, but @nonnullable will only work with class references and anything else will be an error. So directly attaching it to the type (like the questionmark) makes more sense.

> Anyway, my estimation of probability of accepting constructions like "type&", "type*?", etc and defaulting to non-null references is very low, at least for D2.
That's right, but let's use the youth of the language to change this. I guess many will hate me if we do so.

On Wednesday, 3 October 2012 at 16:33:15 UTC, Simen Kjaeraas wrote:
> On 2012-10-03, 18:12,  wrote:
>

> They make sure you never pass null to a function that doesn't expect null - I'd say that's a nice advantage.

No, it is meaningless. If you have a class which is supposed to hold a prime number and you pass it to a function are you going to check each time that the value is indeed prime? That would kill the efficiency of your program guaranteed. So you would be happy to know that the reference is non-null but you would take it for granted the value is indeed prime? Does it make any sense?
I maintain that this non-null "advantage" does not warrant to make the language more complicated even by a tiny bit. It is dwarfed by normal considerations related to program correctness.

With default null references:
A)either null is an expected non-value for the type (like in the chess example), checking for it is part of normal processing then
B) or null is not a valid value, then there is no need to check for it. If you get a null reference it is a bug. It is like getting a 15 for your prime number. You do not put checks like that in your code. You test your prime generation routine not the consumers. If your function gets a null reference when it should not, some other part of your program is buggy. You do not process bugs in your code - you remove them from it.

However with D, dereferencing an uninitialized reference is well defined - null is not random data: you get a well-defined exception and you know you are dealing with unitialized data. This is easy to fix. You just go up the stack and check where the reference comes from. Much easier probably than finding out why your prime numbers turn out to be divisible by 3. How about introducing some syntax that will rule this out?

To quote (loosely) Mr. Walter Bright from another discussion: how many current bugs in dmd are related to default null references?

Franciszek Czekała:

> I maintain that this non-null "advantage" does not warrant to make the language more complicated even by a tiny bit. It is dwarfed by normal considerations related to program correctness.

Surely there are more important things to care about. But a non-null system has right the purpose of allowing you to take more care of the program logic and less about possible null values.


> With default null references:
> A)either null is an expected non-value for the type (like in the chess example), checking for it is part of normal processing then

A well implemented not-nullable system forces you to verify the possible presence of null of nullable values. And if you handle the null value in a "if" clause then the compiler will assume the reference is not null in the other clause. This essentially means the type of that reference is different inside the two clauses.

A small program that shows two or three important things std.typecons.Nullable isn't able to do:


import std.stdio, std.algorithm, std.typecons;

alias Nullable!(int, -1) Position;

void foo(int[] a, Position pos) /*nothrow*/ { // allow this to be nothrow
    if (pos.isNull) {
        return;
    } else {
        a[pos] = 10; // perform no nullness test here, optimization
    }
}

void bar(int[] a, Position pos) {
    a[pos] = 10; // maybe: require a test here?
}

void main() {
    auto data = [1, 2, 3, 4, 5];
    auto p = Position(countUntil(data, 7));
    foo(data, p);
    writeln(data);
}


> However with D, dereferencing an uninitialized reference is well defined - null is not random data: you get a well-defined exception and you know you are dealing with unitialized data. This is easy to fix. You just go up the stack and check where the reference comes from.

Even better: avoid similar problems statically, with a not-nullable extension of the type system, so there is no need to debug your program after a run.


> Much easier probably than finding out why your prime numbers turn out to be divisible by 3. How about introducing some syntax that will rule this out?

If D programs contain prime number classes as often as not-null references then adding a syntax to statically rule out not-prime numbers is an acceptable idea. But the initial assumption is false. Walter&Andrei have added @disable to try to allow programmers to disallow the presence of divisible numbers inside instances of a prime class too. But I am not sure the final result is good enough.


> To quote (loosely) Mr. Walter Bright from another discussion: how many current bugs in dmd are related to default null references?

DMD source code is a compiler, it's not representative of all kinds of D programs. I think there is a class of commercial programs, or pointer-heavy programs that enjoy having not-null references.

Even if null-related bugs are not common in D code, having a language help you reduce one class of bugs helps you program faster. I am sometimes able to write working (and almost "correct") C programs, but D safeties allows me to write them faster. If you ask to Scala programmers they will tell you they are quite happy to not have to worry much about nulls when they write idiomatic Scala code, while they will tell you they have to keep more care when they inter-operate with Java code that sometimes has nulls.

Bye,
bearophile

On Wednesday, 3 October 2012 at 17:37:14 UTC, Franciszek Czekała wrote:
> No, it is meaningless. If you have a class which is supposed to hold a prime number and you pass it to a function are you going to check each time that the value is indeed prime? That would kill the efficiency of your program guaranteed. So you would be happy to know that the reference is non-null but you would take it for granted the value is indeed prime? Does it make any sense?
You have to decide which information should be available at compile- and runtime. And that is not easy. In general, it's always better to have information present at compile time. To come back to your example:

class Number {
    this(int i) {
        this.i = i;
    }

    const int i;
}

class PrimeNumber : Number {
    this(int i) {
        // Check whether i is a prime number or not...
        ....

        super(i);
    }
}

void func(PrimeNumber num);

That's a way to check this at compile time and pass the number without runtime checks between prime number functions. But it requires i to be constant throughout the lifetime of Number.

It always depends on the context if the use of an extra class makes sense.

> I maintain that this non-null "advantage" does not warrant to make the language more complicated even by a tiny bit. It is dwarfed by normal considerations related to program correctness.
>
> With default null references:
> A)either null is an expected non-value for the type (like in the chess example), checking for it is part of normal processing then
Great, use a question mark.

> B) or null is not a valid value, then there is no need to check for it. If you get a null reference it is a bug. It is like getting a 15 for your prime number. You do not put checks like that in your code. You test your prime generation routine not the consumers. If your function gets a null reference when it should not, some other part of your program is buggy. You do not process bugs in your code - you remove them from it.
Contract programming comes into play. But still, you have to write contracts containing all those assertions. In libraries you can't even use contracts in most cases.

> However with D, dereferencing an uninitialized reference is well defined - null is not random data: you get a well-defined exception and you know you are dealing with unitialized data. This is easy to fix. You just go up the stack and check where the reference comes from. Much easier probably than finding out why your prime numbers turn out to be divisible by 3. How about introducing some syntax that will rule this out?
If you write an application, indeed, it's easy to fix.

> To quote (loosely) Mr. Walter Bright from another discussion: how many current bugs in dmd are related to default null references?
DMD is an application, not a library.

On Wednesday, 3 October 2012 at 18:12:51 UTC, Henning Pohl wrote:
> class PrimeNumber : Number {
>    this(int i) {
>        // Check whether i is a prime number or not...
>        ....
>
>        super(i);
>    }
>}

This is a good example in that validity of the data can be checked from inside the class but not the validity of the reference. So indeed these two things are not entirely equivalent and references need more of the support from the language. Still my point is that, in any program lots of things have to be taken on faith. Checking and rechecking everything can sink a Titanic. A line has to be drawn somewhere. Program correctness will never be syntax based only. Default null references are a very reasonable approach. C++ does not check anything and is still going strong :). Let's not get paranoid. I'd rather see bug-free 64bit dmd for windows for the current version of the language and some libraries for easy windows programming than endless ideas for language additions. D already borders on being too complex.

On 2012-10-03, 19:31,  wrote:

> On Wednesday, 3 October 2012 at 16:33:15 UTC, Simen Kjaeraas wrote:
>> On 2012-10-03, 18:12,  wrote:
>>
>
>> They make sure you never pass null to a function that doesn't expect null - I'd say that's a nice advantage.
>
> No, it is meaningless. If you have a class which is supposed to hold a prime number and you pass it to a function are you going to check each time that the value is indeed prime? That would kill the efficiency of your program guaranteed. So you would be happy to know that the reference is non-null but you would take it for granted the value is indeed prime? Does it make any sense?

I don't know the field you're working in, but prime numbers are rarely
a problem in my field. Null pointers, I have to worry about every day.
In our product there is a null pointer bug that has thus far only
occurred on production hardware, where we're not allowed to run
debuggers (gawd dangit), and only once every few weeks. We know where
it happens (third party library) and that non-nullable references would
have ensured it'd never show up.



-- 
Simen

On 10/03/2012 07:31 PM, "Franciszek Czekała" <home@valentimex.com>" wrote:
> ...
>
> To quote (loosely) Mr. Walter Bright from another discussion: how many
> current bugs in dmd are related to default null references?

More than zero.

On 10/03/2012 06:59 PM, Maxim Fomin wrote:
> On Wednesday, 3 October 2012 at 16:36:15 UTC, Henning Pohl wrote:
>> Just put something like a questionmark behind the reference type to
>> indicate that it's nullable.
> ...
>> Not really. It's all about one question mark for example.
>
> How much code would be broken by moving nullable references from current
> state to "question mark notation"?
>

Most of it, and fixing it up quickly may require adding explicit assertions together with the question marks.

> I expect that non-nullable class objects (called references here)
> addition (if there is no objections to idea in general) would not break
> much code and would not request vast syntax changes. And it likely can
> be done by still defaulting to nullable references. For example, it can
> be done with the help of @nonnullable (like immutable) type qualifier
> and semantic check of operations involving references with such
> qualifier. Anyway, my estimation of probability of accepting
> constructions like "type&", "type*?", etc and defaulting to non-null
> references is very low, at least for D2.

'non-null reference' is an abominable term. It shouldn't have 'null' in
it. I'd just call it a 'reference'. Having a reference type whose
values always refer to an object, while allowing dereferencing
nullable-by-default references at the same time is not worth the effort.

Imho, either break all code or leave it out.