On Saturday, 6 October 2012 at 04:10:28 UTC, Chad J wrote:
> On 10/05/2012 08:31 AM, Regan Heath wrote:
>> On Fri, 05 Oct 2012 05:19:13 +0100, Alex Burton
>> <alexibureplacewithzero@gmail.com> wrote:
>>
>>> On Saturday, 15 September 2012 at 17:51:39 UTC, Jonathan M Davis wrote:
>>>> On Saturday, September 15, 2012 19:35:44 Alex Rønne Petersen wrote:
>>>>> Out of curiosity: Why? How often does your code actually accept null as
>>>>> a valid state of a class reference?
>>>>
>>>> I have no idea. I know that it's a non-negligible amount of the time,
>>>> though
>>>> it's certainly true that they normally have values. But null is how you
>>>> indicate that a reference has no value. The same goes for arrays and
>>>> pointers.
>>>> Sometimes it's useful to have null and sometimes it's useful to know
>>>> that a
>>>> value can't be null. I confess though that I find it very surprising
>>>> how much
>>>> some people push for non-nullable references, since I've never really
>>>> found
>>>> null to be a problem. Sure, once in a while, you get a null
>>>> pointer/reference
>>>> and something blows up, but that's very rare in my experience, so I
>>>> can't help
>>>> but think that people who hit issues with null pointers on a regular
>>>> basis are
>>>> doing something wrong.
>>>>
>>>> - Jonathan M Davis
>>>
>>> In my experience this sort of attutide is not workable in projects
>>> with more than one developer.
>>
>> Almost all my work is on projects with multiple developers in C/C++ and
>> making extensive use of null.
>>
>>> It all works OK if everyone knows the 'rules' about when to check for
>>> null and when not to.
>>
>> As every good C/C++ developer does. The rule is simple, always check for
>> nulls on input passed to "public" functions/methods. What you do with
>> internal protected and private functions and methods is up to you (I use
>> assert).
>>
>>> Telling team members that find bugs caused by your null references
>>> that they are doing it wrong and next time should check for null is a
>>> poor substitute for having the language define the rules.
>>
>> Having language defined rules is a nice added /bonus/ it doesn't let you
>> off the hook when it comes to being "null safe" in your code.
>>
>>> A defensive attitude of checking for null everywhere like I have seen
>>> in many C++ projects makes the code ugly.
>>
>> That's a matter of opinion. I like to see null checks at the top of a
>> function or method, it makes it far more likely to be safe and it means
>> I can ignore the possibility of null from then on - making the code much
>> cleaner.
>>
>> R
>>
>
> I find this to be very suboptimal at the least.
>
> This prevents null values from traveling "up" the stack, but still allows them to move "down" (as return values) and allows them to survive multiple unrelated function calls.
>
> It catches null values once they've already ended up in a place they shouldn't be.  Too late.
>
> Nulls can also be placed into variables within structs or classes that then get passed around.  Checking for those can require some complex traversal: impractical for casual one-off checks at the start of a function in some cases.
>
> void main()
> {
> 	void* x = a(b());
> 	c();
> 	while(goobledegook)
> 	{
> 		x = p();
> 		d(x);
> 	}
> 	e(x); /+ Crash! x is null. +/
> }
>
> Where did x's null value come from?  Not a. Not p; the while loop happened to be never executed.  To say "b" would be closer, but still imprecise.  Actually it was created in the q() function that was called by u() that was called by b() which then created a class that held the null value and was passed to a() that then dereferenced the class and returned the value stored in the class that happened to be null.  nulls create very non-local bugs, and that's why they frustrate me to no end sometimes.
>
> What I really want to know is where errant null values come FROM.
>
> I also want to know this /at compile time/, because debugging run-time errors is time consuming and debugging compile-time errors is not.
>
> The above example could yield the unchecked null assignment at compile time if all of the types involved were typed as non-nullable, except for the very bare minimum that needs to be nullable.  If something is explicitly nullable, then its enclosing function/class is responsible for handling null conditions before passing it into non-nullable space.
>  If a function/class with nullable state tries to pass a null value into non-nullable space, then it is a bug.  This contains the non-locality of null values as much as is reasonably possible.
>
> Additionally, it might be nice to have a runtime nullable type that uses its object file's debugging information to remember which file/function/line that its null value originated from (if it happens to be null at all).  This would make for some even better diagnostics when code that HAS to deal with null values eventually breaks and needs to dump a stack trace on some poor unsuspecting sap (that isn't me) or ideally sends an email to the responsible staff (which is me).

>returned the value stored in the class that happened to be null.

Happened? "I was driving carefully and then it happened I drove into the tree, officer." Every function should define its interface, its contract with the outside world. If a() function returns a pointer it is a part of the contract whether it can be null. Two possibilities:

A) The contract says it can be null. Then it is your duty to check for null. Period. Learn to read the signs before you start driving. You assinged the value without checking, it is your fault, not a()'s, not the language's.

B) The description of a() says the return value cannot be null. Then a() should check its return value before returning or make otherwise sure it is not null. If it returns null it is a bug. One of the infinite number of possible bugs that can happen. Again it is not the problem of the language. The problem of divergence of specification and code is a human problem that cannot be solved formally. Insistance on formal tools is a misunderstanding that leads to design bloat and eventually failure (Ada).

D competes directly with C++ as Ada did before. Ada drowned under the weight of its "safety" and so will D if it goes the same route. The only thing needed now are mature compilers and good systems API integration. If anything I would rather consider removing features from the language than adding them.

Franciszek Czekała:

> Insistance on formal tools is a misunderstanding that leads to design bloat and eventually failure (Ada).
>
> D competes directly with C++ as Ada did before. Ada drowned under the weight of its "safety" and so will D if it goes the same route. The only thing needed now are mature compilers and good systems API integration. If anything I would rather consider removing features from the language than adding them.

Ada has not "failed", it's a niche language, but at the moment in its niche (high integrity code) it's used and I think there its usage is growing. (And Ada is used far more than D, there are many important system that use Ada, unlike D).

I think the usage of formal tools is slowly growing (despite being tiny).

Probably Ada has failed to become more widespread mostly because its syntax requires to write too much code and to state too many things two times. And because it's Pascal-like. And maybe a bit because of its military origins too.

And while Ada/Spark are safe, there are more modern ways to obtain some safety that require to write less code. You see this a little even in Rust.

D is not half as safe as Ada, D is C-derived, D syntax allows to write code quite more succinct than C# code. So comparing Ada and D, despite D likes some extra safety compared to C++, is not so meaningful.

Bye,
bearophile

On 10/06/2012 10:18 AM, "Franciszek Czekała" <home@valentimex.com>" wrote:
> Every function should define its interface, its contract
> with the outside world. If a() function returns a pointer it is a part
> of the contract whether it can be null.

The default should be it can't be null. Why would it be null?

> Two possibilities:
>
> A) The contract says it can be null. Then it is your duty to check for
> null. Period. Learn to read the signs before you start driving. You
> assinged the value without checking, it is your fault, not a()'s, not
> the language's.
>

It does not matter whose fault it is. The tree/car/software is broken
already. Google 'automatic braking system'.

> B) The description of a() says the return value cannot be null. Then a()
> should check its return value before returning or make otherwise sure it
> is not null. If it returns null it is a bug. One of the infinite number
> of possible bugs that can happen. Again it is not the problem of the
> language. The problem of divergence of specification and code is a human
> problem that cannot be solved formally.

If the contract does not have to talk about null values when they are
unimportant, the problem does not even occur.

On 10/06/2012 04:18 AM, "Franciszek Czekała" <home@valentimex.com>" wrote:
> On Saturday, 6 October 2012 at 04:10:28 UTC, Chad J wrote:
>>
>> I find this to be very suboptimal at the least.
>>
>> This prevents null values from traveling "up" the stack, but still
>> allows them to move "down" (as return values) and allows them to
>> survive multiple unrelated function calls.
>>
>> It catches null values once they've already ended up in a place they
>> shouldn't be. Too late.
>>
>> Nulls can also be placed into variables within structs or classes that
>> then get passed around. Checking for those can require some complex
>> traversal: impractical for casual one-off checks at the start of a
>> function in some cases.
>>
>> void main()
>> {
>> void* x = a(b());
>> c();
>> while(goobledegook)
>> {
>> x = p();
>> d(x);
>> }
>> e(x); /+ Crash! x is null. +/
>> }
>>
>> Where did x's null value come from? Not a. Not p; the while loop
>> happened to be never executed. To say "b" would be closer, but still
>> imprecise. Actually it was created in the q() function that was called
>> by u() that was called by b() which then created a class that held the
>> null value and was passed to a() that then dereferenced the class and
>> returned the value stored in the class that happened to be null. nulls
>> create very non-local bugs, and that's why they frustrate me to no end
>> sometimes.
>>
>> What I really want to know is where errant null values come FROM.
>>
>> I also want to know this /at compile time/, because debugging run-time
>> errors is time consuming and debugging compile-time errors is not.
>>
>> The above example could yield the unchecked null assignment at compile
>> time if all of the types involved were typed as non-nullable, except
>> for the very bare minimum that needs to be nullable. If something is
>> explicitly nullable, then its enclosing function/class is responsible
>> for handling null conditions before passing it into non-nullable space.
>> If a function/class with nullable state tries to pass a null value
>> into non-nullable space, then it is a bug. This contains the
>> non-locality of null values as much as is reasonably possible.
>>
>> Additionally, it might be nice to have a runtime nullable type that
>> uses its object file's debugging information to remember which
>> file/function/line that its null value originated from (if it happens
>> to be null at all). This would make for some even better diagnostics
>> when code that HAS to deal with null values eventually breaks and
>> needs to dump a stack trace on some poor unsuspecting sap (that isn't
>> me) or ideally sends an email to the responsible staff (which is me).
>
>> returned the value stored in the class that happened to be null.
>
> Happened? "I was driving carefully and then it happened I drove into the
> tree, officer." Every function should define its interface, its contract
> with the outside world. If a() function returns a pointer it is a part
> of the contract whether it can be null. Two possibilities:
>
> A) The contract says it can be null. Then it is your duty to check for
> null. Period. Learn to read the signs before you start driving. You
> assinged the value without checking, it is your fault, not a()'s, not
> the language's.
>

I am unconvinced by the driving analogy.  When driving, most of the important bits become muscle memory (acceleration, braking, turn signals, etc) and the rest falls under the category of "be aware".  The factor in our advantage is that awareness in driving usually only requires you to focus on one thing at a time: "turn your head before changing lanes or turning", "look at the sides of the road", "check your rear view", etc.

Programming involves the management of complex logical relationships. It is more akin to mathematics.  I could continue, but I'll stop here and leave it at "I'm unconvinced".

Even if I grant the premise, I'll expand on what Timon wrote:
We'd have a lot less accidents if well-designed robots drove our vehicles for us (with manual overrides, of course).

> B) The description of a() says the return value cannot be null. Then a()
> should check its return value before returning or make otherwise sure it
> is not null. If it returns null it is a bug. One of the infinite number
> of possible bugs that can happen. Again it is not the problem of the
> language. The problem of divergence of specification and code is a human
> problem that cannot be solved formally. Insistance on formal tools is a
> misunderstanding that leads to design bloat and eventually failure (Ada).
>

As I understand it, you would have written my code snippet this way:

void main()
{
    MyType j = b();
    assert( j !is null );
    assert( j.qux !is null );
    assert( j.qux.yarly !is null ); /+ Crash! yarly is null. +/

    void* x = a(j);
    assert( x !is null );

    c();
    while(goobledegook)
    {
        x = p();
        assert(x !is null);

        d(x);

        // Note: be sure to put this one in!
        //   The previous dev forgot it...
        assert(x !is null);
    }
    e(x);
}

If you feel that this is misrepresentative, then please provide your own.  As it stands: give me non-null types, because I like mine better.

> D competes directly with C++ as Ada did before. Ada drowned under the
> weight of its "safety" and so will D if it goes the same route. The only
> thing needed now are mature compilers and good systems API integration.
> If anything I would rather consider removing features from the language
> than adding them.
>
>

Given the two snippets I currently see in my mind's eye that represent our two different philosophies, I do not see the "weight" of this safety.  I think it makes for much more concise code, and a faster development process because I don't have to worry about inane stuff like checking for nulls.  If I don't have to worry about inane stuff, then I can allocate those short-term memory slots to solving problems that are actually novel and interesting.

On 10/06/2012 04:18 AM, "Franciszek Czekała" <home@valentimex.com>" wrote:
> B) The description of a() says the return value cannot be null. Then a()
> should check its return value before returning or make otherwise sure it
> is not null. If it returns null it is a bug. One of the infinite number
> of possible bugs that can happen. Again it is not the problem of the
> language. The problem of divergence of specification and code is a human
> problem that cannot be solved formally. Insistance on formal tools is a
> misunderstanding that leads to design bloat and eventually failure (Ada).
>
> D competes directly with C++ as Ada did before. Ada drowned under the
> weight of its "safety" and so will D if it goes the same route. The only
> thing needed now are mature compilers and good systems API integration.
> If anything I would rather consider removing features from the language
> than adding them.
>
>

I have another thing to bring up: why the hating on Ada?

Because, if you're hating on Ada because it requires a bunch of extra boilerplate and verbosity that most programmers would find unnecessary, then I will happily join you in hating on Ada (and Pascal and the like).  Keep in mind that one of D's current idiomatic objectives seems to be the elimination of as much boilerplate as possible.

I firmly believe that safety and brevity are NOT exclusive to each other.  Moreover, they even synergize well if the language and tools are designed right: verbose code will be less readable and therefore more prone to error.

> void main()
> {
> 	void* x = a(b());
> 	c();
> 	while(goobledegook)
> 	{
> 		x = p();
> 		d(x);
> 	}
> 	e(x); /+ Crash! x is null. +/
> }
>
> Where did x's null value come from?  Not a. Not p; the while loop happened to be never executed.  To say "b" would be closer, but still imprecise.  Actually it was created in the q() function that was called by u() that was called by b() which then created a class that held the null value and was passed to a() that then dereferenced the class and returned the value stored in the class that happened to be null.  nulls create very non-local bugs, and that's why they frustrate me to no end sometimes.

Since this thread's attracted lots of commotion I thought I'd just drop by and +1 for non-nullable types, and +1 for your arguments.

I keep wondering, though, if it is 'enough' to solve the null problem, or if it would be possible to devise a more general mechanism for solving other problems too, like say, the fact that certain integers have to always be positive, or if you want to go more general, that a certain relationship must hold between two structures...

Not having used D's invariants so far (well, I haven't used D itself for a real project actually)... what's stopping D's invariant mechanism from handling all this?

http://dlang.org/class.html#invariants (as is typical of D documentation, this says nothing about invariants on structs, but the page about structs says that they support invariants with an X.)

I mean, problems are detected at runtime this way, and slightly too late, but still, it would be better than most popular languages that can't do anything about nulls at all. Since D's devs don't even seem to have enough time to implement D as described in TDPL (published more than two years ago), I wouldn't expect to see this feature in the D language in the near future.

On 10/07/2012 02:22 AM, David Piepgrass wrote:
>> void main()
>> {
>> void* x = a(b());
>> c();
>> while(goobledegook)
>> {
>> x = p();
>> d(x);
>> }
>> e(x); /+ Crash! x is null. +/
>> }
>>
>> Where did x's null value come from? Not a. Not p; the while loop
>> happened to be never executed. To say "b" would be closer, but still
>> imprecise. Actually it was created in the q() function that was called
>> by u() that was called by b() which then created a class that held the
>> null value and was passed to a() that then dereferenced the class and
>> returned the value stored in the class that happened to be null. nulls
>> create very non-local bugs, and that's why they frustrate me to no end
>> sometimes.
>
> Since this thread's attracted lots of commotion I thought I'd just drop
> by and +1 for non-nullable types, and +1 for your arguments.
>
> I keep wondering, though, if it is 'enough' to solve the null problem,
> or if it would be possible to devise a more general mechanism for
> solving other problems too, like say, the fact that certain integers
> have to always be positive, or if you want to go more general, that a
> certain relationship must hold between two structures...
>

I agree.

> Not having used D's invariants so far (well, I haven't used D itself for
> a real project actually)... what's stopping D's invariant mechanism from
> handling all this?
>
> http://dlang.org/class.html#invariants (as is typical of D
> documentation, this says nothing about invariants on structs, but the
> page about structs says that they support invariants with an X.)
>
> I mean, problems are detected at runtime this way, and slightly too
> late, but still, it would be better than most popular languages that
> can't do anything about nulls at all. Since D's devs don't even seem to
> have enough time to implement D as described in TDPL (published more
> than two years ago), I wouldn't expect to see this feature in the D
> language in the near future.

Invariants might work... create a proxy struct and then have assignment always check the invariant.  I don't like the idea that they get removed during release mode.  I'd like to be able to control which checks I pay for.  I think this might just mean that the important thing is overloading assignment to do checks, which could be done in principle, and this could work without invariants and thus give the desired control.

I just haven't had much luck creating proxy types in the past.  As of some months ago, D just wasn't there yet.  :(

In another post in this thread I mentioned something similar:

>
> It would be cool to have templates like this:
>
> 51.  bool isPrime(int val)
> 52.  {
> 53.      ...
> 54.  }
>
> 101. Watch!(int,&isPrime) primeNumber = primeGenerator();
> 102. primeNumber = 8;
> 103. doStuff();
> 104. checkPrime(primeNumber); /+ Crash! +/
> Error: checkPrime(primeNumber): primeNumber is not prime.
> primeNumber: isPrime returned false after assignment at
>   (foobar.d, line 102)
>
> ~or~
>
> 101. Constrain!(int,&isPrime) primeNumber = primeGenerator();
> 102. primeNumber = 8; /+ Crash! +/
> 103. doStuff();
> 104. checkPrime(primeNumber);
> foobar.d, line 102: isPrime returned false after assignment.
>
> For convenience one could define this:
> alias Constrain!(int,&isPrime) PrimeInt;

Maybe these could be turned on/off in release mode on a case-by-case basis.  I would probably leave the checks and tracking in always, with the one exception of code that has been shown (with profiling) to need the extra performance.