September 27, 2009
Jeremie Pelletier wrote:
> Ary Borenszweig wrote:
>> Walter Bright wrote:
>>> Daniel Keep wrote:
>>>> "But the user will just assign to something useless to get around that!"
>>>>
>>>> You mean like how everyone wraps every call in try{...}catch(Exception
>>>> e){} to shut the damn exceptions up?
>>>
>>> They do just that in Java because of the checked-exceptions thing. I have a reference to Bruce Eckel's essay on it somewhere in this thread. The observation in the article was it wasn't just moron idiot programmers doing this. It was the guru programmers doing it, all the while knowing it was the wrong thing to do. The end result was the feature actively created the very problems it was designed to prevent.
>>>
>>>
>>>> Or uses pointer arithmetic and
>>>> casts to get at those pesky private members?
>>>
>>> That's entirely different, because privacy is selected by the programmer, not the language. I don't have any issue with a user-defined type that is non-nullable (Andrei has designed a type constructor for that).
>>>
>>>
>>>> If someone is actively trying to break the type system, it's their
>>>> goddamn fault!  Honestly, I don't care about the hacks they employ to
>>>> defeat the system because they're going to go around blindly shooting
>>>> themselves in the foot no matter what they do.
>>>
>>> True, but it's still not a good idea to design a language feature that winds up, in reality, encouraging bad programming practice. It encourages bad practice in a way that is really, really hard to detect in a code review.
>>>
>>> I like programming mistakes to be obvious, not subtle. There's nothing subtle about a null pointer exception. There's plenty subtle about the wrong default value.
>>>
>>>
>>>> And what about the people who AREN'T complete idiots, who maybe
>>>> sometimes just accidentally trip and would quite welcome a safety rail
>>>> there?
>>>
>>> Null pointer seg faults *are* a safety rail. They keep an errant program from causing further damage.
>>
>> Null pointer seg faults *not being able to happen* are much more safe. :)
> 
> There is no such thing as "not being able to happen" :)
> 
> Object thisCannotPossiblyBeNullInAnyWayWhatsoever = cast(Object)null;
> 
> I seem to be the only one who sees Walter's side of things in this thread :o)
> 
> For nonnulls to *really* be enforcable you'd have to get rid of the cast system entirely.

It's a systems programming language. You can screw up the type system if you really want to. But then it would still fall back to the lovely segfault. If you don't screw with it, you're safe with static checking. It's a clean win.


-- 
Tomasz Stachowiak
http://h3.team0xf.com/
h3/h3r3tic on #D freenode
September 27, 2009
Ary Borenszweig:

> Please, please, please, do some fun little project in Java or C# and drop the idea of initializing variables whenever you declare them. Just leave them like this:
[...]
> Until you do that, you won't understand what most people are answering to you.

Something similar happens in other fields too. I have had long discussions with nonbiologists about evolutionary matters. Later I have understood that those discussions weren't very useful, the best thing for them, to understand why and how evolution happens, is to do a week of field etology, studying how insects on a wild lawn interact, compete, fight and cooperate with each other. If you have some expert that shows you things in just a week you can see lot of things. At that point you have some common frame of reference that allows you to understand how evolution happens :-) Practical experience is important.

Bye,
bearophile
September 27, 2009
On 2009-09-26 22:07:00 -0400, Walter Bright <newshound1@digitalmars.com> said:

> [...] The facilities in D enable one to construct a non-nullable type, and they are appropriate for many designs. I just don't see them as a replacement for *all* reference types.

As far as I understand this thread, no one here is arguing that non-nullable references/pointers should replace *all* reference/pointer types. The argument made is that non-nullable should be the default and nullable can be specified explicitly any time you need it.

So if you need a reference you use "Object" as the type, and if you want that reference to be nullable you write "Object?". The static analysis can then assert that your code properly check for null prior dereferencing a nullable type and issues a compilation error if not.

-- 
Michel Fortin
michel.fortin@michelf.com
http://michelf.com/

September 27, 2009
Jeremie Pelletier wrote:
> Ary Borenszweig wrote:
>> Walter Bright wrote:
>>> Daniel Keep wrote:
>>>> "But the user will just assign to something useless to get around that!"
>>>>
>>>> You mean like how everyone wraps every call in try{...}catch(Exception
>>>> e){} to shut the damn exceptions up?
>>>
>>> They do just that in Java because of the checked-exceptions thing. I have a reference to Bruce Eckel's essay on it somewhere in this thread. The observation in the article was it wasn't just moron idiot programmers doing this. It was the guru programmers doing it, all the while knowing it was the wrong thing to do. The end result was the feature actively created the very problems it was designed to prevent.
>>>
>>>
>>>> Or uses pointer arithmetic and
>>>> casts to get at those pesky private members?
>>>
>>> That's entirely different, because privacy is selected by the programmer, not the language. I don't have any issue with a user-defined type that is non-nullable (Andrei has designed a type constructor for that).
>>>
>>>
>>>> If someone is actively trying to break the type system, it's their
>>>> goddamn fault!  Honestly, I don't care about the hacks they employ to
>>>> defeat the system because they're going to go around blindly shooting
>>>> themselves in the foot no matter what they do.
>>>
>>> True, but it's still not a good idea to design a language feature that winds up, in reality, encouraging bad programming practice. It encourages bad practice in a way that is really, really hard to detect in a code review.
>>>
>>> I like programming mistakes to be obvious, not subtle. There's nothing subtle about a null pointer exception. There's plenty subtle about the wrong default value.
>>>
>>>
>>>> And what about the people who AREN'T complete idiots, who maybe
>>>> sometimes just accidentally trip and would quite welcome a safety rail
>>>> there?
>>>
>>> Null pointer seg faults *are* a safety rail. They keep an errant program from causing further damage.
>>
>> Null pointer seg faults *not being able to happen* are much more safe. :)
> 
> There is no such thing as "not being able to happen" :)
> 
> Object thisCannotPossiblyBeNullInAnyWayWhatsoever = cast(Object)null;

Object is not-nullable, Object? (or whatever syntax you like) is nullable. So that line is a compile-time error: you can't cast a null to an Object (because Object *can't* be null).

You might be the only one here that understands Walter's point. But Walter is wrong. ;-)
September 27, 2009
Ary Borenszweig wrote:
> Walter Bright wrote:
>> Denis Koroskin wrote:
>>  > On Sat, 26 Sep 2009 22:30:58 +0400, Walter Bright
>>  > <newshound1@digitalmars.com> wrote:
>>  >> D has borrowed ideas from many different languages. The trick is to
>>  >> take the good stuff and avoid their mistakes <g>.
>>  >
>>  > How about this one:
>>  > http://sadekdrobi.com/2008/12/22/null-references-the-billion-dollar-mistake/ 
>>
>>  >
>>  >
>>  > :)
>>
>> I think he's wrong.
> 
> Please, please, please, do some fun little project in Java or C# and drop the idea of initializing variables whenever you declare them. Just leave them like this:
> 
> int i;
> 
> and then later initialize them when you need them, for example different values depending on some conditions. Then you'll realize how powerful is having the compiler stop variables that are not initialized *in the context of a function, not necessarily in the same line of their declaration*. It's always a win: you get a compile time error, you don't have to wait to get an error at runtime.
> 
> Until you do that, you won't understand what most people are answering to you.
> 
> But I know what you'll answer. You'll say "what about pointers?", "what about ref parameters?", "what about out parameters?", and then someone will say to you "C# has them", etc, etc.
> 
> No point disussing non-null variables without also having the compiler stop uninitialized variables.

All null values are uninitialized, but not all initializers are null, especially the void initializer. You can't always rely on initializers in your algorithms, you can always rely on null.

Kinda like all pointers are references, but not all references are pointers. You can't do pointer arithmetic on references.
September 27, 2009
Jarrett Billingsley wrote:
> On Sat, Sep 26, 2009 at 11:23 PM, Jeremie Pelletier <jeremiep@gmail.com> wrote:
> 
>> There is no such thing as "not being able to happen" :)
>>
>> Object thisCannotPossiblyBeNullInAnyWayWhatsoever = cast(Object)null;
>>
>> I seem to be the only one who sees Walter's side of things in this thread
>> :o)
> 
> Why the hell would the compiler allow that to begin with? Why bother
> implementing nonnull references only to allow the entire system to be
> broken?

Because D is a practical language that let the programmer do whatever he wants, even shoot his own foot if he wants to. Doing so just isn't as implicit as in C.

Walter understands there are some cases where you want to override the type system, that's why casts are in D, too many optimizations rely on it.
September 27, 2009
Jeremie Pelletier wrote:
> Ary Borenszweig wrote:
>> Walter Bright wrote:
>>> Denis Koroskin wrote:
>>>  > On Sat, 26 Sep 2009 22:30:58 +0400, Walter Bright
>>>  > <newshound1@digitalmars.com> wrote:
>>>  >> D has borrowed ideas from many different languages. The trick is to
>>>  >> take the good stuff and avoid their mistakes <g>.
>>>  >
>>>  > How about this one:
>>>  > http://sadekdrobi.com/2008/12/22/null-references-the-billion-dollar-mistake/ 
>>>
>>>  >
>>>  >
>>>  > :)
>>>
>>> I think he's wrong.
>>
>> Please, please, please, do some fun little project in Java or C# and drop the idea of initializing variables whenever you declare them. Just leave them like this:
>>
>> int i;
>>
>> and then later initialize them when you need them, for example different values depending on some conditions. Then you'll realize how powerful is having the compiler stop variables that are not initialized *in the context of a function, not necessarily in the same line of their declaration*. It's always a win: you get a compile time error, you don't have to wait to get an error at runtime.
>>
>> Until you do that, you won't understand what most people are answering to you.
>>
>> But I know what you'll answer. You'll say "what about pointers?", "what about ref parameters?", "what about out parameters?", and then someone will say to you "C# has them", etc, etc.
>>
>> No point disussing non-null variables without also having the compiler stop uninitialized variables.
> 
> All null values are uninitialized, but not all initializers are null, especially the void initializer.

I don't see your point here. "new Object()" is not a null intiializer nor "1"... so?

 You can't always rely on initializers
> in your algorithms, you can always rely on null.

Yes, I can always rely on initializers in my algorithm. I can, if the compiler lets me safely initialize them whenever I want, not necessarily in the line I declare them.

Just out of curiosity: have you ever programmed in Java or C#?
September 27, 2009
Jeremie Pelletier:

> I don't remember the last time I had a segfault on a null reference actually.

I have read that null deference bugs are among the most common problem in Java/C# code. I have no pointers...


> I can see what the point is with nonnull references, but I can also see its not a bulletproof solution. ie "Object foo = cast(Object)null;" would easily bypass the nonnull enforcement, resulting in a segfault the system is trying to avoid.

That's life.


> What about function parameters, a lot of parameters are optional references, which are tested and then used into functions whose parameters aren't optional. It would result in a lot of casts, something that could easily confuse people and easily generate segfaults.

By "optional" I think you mean "nullable" there.

Note that some of those casts can be avoided, because the nonnull nature of a reference can be implicitly inferred by the compiler:

Foo somefunction(Foo? foo) {
  if (foo is null) {
    ... // do something
  } else {
    // here foo can be implicitly converted to
    // a nonnullable reference, because the compiler
    // can infer that here foo can never be null.
    return foo;
}


> Alls I'm saying is, nonnull references would just take the issue from one place to another. Like Walter said, you can put a gas mask to ignore the room full of toxic gas, but that doesn't solve the gas problem in itself, you're just denyinng it exists. Then someday you forget about it, remove the mask, and suffocate.

No solution is perfect, so it's a matter of computing its pro and cons. It's hard to tell how much good a feature is before trying it. That's why I have half-seriously to implement nonullables in a branch of D2, test it and keep it only if it turns out to be good.

Bye,
bearophile
September 27, 2009

Walter Bright wrote:
> Daniel Keep wrote:
>> "But the user will just assign to something useless to get around that!"
>>
>> You mean like how everyone wraps every call in try{...}catch(Exception e){} to shut the damn exceptions up?
> 
> They do just that in Java because of the checked-exceptions thing. I have a reference to Bruce Eckel's essay on it somewhere in this thread. The observation in the article was it wasn't just moron idiot programmers doing this. It was the guru programmers doing it, all the while knowing it was the wrong thing to do. The end result was the feature actively created the very problems it was designed to prevent.

Checked exceptions are a bad example: you can't not use them.  No one is proposing to remove null from the language.  If we WERE, you would be quite correct.

But we're not.

If someone doesn't want to use non-null references, then they don't use them.

>> Or uses pointer arithmetic and
>> casts to get at those pesky private members?
> 
> That's entirely different, because privacy is selected by the programmer, not the language. I don't have any issue with a user-defined type that is non-nullable (Andrei has designed a type constructor for that).

Good grief, that's what non-null references are!

  Object foo = new Object;
      // Dear Mr. Compiler, I would like a non-nullable
      // reference to an Object, please!  Here's the object
      // I want you to use.

  Object? bar;
      // Dear Mr. Compiler, I would like a nullable reference
      // to an object, please!  Just initialise with null, thanks.

How is that not selected by the programmer?  The programmer is in complete control.  We are not asking for the language to unilaterally declare null to be a sin, we want to be given the choice to say we don't want it!

Incidentally, on the subject of non-null as a UDT, that would be a largely acceptable solution for me.  The trouble is that in order to do it, you'd need to be able to block default initialisation,

  **which is precisely what you're arguing against**

You can't have it both ways.

>> If someone is actively trying to break the type system, it's their goddamn fault!  Honestly, I don't care about the hacks they employ to defeat the system because they're going to go around blindly shooting themselves in the foot no matter what they do.
> 
> True, but it's still not a good idea to design a language feature that winds up, in reality, encouraging bad programming practice. It encourages bad practice in a way that is really, really hard to detect in a code review.

Whether or not it encourages it is impossible to determine at this juncture because I can't think of a language comparable to D that has it.

Things that are "like" it don't count.

Ignoring that, you're correct that if someone decides to abuse non-null references, it's going to be less than trivial to detect.

> I like programming mistakes to be obvious, not subtle. There's nothing subtle about a null pointer exception. There's plenty subtle about the wrong default value.

I think this is a fallacy.  You're assuming a person who is actively going out of their way to misuse the type system.  I'll repeat myself:

  Foo bar = arbitrary_default;

is harder to do than

  Foo? bar;

Which does exactly what they want: it relieves them of the need to initialise, and gives a relatively safe default value.

I mean, people could abuse a lot of things in D.  Pointers, certainly. DEFINITELY inline assembler.  But we don't get rid of them because at some point you have to say "you know what?  If you're going to play with fire, that's your own lookout."

The only way you're ever going to have a language that's actually safe no matter how ignorant, stupid or just outright suicidal the programmer is would be to implement a compiler for SIMPLE:

http://esoteric.voxelperfect.net/wiki/SIMPLE

>> And what about the people who AREN'T complete idiots, who maybe sometimes just accidentally trip and would quite welcome a safety rail there?
> 
> Null pointer seg faults *are* a safety rail. They keep an errant program from causing further damage.

Really?

"
I used to work at Boeing designing critical flight systems. Absolutely
the WRONG failure mode is to

**pretend nothing went wrong**

and happily return

**default values**

and show lovely green lights on the instrument panel. The right thing is to

**immediately inform the pilot that something went wrong and INSTANTLY SHUT THE BAD SYSTEM DOWN**

before it does something really, really bad, because now it is in an
unknown state. The pilot then follows the procedure he's trained to,
such as engage the backup.
"

Think of the compiler as the autopilot.

Pretending nothing went wrong is passing a null into a function that doesn't expect it, or shoving it into a field that's not meant to be null.

Null IS a happy default value that can be passed around without consequence from the type system.

Immediately informing the pilot is refusing to compile because the code looks like it's doing something wrong.

A NPE is the thermonuclear option of error handling.  Your program blows up, tough luck, try again.  Debugging is forensics, just like picking through a mound of dead bodies and bits of fuselage; if it's come to that, there's a problem.

Non-nullable references are the compiler (or autopilot) putting up the red flag and saying "are you really sure you want to do this?  I mean, it LOOKS wrong to me!"

>> Finally, let me re-post something I wrote the last time this came up:
>>
>>> The problem with null dereference problems isn't knowing that they're there: that's the easy part.  You helpfully get an exception to the face when that happens. The hard part is figuring out *where* the problem originally occurred. It's not when the exception is thrown that's the issue; it's the point at which you placed a null reference in a slot where you shouldn't have.
> 
> It's a lot harder to track down a bug when the bad initial value gets combined with a lot of other data first. The only time I've had a problem finding where a null came from (because they tend to fail very close to their initialization point) is when the null was caused by another memory corruption problem. Non-nullable references won't mitigate that.

Only when the nulls are assigned and used locally.

I've had code before when a null accidentally snuck into an object through a constructor that was written before the field existed.

The object gets passed around.  No problem; it's not null. It gets stored inside other things, pulled out.  The field itself is pulled out and passed around, put into other things.

And THEN the program blows up.

You can't run a debugger backwards through time, because that's what you need to do to figure out where the bloody thing came from.  The NPE tells you there is a problem, but it doesn't tell you WHY or WHERE.

It's your leg dropping off from necrosis and the doctor going "gee, I guess you're sick."

It's the plane smashing into the ground and killing everyone inside, a specialised team spending a month analysing the wreckage and saying "well, this screw came loose but BUGGERED if we can work out why."

Then, after several more crashes, someone finally realises that it didn't come loose, it was never there to begin with.  "Oh!  THAT'S why they keep crashing!

"Gee, would've been nice if the plane wouldn't have taken off without it."
September 27, 2009
Ary Borenszweig wrote:
> Jeremie Pelletier wrote:
>> Ary Borenszweig wrote:
>>> Walter Bright wrote:
>>>> Daniel Keep wrote:
>>>>> "But the user will just assign to something useless to get around that!"
>>>>>
>>>>> You mean like how everyone wraps every call in try{...}catch(Exception
>>>>> e){} to shut the damn exceptions up?
>>>>
>>>> They do just that in Java because of the checked-exceptions thing. I have a reference to Bruce Eckel's essay on it somewhere in this thread. The observation in the article was it wasn't just moron idiot programmers doing this. It was the guru programmers doing it, all the while knowing it was the wrong thing to do. The end result was the feature actively created the very problems it was designed to prevent.
>>>>
>>>>
>>>>> Or uses pointer arithmetic and
>>>>> casts to get at those pesky private members?
>>>>
>>>> That's entirely different, because privacy is selected by the programmer, not the language. I don't have any issue with a user-defined type that is non-nullable (Andrei has designed a type constructor for that).
>>>>
>>>>
>>>>> If someone is actively trying to break the type system, it's their
>>>>> goddamn fault!  Honestly, I don't care about the hacks they employ to
>>>>> defeat the system because they're going to go around blindly shooting
>>>>> themselves in the foot no matter what they do.
>>>>
>>>> True, but it's still not a good idea to design a language feature that winds up, in reality, encouraging bad programming practice. It encourages bad practice in a way that is really, really hard to detect in a code review.
>>>>
>>>> I like programming mistakes to be obvious, not subtle. There's nothing subtle about a null pointer exception. There's plenty subtle about the wrong default value.
>>>>
>>>>
>>>>> And what about the people who AREN'T complete idiots, who maybe
>>>>> sometimes just accidentally trip and would quite welcome a safety rail
>>>>> there?
>>>>
>>>> Null pointer seg faults *are* a safety rail. They keep an errant program from causing further damage.
>>>
>>> Null pointer seg faults *not being able to happen* are much more safe. :)
>>
>> There is no such thing as "not being able to happen" :)
>>
>> Object thisCannotPossiblyBeNullInAnyWayWhatsoever = cast(Object)null;
> 
> Object is not-nullable, Object? (or whatever syntax you like) is nullable. So that line is a compile-time error: you can't cast a null to an Object (because Object *can't* be null).
> 
> You might be the only one here that understands Walter's point. But Walter is wrong. ;-)

union A {
	Object foo;
	Object? bar;
}

Give me a type system, and I will find backdoors :)

I didn't say Walter was right or wrong, I said I understand his point of view. The sweet spot most likely lie in the middle of both arguments seen in this thread, and that's not an easy one to pinpoint!

I think we should much rather enforce variable initialization in D than nullable/non-nullable types. The error after all is that an unitialized reference triggers a segfault.

What if using 'Object obj;' raises a warning "unitialized variable" and makes everyone wanting non-null references happy, and 'Object obj = null;' raises no warning and makes everyone wanting to keep the current system (all two of us!) happy.

I believe it's a fair compromise.