On 2009-09-26 23:28:30 -0400, Michel Fortin <michel.fortin@michelf.com> said:

> On 2009-09-26 22:07:00 -0400, Walter Bright <newshound1@digitalmars.com> said:
> 
>> [...] The facilities in D enable one to construct a non-nullable type, and they are appropriate for many designs. I just don't see them as a replacement for *all* reference types.
> 
> As far as I understand this thread, no one here is arguing that non-nullable references/pointers should replace *all* reference/pointer types. The argument made is that non-nullable should be the default and nullable can be specified explicitly any time you need it.
> 
> So if you need a reference you use "Object" as the type, and if you want that reference to be nullable you write "Object?". The static analysis can then assert that your code properly check for null prior dereferencing a nullable type and issues a compilation error if not.

I just want to add: some people here are suggesting the compiler adds code to check for null and throw exceptions... I believe like you that this is the wrong approach because, like you said, it makes people add dummy try/catch statements to ignore the error. What you want a prorammer to do is check for null and properly handle the situation before the error occurs, and this is exactly what the static analysis approach I suggest forces.

Take this example where "a" is non-nullable and "b" is nullable:

string test(Object a, Object? b)
{
	auto x = a.toString();
	auto y = b.toString();
	
	return x ~ y;
}

This should result in a compiler error on line 4 with a message telling you that "b" needs to be checked for null prior use. The programmer must then fix his error with an if (or some other control structure), like this:

string test(Object a, Object? b)
{
	audo result = a.toString();
	if (b)
		result ~= b.toString();

	return result;
}

And now the compiler will let it pass. This is what I'd like to see. What do you think?

I'm not totally against throwing exceptions in some cases, but the above approach would be much more useful. Unfortunatly, throwing exceptions it the best you can do with a library type approach.

-- 
Michel Fortin
michel.fortin@michelf.com
http://michelf.com/

Jeremie Pelletier wrote:
> Ary Borenszweig wrote:
>> Jeremie Pelletier wrote:
>>> Ary Borenszweig wrote:
>>>> Walter Bright wrote:
>>>>> Daniel Keep wrote:
>>>>>> "But the user will just assign to something useless to get around that!"
>>>>>>
>>>>>> You mean like how everyone wraps every call in try{...}catch(Exception
>>>>>> e){} to shut the damn exceptions up?
>>>>>
>>>>> They do just that in Java because of the checked-exceptions thing. I have a reference to Bruce Eckel's essay on it somewhere in this thread. The observation in the article was it wasn't just moron idiot programmers doing this. It was the guru programmers doing it, all the while knowing it was the wrong thing to do. The end result was the feature actively created the very problems it was designed to prevent.
>>>>>
>>>>>
>>>>>> Or uses pointer arithmetic and
>>>>>> casts to get at those pesky private members?
>>>>>
>>>>> That's entirely different, because privacy is selected by the programmer, not the language. I don't have any issue with a user-defined type that is non-nullable (Andrei has designed a type constructor for that).
>>>>>
>>>>>
>>>>>> If someone is actively trying to break the type system, it's their
>>>>>> goddamn fault!  Honestly, I don't care about the hacks they employ to
>>>>>> defeat the system because they're going to go around blindly shooting
>>>>>> themselves in the foot no matter what they do.
>>>>>
>>>>> True, but it's still not a good idea to design a language feature that winds up, in reality, encouraging bad programming practice. It encourages bad practice in a way that is really, really hard to detect in a code review.
>>>>>
>>>>> I like programming mistakes to be obvious, not subtle. There's nothing subtle about a null pointer exception. There's plenty subtle about the wrong default value.
>>>>>
>>>>>
>>>>>> And what about the people who AREN'T complete idiots, who maybe
>>>>>> sometimes just accidentally trip and would quite welcome a safety rail
>>>>>> there?
>>>>>
>>>>> Null pointer seg faults *are* a safety rail. They keep an errant program from causing further damage.
>>>>
>>>> Null pointer seg faults *not being able to happen* are much more safe. :)
>>>
>>> There is no such thing as "not being able to happen" :)
>>>
>>> Object thisCannotPossiblyBeNullInAnyWayWhatsoever = cast(Object)null;
>>
>> Object is not-nullable, Object? (or whatever syntax you like) is nullable. So that line is a compile-time error: you can't cast a null to an Object (because Object *can't* be null).
>>
>> You might be the only one here that understands Walter's point. But Walter is wrong. ;-)
> 
> union A {
>     Object foo;
>     Object? bar;
> }
> 
> Give me a type system, and I will find backdoors :)

Ah, nice one.

Well, I see you can always break the type system. The point is to break it as little as possible while obtaining the most out of it without it bothering you.

Jeremie Pelletier wrote:
> ...
> 
> This is something for the runtime or the debugger to deal with. My
> runtime converts access violations on windows or segfaults on linux into
> exception objects, which unwind all the way down to main where it
> catches into the unhandled exception handler (or crash handler) and I
> get a neat popup with a "hello, your program crashed at this point, here
> is a backtrace with resolved symbols and filenames along with current
> registers and loaded modules, would you like a cup of coffee while you
> solve the problem?". I sent that crash handler to D.announce last week too.

See my long explanation that NPEs are only symptoms; very rarely do they put up a big sign saying "what ho; the problem is RIGHT HERE!"

> The compiler won't be able to enforce *every* nonnull reference and segfaults are bound to happen, especially with casting. While it may prevent most of them, any good programmer would too, I don't remember the last time I had a segfault on a null reference actually.

I do.  It took a day and a half to track it back to the source.

> I can see what the point is with nonnull references, but I can also see its not a bulletproof solution. ie "Object foo = cast(Object)null;" would easily bypass the nonnull enforcement, resulting in a segfault the system is trying to avoid.

Why lock the door when someone could break the window?

Why have laws when people could break them?

Why build a wall when someone could park a hydrogen bomb next to it?

Why have a typesystem when you could use casting to put the float representation of 3.14159 into a void* and then dereference it?

Casting is not an argument against non-null references because casting can BREAK ANYTHING.

"Doctor, it hurts when I hammer nails into my shin."

"So stop doing it."

> What about function parameters, a lot of parameters are optional references, which are tested and then used into functions whose parameters aren't optional. It would result in a lot of casts, something that could easily confuse people and easily generate segfaults.

So what you're saying is: better to never, ever do error checking and just start fixing things after they've broken?

And why is everything solved via casting?  Look: here's a solution that's less typing than a cast, AND it's safe.  You could even put nonnull it in object.d!

T notnull(U : T?, T)(U obj)
{
    if( obj is null ) throw new NullException;
    return cast(T) obj;
}

void foo(Quxx o)
{
    o.doStuff;
}

void foo(Quxx? o)
{
    foo(notnull(o));
}

> Alls I'm saying is, nonnull references would just take the issue from one place to another.

YES.

THAT'S THE POINT.

It would take the error from a likely unrelated location in the program's execution and put it RIGHT where the mistake initially occurs!

> Like Walter said, you can put a gas mask to ignore
> the room full of toxic gas, but that doesn't solve the gas problem in
> itself, you're just denyinng it exists. Then someday you forget about
> it, remove the mask, and suffocate.
> 
> Jeremie

That's what NPEs are!  They're a *symptom* of you passing crap in to fields or functions.  They very, VERY rarely actually point out what the underlying mistake is.

Walter Bright Wrote:

> language_fan wrote:
> > The idea behind non-nullable types and other contracts is to catch these errors on compile time. Sure, the code is a bit harder to write, but it is safe and never segfaults. The idea is to minimize the amount of runtime errors of all sorts. That's also how other features of statically typed languages work.
> 
> 
> I certainly agree that catching errors at compile time is preferable by far. Where I disagree is the notion that non-nullable types achieve this. I've argued extensively here that they hide errors, not fix them.

If you argued any cases other than there's no good default initialization, I missed it. I reject the default initialization argument. I find code that relies on default initialization hard to read. I also find C#'s warning of uninitialized variables highly useful. I've also never had a bug that Don's signalling nans would help with. I've had nan bugs that cropped up later though... On top of that, use of a null variable because it was never set are typically easy to find.

> Also, by "safe" I presume you mean "memory safe" which means free of memory corruption. Null pointer exceptions are memory safe. A null pointer could be caused by memory corruption, but it cannot *cause* memory corruption.

I reject this argument too :(
To me, code isn't safe if it crashes. Did Boeing avoid checking for fault modes that were easily and reliably detectable? It seems stupid to argue that it's ok for an altimeter can send bogus data as long as it's easy to detect. All you have to do is turn off autopilot. Who cares, right?

Why should I use D for production code if it's designed to segfault? Software isn't used for important things like autopilot, controlling the brakes in my car, or dispensing medicine in hospitals. There's no problem allowing that stuff to crash. You can always recover the core file, and it's always trivial to reproduce the scenario...

Mix in other things like malfunctioning debug data, and I wonder why I even use D.

Daniel Keep wrote:
> 
> Walter Bright wrote:
>> Daniel Keep wrote:
>>> "But the user will just assign to something useless to get around that!"
>>>
>>> You mean like how everyone wraps every call in try{...}catch(Exception
>>> e){} to shut the damn exceptions up?
>> They do just that in Java because of the checked-exceptions thing. I
>> have a reference to Bruce Eckel's essay on it somewhere in this thread.
>> The observation in the article was it wasn't just moron idiot
>> programmers doing this. It was the guru programmers doing it, all the
>> while knowing it was the wrong thing to do. The end result was the
>> feature actively created the very problems it was designed to prevent.
> A NPE is the thermonuclear option of error handling.  Your program blows
> up, tough luck, try again.  Debugging is forensics, just like picking
> through a mound of dead bodies and bits of fuselage; if it's come to
> that, there's a problem.
> 
> It's your leg dropping off from necrosis and the doctor going "gee, I
> guess you're sick."
> 
> It's the plane smashing into the ground and killing everyone inside, a
> specialised team spending a month analysing the wreckage and saying
> "well, this screw came loose but BUGGERED if we can work out why."
> 
> Then, after several more crashes, someone finally realises that it
> didn't come loose, it was never there to begin with.  "Oh!  THAT'S why
> they keep crashing!
> 
> "Gee, would've been nice if the plane wouldn't have taken off without it."

I like your analogies. :)

Ary Borenszweig wrote:
> Jeremie Pelletier wrote:
>> Ary Borenszweig wrote:
>>> Walter Bright wrote:
>>>> Denis Koroskin wrote:
>>>>  > On Sat, 26 Sep 2009 22:30:58 +0400, Walter Bright
>>>>  > <newshound1@digitalmars.com> wrote:
>>>>  >> D has borrowed ideas from many different languages. The trick is to
>>>>  >> take the good stuff and avoid their mistakes <g>.
>>>>  >
>>>>  > How about this one:
>>>>  > http://sadekdrobi.com/2008/12/22/null-references-the-billion-dollar-mistake/ 
>>>>
>>>>  >
>>>>  >
>>>>  > :)
>>>>
>>>> I think he's wrong.
>>>
>>> Please, please, please, do some fun little project in Java or C# and drop the idea of initializing variables whenever you declare them. Just leave them like this:
>>>
>>> int i;
>>>
>>> and then later initialize them when you need them, for example different values depending on some conditions. Then you'll realize how powerful is having the compiler stop variables that are not initialized *in the context of a function, not necessarily in the same line of their declaration*. It's always a win: you get a compile time error, you don't have to wait to get an error at runtime.
>>>
>>> Until you do that, you won't understand what most people are answering to you.
>>>
>>> But I know what you'll answer. You'll say "what about pointers?", "what about ref parameters?", "what about out parameters?", and then someone will say to you "C# has them", etc, etc.
>>>
>>> No point disussing non-null variables without also having the compiler stop uninitialized variables.
>>
>> All null values are uninitialized, but not all initializers are null, especially the void initializer.
> 
> I don't see your point here. "new Object()" is not a null intiializer nor "1"... so?

Object o = void;

>  You can't always rely on initializers
>> in your algorithms, you can always rely on null.
> 
> Yes, I can always rely on initializers in my algorithm. I can, if the compiler lets me safely initialize them whenever I want, not necessarily in the line I declare them.
> 
> Just out of curiosity: have you ever programmed in Java or C#?

Nope, never got interested in these to tell the truth. I only did C, C++, D and x86 assembly in systems programming, I have quite a background in PHP and JavaScript also.

I played with a lot of languages, but those are the ones I use on a daily basis. I would like to get into Python or Ruby someday, I only hear good things about these two. I know LUA has less overhead than Python, but it's more of a support language to implement easy scripting over C than a standalone language, I already have my LUA bindings for D ready to do just that.

I like extremes :)

Ary Borenszweig wrote:
> Daniel Keep wrote:
>>
>> Walter Bright wrote:
>>> Daniel Keep wrote:
>>>> "But the user will just assign to something useless to get around that!"
>>>>
>>>> You mean like how everyone wraps every call in try{...}catch(Exception
>>>> e){} to shut the damn exceptions up?
>>> They do just that in Java because of the checked-exceptions thing. I
>>> have a reference to Bruce Eckel's essay on it somewhere in this thread.
>>> The observation in the article was it wasn't just moron idiot
>>> programmers doing this. It was the guru programmers doing it, all the
>>> while knowing it was the wrong thing to do. The end result was the
>>> feature actively created the very problems it was designed to prevent.
>> A NPE is the thermonuclear option of error handling.  Your program blows
>> up, tough luck, try again.  Debugging is forensics, just like picking
>> through a mound of dead bodies and bits of fuselage; if it's come to
>> that, there's a problem.
>>
>> It's your leg dropping off from necrosis and the doctor going "gee, I
>> guess you're sick."
>>
>> It's the plane smashing into the ground and killing everyone inside, a
>> specialised team spending a month analysing the wreckage and saying
>> "well, this screw came loose but BUGGERED if we can work out why."
>>
>> Then, after several more crashes, someone finally realises that it
>> didn't come loose, it was never there to begin with.  "Oh!  THAT'S why
>> they keep crashing!
>>
>> "Gee, would've been nice if the plane wouldn't have taken off without it."
> 
> I like your analogies. :)

I also do, but try and picture a plane sophisticated to the point it can notice missing screws and ask yourself the following question: what is making sure such a screw detection system works correctly.

That's really just taking a problem and sending it to another team to solve, at the end of the day, it's still a problem. Besides, explosions are cool!

Daniel Keep wrote:
> 
> Jeremie Pelletier wrote:
>> ...
>>
>> This is something for the runtime or the debugger to deal with. My
>> runtime converts access violations on windows or segfaults on linux into
>> exception objects, which unwind all the way down to main where it
>> catches into the unhandled exception handler (or crash handler) and I
>> get a neat popup with a "hello, your program crashed at this point, here
>> is a backtrace with resolved symbols and filenames along with current
>> registers and loaded modules, would you like a cup of coffee while you
>> solve the problem?". I sent that crash handler to D.announce last week too.
> 
> See my long explanation that NPEs are only symptoms; very rarely do they
> put up a big sign saying "what ho; the problem is RIGHT HERE!"
> 
>> The compiler won't be able to enforce *every* nonnull reference and
>> segfaults are bound to happen, especially with casting. While it may
>> prevent most of them, any good programmer would too, I don't remember
>> the last time I had a segfault on a null reference actually.
> 
> I do.  It took a day and a half to track it back to the source.

Happens to me on some issues too, I don't ask for a workaround in the compiler, I just learn my lesson and never repeat that error.

>> I can see what the point is with nonnull references, but I can also see
>> its not a bulletproof solution. ie "Object foo = cast(Object)null;"
>> would easily bypass the nonnull enforcement, resulting in a segfault the
>> system is trying to avoid.
> 
> Why lock the door when someone could break the window?

Easier to prove someone broke in when the window is shattered than if someone just went through the door, stole your stuff and left without any traces.

> Why have laws when people could break them?

People break the law, some of them only for the challenge of it, some of them to survive, some just don't care. Remove the laws and you remove most of these behaviors you're trying to prohibit in the first place. Most of the time laws are there so corporate criminals can get rid of street criminals legally.

> Why build a wall when someone could park a hydrogen bomb next to it?

They keep most people out, or in. Hydrogen bombs are not something you expect the first guy on the street to own.

> Why have a typesystem when you could use casting to put the float
> representation of 3.14159 into a void* and then dereference it?

Because it also allows for countless different optimizations, at the price of also being able to shoot your own foot.

There, four similar questions and four completely different answers. My point is, there is no perfect all-around solution.

> Casting is not an argument against non-null references because casting
> can BREAK ANYTHING.
> 
> "Doctor, it hurts when I hammer nails into my shin."
> 
> "So stop doing it."

Why tell him to stop it? The guy will just kill himself at some point and raise the collective IQ of mankind in the process. Same for programming or anything else, if someone is dumb enough to repeat the same mistake over and over, he should find a new domain to work in.

>> What about function parameters, a lot of parameters are optional
>> references, which are tested and then used into functions whose
>> parameters aren't optional. It would result in a lot of casts, something
>> that could easily confuse people and easily generate segfaults.
> 
> So what you're saying is: better to never, ever do error checking and
> just start fixing things after they've broken?

No, but you shouldn't rule out the fact that they may break, no matter what system you're working with.

> And why is everything solved via casting?  Look: here's a solution
> that's less typing than a cast, AND it's safe.  You could even put
> nonnull it in object.d!
> 
> T notnull(U : T?, T)(U obj)
> {
>     if( obj is null ) throw new NullException;
>     return cast(T) obj;
> }
> 
> void foo(Quxx o)
> {
>     o.doStuff;
> }
> 
> void foo(Quxx? o)
> {
>     foo(notnull(o));
> }

Also slower than a cast if the compiler doesn't use -inline. Debug builds are already painful enough as it is with realtime code.

>> Alls I'm saying is, nonnull references would just take the issue from
>> one place to another.
> 
> YES.
> 
> THAT'S THE POINT.
> 
> It would take the error from a likely unrelated location in the
> program's execution and put it RIGHT where the mistake initially occurs!

That's a case for variable initialization, not nullable/non-null types.

A nonnull type does not guarantee the value will *never* be null, even the simplest hack can get around it.

>> Like Walter said, you can put a gas mask to ignore
>> the room full of toxic gas, but that doesn't solve the gas problem in
>> itself, you're just denyinng it exists. Then someday you forget about
>> it, remove the mask, and suffocate.
>>
>> Jeremie
> 
> That's what NPEs are!  They're a *symptom* of you passing crap in to
> fields or functions.  They very, VERY rarely actually point out what the
> underlying mistake is.

There again, I favor stronger initialization semantics over nonnull types. This will get rid of most of these errors and still keep you on your toes when a segfault arise, if you only see a segfault once a year how will you know how to handle it :)

Most segfaults I have take me at most a few minutes to pinpoint. Its finding backdoors to compiler enforcements thats annoying.

Walter Bright wrote:
> language_fan wrote:
>> Maybe Walter has not yet transitioned from the good olde Pascal/C style programming to the C++/D/Java style?
> 
> Heh, there's still a Fortran influence in my code <g>.

This may be a good time to ask about how these variables which can be declared anywhere in the function scope are implemented.

void bar(bool foo) {
	if(foo) {
		int a = 1;
		...
	}
	else {
		int a = 2;
		...
	}

}

is the stack frame using two ints, or is the compiler seeing only one? I never bothered to check it out and just declared 'int a = void;' at the beginning of the routine to keep the stack frames as small as possible.

Walter Bright wrote:
> Yigal Chripun wrote:
>> An exception trace is *far* better than a segfault and that does not require null values.
> 
> Seg faults are exceptions, too. You can even catch them (on windows)!

Walter, check the crash handler I submitted to D.announce, it has signal handlers on linux to convert segfaults into D exception objects and throw them so the code can unwind properly and even catch it.

It has made my life so much easier, I barely need to run within a debugger anymore for most crashes. I don't know enough of phobos and druntime to port it, but its under a public domain license so anyone is free to do it!

</shameless plug>