On 1/17/2014 4:15 PM, Andrei Alexandrescu wrote:
> One problem with null is it's not "proportional response", i.e. it takes the
> application in the back and shoots it in the head instead of e.g. a stale
> display. That _does_ make null special; there's a large category of application
> for which aborting is simply not an option.


I'm aware of that, I was responding to the idea that null pointers are harder to track down once detected.

In particular, I don't see how a stale display is an easier bug to track down, although granted it is less annoying to the end user.

On 1/17/2014 4:44 PM, "Ola Fosheim Grøstad" <ola.fosheim.grostad+dlang@gmail.com>" wrote:
> It isn't special in the theoretical sense, but since it in practice is used for
> a wide variety of things  it becomes a sensitive issue in situations where you
> cannot replay input. Especially since it is used for linking together
> subsystems. In multi user games you can do fine with wrong "float values",
> objects might drift, you might get odd effects, but those bugs can be cool. They
> might even become features. A wrong "null" makes the client or server shut down,
> it is disruptive to the service.
>
> So what can you do? You can log all input events in a ring buffer and send a big
> dump to the developers when the game client crash and try to find correlation
> between stack trace and input events (if you get multiple reports).  And on a
> server with say 1000 concurrent users that interact and trigger a variety of
> bugs in one big multi-threaded global state world sim… You want to use a safe
> language for world content, where you disable features if needed, but keep the
> world running.
>
> The same goes for a web service. You don't want to shut down the whole service
> just because the "about page" handler fails to test for null.
>
> Big systems have to live with bugs, it is inevitable that they run with bugs.
> Erlang was created for stuff like that. Can you imagine a telephone central
> going down just because the settings for one subscriber was wrong and triggered
> a bug?

First off, in all these scenarios you describe, how does not having a null make it EASIER to track down the bug?

Next, it's one thing to have a game degrade but keep playing if there's a bug in the code. It's quite another to have a critical system start behaving erratically because of a bug.

Remember, a bug means the program has entered an undefined, unanticipated state. If it is a critical system, continuing to run that system is absolutely, positively, the wrong thing to do. The correct way is when the bug is detected, that software is IMMEDIATELY SHUT DOWN and the backup is engaged. If you don't have such a backup, you have a very, very badly designed system.

I'm not just making that stuff up; it is not my uninformed opinion. This is how, for example, Boeing designs things for airliners. You wouldn't want it any other way. Fukushima and Deepwater Horizon are what happens when you don't have a system with independent backups.

Again, granted, this doesn't apply to a game. Nothing bad happens when a game goes awry. But it definitely applies when there's money involved, like transaction processing, or trading software. And it certainly applies when failure means people die.

On Saturday, 18 January 2014 at 01:05:50 UTC, Walter Bright wrote:
> Because I've tracked down the cause of many, many null pointers, and I've tracked down the cause of many, many other kinds of invalid values in a variable. Null pointers tend to get detected much sooner, hence closer to where they were set.

To be fair, I do think we should require explicit initialization for char for instance instead of putting in them an invalid codepoint. Still, the consequences for null are much more dramatic, which make it more worthwhile to change. They don't have the same ROI as they don't have the same cost.

On Saturday, 18 January 2014 at 01:05:50 UTC, Walter Bright wrote:
> Because I've tracked down the cause of many, many null pointers, and I've tracked down the cause of many, many other kinds of invalid values in a variable. Null pointers tend to get detected much sooner, hence closer to where they were set.

Some null pointers. The ones that were set in the current call-chain, but not the long-lived ones.

Anyway, pointers are special since they link together subsystems. Yes, it is better to have a null-value than a pointer to a random location, so "null" isn't the worst value for a pointer. It is the best arbitrary wrong value, but there is nothing wrong with saying "Hey, when we are wrong about User identity, we want to keep the system running assuming a Guest identity".

However C-like null values conflate a wide array of semantics, I think you can have near half a dozen types of null values in a database. In c-like languages null can mean: not-yet-initialized (used too early), failed-computation (bottom), entity-shall-not-have-subentity (deliberate lack of), empty-container (list is empty), service-not-ready (try again), etc.

So yeah, it would be nice if the language is able to distinguish between the different semantics of "null" of different pointer types and convert them into something sensible when "cast" to a different type.

For some pointer types it might make sense to kill the thread, for others it would make sense to throw an exception, for others it might make sense to allocate an empty object, for others it might make sense to use a dummy/sentinel.

It isn't universally true that the best option is to core dump, but you could do both. You could fork(), core-dump the fork, and recover the original.

On 1/17/2014 4:44 PM, "Ola Fosheim Grøstad" <ola.fosheim.grostad+dlang@gmail.com>" wrote:
> Big systems have to live with bugs, it is inevitable that they run with bugs.

It's a dark and stormy night. You're in a 747 on final approach, flying on autopilot.

Scenario 1
----------

The autopilot software was designed by someone who thought it should keep operating even if it detects faults in the software. The software runs into a null object when there shouldn't be one, and starts feeding bad values to the controls. The plane flips over and crashes, everybody dies. But hey, the software kept on truckin'!

Scenario 2
----------

The autopilot software was designed by Boeing. Actually, there are two autopilots, each independently developed, with different CPUs, different hardware, different algorithms, different languages, etc. One has a null pointer fault. A deadman circuit sees this, and automatically shuts that autopilot down. The other autopilot immediately takes over. The pilot is informed that one of the autopilots failed, and the pilot immediately shuts off the remaining autopilot and lands manually. The passengers all get to go home.


Note that in both scenarios there are bugs in the software. Yes there have been incidents with earlier autopilots where bugs in it caused the airplane to go inverted.

Consider also the Toyota. My understanding from reading reports (admittedly journalists botch up the facts) is that a single computer controls the brakes, engine, throttle, ignition switch, etc. Oh joy. I wouldn't want to be in that car when it keeps on going despite having self-detected faults. It could, you know, start going at full throttle and ignore all signals to brake or turn off, only stopping when it crashes or runs out of gas.

On 1/17/2014 4:25 PM, Andrei Alexandrescu wrote:
> On 1/17/14 4:21 PM, Adam D. Ruppe wrote:
>> On Saturday, 18 January 2014 at 00:12:16 UTC, Andrei Alexandrescu wrote:
>>> Yes, improving the language is in the cards. I have collected enough
>>> hard evidence to convince myself that null references are a real and
>>> important issue (previously I'd agreed with Walter who is considering
>>> it not particularly remarkable).
>>
>> Have they tried using a NotNull!T yet?
>
> Not a D project (Java/Android). However there is work underway on a static
> analyzer for Java. I cannot give exact stats, but we have gathered extensive
> data that null pointer exceptions form a major, major problem on Facebook's
> Android app, and that virtually all can be solved by static analysis.
>
> No amount of speculation and hypothesizing will talk me out of that.

I don't disagree that some form of static analysis that can detect null dereferencing at compile time (i.e. static analysis) is a good thing. I concur that detecting bugs at compile time is better.

I don't disagree that for an app like a game, soldiering on after the program is in an invalid state doesn't do any particular harm, and so one can make a good case for using the null object pattern.

I do not agree with the notion that null pointers detected at runtime are harder to track down than other kinds of invalid values detected at runtime.

I strong, strongly, disagree with the notion that critical systems should soldier on once they have entered an invalid state. Such is absolutely the wrong way to go about making a fault tolerant system. For hard evidence, I submit the safety record of airliners.

Walter Bright:

> I strong, strongly, disagree with the notion that critical systems should soldier on once they have entered an invalid state.

The idea is to design the language and its type system (and static analysis tools) to be able to reduce the frequency (or probability) of such invalid states, because many of them are removed while you write the program.

Bye,
bearophile

On 2014-01-18 01:22:49 +0000, Walter Bright <newshound2@digitalmars.com> said:

> First off, in all these scenarios you describe, how does not having a null make it EASIER to track down the bug?

Implemented well, it makes it a compilation error. It works like this:

- can't pass a likely-null value to a function that wants a not-null argument.
- can't assign a likely-null value to a not-null variable.
- can't dereference a likely-null value.

You have to check for null first, and the check changes the value from likely-null to not-null in the branch taken when the pointer is valid.

-- 
Michel Fortin
michel.fortin@michelf.ca
http://michelf.ca

On Saturday, 18 January 2014 at 01:46:55 UTC, Walter Bright wrote:
> The autopilot software was designed by someone who thought it should keep operating even if it detects faults in the software.

I would not write autopilot or life-support software in D. So that is kind of out-of-scope for the language. But:

Keep the system simple, select a high level language and verify correctness by an automated proof system.

Use 3 independently implemented systems and shut down the one that produces deviant values. That covers more ground than the unlikely null-pointers in critical systems. No need to self-detect anything.

> Consider also the Toyota. My understanding from reading reports (admittedly journalists botch up the facts) is that a single computer controls the brakes, engine, throttle, ignition switch, etc. Oh joy. I wouldn't want to be in that car when it keeps on going despite having self-detected faults.

So you would rather have the car drive off the road because the anti-skid software abruptly turned itself off during an emergency manoeuvre?

But would you stay in a car where the driver talks in a cell-phone while driving, or would you tell him to stop? Probably much more dangerous if you measured correlation between accidents and system features. So you demand perfection from a computer, but not from a human being that is exhibiting risk-like behaviour. That's an emotional assessment.

The rational action would be to improve the overall safety of the system, rather than optimizing a single part. So spend the money on installing a cell-phone jammer and an accelerator limiter rather than investing in more computers. Clearly, the computer is not the weakest link, the driver is. He might not agree, but he is and he should be forced to exhibit low risk behaviour. Direct effort to where it has most effect.

(From a system analytical point of view. It might not be a good sales tactic, because car buyers aren't that rational.)

Ola Fosheim Grøstad:

> I would not write autopilot or life-support software in D. So that is kind of out-of-scope for the language.

Do you want to use Ada for those purposes?

Bye,
bearophile