On 1/17/14 11:43 AM, Walter Bright wrote:
> On 1/17/2014 6:06 AM, "Ola Fosheim Grøstad"
> <ola.fosheim.grostad+dlang@gmail.com>" wrote:
>> On Friday, 17 January 2014 at 01:42:38 UTC, Andrei Alexandrescu wrote:
>>> One common idiom to replace null pointer exceptions with milder
>>> reproducible
>>> errors is the null object pattern, i.e. there is
>>
>> Usually null failures are hard to track when some function returned a
>> null value
>> as an indication of error when the programmer expected an exception.
>
> I've almost never had a problem tracking down the cause of a null
> pointer. Usually just a few minutes with a debugger and getting a
> backtrace.

I think this is bias that is countered by extensive experience of groups at Facebook that have no axe to grind in the matter. This bias makes it all the more difficult to recognize the importance of the problem.

Andrei

On 1/17/14 12:42 PM, Michel Fortin wrote:
> On 2014-01-17 20:21:36 +0000, Jacob Carlborg <doob@me.com> said:
>
>> On 2014-01-17 16:05, Michel Fortin wrote:
>>
>>> This is basically what everyone would wish for a not-null type to do.
>>> The syntax is clean and the control flow forces you to check for null
>>> before use. Misuses result in a compile-time error.
>>
>> Yes, exactly. But the issue is that the compiler needs to be modified
>> for this.
>
> Andrei's post was referring at language/compiler changes too: allowing
> init to be defined per-class, with a hint about disabling init. I took
> the hint that modifying the compiler to add support for non-null was in
> the cards and proposed something more useful and less clunky to use.

Yes, improving the language is in the cards. I have collected enough hard evidence to convince myself that null references are a real and important issue (previously I'd agreed with Walter who is considering it not particularly remarkable).

Andrei

On 1/17/14 1:29 PM, Walter Bright wrote:
> On 1/17/2014 1:10 PM, H. S. Teoh wrote:
>> On Fri, Jan 17, 2014 at 11:43:59AM -0800, Walter Bright wrote:
>> [...]
>>> I've almost never had a problem tracking down the cause of a null
>>> pointer. Usually just a few minutes with a debugger and getting a
>>> backtrace.
>>
>> I think this depends on the kind of code you write.
>>
>> In callback-heavy code, usually when you're multiplexing between many
>> simultaneous request/response chains, these kinds of problems are very
>> hard to track down.  You'll see the null pointer somewhere in your
>> callback's context structure, but no amount of backtrace will help you
>> go any further because they all end at the event dispatch loop pretty
>> shortly up the stack, which doesn't tell you where in the chain of
>> events the null came from.
>
> What you do then is go back as far as practical, then put asserts in.

That hardly helps. The source of the problem is not detecting that it occurs, it's avoiding it in the first place by initializing data appropriately.

Andrei

On 1/17/14 1:31 PM, Walter Bright wrote:
> On 1/17/2014 12:35 PM, "Ola Fosheim Grøstad"
> <ola.fosheim.grostad+dlang@gmail.com>" wrote:
>> On Friday, 17 January 2014 at 19:43:58 UTC, Walter Bright wrote:
>>> I've almost never had a problem tracking down the cause of a null
>>> pointer.
>>> Usually just a few minutes with a debugger and getting a backtrace.
>>
>> Doesn't work if the unexpected "null" sits in a graph and the source
>> of it is
>> hard to pinpoint or occurs "randomly". E.g. if you are using a "black
>> box"
>> framework or it happens spuriously on a server because it is triggered
>> by a
>> database timeout which never happens on the dev server.
>
> As I replied elsewhere, tracking down the source of a bad value in any
> variable is a standard debugging problem. There isn't anything special
> about null in this regard.

One problem with null is it's not "proportional response", i.e. it takes the application in the back and shoots it in the head instead of e.g. a stale display. That _does_ make null special; there's a large category of application for which aborting is simply not an option.

Andrei

On 1/17/14 3:37 PM, Walter Bright wrote:
> Come on. Every type in D has a default initializer. There's still
> nothing special about null.

There is. Null pointers cause the process to finish.

> Even if you got rid of all the nulls and instead use the null object
> pattern, you're not going to find it any easier to track it down, and
> you're in even worse shape because now it can fail and you may not even
> detect the failure, or may discover the error much, much further from
> the source of the bug.

How do you know all that?


Andrei

On Saturday, 18 January 2014 at 00:12:16 UTC, Andrei Alexandrescu wrote:
> Yes, improving the language is in the cards. I have collected enough hard evidence to convince myself that null references are a real and important issue (previously I'd agreed with Walter who is considering it not particularly remarkable).

Have they tried using a NotNull!T yet? I wrote one ages ago for a phobos pull request that didn't really go anywhere, but I'm still keeping my file updated with new ideas (most recently, the if(auto nn = x.checkNull) {} thing):

http://arsdnet.net/dcode/notnull.d


I really think that is very close to meeting all the requirements here, but tbh I don't use it much in the real world myself so I could be missing a major flaw.

Of course, such library solutions don't change the default nullability of references.

On 1/17/14 4:21 PM, Adam D. Ruppe wrote:
> On Saturday, 18 January 2014 at 00:12:16 UTC, Andrei Alexandrescu wrote:
>> Yes, improving the language is in the cards. I have collected enough
>> hard evidence to convince myself that null references are a real and
>> important issue (previously I'd agreed with Walter who is considering
>> it not particularly remarkable).
>
> Have they tried using a NotNull!T yet?

Not a D project (Java/Android). However there is work underway on a static analyzer for Java. I cannot give exact stats, but we have gathered extensive data that null pointer exceptions form a major, major problem on Facebook's Android app, and that virtually all can be solved by static analysis.

No amount of speculation and hypothesizing will talk me out of that.


Andrei

On Friday, 17 January 2014 at 21:31:16 UTC, Walter Bright wrote:
> As I replied elsewhere, tracking down the source of a bad value in any variable is a standard debugging problem. There isn't anything special about null in this regard.

It isn't special in the theoretical sense, but since it in practice is used for a wide variety of things  it becomes a sensitive issue in situations where you cannot replay input. Especially since it is used for linking together subsystems. In multi user games you can do fine with wrong "float values", objects might drift, you might get odd effects, but those bugs can be cool. They might even become features. A wrong "null" makes the client or server shut down, it is disruptive to the service.

So what can you do? You can log all input events in a ring buffer and send a big dump to the developers when the game client crash and try to find correlation between stack trace and input events (if you get multiple reports).  And on a server with say 1000 concurrent users that interact and trigger a variety of bugs in one big multi-threaded global state world sim… You want to use a safe language for world content, where you disable features if needed, but keep the world running.

The same goes for a web service. You don't want to shut down the whole service just because the "about page" handler fails to test for null.

Big systems have to live with bugs, it is inevitable that they run with bugs. Erlang was created for stuff like that. Can you imagine a telephone central going down just because the settings for one subscriber was wrong and triggered a bug?

On Saturday, 18 January 2014 at 00:25:49 UTC, Andrei Alexandrescu
wrote:
> On 1/17/14 4:21 PM, Adam D. Ruppe wrote:
>> On Saturday, 18 January 2014 at 00:12:16 UTC, Andrei Alexandrescu wrote:
>>> Yes, improving the language is in the cards. I have collected enough
>>> hard evidence to convince myself that null references are a real and
>>> important issue (previously I'd agreed with Walter who is considering
>>> it not particularly remarkable).
>>
>> Have they tried using a NotNull!T yet?
>
> Not a D project (Java/Android). However there is work underway on a static analyzer for Java. I cannot give exact stats, but we have gathered extensive data that null pointer exceptions form a major, major problem on Facebook's Android app, and that virtually all can be solved by static analysis.
>
> No amount of speculation and hypothesizing will talk me out of that.
>
>
> Andrei

I remember mentioning that a year or two month ago, and you
dismissing it.

Happy to see you changed your mind on that.

On 1/17/2014 4:17 PM, Andrei Alexandrescu wrote:
> On 1/17/14 3:37 PM, Walter Bright wrote:
>> Come on. Every type in D has a default initializer. There's still
>> nothing special about null.
>
> There is. Null pointers cause the process to finish.

This particular subthread is about finding the reason why there was a null, not about what happens when the invalid value is detected.


>> Even if you got rid of all the nulls and instead use the null object
>> pattern, you're not going to find it any easier to track it down, and
>> you're in even worse shape because now it can fail and you may not even
>> detect the failure, or may discover the error much, much further from
>> the source of the bug.
>
> How do you know all that?

Because I've tracked down the cause of many, many null pointers, and I've tracked down the cause of many, many other kinds of invalid values in a variable. Null pointers tend to get detected much sooner, hence closer to where they were set.