On 2014-01-17 16:05, Michel Fortin wrote:

> This is basically what everyone would wish for a not-null type to do.
> The syntax is clean and the control flow forces you to check for null
> before use. Misuses result in a compile-time error.

Yes, exactly. But the issue is that the compiler needs to be modified for this.

-- 
/Jacob Carlborg

On Friday, 17 January 2014 at 19:43:58 UTC, Walter Bright wrote:
> I've almost never had a problem tracking down the cause of a null pointer. Usually just a few minutes with a debugger and getting a backtrace.

Doesn't work if the unexpected "null" sits in a graph and the source of it is hard to pinpoint or occurs "randomly". E.g. if you are using a "black box" framework or it happens spuriously on a server because it is triggered by a database timeout which never happens on the dev server.

On 2014-01-17 20:21:36 +0000, Jacob Carlborg <doob@me.com> said:

> On 2014-01-17 16:05, Michel Fortin wrote:
> 
>> This is basically what everyone would wish for a not-null type to do.
>> The syntax is clean and the control flow forces you to check for null
>> before use. Misuses result in a compile-time error.
> 
> Yes, exactly. But the issue is that the compiler needs to be modified for this.

Andrei's post was referring at language/compiler changes too: allowing init to be defined per-class, with a hint about disabling init. I took the hint that modifying the compiler to add support for non-null was in the cards and proposed something more useful and less clunky to use.

-- 
Michel Fortin
michel.fortin@michelf.ca
http://michelf.ca

On Fri, Jan 17, 2014 at 11:43:59AM -0800, Walter Bright wrote: [...]
> I've almost never had a problem tracking down the cause of a null pointer. Usually just a few minutes with a debugger and getting a backtrace.

I think this depends on the kind of code you write.

In callback-heavy code, usually when you're multiplexing between many simultaneous request/response chains, these kinds of problems are very hard to track down.  You'll see the null pointer somewhere in your callback's context structure, but no amount of backtrace will help you go any further because they all end at the event dispatch loop pretty shortly up the stack, which doesn't tell you where in the chain of events the null came from. The callback could've been invoked from any number of places (usually callbacks are factored into generic functions so that you don't have to deal with writing 500 callbacks just to handle the response to each event type), so you have to deduce which of the n number of possibilities may have caused it. Usually, that only leads to finding that the previous event handler was only passing things along, so you have to go back yet another step in the chain.

When there are n possible ancestors for each step in the chain, you're talking about n^k possible paths to investigate before you find the ultimate culprit. A few minutes is not going to suffice, even for moderate values of k.

If you're very very lucky, you may have a log of events that help narrow k to a sufficiently small number that allows quick deduction of where the problem is. Often, though, all you have is a stack trace from an inaccessible customer production server, and you'll have to explore n^k possibilities before you can find the problem. (Even a stack trace should already be counted as lucky; I've had to fix *race conditions* with no stack trace and only indirect evidence that a daemon died and got restarted by init. It took months before we could even reproduce the problem -- it was highly dependent on precise network timing -- much less guess at what went wrong.)

Nipping the null at the bud (i.e., know where in the code it was first set as null) is a real life-saver in these kinds of situations.


T

-- 
Кто везде - тот нигде.

On 1/17/2014 1:10 PM, H. S. Teoh wrote:
> On Fri, Jan 17, 2014 at 11:43:59AM -0800, Walter Bright wrote:
> [...]
>> I've almost never had a problem tracking down the cause of a null
>> pointer. Usually just a few minutes with a debugger and getting a
>> backtrace.
>
> I think this depends on the kind of code you write.
>
> In callback-heavy code, usually when you're multiplexing between many
> simultaneous request/response chains, these kinds of problems are very
> hard to track down.  You'll see the null pointer somewhere in your
> callback's context structure, but no amount of backtrace will help you
> go any further because they all end at the event dispatch loop pretty
> shortly up the stack, which doesn't tell you where in the chain of
> events the null came from.

What you do then is go back as far as practical, then put asserts in.


> When there are n possible ancestors for each step in the chain, you're
> talking about n^k possible paths to investigate before you find the
> ultimate culprit. A few minutes is not going to suffice, even for
> moderate values of k.

Put asserts in, maybe a printf too. I do it all the time to track down problems. Memory corruption can be a difficult one to track down, but I've never had trouble tracking down a null pointer's source. They aren't any harder than any "I have a bad value in this field, I wonder who set it" problem.

I.e. it isn't special.


> Nipping the null at the bud (i.e., know where in the code it was first
> set as null) is a real life-saver in these kinds of situations.

It's better for any bug to catch it at compile time.

On 1/17/2014 12:35 PM, "Ola Fosheim Grøstad" <ola.fosheim.grostad+dlang@gmail.com>" wrote:
> On Friday, 17 January 2014 at 19:43:58 UTC, Walter Bright wrote:
>> I've almost never had a problem tracking down the cause of a null pointer.
>> Usually just a few minutes with a debugger and getting a backtrace.
>
> Doesn't work if the unexpected "null" sits in a graph and the source of it is
> hard to pinpoint or occurs "randomly". E.g. if you are using a "black box"
> framework or it happens spuriously on a server because it is triggered by a
> database timeout which never happens on the dev server.

As I replied elsewhere, tracking down the source of a bad value in any variable is a standard debugging problem. There isn't anything special about null in this regard.

On Fri, Jan 17, 2014 at 01:29:53PM -0800, Walter Bright wrote:
> On 1/17/2014 1:10 PM, H. S. Teoh wrote:
> >On Fri, Jan 17, 2014 at 11:43:59AM -0800, Walter Bright wrote: [...]
> >>I've almost never had a problem tracking down the cause of a null pointer. Usually just a few minutes with a debugger and getting a backtrace.
> >
> >I think this depends on the kind of code you write.
> >
> >In callback-heavy code, usually when you're multiplexing between many simultaneous request/response chains, these kinds of problems are very hard to track down.  You'll see the null pointer somewhere in your callback's context structure, but no amount of backtrace will help you go any further because they all end at the event dispatch loop pretty shortly up the stack, which doesn't tell you where in the chain of events the null came from.
> 
> What you do then is go back as far as practical, then put asserts in.

Of course, but that requires a few rounds of recompilation, repacking the EEPROM image, installation on device, and repeating a user interaction that possibly only randomly hits the bug. Hardly the work of a few minutes.


> >When there are n possible ancestors for each step in the chain, you're talking about n^k possible paths to investigate before you find the ultimate culprit. A few minutes is not going to suffice, even for moderate values of k.
> 
> Put asserts in, maybe a printf too. I do it all the time to track down problems. Memory corruption can be a difficult one to track down, but I've never had trouble tracking down a null pointer's source. They aren't any harder than any "I have a bad value in this field, I wonder who set it" problem.
> 
> I.e. it isn't special.

True.

But in many cases, the source of a bad value is easy to find because of its uniqueness (hmm, why does variable x have a value of 24576? oh I know, search for 24576 in the code). Nulls, though, can occur in many places, and in D they are implicit. There's no easy way to search for that.


> >Nipping the null at the bud (i.e., know where in the code it was first set as null) is a real life-saver in these kinds of situations.
> 
> It's better for any bug to catch it at compile time.
[...]

Currently D doesn't allow catching nulls at compile-time.


T

-- 
Give me some fresh salted fish, please.

On 1/17/2014 2:05 PM, H. S. Teoh wrote:
> Of course, but that requires a few rounds of recompilation, repacking
> the EEPROM image, installation on device, and repeating a user
> interaction that possibly only randomly hits the bug. Hardly the work of
> a few minutes.

I've done EEPROM programming & debugging, I know how to do it. Nulls still aren't a special kind of bug.

BTW, even back in the 1970's, people developing EEPROM systems would plug a special device into the board which would redirect the EEPROM access to some off-board RAM, or would debug using emulators. Maybe this technology has been forgotten :-)


> But in many cases, the source of a bad value is easy to find because of
> its uniqueness (hmm, why does variable x have a value of 24576? oh I
> know, search for 24576 in the code). Nulls, though, can occur in many
> places, and in D they are implicit. There's no easy way to search for
> that.

Come on. Every type in D has a default initializer. There's still nothing special about null.

Even if you got rid of all the nulls and instead use the null object pattern, you're not going to find it any easier to track it down, and you're in even worse shape because now it can fail and you may not even detect the failure, or may discover the error much, much further from the source of the bug.

On Fri, Jan 17, 2014 at 03:37:22PM -0800, Walter Bright wrote:
> On 1/17/2014 2:05 PM, H. S. Teoh wrote:
> >Of course, but that requires a few rounds of recompilation, repacking the EEPROM image, installation on device, and repeating a user interaction that possibly only randomly hits the bug. Hardly the work of a few minutes.
> 
> I've done EEPROM programming & debugging, I know how to do it. Nulls still aren't a special kind of bug.

I wasn't arguing it is. I'm just saying it's not as trivial as you made it sound.


> BTW, even back in the 1970's, people developing EEPROM systems would plug a special device into the board which would redirect the EEPROM access to some off-board RAM, or would debug using emulators. Maybe this technology has been forgotten :-)

Well, nowadays people use VMware, which is essentially a glorified emulator. :) There are times when you have to test on the actual device, though, when emulation is imperfect.


> >But in many cases, the source of a bad value is easy to find because of its uniqueness (hmm, why does variable x have a value of 24576? oh I know, search for 24576 in the code). Nulls, though, can occur in many places, and in D they are implicit. There's no easy way to search for that.
> 
> Come on. Every type in D has a default initializer. There's still nothing special about null.
> 
> Even if you got rid of all the nulls and instead use the null object pattern, you're not going to find it any easier to track it down, and you're in even worse shape because now it can fail and you may not even detect the failure, or may discover the error much, much further from the source of the bug.

If the null object remembers who created it (file + line), this would short-circuit a lot of time-consuming detective work you'd have to do otherwise. That was the main thrust behind my original post. I didn't really find the null object pattern, as it is generally used (dummy values for fields, no-ops for methods), particularly useful as a programming technique. It is, after all, just the OO version of a sentinel, and sentinels have been around long before OO was. But if the null object can be made smarter -- remember which piece of code created it, for example -- then it might begin to be useful.

Otherwise, I don't see much value in Andrei's proposal.


T

-- 
We've all heard that a million monkeys banging on a million typewriters will eventually reproduce the entire works of Shakespeare.  Now, thanks to the Internet, we know this is not true. -- Robert Wilensk

On 17 Jan 2014 01:45, "Andrei Alexandrescu" <SeeWebsiteForEmail@erdani.org> wrote:
>
> Walter and I were talking today about the null pointer issue and he had
the following idea.
>
> One common idiom to replace null pointer exceptions with milder
reproducible errors is the null object pattern, i.e. there is one object that is used in lieu of the null reference to initialize all otherwise uninitialized references. In D that would translate naturally to:
>
> class Widget
> {
>     private int x;
>     private Widget parent;
>     this(int y) { x = y; }
>     ...
>     // Here's the interesting part
>     static Widget init = new Widget(42);
> }
>
> Currently the last line doesn't compile, but we can make it work if the
respective constructor is callable during compilation. The compiler will allocate a static buffer for the "new"ed Widget object and will make init point there.
>
> Whenever a Widget is to be default-initialized, it will point to
Widget.init (i.e. it won't be null). This beautifully extends the language
because currently (with no init definition) Widget.init is null.
>
> So the init Widget will satisfy:
>
> assert(x == 42 && parent is Widget.init);
>
> Further avenues are opened by thinking what happens if e.g. init is
private or @disable-d.
>
> Thoughts?

I would have thought that sort of code would be doable since someone thought it be a good idea to introduce ClassReferenceExp to the frontend implementation. :)