My personal policy is to assert early and assert often.  I see many functions taking pointer args and dereferencing them without checking for NULL first.  I know Walter likes to claim that dmd doesn't have any NPEs because he is a careful coder; but the fact is, he is not the only coder, and most functions don't explicitly document which arguments are allowed to be NULL and/or when.  Of course, it would be better for non-null arguments to just pass by reference, but fixing that would be quite disruptive in most cases (and outright infeasible in others, like void*).

However, I also know that asserting in every function is a religious issue for some, so I thought I would ask the dev community about thoughts on this topic.

Dave

_______________________________________________
dmd-internals mailing list
dmd-internals@puremagic.com
http://lists.puremagic.com/mailman/listinfo/dmd-internals

On Thursday, November 08, 2012 23:40:40 David Held wrote:
> My personal policy is to assert early and assert often.  I see many functions taking pointer args and dereferencing them without checking for NULL first.  I know Walter likes to claim that dmd doesn't have any NPEs because he is a careful coder; but the fact is, he is not the only coder, and most functions don't explicitly document which arguments are allowed to be NULL and/or when.  Of course, it would be better for non-null arguments to just pass by reference, but fixing that would be quite disruptive in most cases (and outright infeasible in others, like void*).
> 
> However, I also know that asserting in every function is a religious issue for some, so I thought I would ask the dev community about thoughts on this topic.

I would point out that asserting that something is non-null doesn't really buy you much. The compiler will just segfault if it tries to dereference a null pointer. Even if you don't have core dumps turned on, compilers are highly deterministic, so all you have to do is try and compile the same program again in a debugger, and you'll see exactly what happened (which is what you'd probably have to do anyway if it failed an assertion). So, there really isn't much point in adding a bunch of assertions to check that pointers aren't null.

- Jonathan M Davis
_______________________________________________
dmd-internals mailing list
dmd-internals@puremagic.com
http://lists.puremagic.com/mailman/listinfo/dmd-internals

On 11/8/2012 11:40 PM, David Held wrote:
> My personal policy is to assert early and assert often.  I see many functions taking pointer args and dereferencing them without checking for NULL first.  I know Walter likes to claim that dmd doesn't have any NPEs because he is a careful coder;

Not exactly. My argument is that the assert:

void test(T* p) {
    assert(p != null);
    ... *p ...
}

is redundant because the hardware will do that for you. Compile with debug on, run it under the debugger, and you'll get the file/line plus a lovely stack trace.

This works so well I'll often insert:

   *cast(char*)0=0;

into code just to see how it got there.

> but the fact is, he is not the only coder, and most functions don't explicitly document which arguments are allowed to be NULL and/or when.  Of course, it would be better for non-null arguments to just pass by reference, but fixing that would be quite disruptive in most cases (and outright infeasible in others, like void*).

Unfortunately, references can also be null pointers.

>
> However, I also know that asserting in every function is a religious issue for some, so I thought I would ask the dev community about thoughts on this topic.

I like using asserts for non-obvious checks for logic errors, such as what I call loop-back tests. For example, calculating the number of bytes that will be written, and then at the end verifying that the number of bytes written matches. Those kinds of asserts detect loads of bugs.

Having a null pointer in the wrong spot is just another bug, I don't really understand all the press it gets. It's not even a particularly bad one, as it won't result in silently corrupted data. I find array overflows and uninitialized data to be far, far, far worse in terms of consuming my time. But fortunately, valgrind has ridden to the rescue there and has saved me enormous effort. So again, I tend not to focus on using asserts for those issues. In my not-so-humble (and probably unique) opinion, valgrind has kept C (and maybe even C++) off of the ash heap of history.
_______________________________________________
dmd-internals mailing list
dmd-internals@puremagic.com
http://lists.puremagic.com/mailman/listinfo/dmd-internals

On Fri, Nov 9, 2012 at 8:40 AM, David Held <dmd@wyntrmute.com> wrote:
> My personal policy is to assert early and assert often.  I see many functions taking pointer args and dereferencing them without checking for NULL first.  I know Walter likes to claim that dmd doesn't have any NPEs because he is a careful coder; but the fact is, he is not the only coder, and most functions don't explicitly document which arguments are allowed to be NULL and/or when.  Of course, it would be better for non-null arguments to just pass by reference, but fixing that would be quite disruptive in most cases (and outright infeasible in others, like void*).
>
> However, I also know that asserting in every function is a religious issue for some, so I thought I would ask the dev community about thoughts on this topic.
>
> Dave

I don't hack on DMD much, but here's my take on the matter: Asserting every function parameter (where applicable of course) is good practice because it helps you catch bugs as early as possible. If you don't do this, you could end up with null pointers far into the program where you then have to spend time figuring out where it came from. This is especially nasty when e.g. building tree-like data structures and you can't just look at the call stack to figure out where you got a null pointer from.

I also think the documentation aspect is important. I've often found myself wondering whether some parameter is allowed to be null when hacking on DMD.

Regards,
Alex
_______________________________________________
dmd-internals mailing list
dmd-internals@puremagic.com
http://lists.puremagic.com/mailman/listinfo/dmd-internals

On 9 November 2012 08:40, David Held <dmd@wyntrmute.com> wrote:
> My personal policy is to assert early and assert often.  I see many functions taking pointer args and dereferencing them without checking for NULL first.  I know Walter likes to claim that dmd doesn't have any NPEs because he is a careful coder; but the fact is, he is not the only coder, and most functions don't explicitly document which arguments are allowed to be NULL and/or when.  Of course, it would be better for non-null arguments to just pass by reference, but fixing that would be quite disruptive in most cases (and outright infeasible in others, like void*).
>
> However, I also know that asserting in every function is a religious issue for some, so I thought I would ask the dev community about thoughts on this topic.
>
> Dave

I don't think it buys much -- as Walter says, you get hardware asserts
anyway. Historically, there have never been many DMD segfaults in
Bugzilla , and they are the easiest type of bug to fix. More
importantly, a significant fraction of the DMD segfaults I've seen
were _not_ null pointers. Instead, the nasty ones are always caused by
an incorrect cast, resulting in a wrong vtable.
There are a *huge* number of casts in the code, it would be great to
reduce the number. The changes to the Array struct were a huge
improvement. I'm sure we can do more.
_______________________________________________
dmd-internals mailing list
dmd-internals@puremagic.com
http://lists.puremagic.com/mailman/listinfo/dmd-internals

On 11/9/2012 12:52 AM, Alex Rønne Petersen wrote:
> I don't hack on DMD much, but here's my take on the matter: Asserting
> every function parameter (where applicable of course) is good practice
> because it helps you catch bugs as early as possible. If you don't do
> this, you could end up with null pointers far into the program where
> you then have to spend time figuring out where it came from. This is
> especially nasty when e.g. building tree-like data structures and you
> can't just look at the call stack to figure out where you got a null
> pointer from.

That is true for some programs. It isn't true for dmd, where tracking down the source of a null pointer has never been particularly difficult.

There are a lot of refactoring things that dmd would benefit from. Adding assert(p!=NULL) everywhere will not add much value.
_______________________________________________
dmd-internals mailing list
dmd-internals@puremagic.com
http://lists.puremagic.com/mailman/listinfo/dmd-internals

Relying on the debugger to tell you where something went wrong is pretty ghetto, IMO.  For instance, the debugger makes all of the commented-out printfs() in the code also redundant, and yet I see thousands of these.  So which is it...should the compiler give you diagnostic information, or should you rely on the debugger?  Since we're on this topic, let's then switch it to the nature of diagnostics in dmd.

Since I work on apps professionally that need to be running 24/7, I am naturally inclined to rely on logging rather than debugging. Especially when there may be hundreds of instances running, while only a few actually experience a problem, logging usually turns out to be the better choice.  Then consider that logging is also more useful for bug reporting, as well as visualizing the code flow even in non-error cases.

One of my TODO items was to take the commented-out printfs and turn them into a first-class logging solution with configurable log levels/modules/etc. (so you can turn on logging in just one subsystem if you want).  If the reigning philosophy is instead to lean on the debugger, then that would be a waste of time.  What do y'all think?

Dave


On 11/9/2012 3:11 PM, Walter Bright wrote:
>
> On 11/9/2012 12:52 AM, Alex Rønne Petersen wrote:
>> I don't hack on DMD much, but here's my take on the matter: Asserting
>> every function parameter (where applicable of course) is good practice
>> because it helps you catch bugs as early as possible. If you don't do
>> this, you could end up with null pointers far into the program where
>> you then have to spend time figuring out where it came from. This is
>> especially nasty when e.g. building tree-like data structures and you
>> can't just look at the call stack to figure out where you got a null
>> pointer from.
>
> That is true for some programs. It isn't true for dmd, where tracking down the source of a null pointer has never been particularly difficult.
>
> There are a lot of refactoring things that dmd would benefit from. Adding assert(p!=NULL) everywhere will not add much value.
> _______________________________________________
> dmd-internals mailing list
> dmd-internals@puremagic.com
> http://lists.puremagic.com/mailman/listinfo/dmd-internals

_______________________________________________
dmd-internals mailing list
dmd-internals@puremagic.com
http://lists.puremagic.com/mailman/listinfo/dmd-internals

On Friday, November 09, 2012 22:10:48 David Held wrote:
> Relying on the debugger to tell you where something went wrong is pretty ghetto, IMO.  For instance, the debugger makes all of the commented-out printfs() in the code also redundant, and yet I see thousands of these. So which is it...should the compiler give you diagnostic information, or should you rely on the debugger?  Since we're on this topic, let's then switch it to the nature of diagnostics in dmd.
> 
> Since I work on apps professionally that need to be running 24/7, I am naturally inclined to rely on logging rather than debugging. Especially when there may be hundreds of instances running, while only a few actually experience a problem, logging usually turns out to be the better choice.  Then consider that logging is also more useful for bug reporting, as well as visualizing the code flow even in non-error cases.
> 
> One of my TODO items was to take the commented-out printfs and turn them into a first-class logging solution with configurable log levels/modules/etc. (so you can turn on logging in just one subsystem if you want).  If the reigning philosophy is instead to lean on the debugger, then that would be a waste of time.  What do y'all think?

Logging is extremely useful for applications which are constantly up and/or which involve a lot of network traffic or user interaction (which is typically non-repeatable and often can't be examined with the debugge running). However, that doesn't apply at all to a compiler. Compilers are incredibly deterministic, and errors are very, very repeatable. All you have to do is run the compiler on the same input, and you'll see the problem again, and stopping the compiler in a debugger causes no problems. So, I'd say that logging is completely inappropriate for a compiler.

- Jonathan M Davis
_______________________________________________
dmd-internals mailing list
dmd-internals@puremagic.com
http://lists.puremagic.com/mailman/listinfo/dmd-internals

On 11/9/2012 10:17 PM, Jonathan M Davis wrote:
> [...]
> Logging is extremely useful for applications which are constantly up and/or
> which involve a lot of network traffic or user interaction (which is typically
> non-repeatable and often can't be examined with the debugge running). However,
> that doesn't apply at all to a compiler. Compilers are incredibly
> deterministic, and errors are very, very repeatable. All you have to do is run
> the compiler on the same input, and you'll see the problem again, and stopping
> the compiler in a debugger causes no problems. So, I'd say that logging is
> completely inappropriate for a compiler.

If there were no logging statements in the compiler, you might have a point.  The fact that the dmd source is littered with them puts the lie to your insistence that they are "inappropriate". Obviously, Walter found them very useful at times.

Also, the compiler is only deterministic because it isn't yet multi-threaded.  That doesn't mean Walter hasn't attempted to make it such on more than one occasion.  If the compiler had more immutable data structures, this would probably be an easier effort. ;)

Finally, what the debugger cannot do is provide you with a history of what happened, except insofar as you are willing to manually capture the state change of various memory locations as you step through the program.  Although I tend to use debuggers as the tool of last resort, I will still insist that logs are even more powerful when used with the debugger, because they can often give you a good idea where you should set breakpoints, avoiding having to step through large portions of the code which aren't relevant to the problem at hand (after all, not all bugs are as obvious as a segfault/AV).

Dave

_______________________________________________
dmd-internals mailing list
dmd-internals@puremagic.com
http://lists.puremagic.com/mailman/listinfo/dmd-internals

On 11/9/2012 10:10 PM, David Held wrote:
> Relying on the debugger to tell you where something went wrong is pretty ghetto, IMO.  For instance, the debugger makes all of the commented-out printfs() in the code also redundant, and yet I see thousands of these.  So which is it...should the compiler give you diagnostic information, or should you rely on the debugger?  Since we're on this topic, let's then switch it to the nature of diagnostics in dmd.

I'll often use printf because although the debugger knows types, it rarely shows the values in a form I particularly need to track down a problem, which tends to be different every time. And besides, throwing in a few printfs is fast and easy, whereas setting break points and stepping through a program is an awfully tedious process. Or maybe I never learned to use debugger properly, which is possible since I've been forced to debug systems where no debugger whatsoever was available - I've even debugged programs using an oscilloscope, making clicks on a speaker, blinking an LED, whatever is available.

Note that getting a call stack for a seg fault does not suffer from these problems. I just:

   gdb --args dmd foo.d

and whammo, I got my stack trace, complete with files and line numbers.


>
> Since I work on apps professionally that need to be running 24/7, I am naturally inclined to rely on logging rather than debugging.

> Especially when there may be hundreds of instances running, while only a few actually experience a problem, logging usually turns out to be the better choice.  Then consider that logging is also more useful for bug reporting, as well as visualizing the code flow even in non-error cases.

Sure, but that doesn't apply to dmd. What's best practice for one kind of program isn't for another.

>
> One of my TODO items was to take the commented-out printfs and turn them into a first-class logging solution with configurable log levels/modules/etc. (so you can turn on logging in just one subsystem if you want).  If the reigning philosophy is instead to lean on the debugger, then that would be a waste of time.  What do y'all think?

I've tried that (see the LOG macros in template.c). It doesn't work very well, because the logging data tends to be far too voluminous. I like to tailor it to each specific problem. It's faster for me, and works.

_______________________________________________
dmd-internals mailing list
dmd-internals@puremagic.com
http://lists.puremagic.com/mailman/listinfo/dmd-internals