On Saturday, 4 January 2014 at 22:08:59 UTC, Andrej Mitrovic wrote:
> On 1/4/14, Adam D. Ruppe <destructionator@gmail.com> wrote:
>> The big thing people have asked for before is
>>
>> Object foo;
>> if(auto obj = checkNull(foo)) {
>>     obj == NotNull!Object
>> } else {
>>    // foo is null
>> }
>>
>> and i haven't figured that out yet...
>
> Here you go:
>
> -----
> import std.stdio;
>
> struct NotNull(T) { T obj; }
>
> struct CheckNull(T)
> {
>     private T _payload;
>
>     auto opCast(X = bool)() { return _payload !is null; }
>
>     @property NotNull!T getNotNull() { return NotNull!T(_payload); }
>     alias getNotNull this;
> }
>
> CheckNull!T checkNull(T)(T obj)
> {
>     return CheckNull!T(obj);
> }
>
> class C { }
>
> void main()
> {
>     Object foo;
>     if (auto obj = checkNull(foo))
>     {
>         writeln("foo is not null");
>     }
>     else
>     {
>         writeln("foo is null");
>     }
>
>     foo = new C;
>
>     if (auto obj = checkNull(foo))
>     {
>         // note: ":" rather than "==" due to alias this.
>         static assert(is(typeof(obj) : NotNull!Object));
>
>         // assignment will work of course (alias this)
>         NotNull!Object obj2 = obj;
>
>         writeln("foo is not null");
>     }
>     else
>     {
>         writeln("foo is null");
>     }
> }
> -----


Excuse me, not so fast ...

1. static assert asserts at compile time, so let's not put it in a runtime check of obj (converted to bool).

2.
    Object foo;
    assert(foo is null);
    NotNull!Object test = NotNull!Object(foo);
    assert(is(typeof(test) : NotNull!Object));
    static assert(is(typeof(test) : NotNull!Object));

always passes for a null reference, with or without static, as it should. So what good is NotNull?

3. To wit: if you replace main with

void main() {
    Object foo;

    auto obj = checkNull(foo);
    if (obj) {
        static assert(is(typeof(obj) : NotNull!Object)); //let's leave this inside the dynamic if, just to demonstrate (yuck!)
        assert(is(typeof(obj) : NotNull!Object));
        NotNull!Object obj2 = obj;
        writeln("foo is not null");
    } else {
        static assert(is(typeof(obj) : NotNull!Object));
        assert(is(typeof(obj) : NotNull!Object));
        writeln("foo is null");
    }

    foo = new C;

    obj = checkNull(foo);
    if (obj) {
        static assert(is(typeof(obj) : NotNull!Object));
        assert(is(typeof(obj) : NotNull!Object));
        NotNull!Object obj2 = obj;
        writeln("foo is not null");
    } else {
        static assert(is(typeof(obj) : NotNull!Object));
        assert(is(typeof(obj) : NotNull!Object));
        writeln("foo is null");
    }
}

all asserts pass, be they static or dynamic. The output is correct as in the original, but it has nothing to do with obj being convertible to NotNull!Object or not. It always is. The output is correct due to the simple runtime check whether obj is null or not.

Greets.
(PS: I think I just poked me in the eye with my pencil.)

* correct: whether obj converts to true or false

Just found out that when I replace

struct NotNull(T) { T obj; }

with (http://arsdnet.net/dcode/notnull.d)'s definition of NotNull it all makes sense.

Greets.

On Sunday, 5 January 2014 at 16:21:31 UTC, Jacob Carlborg wrote:
> On 2014-01-05 13:58, Paulo Pinto wrote:
>
>> Well it depends. On my case, the technology stack is always choosen from
>> the customers.
>>
>> Our freedom to choose is quite limited.
>
> One could think that the technology stack is chosen based on the task it should solve.

I oversimplified our use case.

Usually on my line of work, the company gets a request for proposal given a certain problem and corresponding technology being used.

We then look for developers with the skill sets being asked for.

The teams are usually a mix of people with the requested skill sets and a new ones that will learn on the job as a means to acquire those skills, in case similar projects appear.

So the direction of what technologies the company masters is driven by customer requests, not by what we might suggest.


--
Paulo

On Monday, 6 January 2014 at 04:16:56 UTC, H. S. Teoh wrote:
> Since a null pointer implies that there's some kind of logic error in
> the code, how much confidence do you have that the other 99 concurrent
> requests aren't being wrongly processed too?

That doesn't matter if the service isn't critical, it only matters if it destructively writes to a database. You can also shut down parts of the service rather than the entire service.

> Based on this, I'm inclined to say that if a web request process
> encountered a NULL pointer, it's probably better to just reset back to a known-good state by restarting.

I many cases it might be, but it should be up to the project management or the organization to set the policy, not the language designer. This is an issue I have with many of the "c++ wannabe languages". They enforce policies that shouldn't be done on the level of a tool (it could be a compiler option though). My pet peeve is Go and its banning of assert() because many programmers use it in an appropriate manner. In D you have the overloading of conditionals and others. With Ada and Rust, it is ok, because they exist to enforce a policy for existing organizations (DoD, Mozilla). Generic programming languages that claim should be more adaptable.

> No, usually you'd set things up so that if the webserver goes down, an init script would restart it. Restarting is preferable, because it resets the program back to a known-good state.

The program might be written in such a way that you know that it is a good state when you catch the null exception.

> careless bug, but a symptom of somebody attempting to inject a root
> exploit?  Blindly continuing will only play into the hand of the
> attacker.

Protection against root exploits should be done on lower level (jail).

> The thing is, a null pointer error isn't just an exceptional condition
> caused by bad user data; it's a *logic* error in the code. It's a sign
> that something is wrong with the program logic.

And so is array-out-of bounds, or division-by-zero.

> Tell the client not to do that again? *That* sounds like the formula for
> a DoS vector (a rogue client deliberately sending the crashing request
> over and over again).

What else can you do? You return an error and block subsequent requests if appropriate.

In a networked  computer game you log misbehaviour, you drop the client after a random delay and you can block the offender. What you do not want is to disable the entire service. It is better to run a somewhat faulty service that entertain and retain your customers than shutting down until a bug fix appears. If it takes 15-30 seconds to bring the server back up then you cannot afford to reset all the time.

I can point to many launches of online computer games that has resulted in massive losses due to servers going down during the first few weeks. That is actually one good reason to not use C++ in game servers, the lack of robustness to failure. In some domains the ability to keep the service running, and the ability to turn off parts of the service, is more important than correctness. What you want is a log of player-resources so that you post-failure can restore game balance.

> data and start over. This is a case of a problem with the *code*, which
> means you cannot trust the program will continue doing what you

That depends on how the program is written and in which area the null exception happend. It might even be a known bug that might take a long time to locate and fix, but that is known to be innocent.

> things will still work the way you think they work, will only lead to
> your program running the exploit code that has been injected into the
> corrupted stack.

Pages with execution bit set should be write protected. You can only jump into existing code, injection of code isn't really possible. So if the existing code is unknown to the attacker that attack vector is weak.

> The safest recourse is to reset the program back to a known state.

I see no problem with trapping None-failures in pure Python and keeping the service running. The places where it can happen tend to be when you are looking up a non-existing object in a database. Quite innocent if you can backtrack all the way down to the request handler and return an appropriate status code.

If you use the safe subset of D, why should it be different?

On Monday, 6 January 2014 at 02:03:03 UTC, Walter Bright wrote:
> On 1/5/2014 4:20 PM, deadalnix wrote:
>> On Monday, 6 January 2014 at 00:13:19 UTC, Walter Bright wrote:
>>> I'd still like to see an example, even a contrived one.
>>
>> void foo(int* ptr) {
>>     *ptr;
>>     if (ptr is null) {
>>         // do stuff
>>     }
>>
>>     // do stuff.
>> }
>>
>> The code look stupid, but this is quite common after a first pass of
>> optimization/inlining, do end up with something like that when a null check if
>> forgotten.
>
> The code is fundamentally broken. I don't know of any legitimate optimization transforms that would move a dereference from after a null check to before, so I suspect the code was broken before that first pass of optimization/inlining.
>

I know. But his code will behave in random ways, not instant fail. This example show that the instant fail approach you seem to like is inherently flawed.

> If you're writing code where you expect undefined behavior to cause a crash, then that code has faulty assumptions.
>
> This is why many languages work to eliminate undefined behavior - but still, as a professional programmer, you should not be relying on undefined behavior, and it is not the optimizer's fault if you did. If you deliberately rely on UB (and I do on occasion) then you should be prepared to take your lumps if the compiler changes.

Are you saying that dereferencing null must be undefined behavior, and not instant failure ? That contradict the position you gave before.

On Monday, 6 January 2014 at 09:59:55 UTC, Organic Farmer wrote:
> Just found out that when I replace
>
> struct NotNull(T) { T obj; }
>
> with (http://arsdnet.net/dcode/notnull.d)'s definition of NotNull it all makes sense.

Yes, it is very important to use the full type so you get the checks. The reason this is better than the segfault is that here, the run-time error occurs closer to the point of assignment instead of at the point of use.

my_function(enforceNotNull(obj)); // throw right here if it is null

This especially matters if the function stores the object somewhere. Having an unexpected null in the middle of a container can be a hidden bug for some time. Fixing it means finding how null got in there in the first place, and the segfault stack trace is almost no help at all. The not null things though catch it early and then the type system (almost*) ensures it stays that way.

* it is still possible to use casts and stuff to get a null in there but surely nobody would actually do that!

On Monday, 6 January 2014 at 00:20:59 UTC, deadalnix wrote:
> On Monday, 6 January 2014 at 00:13:19 UTC, Walter Bright wrote:
>> I'd still like to see an example, even a contrived one.
>
> void foo(int* ptr) {
>     *ptr;
>     if (ptr is null) {
>         // do stuff
>     }
>
>     // do stuff.
> }

> The problem here is that the if can be removed, as you can't reach that point if the pointer is null, but *ptr can also be removed later as it is a dead load.
>
> The resulting code won't crash and do random shit instead.

"Code can't be reached if pointer is null" means "The code could fail before reaching here". Honestly, this looks like an optimizer issue to me. Who the **** would remove code that could fail?

On Saturday, 4 January 2014 at 11:36:20 UTC, bearophile wrote:
[cut]
> Why aren't they using Ada?[cut]

Because the "software world" is unfortunately very much a "fashion world": if it's old, it's not interesting.. :-(

renoX

On Monday, 6 January 2014 at 15:56:09 UTC, fra wrote:
> On Monday, 6 January 2014 at 00:20:59 UTC, deadalnix wrote:
>> On Monday, 6 January 2014 at 00:13:19 UTC, Walter Bright wrote:
>>> I'd still like to see an example, even a contrived one.
>>
>> void foo(int* ptr) {
>>    *ptr;
>>    if (ptr is null) {
>>        // do stuff
>>    }
>>
>>    // do stuff.
>> }
>
>> The problem here is that the if can be removed, as you can't reach that point if the pointer is null, but *ptr can also be removed later as it is a dead load.
>>
>> The resulting code won't crash and do random shit instead.
>
> "Code can't be reached if pointer is null" means "The code could fail before reaching here". Honestly, this looks like an optimizer issue to me. Who the **** would remove code that could fail?

That is the whole node of the issue.

As a matter of fact, any load can trap. Considering this, we either want the optimizer to prove that the load won't trap before to optimize it away OR we consider the trap as a special case that can be removed by the optimizer.

The thing is that the first option is highly undesirable for performance reason, as the optimizer won't be able to remove most loads. This isn't something small as memory is WAY slower than CPU nowadays (a cache miss at the very least 200 cycles, typically in the 300, add limitation in memory bandwidth and you have an idea).

That is what I cover in short in one of my previous posts. We could decide that the optimizer can't remove load unless it can prove that they do not trap. That mean that most loads can't be optimized away anymore. Or, we can decide that trapping in not guaranteed, and then dereferencing null is undefined behavior, which is much worse that a compile time failure.