On 03/06/2012 03:39 PM, Mantis wrote:
> 06.03.2012 8:04, Chad J пишет:
>> On 03/06/2012 12:07 AM, Jonathan M Davis wrote:
>>>
>>> If you dereference a null pointer, there is a serious bug in your
>>> program.
>>> Continuing is unwise. And if it actually goes so far as to be a segfault
>>> (since the hardware caught it rather than the program), it is beyond
>>> a doubt
>>> unsafe to continue. On rare occasion, it might make sense to try and
>>> recover
>>> from dereferencing a null pointer, but it's like catching an
>>> AssertError. It's
>>> rarely a good idea. Continuing would mean trying to recover from a
>>> logic error
>>> in your program. Your program obviously already assumed that the
>>> variable
>>> wasn't null, or it would have checked for null. So from the point of
>>> view of
>>> your program's logic, you are by definition in an undefined state, and
>>> continuing will have unexpected and potentially deadly behavior.
>>>
>>> - Jonathan M Davis
>>
>> This could be said for a lot of things: array-out-of-bounds
>> exceptions, file-not-found exceptions, conversion exception, etc. If
>> the programmer thought about it, they would have checked the array
>> length, checked for file existence before opening it, been more
>> careful about converting things, etc.
>>
> It's different: with array-out-of-bounds there's no hardware detection,
> so its either checked in software or unchecked (in best case you'll have
> access violation or segfault, but otherwise going past the bounds of
> array leads to undefined behavior). Both file-not-found and conv
> exceptions often rely on user's input, in which case they do not
> necessarily mean bug in a program.
>

Alright.

>> To me, the useful difference between fatal and non-fatal things is how
>> well isolated the failure is. Out of memory errors and writes into
>> unexpected parts of memory are very bad things and can corrupt
>> completely unrelated sections of code. The other things I've
>> mentioned, null-dereference included, cannot do this.
>>
>> Null-dereferences and such can be isolated to sections of code. A
>> section of code might become compromised by the dereference, but the
>> code outside of that section is still fine and can continue working.
>>
>> Example:
>> [...]
>> And if riskyShenanigans were to modify global state... well, it's no
>> longer so well isolated anymore. This is just a disadvantage of global
>> state, and it will be true with many other possible exceptions too.
>>
>> Long story short: I don't see how an unexpected behavior in one part
>> of a program will necessarily create unexpected behavior in all parts
>> of the program, especially when good encapsulation is practiced.
>>
>> Thoughts?
>>
> If riskyShenanigans nullifies reference in a process, then it must check
> it before dereferencing. There's obviously a bug, and if program will
> leave a proper crash log you shouldn't have problems finding and fixing
> this bug. If you don't have access to function's source, then you cannot
> guarantee it's safeness and isolation, so recovering from exception is
> unsafe.

But what do you say to the notion of isolation?  someFunc is isolated from riskyShenanigans becuase it /knows/ what state is touched by riskyShenanigans.  If riskyShenanigans does something strange and unexpected, and yes, it does have a bug in it, then I feel that someFunc should be able to reset the state touched by riskyShenanigans and continue.

The thing I find really strange here is that there's this belief that if feature A is buggy then the unrelated feature B shouldn't work either. Why?  Shouldn't the user be able to continue using feature B?

Btw, crashing a program is bad.  That can lose data that the user has entered but not yet stored.  I should have a very good reason before I let this happen.

It would also be extremely frustrating for a user to have a program become crippled because some feature they don't even use will occasionally dereference null and crash the thing.  Then they have to wait for me to fix it, and I'm busy, so it could be awhile.

My impression so far is that this hinges on some kind of "where there's one, there's more" argument.  I am unconvinced because programs tend to have bugs anyways.  riskyShenanigans doing a null-dereference once doesn't mean it's any more likely to produce corrupt results the rest of the time: it can produce corrupt results anyways, because it is a computer program written by a fallible human being.  Anyone trying to be really careful should validate the results in someFunc.

On 03/06/2012 12:19 PM, Timon Gehr wrote:
> On 03/06/2012 04:46 PM, foobar wrote:
>> On Tuesday, 6 March 2012 at 10:19:19 UTC, Timon Gehr wrote:
>>
>>> This is quite close, but real support for non-nullable types means
>>> that they are the default and checked statically, ideally using data
>>> flow analysis.
>>
>> I agree that non-nullable types should be made the default and
>> statically checked but data flow analysis here is redundant.
>> consider:
>> T foo = ..; // T is not-nullable
>> T? bar = ..; // T? is nullable
>> bar = foo; // legal implicit coercion T -> T?
>> foo = bar; // compile-time type mismatch error
>> //correct way:
>> if (bar) { // make sure bar isn't null
>> // compiler knows that cast(T)bar is safe
>> foo = bar;
>> }
>>
>
> Right. This example already demonstrates some simplistic data flow
> analysis.
>
>
>> of course we can employ additional syntax sugar such as:
>> foo = bar || <default_value>;
>>
>> furthermore:
>> foo.method(); // legal
>> bar.method(); // compile-time error
>>
>> it's all easily implementable in the type system.
>
> Actually it requires some thinking because making initialization of
> non-null fields safe is not entirely trivial.
>
> For example:
> http://pm.inf.ethz.ch/publications/getpdf.php/bibname/Own/id/SummersMuellerTR11.pdf
>
>
> CTFE and static constructors solve that issue for static data.

I can't seem to download the PDF... it always gives me just two bytes.

But to initialize non-null fields, I suspect we would need to be able to do stuff like this:

class Foo
{
	int dummy;
}

class Bar
{
	Foo foo = new Foo();

	this() { foo.dummy = 5; }
}

Which would be lowered by the compiler into this:

class Bar
{
	// Assume we've already checked for bogus assignments.
	// It is now safe to make this nullable.
	Nullable!(Foo) foo;

	this()
	{
		// Member initialization is done first.
		foo = new Foo();
		
		// Then programmer-supplied ctor code runs after.
		foo.dummy = 5;
	}
}

I remember C# being able to do this.  I never understood why D doesn't allow this.  Without it, I have to repeat myself a lot, and that is just wrong ;).  Allowing this kind of initialization might also make it possible for us to have zero-argument struct constructors.

On 3/6/2012 5:29 PM, Chad J wrote:
> But what do you say to the notion of isolation? someFunc is isolated from
> riskyShenanigans becuase it /knows/ what state is touched by riskyShenanigans.
> If riskyShenanigans does something strange and unexpected, and yes, it does have
> a bug in it, then I feel that someFunc should be able to reset the state touched
> by riskyShenanigans and continue.


That's the theory. But in practice, when you get a seg fault, there's (at minimum) a logical error in your program, and it is in an undefined state. Since memory is all shared, you have no idea whether that error is isolated or not, and you *cannot* know, because there's a logic error you didn't know about.

Continuing on after the program has entered an unknown an undefined state is just a recipe for disaster.

On Tue, Mar 06, 2012 at 08:29:35PM -0500, Chad J wrote: [...]
> But what do you say to the notion of isolation?  someFunc is isolated from riskyShenanigans becuase it /knows/ what state is touched by riskyShenanigans.  If riskyShenanigans does something strange and unexpected, and yes, it does have a bug in it, then I feel that someFunc should be able to reset the state touched by riskyShenanigans and continue.
>
> The thing I find really strange here is that there's this belief that if feature A is buggy then the unrelated feature B shouldn't work either. Why?  Shouldn't the user be able to continue using feature B?

If feature A is buggy and the user is trying to use it, then there's a problem. If the user doesn't use feature A or knows that feature A is buggy and so works around it, then feature A doesn't (shouldn't) run and won't crash.


> Btw, crashing a program is bad.  That can lose data that the user has entered but not yet stored.  I should have a very good reason before I let this happen.

I don't know what your software design is, but when I write code, if there is the possibility of data loss, I always make the program backup the data at intervals. I don't trust the integrity of user data after a major problem like dereferencing a null pointer happens. Obviously there's a serious logic flaw in the program that led to this, so all bets are off as to whether the user's data is even usable.


> It would also be extremely frustrating for a user to have a program become crippled because some feature they don't even use will occasionally dereference null and crash the thing.  Then they have to wait for me to fix it, and I'm busy, so it could be awhile.

The fact that the unused feature running even though the user isn't using it is, to me, a sign that something like a null pointer dereference should be fatal, because it means that what you assumed the unused feature was doing before in the background was consistent, but it turns out to be false, so who knows what else it has been doing wrong before it hit the null pointer. I should hate for the program to continue running after that, since consistency has been compromised; continuing will probably only worsen the problem.


> My impression so far is that this hinges on some kind of "where there's one, there's more" argument.  I am unconvinced because programs tend to have bugs anyways.  riskyShenanigans doing a null-dereference once doesn't mean it's any more likely to produce corrupt results the rest of the time: it can produce corrupt results anyways, because it is a computer program written by a fallible human being.  Anyone trying to be really careful should validate the results in someFunc.

It sound like what you want is some kind of sandbox isolation function, and null pointers are just the most obvious problem among other things that could go wrong.

We could have a std.sandbox module that can run some given code (say PossiblyBuggyFeatureA) inside a sandbox, so that if it dereferences a null pointer, corrupts memory, or whatever, it won't affect UnrelatedFeatureB which runs in a different sandbox, or the rest of the system. This way you can boldly charge forward in spite of any problems, because you know that only the code inside the sandbox is in a bad state, and the rest of the program (presumably) is still in good working condition.

In Linux this is easily implemented by fork() and perhaps chroot() (if
you're *really* paranoid) and message-passing (so the main program is
guaranteed to have no corruption even when BadPluginX goes crazy and
starts trashing memory everywhere). I don't know about Windows, but I
assume there is some way to do sandboxing as well.


T

-- 
Customer support: the art of getting your clients to pay for your own incompetence.

Chad J:

> I can't seem to download the PDF... it always gives me just two bytes.
> 
> But to initialize non-null fields, I suspect we would need to be able to do stuff like this:

There are some links here: http://d.puremagic.com/issues/show_bug.cgi?id=4571

Bye,
bearophile

On Mar 6, 2012, at 6:29 PM, Walter Bright <newshound2@digitalmars.com> wrote:

> On 3/6/2012 5:29 PM, Chad J wrote:
>> But what do you say to the notion of isolation? someFunc is isolated from riskyShenanigans becuase it /knows/ what state is touched by riskyShenanigans. If riskyShenanigans does something strange and unexpected, and yes, it does have a bug in it, then I feel that someFunc should be able to reset the state touched by riskyShenanigans and continue.
> 
> 
> That's the theory. But in practice, when you get a seg fault, there's (at minimum) a logical error in your program, and it is in an undefined state. Since memory is all shared, you have no idea whether that error is isolated or not, and you *cannot* know, because there's a logic error you didn't know about.

Minor point, but some apps are designed such that segfaults are intended. I worked on a DB that dynamically mapped memory in the segfault handler and then resumed execution.  Since D is a systems languages, very few assumptions can be made about error conditions.

On 3/6/2012 7:08 PM, Sean Kelly wrote:
> Minor point, but some apps are designed such that segfaults are intended. I
> worked on a DB that dynamically mapped memory in the segfault handler and
> then resumed execution.  Since D is a systems languages, very few assumptions
> can be made about error conditions.

Yes, and I've written a GC implementation that relied on intercepting invalid page writes to construct its list of 'dirty' pages.

There's nothing in D preventing one from doing that, although for sure such code will be very, very system specific.

What I'm talking about is the idea that one can recover from seg faults resulting from program bugs.

On 3/6/2012 8:05 PM, Walter Bright wrote:
> What I'm talking about is the idea that one can recover from seg faults
> resulting from program bugs.

I've written about this before, but I want to emphasize that attempting to recover from program BUGS is the absolutely WRONG way to go about writing fail-safe, critical, fault-tolerant software.

Oh alright. Then we're in complete agreement.

On Mar 6, 2012, at 8:05 PM, Walter Bright <newshound2@digitalmars.com> wrote:

> On 3/6/2012 7:08 PM, Sean Kelly wrote:
>> Minor point, but some apps are designed such that segfaults are intended. I worked on a DB that dynamically mapped memory in the segfault handler and then resumed execution.  Since D is a systems languages, very few assumptions can be made about error conditions.
> 
> Yes, and I've written a GC implementation that relied on intercepting invalid page writes to construct its list of 'dirty' pages.
> 
> There's nothing in D preventing one from doing that, although for sure such code will be very, very system specific.
> 
> What I'm talking about is the idea that one can recover from seg faults resulting from program bugs.

On 03/07/2012 02:40 AM, Chad J wrote:
> On 03/06/2012 12:19 PM, Timon Gehr wrote:
...
>> For example:
>> http://pm.inf.ethz.ch/publications/getpdf.php/bibname/Own/id/SummersMuellerTR11.pdf
>>
>>
>>
>> CTFE and static constructors solve that issue for static data.
>
> I can't seem to download the PDF... it always gives me just two bytes.
>

Same here. Strange. Interestingly, it works if you copy and paste the link into google.