On Wednesday, 31 May 2017 at 13:04:52 UTC, Steven Schveighoffer wrote:
> I have discovered an annoyance in using vibe.d instead of another web framework. Simple errors in indexing crash the entire application.
Is this option useful for you?

VibeDebugCatchAll 	Enables catching of exceptions that derive from Error. This can be useful during application development to get useful error information while keeping the application running, but can generally be dangerous, because the application may be left in a bad state after an Error has been thrown.

From: http://vibed.org/docs#compile-time-configuration

On Thursday, 1 June 2017 at 09:46:09 UTC, Ola Fosheim Grøstad wrote:
> On Thursday, 1 June 2017 at 09:18:24 UTC, Guillaume Piolat wrote:
>> Even with consumer software, you may want to crash immediately so that you actually get complaints from testers/buyers instead of having a silent, invisible bug that no one will report ever.
>
> No. You don't want to crash immediately. In fact, you want to save and recover. Preferably without much work lost and without the user being bothered by it.

Solved by auto-saving, _before_ the crash

On Thursday, 1 June 2017 at 10:26:24 UTC, Steven Schveighoffer wrote:
> On 5/31/17 9:05 PM, Walter Bright wrote:
>> On 5/31/2017 6:04 AM, Steven Schveighoffer wrote:
>>> Technically this is a programming error, and a bug. But memory hasn't
>>> actually been corrupted.
>>
>> Since you don't know where the bad index came from, such a conclusion
>> cannot be drawn.
>
> You could say that about any error. You could say that about malformed unicode strings, malformed JSON data, file not found. In this mindset, everything should be an Error, and nothing should be recoverable.
>
>>
>>
>>> This seems like a large penalty for "almost" corrupting memory. No
>>> other web framework I've used crashes the entire web server for such a
>>> simple programming error.
>>
>> Hence the endless vectors for malware insertion in those other frameworks.
>
> No, those are due to the implementation of the interpreter. If the interpreter is implemented in @safe D, then you don't have those problems.
>
>>> Compare this to, let's say, a malformed unicode string (exception),
>> malformed JSON data (exception), file not found (exception), etc.
>>
>> That's because those are input and environmental errors, not programming
>> bugs.
>
> Not necessarily. A file name could be sourced from the program, but have a typo. An index could come from the environment. The library can't know, but makes assumptions one way or the other. Just like we assume you want to use the GC, these assumptions are harmful for those who need it to be the other way.
>
>> There can be grey areas in classifying problems as input errors or
>> programming bugs, and those will need some careful thought by the
>> programmer as to which bin they fall into, and then code accordingly.
>>
>> Array overflows are not a grey area, however. They are always
>> programming bugs.
>
> Of course, programming bugs cause all kinds of Errors and Exceptions alike. Environmental bugs can cause Array overflows.

I think the idea is that no, array overflows can never be caused by the environment in a correct program. If you don't adequately screen the environmental input, your program is incorrect.


This is how I think about it:

There are 3 categories of bugs: known safe to survive, known unsafe to survive, unknown safety.

Range Errors are an example of errors that can be considered "unknown safety", so by default we assume it is unsafe to continue.

If you - as the human developer - decide that the specific RangeError bug from this place in the code is actually known safe to survive, you should add screening for the bad value and throw an Exception instead, or if that's difficult to do then catch the Error and then throw an Exception*. Note that these aren't fixes for the bug, these are explicit recognition of the continued existence of the bug while promising (@trusted style) that everything will still be OK.

If you decide it is truly an "unsafe to continue" bug, then let it carry on crashing there.

Ultimately of course you screen the environment at the appropriate level or fix the bug, do the "right thing" whatever that may be.

*note that you could abstract this away into an array type that throws Exceptions, but where would you know it was safe to use? Perhaps not so many places.

Tldr; if you know that a bug is safe to continue/recover from, put in the necessary code to do so.


I would be interested to see ideas of how to implement some sort of logical sandboxing in D. Perhaps if one calls a strongly pure @safe function, there is no way it can mess up shared state, so you know that as long as you disregard the result it will always be safe to continue... Effectively it's a "process within a process" or something like that. Of course you'd need to be able to guarantee you can catch Errors, plus even though the function you've called can't have *caused* the problem, it might be the only place where you *find* the problem and that might be bad enough to not want to continue from...

On Thursday, 1 June 2017 at 14:10:21 UTC, John Colvin wrote:

> I would be interested to see ideas of how to implement some sort of logical sandboxing in D. Perhaps if one calls a strongly pure @safe function, there is no way it can mess up shared state,

Oh yes, there is a way: http://forum.dlang.org/post/psdamamjecdwfeiuvqsz@forum.dlang.org

On Thursday, 1 June 2017 at 14:21:35 UTC, Stanislav Blinov wrote:
> On Thursday, 1 June 2017 at 14:10:21 UTC, John Colvin wrote:
>
>> I would be interested to see ideas of how to implement some sort of logical sandboxing in D. Perhaps if one calls a strongly pure @safe function, there is no way it can mess up shared state,
>
> Oh yes, there is a way: http://forum.dlang.org/post/psdamamjecdwfeiuvqsz@forum.dlang.org

Sure, @safe has some holes as it currently stands.

On Thu, Jun 01, 2017 at 02:04:40PM +0000, Guillaume Piolat via Digitalmars-d wrote:
> On Thursday, 1 June 2017 at 09:46:09 UTC, Ola Fosheim Grøstad wrote:
> > On Thursday, 1 June 2017 at 09:18:24 UTC, Guillaume Piolat wrote:
> > > Even with consumer software, you may want to crash immediately so that you actually get complaints from testers/buyers instead of having a silent, invisible bug that no one will report ever.
> > 
> > No. You don't want to crash immediately. In fact, you want to save and recover. Preferably without much work lost and without the user being bothered by it.
> 
> Solved by auto-saving, _before_ the crash

Yes.  Saving *after* a crash was detected is stupid, because you no longer can guarantee the user data you're saving hasn't already been corrupted.  I've experienced over-zealous "crash recovery" code in applications overwrite the last known good copy of my data with the latest, most up-to-date, and also most-corrupted data after it detected a problem. Not nice at all.


T

-- 
Question authority. Don't ask why, just do it.

On Thu, Jun 01, 2017 at 10:11:19AM +0000, Kagamin via Digitalmars-d wrote: [...]
> Sad reality is that d programmers are still comfortable writing code in 70s style playing with void* pointers and don't enable bound checks early enough, see https://issues.dlang.org/show_bug.cgi?id=13367

Huh? There is no void* in that bug report, and it was closed 3 years ago. What's your point?


T

-- 
Ph.D. = Permanent head Damage

On Thu, Jun 01, 2017 at 06:26:24AM -0400, Steven Schveighoffer via Digitalmars-d wrote: [...]
> Of course, programming bugs cause all kinds of Errors and Exceptions alike.  Environmental bugs can cause Array overflows.
> 
> I can detail exactly what happened in my code -- I am accepting dates from a given week from a web request. One of the dates fell outside the week, and so tried to access a 7 element array with index 9. Nothing corrupted memory, but the runtime corrupted my entire process, forcing a shutdown.
[...]

Isn't this a case of failing to sanitize user input adequately before using it for internal processing?  And failing to test the code with pathological data to ensure resilience before deploying to a live server?

In this case, nothing worse happened than an out-of-bounds array index. But we all know what *could* happen with unsanitized user input in other cases...


T

-- 
Stop staring at me like that! It's offens... no, you'll hurt your eyes!

On Thursday, June 01, 2017 14:40:59 John Colvin via Digitalmars-d wrote:
> On Thursday, 1 June 2017 at 14:21:35 UTC, Stanislav Blinov wrote:
> > On Thursday, 1 June 2017 at 14:10:21 UTC, John Colvin wrote:
> >> I would be interested to see ideas of how to implement some sort of logical sandboxing in D. Perhaps if one calls a strongly pure @safe function, there is no way it can mess up shared state,
> >
> > Oh yes, there is a way: http://forum.dlang.org/post/psdamamjecdwfeiuvqsz@forum.dlang.org
>
> Sure, @safe has some holes as it currently stands.

It's far better than nothing, but it definitely has holes. DIP 1000 is fixing a lot of those holes. Unfortunately, the only way to absolutely guarantee that it doesn't have any holes is to do it via whitelisting operations and then vetting every operation to make sure that it's safe for the compiler to say that it's @safe, whereas it's implemented by blacklisting operations that are determined to be unsafe. So, we'll probably always be at risk of having holes in @safe, but the situation is improving.

- Jonathan M Davis

On 06/01/2017 09:54 AM, Martin Tschierschke wrote:
> On Wednesday, 31 May 2017 at 13:04:52 UTC, Steven Schveighoffer wrote:
>> I have discovered an annoyance in using vibe.d instead of another web framework. Simple errors in indexing crash the entire application.
> Is this option useful for you?
> 
> VibeDebugCatchAll     Enables catching of exceptions that derive from Error. This can be useful during application development to get useful error information while keeping the application running, but can generally be dangerous, because the application may be left in a bad state after an Error has been thrown.
> 
> From: http://vibed.org/docs#compile-time-configuration
> 
> 

All that would do is *cause* corruption due to the way the runtime handles (or more precisely, doesn't handle) a thrown Error.