On Wednesday, 31 May 2017 at 20:23:21 UTC, Nick Sabalausky (Abscissa) wrote:
> On 05/31/2017 03:17 PM, Moritz Maxeiner wrote:
>> in general you have to assume that the index *being* out of bounds is itself the *result* of *already occurred* data corruption;
> Of course not, that's absurd. Where do people get the idea that out-of-bounds *implies* pre-existing data corruption?

You assume something I did not write. What I wrote is that the runtime cannot *in general* (i.e. without further information about the semantics of your specific program) assume that it was *not* preexisting data corruption.

> Most of  the time, out-of-bounds comes from a bug (especially in D, what with all of its safeguards).

Unfortunately the runtime has no way to know *if* the out of bounds comes from a bug or a data corruption, which was my point; only a human can know that. What is the most likely culprit is irrelevant for the default behaviour, because as long as it *could* be data corruption, the runtime cannot by default assume that it is not; that would be unsafe.

>
> Sure, data corruption is one possible cause of out-of-bounds, but data corruption is one possible cause of *ANYTHING*. So just to be safe, let's just abort on all exceptions, and upon everything else for that matter.

No, abort on Errors where the runtime cannot know if data corruption has already occured, i.e. the program is in an undefined state. If you, as the programmer, know that it is safe, you have to code that in.

On 5/31/17 4:53 PM, Kagamin wrote:
> On Wednesday, 31 May 2017 at 13:04:52 UTC, Steven Schveighoffer wrote:
>> This seems like a large penalty for "almost" corrupting memory. No
>> other web framework I've used crashes the entire web server for such a
>> simple programming error.
>
> On windows you can set up service restart settings in case it crashes.
> Useful for services that crash regularly.

That *would* be a feature on Windows ;)

No, this is Linux, so I'll have to research how to properly do it with systemd.

-Steve

On Wednesday, 31 May 2017 at 21:03:02 UTC, Steven Schveighoffer wrote:
>
> No, this is Linux, so I'll have to research how to properly do it with systemd.

OT: *with whatever process supervisor floats your boat.

On Wednesday, 31 May 2017 at 21:02:06 UTC, Steven Schveighoffer wrote:
>
> Nope, an autonomous system did not type out my code that caused the out of bounds error, I did :)

Same as the human who typed out the code of the autonomous system.

On 31.05.2017 22:45, Moritz Maxeiner wrote:
> On Wednesday, 31 May 2017 at 20:09:16 UTC, Nick Sabalausky (Abscissa) wrote:
>> [...]
>>> program is in an undefined state and should terminate asap.
>>
>> Then out-of-bounds and assert failures should be Exception not Error. Frankly, even out-of-memory, arguably. And then there's null dereference... In other words, basically everything.
> 
> No, because as I stated in my other post, the runtime *cannot* assume that it is safe *in all cases*. If there is even one single case in which it is unsafe, it must abort.

Hence all programs must abort on startup.

On 05/31/2017 02:00 PM, Steven Schveighoffer wrote:
> On 5/31/17 3:17 PM, Moritz Maxeiner wrote:

>> It is not that accessing the array out of bounds *leading* to data
>> corruption that is the issue here, but that in general you have to
>> assume that the index *being* out of bounds is itself the *result* of
>> *already occurred* data corruption;
>
> To be blunt, no this is completely wrong.

Blunter: Moritz is right. :)

> Memory corruption *already having happened* can cause any
> number of errors.

True.

> The point of bounds checking is to prevent memory corruption in
> the first place.

That's just one goal. It also maintains an invariant of arrays: The index value must be within bounds.

> I could memory corrupt the length of the array also (assuming a
> dynamic array), and bounds checking merrily does nothing to
> stop further memory corruption.

That's true but the language provides no tool to check for that. The fact that program correctness is not achievable in general should not have any bearing on bounds checking.

>> and if data corruption occurred for
>> the index, you *cannot* assume that *only* the index has been affected.
>> The runtime cannot simply assume the index being out of bounds is not
>> the result of already occurred data corruption, because that is
>> inherently unsafe, so it *must* terminate asap as the default.
>
> The runtime should not assume that crashing the whole program is
> necessary when an integer is out of range. Preventing actual corruption,
> yes that is good. But an Exception would have done the job just fine.

How could an Exception work in this case? Catch it and repeat the same bug over and over again? What would the program be achieving? (I assume the exception handler will not arbitrarily decrease index values.)

Ali

On Wed, May 31, 2017 at 11:29:53PM +0200, Timon Gehr via Digitalmars-d wrote:
> On 31.05.2017 22:45, Moritz Maxeiner wrote:
[...]
> > No, because as I stated in my other post, the runtime *cannot* assume that it is safe *in all cases*. If there is even one single case in which it is unsafe, it must abort.
> 
> Hence all programs must abort on startup.

If D had *true* garbage collection, it would have done this upon starting up any buggy program. :-D


T

-- 
Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? -- Michael Beibl

On Wednesday, 31 May 2017 at 21:30:05 UTC, Ali Çehreli wrote:
> How could an Exception work in this case? Catch it and repeat the same bug over and over again? What would the program be achieving? (I assume the exception handler will not arbitrarily decrease index values.)

How is this different from a file system exception?
The file system is memory too...

On Wednesday, 31 May 2017 at 21:00:43 UTC, Steven Schveighoffer wrote:
> On 5/31/17 3:17 PM, Moritz Maxeiner wrote:
>> On Wednesday, 31 May 2017 at 13:04:52 UTC, Steven Schveighoffer wrote:
>>> [...]
>>>
>>> What are your thoughts? Have you run into this? If so, how did you
>>> solve it?
>>>
>>
>> It is not that accessing the array out of bounds *leading* to data
>> corruption that is the issue here, but that in general you have to
>> assume that the index *being* out of bounds is itself the *result* of
>> *already occurred* data corruption;
>
> To be blunt, no this is completely wrong.

I disagree.

> Memory corruption *already having happened* can cause any number of errors.

Correct, of which out of bounds array is *one*.

> The point of bounds checking is to prevent memory corruption in the first place.

That is *one* of the purposes. The other is to abort in case of already occurred memory corruption.

> I could memory corrupt the length of the array also (assuming a dynamic array), and bounds checking merrily does nothing to stop further memory corruption.

Yes, that is one case against out of bounds checks do not help; but that changes nothing for the case we were talking about.

>
> The runtime should not assume that crashing the whole program is necessary when an integer is out of range.

Without *any* other information, I think it should.

> Preventing actual corruption, yes that is good. But an Exception would have done the job just fine.

If it were only about further memory corruption, yes, but as I said, my argument about preexisting corruption remains.

>
> But that ship, as I said elsewhere, has sailed. We can't change it to Exception now, as that would break just about all nothrow code in existence.

Sure.

>
>> So in your specific use case I would say use a wrapper. This is one of
>> the reasons why I am working on my own library for data structures (libds).
>
> That is my conclusion too. Is your library in a usable state?

Well, since I really needed only a single data structure at the time, it only contains a binary heap so far, but I believe it to be usable. I intend to add a dynamic array implementation next.

> Perhaps we should not repeat efforts, though I wasn't planning on making a robust public library for it :)

Well, you can take a look at the binary heap implementation[1] and decide if that a style you are interested in, but it does currently use errors for things such as removing an element when the heap is empty; I am not sure there, what I intend to do here, but I might make it configurable via the Conf template parameter in a design-by-introspection style.

[1] https://github.com/Calrama/libds

On Wed, May 31, 2017 at 02:30:05PM -0700, Ali Çehreli via Digitalmars-d wrote:
> On 05/31/2017 02:00 PM, Steven Schveighoffer wrote:
[...]
> > The runtime should not assume that crashing the whole program is necessary when an integer is out of range. Preventing actual corruption, yes that is good. But an Exception would have done the job just fine.
> 
> How could an Exception work in this case? Catch it and repeat the same bug over and over again? What would the program be achieving? (I assume the exception handler will not arbitrarily decrease index values.)
[...]

In this particular case, the idea is that the fibre that ran into the bug would throw an Exception to the main loop, which catches it and terminates the fibre (presumably also sending an error response to the client browser), while continuing to process other, possibly-ongoing requests normally.

Rather than having the one bad request triggering the buggy code and causing *all* currently in-progress requests to terminate because the entire program has aborted.

An extreme example of this is if you had a vibe.d server hosting multiple virtual domains belonging to different customers. It's bad enough that one customer's service would crash when it encounters a bug, but it's far worse to have *all* customers' services crash just because *one* of them encountered a bug.

This is an interesting use case, because conceptually speaking, each vibe.d fibre actually represents an independent computation, so any fatal errors like out-of-bounds bugs should cause the termination of the *fibre*, rather than *everything* that just happens to be running in the same process.  If vibe.d had been implemented with, say, forked processes instead, this wouldn't have been an issue.  But of course, the fibre implementation was chosen for performance (and possibly other) reasons. Forking would give you the per-request isolation needed to handle this kind of problem cleanly, but it also comes with a hefty performance price tag.  Like all things in practical engineering, it's a tradeoff.

I'd say creating a custom type that throws Exception instead of Error is probably the best solution here, given what we have.


T

-- 
The fact that anyone still uses AOL shows that even the presence of options doesn't stop some people from picking the pessimal one. - Mike Ellis