On 10/31/2013 2:24 PM, eles wrote:
> On Thursday, 31 October 2013 at 18:46:07 UTC, Walter Bright wrote:
>> On 10/31/2013 9:00 AM, eles wrote:
>> What if the hardware fails? Such as a bad memory bit that flips a bit in the
>> perfect software, and now it decides to launch nuclear missiles?
>
> If that happens, any software verification could become useless. On the latest
> project that I'm working on, we simply went with two identical (but not
> independently-developed, just identical) hardwares, embedded software on them.
>
> A comparator compares the two outputs. Any difference results in an emergency
> procedure (either a hardware reboot through a watchdog, either a controlled
> shutdown - to avoid infinite loop reboot).

What I posted on HN:

------------------

All I know in detail is the 757 system, which uses triply-redundant hydraulic systems. Any computer control of the flight control systems (such as the autopilot) can be quickly locked out by the pilot who then reverts to manual control.
The computer control systems were dual, meaning two independent computer boards. The boards were designed independently, had different CPU architectures on board, were programmed in different languages, were developed by different teams, the algorithms used were different, and a third group would check that there was no inadvertent similarity.

An electronic comparator compared the results of the boards, and if they differed, automatically locked out both and alerted the pilot. And oh yea, there were dual comparators, and either one could lock them out.

This was pretty much standard practice at the time.

Note the complete lack of "we can write software that won't fail!" nonsense. This attitude permeates everything in airframe design, which is why air travel is so incredibly safe despite its inherent danger.

https://news.ycombinator.com/item?id=6639097

On Thursday, 31 October 2013 at 19:45:17 UTC, Walter Bright wrote:
> On 10/31/2013 7:57 AM, H. S. Teoh wrote:
>> You don't know how thankful I am for having learnt the concept of
>> pumping the brakes, ABS or not. I'm afraid too many driving instructors
>> nowadays just advise slamming the brakes and relying on the ABS to do
>> the job. It doesn't *always* work!
>
> Pumping the brakes is not how to get the max braking effect.
>
> The way to do it is to push on the pedal to about 70-80% of braking force. This causes the car to push its weight onto the front tires and load them up. Then go 100%. You'll stop a lot faster, because with more weight on the front tires they have more grip. (I think this is called 2 step braking.)
>
> You lose about 30% of braking force when the tires break loose. The trick is to press the pedal just short of that happening, which can be found with a bit of practice.
>
> The downside of just slamming the brakes on and letting the ABS take care of it is you lose the 2-step effect.
>
> There are also cases where you *want* to lock the tires. That case is when you're in a skid and the car is at a large angle relative to its velocity vector. This will cause the car to slide in a straight line, meaning that other cars can avoid you. If you don't lock the wheels, the wheels can arbitrarily "grab" and shoot the car off in an unexpected direction - like over the embankment, or into the car that was dodging you. The car will also stop faster than if the wheels suddenly grab when you're at a 30 degree angle.
>
> But yeah, I'd guess less than 1% of drivers know this stuff. And even if you know it, you have to practice it now and then to be proficient at it.

I still think that, although software can help to make things safer, common sense should not be turned off while driving. If it's raining heavily, slow down. Sounds simple, but many drivers don't do it, with or without software.

On Thursday, 31 October 2013 at 21:36:11 UTC, eles wrote:
>
> Yeah, maybe is a corporation culture to avoid the term "bug", but we always use the term "change request". Maybe it has a better image :)
>
Lately, I've instead been reframing my thinking toward parity with Dijkstra.  EWD1036 [0] is particularly relevant to this topic:

"We could, for instance, begin with cleaning up our language by no longer calling a bug a bug but by calling it an error. It is much more honest because it squarely puts the blame where it belongs, viz. with the programmer who made the error. The animistic metaphor of the bug that maliciously sneaked in while the programmer was not looking is intellectually dishonest as it disguises that the error is the programmer's own creation. The nice thing of this simple change of vocabulary is that it has such a profound effect: while, before, a program with only one bug used to be 'almost correct', afterwards a program with an error is just 'wrong' (because in error)."

As a bonus, my experience is it more readily encourages management types to accept that fixing them is important.

> Normally, it is assumed that passing the tests proves that specifications are accomplished, so the software is perfect.
>
> This, of course, if the tests themselves would be correct 100% and *really* extensive.
>
Again from EWD1036:

"Besides the notion of productivity, also that of quality control continues to be distorted by the reassuring illusion that what works with other devices works with programs as well. It is now two decades since it was pointed out that program testing may convincingly demonstrate the presence of bugs, but can never demonstrate their absence. After quoting this well-publicized remark devoutly, the software engineer returns to the order of the day and continues to refine his testing strategies, just like the alchemist of yore, who continued to refine his chrysocosmic purifications."

This passage comes just after he laments that "software engineer" had been diluted so thoroughly as to be meaningless.  (I'd greatly appreciate if this term could be reclaimed, honestly.  Experience has shown me quite clearly that not every programmer is an engineer.)

-Wyatt

[0] http://www.cs.utexas.edu/users/EWD/transcriptions/EWD10xx/EWD1036.html

Walter Bright:

> ...

Everyone who writes safety critical software should really avoid languages unable to detect integral overflows (at compile-time or run-time) in all normal numerical operations, and languages that have undefined operations in their basic semantics.

So Ada language is OK, C and D are not OK for safety critical software.

Bye,
bearophile

On Friday, 1 November 2013 at 13:52:01 UTC, Wyatt wrote:
> On Thursday, 31 October 2013 at 21:36:11 UTC, eles wrote:
>>
>> Yeah, maybe is a corporation culture to avoid the term "bug", but we always use the term "change request". Maybe it has a better image :)
>>
> Lately, I've instead been reframing my thinking toward parity with Dijkstra.  EWD1036 [0] is particularly relevant to this topic:
>
> "We could, for instance, begin with cleaning up our language by no longer calling a bug a bug but by calling it an error. It is much more honest because it squarely puts the blame where it belongs, viz. with the programmer who made the error. The animistic metaphor of the bug that maliciously sneaked in while the programmer was not looking is intellectually dishonest as it disguises that the error is the programmer's own creation. The nice thing of this simple change of vocabulary is that it has such a profound effect: while, before, a program with only one bug used to be 'almost correct', afterwards a program with an error is just 'wrong' (because in error)."
>
> As a bonus, my experience is it more readily encourages management types to accept that fixing them is important.
>
>> Normally, it is assumed that passing the tests proves that specifications are accomplished, so the software is perfect.
>>
>> This, of course, if the tests themselves would be correct 100% and *really* extensive.
>>
> Again from EWD1036:
>
> "Besides the notion of productivity, also that of quality control continues to be distorted by the reassuring illusion that what works with other devices works with programs as well. It is now two decades since it was pointed out that program testing may convincingly demonstrate the presence of bugs, but can never demonstrate their absence. After quoting this well-publicized remark devoutly, the software engineer returns to the order of the day and continues to refine his testing strategies, just like the alchemist of yore, who continued to refine his chrysocosmic purifications."
>
> This passage comes just after he laments that "software engineer" had been diluted so thoroughly as to be meaningless.  (I'd greatly appreciate if this term could be reclaimed, honestly.  Experience has shown me quite clearly that not every programmer is an engineer.)
>
> -Wyatt
>
> [0] http://www.cs.utexas.edu/users/EWD/transcriptions/EWD10xx/EWD1036.html

No, not every programmer is an engineer. But not every programmer writes safety critical code. If Firefox crashes, nobody dies as a consequence (hopefully!).

On Friday, 1 November 2013 at 13:52:01 UTC, Wyatt wrote:
> On Thursday, 31 October 2013 at 21:36:11 UTC, eles wrote:

> much more honest because it squarely puts the blame where it belongs, viz. with the programmer who made the error. The

That's in an ideal world. When different people work on the same code base, it is not so easy to tell who made the error. Look at a race condition when neither of two or three developers takes the mutex. Who made the error then? All that you have is a buggy program (btw, error implies something about being systematic, while bugs are not necessarily) or a program with errors. But, telling *who* made the error is not that simple. And, in most of the cases, would be also quite useless. We do not hunt people, but bugs :p (sorry, it sounds better than hunting errors :)

> testing may convincingly demonstrate the presence of bugs, but can never demonstrate their absence.

Everybody knows that. Alas, testing is not the silver bullet, but at least is a bullet. Just imagine how software shipped without any testing will behave: "it compiles! let's ship it!" Corporations are not chasing the phyilosophical perfections, they are pragmatic. The thing that somewhat works and they have on the table is testing. In a perfect world, you'd have perfect programmers, perfect programs. The thing is, you are not living in a perfect world. Tests are not perfect neither but are among the best that you can get.

On Friday, 1 November 2013 at 15:03:47 UTC, bearophile wrote:
> Walter Bright:

> avoid languages unable to detect integral overflows (at compile-time or run-time) in all normal numerical operations,

Yeah, after the scope() statement, this is the thing that I'd want the most in my C, at least during debugging and testing. At least for some variables that we assume should never overflow (and most of them are like that).

eles:

> Yeah, after the scope() statement, this is the thing that I'd want the most in my C,

The latest versions of Clang are able to catch some integral overflows at run-time. It is of course only a small part of the whole amount of things needed to produce correct code, but for high integrity software it's useful.

Bye,
bearophile

On 11/1/2013 8:03 AM, bearophile wrote:
> Everyone who writes safety critical software should really avoid languages
> unable to detect integral overflows (at compile-time or run-time) in all normal
> numerical operations, and languages that have undefined operations in their
> basic semantics.
>
> So Ada language is OK, C and D are not OK for safety critical software.

I think you're missing the point. Improving the quality of the software is not the answer to making fail safe systems.

On 11/1/2013 6:52 AM, Wyatt wrote:
> "We could, for instance, begin with cleaning up our language by no longer
> calling a bug a bug but by calling it an error. It is much more honest because
> it squarely puts the blame where it belongs, viz. with the programmer who made
> the error.

Although it is tempting to do so, creating a culture of "blame the programmer" for the mistakes he's made also creates a culture of denial of problems.

If you want to create quality of software, a far better culture is one that recognizes that people are imperfect, and looks for collaborative ways to engineer the possibility of errors out of the system. That doesn't work if you're trying to pin the blame on somebody.