What is the right idiom for testing that a function call does segfault when you want it to?

-- 
Russel. ============================================================================= Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder@ekiga.net 41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel@winder.org.uk London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder

On 06-12-2012 10:02, Russel Winder wrote:
> What is the right idiom for testing that a function call does segfault
> when you want it to?
>

There is basically no portable, clean way to do that. Why are you relying on segmentation faults? It's generally speaking a very bad idea because behavior depends entirely on the platform and architecture...

-- 
Alex Rønne Petersen
alex@lycus.org
http://lycus.org

On Thursday, 6 December 2012 at 10:31:43 UTC, Alex Rønne Petersen wrote:
> On 06-12-2012 10:02, Russel Winder wrote:
>> What is the right idiom for testing that a function call does segfault
>> when you want it to?
>>
>
> There is basically no portable, clean way to do that. Why are you relying on segmentation faults? It's generally speaking a very bad idea because behavior depends entirely on the platform and architecture...

I am not relying on segfaults, that would just be silly ;-) The issue is that unit tests should test error as well as success. I want to know if I get a segfault when I have an infinite recursion in an algorithm (due to incorrect parameters slipping through data validation).

I am also trying to investigate tail recursion optimization and getting a segfault is a way of detecting that it isn't happening.

(Sorry for the delay in replying I lost the email in my client.)

On Tuesday, December 11, 2012 17:16:58 Russel Winder wrote:
> I am not relying on segfaults, that would just be silly ;-) The issue is that unit tests should test error as well as success. I want to know if I get a segfault when I have an infinite recursion in an algorithm (due to incorrect parameters slipping through data validation).
> 
> I am also trying to investigate tail recursion optimization and getting a segfault is a way of detecting that it isn't happening.

I'd argue that if you want an error condition that you test for, you should actually do something in your code that generates an error condition (e.g. throw an exception) and that it really makes no sense to test for segfaults. That's not really an error condition. That just means that your code is broken. And such cases aren't generally worth testing for IMHO.

You test that a function works correctly with various inputs, but if it's given something that it's not supposed to accept, and you're not going to throw or assert that it's acceptable input, then it doesn't really matter what happens. Either you're doing defensive programming and checking inputs (in which case, your unit tests will catch error cases that don't fail properly when the exception isn't thrown), or you're doing design by contract, in which case, it doesn't matter what happens when the caller passes invalid input, because they've violated the contract. The unit tests on the caller should then catch that as opposed to the unit tests on the function being called. Testing that the function doesn't do weird things when the contract is broken is a waste of time. If you want to avoid weird stuff happening when contracts are broken, then use defensive programming and protect the function against such bad input.

- Jonathan M Davis

On Tue, 2012-12-11 at 18:58 +0100, Jonathan M Davis wrote: […]
> I'd argue that if you want an error condition that you test for, you should actually do something in your code that generates an error condition (e.g. throw an exception) and that it really makes no sense to test for segfaults. That's not really an error condition. That just means that your code is broken. And such cases aren't generally worth testing for IMHO.

I appreciate where you are coming from with respect to general testing strategy, and am basically in agreement with what you say.  However in this specific case a segfault is just another exception.  In terms of recursion the exception is "recursion stack limit reached" it just happens to be spelt "segfault"

[…]

-- 
Russel. ============================================================================= Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder@ekiga.net 41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel@winder.org.uk London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder

On 2012-12-11 20:21, Russel Winder wrote:

> I appreciate where you are coming from with respect to general testing
> strategy, and am basically in agreement with what you say.  However in
> this specific case a segfault is just another exception.  In terms of
> recursion the exception is "recursion stack limit reached" it just
> happens to be spelt "segfault"

If you actually want to rely on segfaults there are code out there that can catch a segfault and convert it in to an exception. Actully, this got recently merged into druntime: https://github.com/D-Programming-Language/druntime/commit/ae3ef074f4cad03cc993737dcae17d2e1e16779f

-- 
/Jacob Carlborg