On Jul 6, 2010, at 9:05 AM, Michel Fortin wrote:

> Le 2010-07-06 ? 11:28, Sean Kelly a ?crit :
> 
>> On Jul 5, 2010, at 11:35 PM, Walter Bright wrote:
>>> 
>>> Sean Kelly wrote:
>>>> That's probably my fault.
>>> 
>>> Every team needs one of those people. Thanks for filling that role.
>> 
>> I like to maintain a steady level of failure to keep things positive ;-)
>> 
>> How about this... the unittest handler only returns an error code and if it's nonzero then the app won't run.
> 
> This is what I was expecting the change would do originally. Running the program should be conditional to the unit tests being successful, otherwise it's too easy to ignore them.
> 
> Another way to put it is: do you want your failed unit tests to behave as warnings or as errors? I choose errors.

Yeah, the only thing I've been wondering about is whether there's a case where the unittests would want to return 0 but have the app still not run.  I thought maybe with a custom build meant to run unit tests, but in that case I'd have an empty main() that simply returned 0 and would specify that main should run.  Still trying to decide whether there are any justifiable cases that the new behavior wouldn't cover.

Would it be worth considering allowing the user to give a function like OnUnitTest0() that gets called in the case of a 0 return?  If such a function is not specified, default behavior is to stop.  Or perhaps setting a global to the AND of all ut results and letting them check the result?


----- Original Message ----
> From: Sean Kelly <sean at invisibleduck.org>
> To: Discuss the phobos library for D <phobos at puremagic.com>
> Sent: Tue, July 6, 2010 9:12:57 AM
> Subject: Re: [phobos] Unittest errors on windows with latest phobos sources
> 
> On Jul 6, 2010, at 9:05 AM, Michel Fortin wrote:

> Le 2010-07-06 ? 11:28, Sean Kelly a ?crit :
> 
>> On Jul 5, 2010, at 11:35 PM,
> Walter Bright wrote:
>>> 
>>> Sean Kelly
> wrote:
>>>> That's probably my fault.
>>> 
> 
>>> Every team needs one of those people. Thanks for filling that
> role.
>> 
>> I like to maintain a steady level of failure to
> keep things positive ;-)
>> 
>> How about this... the unittest
> handler only returns an error code and if it's nonzero then the app won't run.
> 
> This is what I was expecting the change would do originally. Running the program should be conditional to the unit tests being successful, otherwise it's too easy to ignore them.
> 
> Another way to put it is: do you want your failed unit tests to behave as warnings or as errors? I choose errors.

Yeah, the only thing I've been wondering about
> is whether there's a case where the unittests would want to return 0 but have the app still not run.  I thought maybe with a custom build meant to run unit tests, but in that case I'd have an empty main() that simply returned 0 and would specify that main should run.  Still trying to decide whether there are any justifiable cases that the new behavior wouldn't cover.
_______________________________________________
phobos mailing
> list

> href="mailto:phobos at puremagic.com">phobos at puremagic.com
http://lists.puremagic.com/mailman/listinfo/phobos

Sean Kelly wrote:
>
>
> How about this... the unittest handler only returns an error code and if it's nonzero then the app won't run.  That's consistent with the pre-assert change behavior.  The difference being that even if the unittests are successful, to prevent the app from running, a non-zero error should be returned.  It's been so long since I originally implemented this (it was done for Gregor when I was working on Tango) that I don't recall the exact use case he presented, but it seems reasonable that if you run an app and it runs some tests but doesn't actually execute then the return value should be nonzero, even if all the tests succeed.  What do you think?
>
> 

I don't see the point in doing anything more complicated than:

   run unit tests
    if (any unit tests failed)
           exit (1)

On Jul 6, 2010, at 10:31 AM, Walter Bright wrote:
> 
> Sean Kelly wrote:
>> 
>> 
>> How about this... the unittest handler only returns an error code and if it's nonzero then the app won't run.  That's consistent with the pre-assert change behavior.  The difference being that even if the unittests are successful, to prevent the app from running, a non-zero error should be returned.  It's been so long since I originally implemented this (it was done for Gregor when I was working on Tango) that I don't recall the exact use case he presented, but it seems reasonable that if you run an app and it runs some tests but doesn't actually execute then the return value should be nonzero, even if all the tests succeed.  What do you think?
>> 
>> 
> 
> I don't see the point in doing anything more complicated than:
> 
>  run unit tests
>   if (any unit tests failed)
>          exit (1)

I think the original idea was to allow standalone unit testing to occur using the application binary.  I'm not entirely sure what's to be gained over this approach versus having a separate build with -unittest turned on and a different main() however, since I can't see it being desirable to run unit tests before every execution of an app (except possibly when the unit tests are used to determine whether the environment is configured correctly rather than detailed white-box testing).  In short, I'm going to change it to work pretty much exactly how you want.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.puremagic.com/pipermail/phobos/attachments/20100706/72f66a48/attachment.html>

version(unittest)
{
   void main() {}
}
else
{
   int main(string[] args)
   {
      ...
   }
}

FWIW, I never want to run my application when running unit tests.

-Steve

----- Original Message ----

> From: Sean Kelly <sean at invisibleduck.org>
> To: Discuss the phobos library for D <phobos at puremagic.com>
> Sent: Tue, July 6, 2010 12:12:57 PM
> Subject: Re: [phobos] Unittest errors on windows with latest phobos sources
> 
> On Jul 6, 2010, at 9:05 AM, Michel Fortin wrote:

> Le 2010-07-06 ? 11:28, Sean Kelly a ?crit :
> 
>> On Jul 5, 2010, at 11:35 PM,
> Walter Bright wrote:
>>> 
>>> Sean Kelly
> wrote:
>>>> That's probably my fault.
>>> 
> 
>>> Every team needs one of those people. Thanks for filling that
> role.
>> 
>> I like to maintain a steady level of failure to
> keep things positive ;-)
>> 
>> How about this... the unittest
> handler only returns an error code and if it's nonzero then the app won't run.
> 
> This is what I was expecting the change would do originally. Running the program should be conditional to the unit tests being successful, otherwise it's too easy to ignore them.
> 
> Another way to put it is: do you want your failed unit tests to behave as warnings or as errors? I choose errors.

Yeah, the only thing I've been wondering about
> is whether there's a case where the unittests would want to return 0 but have the app still not run.  I thought maybe with a custom build meant to run unit tests, but in that case I'd have an empty main() that simply returned 0 and would specify that main should run.  Still trying to decide whether there are any justifiable cases that the new behavior wouldn't cover.
_______________________________________________
phobos mailing
> list

> href="mailto:phobos at puremagic.com">phobos at puremagic.com
http://lists.puremagic.com/mailman/listinfo/phobos

FWIW rdmd has a --main flag that adds a dummy main for just unittesting.

Andrei

Sean Kelly wrote:
> On Jul 6, 2010, at 9:05 AM, Michel Fortin wrote:
> 
>> Le 2010-07-06 ? 11:28, Sean Kelly a ?crit :
>>
>>> On Jul 5, 2010, at 11:35 PM, Walter Bright wrote:
>>>> Sean Kelly wrote:
>>>>> That's probably my fault.
>>>> Every team needs one of those people. Thanks for filling that role.
>>> I like to maintain a steady level of failure to keep things positive ;-)
>>>
>>> How about this... the unittest handler only returns an error code and if it's nonzero then the app won't run.
>> This is what I was expecting the change would do originally. Running the program should be conditional to the unit tests being successful, otherwise it's too easy to ignore them.
>>
>> Another way to put it is: do you want your failed unit tests to behave as warnings or as errors? I choose errors.
> 
> Yeah, the only thing I've been wondering about is whether there's a case where the unittests would want to return 0 but have the app still not run.  I thought maybe with a custom build meant to run unit tests, but in that case I'd have an empty main() that simply returned 0 and would specify that main should run.  Still trying to decide whether there are any justifiable cases that the new behavior wouldn't cover.
> _______________________________________________
> phobos mailing list
> phobos at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/phobos

Steve Schveighoffer <schveiguy at yahoo.com> wrote:

> FWIW, I never want to run my application when running unit tests.

Yeah, me neither. Always thought that was a sorta weird feature.

-- 
Simen

Sean Kelly wrote:
>
>  In short, I'm going to change it to work pretty much exactly how you
> want.


Thanks.