On Thursday, July 15, 2010 15:43:47 Andrei Alexandrescu wrote:
> 
> There's one more problem with this, which should be discussed - an assert inside e.g. sqrt or any function that sqrt transitively calls _does_ stop the entire thing. So assert has different semantics depending on locus, which I hope we agree is not a desirable trait. Also, as you mentioned, if someone throws, we're again back to a different semantics.
> 
> I think there's a disadvantage there. FWIW changing the semantics of
> assert that way will translate into a disincentive to use it inside
> unittests ("Hmm, I better use enforce() here because assert() is just
> weird.")

If there are folks (like Walter) who want to report a failure and continue with the assertions without creating separate unit test blocks, then I'd suggest creating a function specifically for that - such as assert_nothrow(). Since the tests are already indicating failure based on a flag, such a function could set that flag so that failure could be indicated, while assert would continue to have its normal semantics, throwing from the block, and indicating failure.

I think that it's a huge mistake to make assert have different semantics in unit tests. And honestly, since every other unit testing framework has functions instead of unit test blocks, and those functions stop execution when an assertion fails (since an exception is thrown), you have to create whole functions for each test that you want separate. That's a lot to worry about than simply adding an extra unittest block - especially if you make the whole thing one line.

If D chooses to have assertions not stop the execution of a particular unit test, block then that will confuse new users - both with regards to the behavior of assertion being different and the fact that the unittest block didn't cease executing after the first failure like occurs in most unit testing frameworks. In fact, if I it hadn't seen discussions about it in the newsgroup, then I would have thought that assertion failing to cease the excecution of the unittest block was an error that needed fixing, and I would have reported it to bugzilla. It totally breaks the normal behavior of assert.

I think that it would be a huge mistake for assert's semantics to change based on context. If we really need a non-throwing assert, we should add a function specifically for that rather than subverting assert.

- Jonathan M Davis

On Jul 15, 2010, at 3:43 PM, Andrei Alexandrescu wrote:
> 
> I think there's a disadvantage there. FWIW changing the semantics of assert that way will translate into a disincentive to use it inside unittests ("Hmm, I better use enforce() here because assert() is just weird.")

Is there a disadvantage in providing a separate routine that reports and doesn't throw?  I know it's another global symbol (assuming it's in object.di), but...

Sean Kelly wrote:
> On Jul 15, 2010, at 3:00 PM, Walter Bright wrote:
> 
>>> NOT YET:
>>>
>>> 4. DESIRED: assert is NOT hacked, any failing assert ends the current unittest, the failure message is printed, execution continues with the next unittest, program ends with error code if at least one assert failed, everybody's happy.
>>>
>>> 
>> Everybody's happy except the guys like me who want to run all the unit tests in one go.
>> 
>
> All the unit tests or all the asserts?

All the asserts.

>   All the unit tests are run with this approach:
>
>     module A;
>
>     unittest {
>         assert(false, "test 1a failed");
>         assert(false, "test 1b failed");
>     }
>
>     unittest {
>         assert(false, "test 2 failed");
>     }
>
> This should print:
>
>     test 1a failed
>     test 2 failed
>
> I'm not sure I understand the problem of breaking unit tests into a collection of atomic blocks.

Extra work.

>   With version(unittest) globals can even be used if state should be retained across tests.
> _______________________________________________
> phobos mailing list
> phobos at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/phobos
>
>
> 

On 07/15/2010 06:11 PM, Walter Bright wrote:
>> I'm not sure I understand the problem of breaking unit tests into a collection of atomic blocks.
>
> Extra work.

Unfortunately, the converse entails extra work for the others.

Andrei

On 07/15/2010 06:00 PM, Sean Kelly wrote:
> On Jul 15, 2010, at 3:43 PM, Andrei Alexandrescu wrote:
>>
>> I think there's a disadvantage there. FWIW changing the semantics
>> of assert that way will translate into a disincentive to use it
>> inside unittests ("Hmm, I better use enforce() here because
>> assert() is just weird.")
>
> Is there a disadvantage in providing a separate routine that reports and doesn't throw?  I know it's another global symbol (assuming it's in object.di), but...

Good question. expect() comes to mind.

Andrei

Hello Walter,

 > Andrei Alexandrescu wrote:
 >
 >> 4. DESIRED: assert is NOT hacked, any failing assert ends the current
 >> unittest, the failure message is printed, execution continues with
 >> the next unittest, program ends with error code if at least one
 >> assert failed, everybody's happy.
 >>
 > Everybody's happy except the guys like me who want to run all the unit
 > tests in one go.
 >

You still can't. If a failed assert ever returns, you can end up with even less results than case 4:

unittest
{
    char* c = some func();
    assert(c !is null);
    assert(c[0] == 'a');  // SEG-V
}

Beside, I don't now nor will I ever care about ANYTHING after the first failed assert in any given block. If code B should run even if an assert in code A fails, I'll put it in another unittest block. If anything BUT option 4 gets implemented I'll start using "if(!test) throw..." rather than assert.

On Thursday, July 15, 2010 16:00:01 Sean Kelly wrote:
> Is there a disadvantage in providing a separate routine that reports and doesn't throw?  I know it's another global symbol (assuming it's in object.di), but...

I can't think of any (though obviously someone else may). And if you name it with a name that starts with assert - like assert_nothrow() - then it's not terribly likely that it would conflict with a symbol name that someone wanted to use in their code anyway. Certainly, it seems like the best solution to me at the moment. It allows Walter (and anyone else like him) to have all of his assertions fail without stopping the unittest they're in, while assert itself functions normally. You get the bost of both worlds. It just means that assertions that aren't supposed to throw have to use a different function. I don't see any real downside to that solution unless Walter just hates the idea of using something other than assert in unittests.

- Jonathan M Davis

On 07/15/2010 06:59 PM, Benjamin Shropshire wrote:
> Beside, I don't now nor will I ever care about ANYTHING after the first failed assert in any given block.

Walter, it would be great if you could substantiate some concrete examples by pasting from your tests. Thanks!

Andrei

On 07/15/2010 05:59 PM, Jonathan M Davis wrote:
> I think that it's a huge mistake to make assert have different semantics in unit tests.

I agree that this is a strong argument against the current behavior.

Andrei

Walter Bright wrote:
> Extra work.

and making sure that a failed assert not stopping a test (e.g. "unisttest {...}") doesn't cause total disaster isn't?