September 12, 2015 Re: std.experimental.testing formal review | ||||
---|---|---|---|---|
| ||||
Posted in reply to Jacob Carlborg | On Friday, 11 September 2015 at 15:54:57 UTC, Jacob Carlborg wrote:
> On 2015-09-11 13:27, Atila Neves wrote:
>
>> Unit tests should run in a fraction of a second... no, there's no such
>> functionality.
>
> Why would I not use this for other kinds of tests?
I guess you're right. In any case, getting what's in there to get accepted is work enough as it is.
Atila
|
September 12, 2015 Re: std.experimental.testing formal review | ||||
---|---|---|---|---|
| ||||
Posted in reply to Robert burner Schadek | On Wednesday, 9 September 2015 at 15:20:41 UTC, Robert burner Schadek wrote:
> This post marks the start of the two week review process of std.experimental.testing.
>
> PR: https://github.com/D-Programming-Language/phobos/pull/3207
> Dub: http://code.dlang.org/packages/unit-threaded
> Doc: See CyberShadow/DAutoTest for up-to-date documentation build
>
> Previous Thread: http://forum.dlang.org/post/uzocokshugchescbawlj@forum.dlang.org
My opinion hasn't changed since last review - it is a very solid and useful library but I don't want provided API to become standard (== part of Phobos).
I also don't like mixing unittest and higher level functional tests (with setup and cleanup phases) into the same buckets - this doesn't fit nice with D module system. Latter should be placed in a separate modules/package to avoid being picked up by rdmd & Co when compiled as dependency.
At the same time actual test running facilities (parallel, randomized, etc) look generally applicable and uncontroversial.
|
September 12, 2015 Re: std.experimental.testing formal review | ||||
---|---|---|---|---|
| ||||
Posted in reply to Atila Neves | On 2015-09-12 11:52, Atila Neves wrote: > I guess you're right. In any case, getting what's in there to get > accepted is work enough as it is. Yes, absolutely. I wasn't arguing for the feature, just the reason not to add it :) -- /Jacob Carlborg |
September 12, 2015 Re: std.experimental.testing formal review | ||||
---|---|---|---|---|
| ||||
Posted in reply to Dicebot | On 2015-09-12 15:34, Dicebot wrote: > I also don't like mixing unittest and higher level functional tests > (with setup and cleanup phases) into the same buckets - this doesn't fit > nice with D module system. Latter should be placed in a separate > modules/package to avoid being picked up by rdmd & Co when compiled as > dependency. Not sure I understand the problem. Does this prevent one from writing functional tests in a completely separate directory? -- /Jacob Carlborg |
September 12, 2015 Re: std.experimental.testing formal review | ||||
---|---|---|---|---|
| ||||
Posted in reply to Jacob Carlborg | On Saturday, 12 September 2015 at 14:50:32 UTC, Jacob Carlborg wrote:
> Not sure I understand the problem. Does this prevent one from writing functional tests in a completely separate directory?
Nope but the fact that they are treated the same (no separate category, no output separation, no documentation warning) encourages to do so - while in practice it is almost always a bad idea.
|
September 13, 2015 Re: std.experimental.testing formal review | ||||
---|---|---|---|---|
| ||||
Posted in reply to Jacob Carlborg | On Saturday, 12 September 2015 at 14:50:32 UTC, Jacob Carlborg wrote:
> On 2015-09-12 15:34, Dicebot wrote:
>
>> I also don't like mixing unittest and higher level functional tests
>> (with setup and cleanup phases) into the same buckets - this doesn't fit
>> nice with D module system. Latter should be placed in a separate
>> modules/package to avoid being picked up by rdmd & Co when compiled as
>> dependency.
>
> Not sure I understand the problem. Does this prevent one from writing functional tests in a completely separate directory?
On related topic - there are 2 things that are currently missing in `TestCase` when applied for functional test purpose:
1) being able to mark test case as fatal (i.e. if internal handshake or sanity check fails there is no point in trying to run other tests)
2) being able to do weak ordering of tests (by defining strict sequence of groups so that parallelization/randomization only happens within such group) - I have used something as simple as numerical priority value so far for my needs
|
September 13, 2015 Re: std.experimental.testing formal review | ||||
---|---|---|---|---|
| ||||
Posted in reply to Dicebot | On Sunday, 13 September 2015 at 09:59:18 UTC, Dicebot wrote: > On Saturday, 12 September 2015 at 14:50:32 UTC, Jacob Carlborg wrote: >> On 2015-09-12 15:34, Dicebot wrote: >> >>> I also don't like mixing unittest and higher level functional tests >>> (with setup and cleanup phases) into the same buckets - this doesn't fit >>> nice with D module system. Latter should be placed in a separate >>> modules/package to avoid being picked up by rdmd & Co when compiled as >>> dependency. >> >> Not sure I understand the problem. Does this prevent one from writing functional tests in a completely separate directory? > > On related topic - there are 2 things that are currently missing in `TestCase` when applied for functional test purpose: > > 1) being able to mark test case as fatal (i.e. if internal handshake or sanity check fails there is no point in trying to run other tests) I've never heard of functionality like that, but should be easy to implement. > 2) being able to do weak ordering of tests (by defining strict sequence of groups so that parallelization/randomization only happens within such group) - I have used something as simple as numerical priority value so far for my needs There's `@singleThreaded` for that: all tests in a module with that UDA run in series (other modules are still run in parallel). I didn't think one was needed for random ordering. Atila Atila |
September 13, 2015 Re: std.experimental.testing formal review | ||||
---|---|---|---|---|
| ||||
Posted in reply to Atila Neves | On 09/11/2015 01:27 PM, Atila Neves wrote: >> How about Fuzz-tests, randomize input for test on each run? > > Like QuickCheck? Robert has something for that. There is also https://github.com/MartinNowak/qcheck for that. |
September 13, 2015 Re: std.experimental.testing formal review | ||||
---|---|---|---|---|
| ||||
Posted in reply to Atila Neves | On 2015-09-13 12:44, Atila Neves wrote: > I've never heard of functionality like that, but should be easy to > implement. We're using that at work, but on a different level. We have two separate jobs in Jenkins, one depends on the other one. If the first one fails, the second one is not run. -- /Jacob Carlborg |
September 15, 2015 Re: std.experimental.testing formal review | ||||
---|---|---|---|---|
| ||||
Posted in reply to Dicebot | On Sunday, 13 September 2015 at 09:59:18 UTC, Dicebot wrote: > On Saturday, 12 September 2015 at 14:50:32 UTC, Jacob Carlborg wrote: >> On 2015-09-12 15:34, Dicebot wrote: >> >>> I also don't like mixing unittest and higher level functional tests >>> (with setup and cleanup phases) into the same buckets - this doesn't fit >>> nice with D module system. Latter should be placed in a separate >>> modules/package to avoid being picked up by rdmd & Co when compiled as >>> dependency. >> >> Not sure I understand the problem. Does this prevent one from writing functional tests in a completely separate directory? > > On related topic - there are 2 things that are currently missing in `TestCase` when applied for functional test purpose: > > 1) being able to mark test case as fatal (i.e. if internal handshake or sanity check fails there is no point in trying to run other tests) I'll see if I can add this before the end of the review phase. > 2) being able to do weak ordering of tests (by defining strict sequence of groups so that parallelization/randomization only happens within such group) - I have used something as simple as numerical priority value so far for my needs How about I change the name from @singleThreaded to @serial and make sure it works with randomised runs as well (it probably already does, I just have to check)? Atila |
Copyright © 1999-2021 by the D Language Foundation