On Monday, March 12, 2012 11:04:54 H. S. Teoh wrote:
> Tangential note: writing unit tests may be tedious, but D's inline unittest syntax has alleviated a large part of that tedium. So much so that I find myself writing as much code in unittests as real code. Which is a good thing, because in the past I'd always been too lazy to write any unittests at all.

D doesn't make writing unit tests easy, since there's an intrinsic amount of effort required to write them, just like there is with any code, but it takes away all of the extraneous effort in having to set up a unit test framework and the like. And by removing pretty much anything from the effort which is not actually required, it makes writing unit testing about as easy as it can be.

I believe that Walter likes to say that it takes away your excuse _not_ to write them because of how easy it is to write unit tests in D.

- Jonathan M Davis

On Mon, Mar 12, 2012 at 02:10:23PM -0400, Jonathan M Davis wrote:
> On Monday, March 12, 2012 11:04:54 H. S. Teoh wrote:
> > Tangential note: writing unit tests may be tedious, but D's inline unittest syntax has alleviated a large part of that tedium. So much so that I find myself writing as much code in unittests as real code. Which is a good thing, because in the past I'd always been too lazy to write any unittests at all.
> 
> D doesn't make writing unit tests easy, since there's an intrinsic amount of effort required to write them, just like there is with any code, but it takes away all of the extraneous effort in having to set up a unit test framework and the like. And by removing pretty much anything from the effort which is not actually required, it makes writing unit testing about as easy as it can be.

I would argue that D *does* make unit tests easier to write, in that you can write them in straight D code inline (as opposed to some testing frameworks that require external stuff like Expect, Python, intermixed with native code), so you don't need to put what you're writing on hold while you go off and write unittests. You can just insert a unittest block after the function/class/etc immediately while the code is still fresh in your mind. I often find myself writing unittests simultaneously with real code, since while writing the code I see a possible boundary condition to test for, and immediately put that in a unittest to ensure I don't forget about it later.  This improves the quality of both the code and the unittests.


> I believe that Walter likes to say that it takes away your excuse _not_ to write them because of how easy it is to write unit tests in D.
[...]

Yep.  They're so easy to write in D that I'd be embarrassed to *not* write them.


T

-- 
Famous last words: I *think* this will work...

On 12-03-2012 19:04, H. S. Teoh wrote:
> On Mon, Mar 12, 2012 at 01:55:33PM -0400, Jonathan M Davis wrote:
> [...]
>> So, no, I don't think that @ctfe would really work. And while I agree
>> that the situation isn't exactly ideal, I don't really see a way
>> around it. Unit tests _do_ catch it for you though. The only thing
>> that they can't catch is whether the template is going to be pure,
>> nothrow, @safe, and/or CTFEable with _your_ arguments to it, but as
>> long as it's pure, nothrow, @safe, and/or CTFEable with _a_ set of
>> arguments, it will generally be the fault of the arguments when such a
>> function fails to be pure, nothrow, @safe, and/or CTFEable as
>> expected. If the unit tests don't hit all of the possible static
>> if-else blocks and all of the possible code paths for CTFE, it could
>> still be a problem, but that just means that the unit tests aren't
>> thorough enough, and more thorough unit tests will fix the problem, as
>> tedious as it may be to do that.
> [...]
>
> Tangential note: writing unit tests may be tedious, but D's inline
> unittest syntax has alleviated a large part of that tedium. So much so
> that I find myself writing as much code in unittests as real code.
> Which is a good thing, because in the past I'd always been too lazy to
> write any unittests at all.
>
>
> T
>

I stopped writing inline unit tests in larger code bases. If I do that, I have to maintain a separate build configuration just for test execution, which is not practical. Furthermore, I want to test my code in debug and release mode, which... goes against having a test configuration.

So, I've ended up moving all unit tests to a separate executable that links in all my libraries and runs their tests in debug/release mode. Works much better.

I don't feel that unittest in D was really thought through properly for large projects targeting actual end users...

-- 
- Alex

On 12-03-2012 18:55, Jonathan M Davis wrote:
> On Monday, March 12, 2012 18:44:06 Alex Rønne Petersen wrote:
>>> Now, that _does_ introduce the possibility of a template being to be pure
>>> and then not being able to be pure thanks to a change that's made to it
>>> or something that it uses, and that makes impossible for any code using
>>> it to be pure. CTFE has the same problem. It's fairly easy to have a
>>> function which is CTFEable cease to be CTFEable thanks to a change to it,
>>> and no one notices. We've had issues with this in the past.
>>
>> That could be solved with a @ctfe attribute or something, no? Like, if
>> the function has @ctfe, go through all possible CTFE paths (excluding
>> !__ctfe paths of course) and make sure they are CTFEable.
>
> 1. That goes completely against how CTFE was designed in that part of the idea
> was that you _wouldn't_ have to annotate it.

Though, rarely, functions written with runtime execution in mind actually Just Work in CTFE. You usually have to change code or special-case things for it to work.

In my experience, anyway...

>
> 2. I don't really know how feasible that would be. At minimum, the fact that
> CTFE works with classes now would probably render it completely infeasible for
> classes, since they're polymorphic, and the compiler can't possibly know all
> of the possible types that could be passed to the function. Templates would
> screw it over too for the exact same reasons that they can have issues with
> pure, @safe, and nothrow. It may or may not be feasible without classes or
> templates being involved.

I hadn't thought of classes at all. In practice, it's impossible then.

>
> So, no, I don't think that @ctfe would really work. And while I agree that the
> situation isn't exactly ideal, I don't really see a way around it. Unit tests
> _do_ catch it for you though. The only thing that they can't catch is whether
> the template is going to be pure, nothrow, @safe, and/or CTFEable with _your_
> arguments to it, but as long as it's pure, nothrow, @safe, and/or CTFEable
> with _a_ set of arguments, it will generally be the fault of the arguments
> when such a function fails to be pure, nothrow, @safe, and/or CTFEable as
> expected. If the unit tests don't hit all of the possible static if-else
> blocks and all of the possible code paths for CTFE, it could still be a
> problem, but that just means that the unit tests aren't thorough enough, and
> more thorough unit tests will fix the problem, as tedious as it may be to do
> that.
>
> - Jonathan M Davis


-- 
- Alex

On Mon, Mar 12, 2012 at 07:41:39PM +0100, Alex Rønne Petersen wrote:
> On 12-03-2012 19:04, H. S. Teoh wrote:
[...]
> >Tangential note: writing unit tests may be tedious, but D's inline unittest syntax has alleviated a large part of that tedium. So much so that I find myself writing as much code in unittests as real code. Which is a good thing, because in the past I'd always been too lazy to write any unittests at all.
[...]
> I stopped writing inline unit tests in larger code bases. If I do that, I have to maintain a separate build configuration just for test execution, which is not practical. Furthermore, I want to test my code in debug and release mode, which... goes against having a test configuration.
[...]

Hmm. Sounds like what you want is not really unittests, but global program startup self-checks. In my mind, unittests is for running specific checks against specific functions, classes/structs inside a module. I frequently write lots of unittests that instantiates all sorts of templates never used by the real program, contrived data objects, etc., that may potentially have long running times, or creates files in the working directory or other stuff like that.  IOW, stuff that are not suitable to be used for release builds at all. It's really more of a way of forcing the program to refuse to start during development when a code change breaks the system, so that the developer notices the breakage immediately. Definitely not for the end-user.

If I wanted release-build self-consistency checking, then yeah, I'd use a different framework than unittests.

As for build configuration, I've given up on make a decade ago for something saner, which can handle complicated build options properly. But that belongs to another topic.


T

-- 
Error: Keyboard not attached. Press F1 to continue. -- Yoon Ha Lee, CONLANG

On 3/12/2012 11:04 AM, H. S. Teoh wrote:
> Tangential note: writing unit tests may be tedious, but D's inline
> unittest syntax has alleviated a large part of that tedium. So much so
> that I find myself writing as much code in unittests as real code.
> Which is a good thing, because in the past I'd always been too lazy to
> write any unittests at all.


That's exactly how it was intended! It seems like such a small feature, really just a syntactic convenience, but what a difference it makes.

On 12-03-2012 20:08, H. S. Teoh wrote:
> On Mon, Mar 12, 2012 at 07:41:39PM +0100, Alex Rønne Petersen wrote:
>> On 12-03-2012 19:04, H. S. Teoh wrote:
> [...]
>>> Tangential note: writing unit tests may be tedious, but D's inline
>>> unittest syntax has alleviated a large part of that tedium. So much
>>> so that I find myself writing as much code in unittests as real code.
>>> Which is a good thing, because in the past I'd always been too lazy
>>> to write any unittests at all.
> [...]
>> I stopped writing inline unit tests in larger code bases. If I do
>> that, I have to maintain a separate build configuration just for test
>> execution, which is not practical. Furthermore, I want to test my code
>> in debug and release mode, which... goes against having a test
>> configuration.
> [...]
>
> Hmm. Sounds like what you want is not really unittests, but global
> program startup self-checks. In my mind, unittests is for running
> specific checks against specific functions, classes/structs inside a

That's what I do. I simply moved my unittest blocks to a separate executable.

> module. I frequently write lots of unittests that instantiates all sorts
> of templates never used by the real program, contrived data objects,
> etc., that may potentially have long running times, or creates files in
> the working directory or other stuff like that.  IOW, stuff that are not

You never know if some code that seems to work fine in debug mode breaks in release mode then (until your user runs into a bug). This is why I want full coverage in all configurations.

> suitable to be used for release builds at all. It's really more of a way
> of forcing the program to refuse to start during development when a code
> change breaks the system, so that the developer notices the breakage
> immediately. Definitely not for the end-user.

Right. That's why my tests are in a separate executable from the actual program.

>
> If I wanted release-build self-consistency checking, then yeah, I'd use
> a different framework than unittests.

IMHO unittest works fine for both debug and release, just not inline.

>
> As for build configuration, I've given up on make a decade ago for
> something saner, which can handle complicated build options properly.
> But that belongs to another topic.

I used to use Make for this project, then switched to Waf. It's an amazing build tool.

>
>
> T
>


-- 
- Alex

On 2012-03-12 19:41, Alex Rønne Petersen wrote:
> I stopped writing inline unit tests in larger code bases. If I do that,
> I have to maintain a separate build configuration just for test
> execution, which is not practical. Furthermore, I want to test my code
> in debug and release mode, which... goes against having a test
> configuration.

I don't inline my unit test either.

> So, I've ended up moving all unit tests to a separate executable that
> links in all my libraries and runs their tests in debug/release mode.
> Works much better.
>
> I don't feel that unittest in D was really thought through properly for
> large projects targeting actual end users...

I agree. I've also started to do more high level testing of some of my command line tools using Cucumber and Aruba. But these test are written in Ruby because of Cucumber and Aruba.

http://cukes.info/
https://github.com/cucumber/aruba

-- 
/Jacob Carlborg

> That could be solved with a @ctfe attribute or something, no? Like, if the function has @ctfe, go through all possible CTFE paths (excluding !__ctfe paths of course) and make sure they are CTFEable.
>
Everything that's pure should be CTFEable which doesn't imply that you
can turn every CTFEable function into a pure one.

On Monday, March 12, 2012 21:36:21 Martin Nowak wrote:
> > That could be solved with a @ctfe attribute or something, no? Like, if the function has @ctfe, go through all possible CTFE paths (excluding !__ctfe paths of course) and make sure they are CTFEable.
> 
> Everything that's pure should be CTFEable which doesn't imply that you can turn every CTFEable function into a pure one.

I don't think that that's quite true. pure doesn't imply @safe, so you could do pointer arithmetic and stuff and the like - which I'm pretty sure CTFE won't allow. And, of course, if you mark a C function as pure or subvert pure through casts, then pure _definitely_ doesn't imply CTFEability.

- Jonathan M Davis