On 2015-04-01 21:25, Dicebot wrote:

> I 100% disagree. Having built-in unittest blocks have been a huge win
> for the language and greatly improved quality of library ecosystem.
> Value of standardization and availability is tremendous here.
>
> Only problem is that development of the feature has stopped half way and
> there are still small bits missing here and there. All your requested
> features can be implemented within existing unittest feature via custom
> runner - while still running tests properly with default one!

The the unittest block itself could have been easily implemented as a library solution, if we had the following features:

* Trailing delegate syntax
* Executable code at module scope

module bar;

unittest("foo bar")
{
}

Would be lowered to:

unittest("foo bar", {

});

These two features are useful for other things, like benchmarking:

benchmark
{
}

-- 
/Jacob Carlborg

On 2015-04-01 21:31, Dicebot wrote:
> P.S. I hate all the Ruby testing facilities, hate with bloody passion.

The unit test framework in the Ruby standard library:

class FooTest
  def test_foo_bar
    assert 3 == 3
  end
end

Looks likes most other testing frameworks out there to me.

-- 
/Jacob Carlborg

On Thu, 02 Apr 2015 08:37:26 +0200, Jacob Carlborg wrote:

> On 2015-04-01 21:25, Dicebot wrote:
> 
>> I 100% disagree. Having built-in unittest blocks have been a huge win for the language and greatly improved quality of library ecosystem. Value of standardization and availability is tremendous here.
>>
>> Only problem is that development of the feature has stopped half way and there are still small bits missing here and there. All your requested features can be implemented within existing unittest feature via custom runner - while still running tests properly with default one!
> 
> The the unittest block itself could have been easily implemented as a library solution, if we had the following features:
> 
> * Trailing delegate syntax * Executable code at module scope
> 
> module bar;
> 
> unittest("foo bar")
> {
> }
> 
> Would be lowered to:
> 
> unittest("foo bar", {
> 
> });
> 
> These two features are useful for other things, like benchmarking:
> 
> benchmark {
> }

executable code at module scope isn't really necessary, i believe. it's just a little messier with mixin templates, but i believe that it's acceptable:

mixin template foo(alias fn) {
  static this () {
    import iv.writer;
    writeln("hello! i'm a test!");
    fn();
    writeln("test passed");
  }
}


// and do it
mixin foo!({assert(42);});

// or with trailing delegate sugar:
mixin foo!{
  assert(42);
};


sure, `static this` can do anything we want; registering unittest delegate in some framework, for example.

On Thu, 02 Apr 2015 06:52:14 +0000, ketmar wrote:

p.s. ah, sure, it should be `shared static this()`. %$Y^%@%.

On Wednesday, 1 April 2015 at 18:04:31 UTC, Ary Borenszweig wrote:
> However, when you have a big complex project you start having
> other needs:
> 1. Named unit-tests, so you can better find what failed
> 2. Better error messages for assertions
> 3. Better output to rerun failed tests
> 4. Setup and teardown hooks
> 5. Different outputs depending on use case

There are test frameworks for D: http://wiki.dlang.org/Libraries_and_Frameworks#Unit_Testing_Framework

> I'd say, deprecate "unittest" and write a good test library.

Test frameworks don't conflict with unittest blocks at all. They have different use cases.

On 2015-04-02 08:52, ketmar wrote:

> executable code at module scope isn't really necessary, i believe.

I think so to have the same syntax.

-- 
/Jacob Carlborg

On Wed, Apr 1, 2015 at 7:31 AM, Johannes Totz via Digitalmars-d < digitalmars-d@puremagic.com> wrote:

> On 31/03/2015 19:24, Andrei Alexandrescu wrote:
> > Probably not; we're looking at two different builds. The build to be deployed has no unittest code at all.
>
> I'm starting to see this differently these days (basically since I
> started to use jenkins for everything):
> A build you haven't unit tested has implicitly failed. That means the
> release build that does not have any unit test bits is not deployable.
> Instead, compile as usual (both debug and release), and run unit tests
> against both (e.g. to catch compiler bugs in the optimiser).
> Then for deployment, drop/strip/remove/dont-package the unit test code.
>
>
This.

I want to run unit tests as part of the build process, and I want my release build to have unit tests run against it.  If unit tests haven't passed for a build, it's not release ready.  But, I don't want my release build to be bloated with unit test code.

Related, unit tests often have dependencies that I _don't_ want as part of my release build.  Mocking frameworks are a good example.

On Thu, Apr 02, 2015 at 11:19:32AM -0700, Jeremy Powers via Digitalmars-d wrote:
> On Wed, Apr 1, 2015 at 7:31 AM, Johannes Totz via Digitalmars-d < digitalmars-d@puremagic.com> wrote:
> 
> > On 31/03/2015 19:24, Andrei Alexandrescu wrote:
> > > Probably not; we're looking at two different builds. The build to be deployed has no unittest code at all.
> >
> > I'm starting to see this differently these days (basically since I started to use jenkins for everything): A build you haven't unit tested has implicitly failed. That means the release build that does not have any unit test bits is not deployable.  Instead, compile as usual (both debug and release), and run unit tests against both (e.g. to catch compiler bugs in the optimiser).  Then for deployment, drop/strip/remove/dont-package the unit test code.
> >
> >
> This.
> 
> I want to run unit tests as part of the build process, and I want my release build to have unit tests run against it.  If unit tests haven't passed for a build, it's not release ready.  But, I don't want my release build to be bloated with unit test code.
> 
> Related, unit tests often have dependencies that I _don't_ want as part of my release build.  Mocking frameworks are a good example.

So what do you want the compiler to do? Emit two executables, one containing the release, the other containing the unittests? Isn't that just a matter of running dmd with/without -unittest?


T

-- 
You have to expect the unexpected. -- RL

On Thursday, 2 April 2015 at 06:33:50 UTC, Jacob Carlborg wrote:
> On 2015-04-01 21:18, Kapps wrote:
>
>> The only issue I have with the way unittests are done right now, is the
>> incredibly annoying requirement of having a main function and that main
>> gets called. It makes generic tooling and CI systems much more annoying,
>> as you have to try and guess whether you need to create a fake main()
>> (or pass in -main), and worry about if the code is going to keep running
>> after tests complete.
>
> I just don't compile the module congaing the main function. Although I place the my tests in a completely separate directory.

Which is okay for running your own code, but the problem is it's not really a good solution for a CI system. I made a plugin for Bamboo to automatically test my dub projects, and essentially just rely on the fact that 'dub test' works. Without that, I would have no way of knowing whether or not there's a main function, whether or not that main function actually does something, etc. Being able to prevent main from being required and running is, for me, the biggest issue with the current unittest system.

On 4/2/15 11:44 AM, H. S. Teoh via Digitalmars-d wrote:
> On Thu, Apr 02, 2015 at 11:19:32AM -0700, Jeremy Powers via Digitalmars-d wrote:
>> On Wed, Apr 1, 2015 at 7:31 AM, Johannes Totz via Digitalmars-d <
>> digitalmars-d@puremagic.com> wrote:
>>
>>> On 31/03/2015 19:24, Andrei Alexandrescu wrote:
>>>> Probably not; we're looking at two different builds. The build to
>>>> be deployed has no unittest code at all.
>>>
>>> I'm starting to see this differently these days (basically since I
>>> started to use jenkins for everything): A build you haven't unit
>>> tested has implicitly failed. That means the release build that does
>>> not have any unit test bits is not deployable.  Instead, compile as
>>> usual (both debug and release), and run unit tests against both
>>> (e.g. to catch compiler bugs in the optimiser).  Then for
>>> deployment, drop/strip/remove/dont-package the unit test code.
>>>
>>>
>> This.
>>
>> I want to run unit tests as part of the build process, and I want my
>> release build to have unit tests run against it.  If unit tests
>> haven't passed for a build, it's not release ready.  But, I don't want
>> my release build to be bloated with unit test code.
>>
>> Related, unit tests often have dependencies that I _don't_ want as
>> part of my release build.  Mocking frameworks are a good example.
>
> So what do you want the compiler to do? Emit two executables, one
> containing the release, the other containing the unittests? Isn't that
> just a matter of running dmd with/without -unittest?

The way I see it, the notion of having one build with strippable unittests is a nice idea but technically challenging. It's also low impact - today concurrent CPU is cheap so running two concurrent unrelated builds can be made as fast as one.

The simple effective step toward improvement is to uniformize the format of assertion errors in unittests and to make it easy with tooling to create unittest and non-unittest builds that are gated by the unittests succeeding.


Andrei