On 2014-04-30 23:35, Andrei Alexandrescu wrote:

> Agreed. I think we should look into parallelizing all unittests. -- Andrei

I recommend running the tests in random order as well.

-- 
/Jacob Carlborg

On 2014-04-30 21:12, bearophile wrote:

> Are UDAs enough?
>
> @uname("foo") unittest {}
>
> What I'd like is to tie one or more unittests to other entities, like
> all the unittests of a specific function.

Something similar is done in RSpec. A BDD framework in Ruby. In D it might look like this:

@describe("foo")
{
    @it("does something useful") unittest
    {
    }

    @it("also does some other stuff") unittest
    {
    }
}

-- 
/Jacob Carlborg

On 2014-05-01 09:10, Dicebot wrote:

> It is not strictly necessary but you can't reliably get all unit test
> blocks during compile-time (must be transitively imported) and current
> run-time reflection for tests is missing any data but actual function
> pointers. I am personally perfectly satisfied with "single root module
> imports all" approach but it is likely to be complained if proposed as
> "standard" way.

The current runner wouldn't work. It needs to be replace with one that uses __traits(getUnitTests).

-- 
/Jacob Carlborg

On 2014-05-01 11:37, Dicebot wrote:

> You only need to make sure all modules are transitively imported from
> initial one.

The solution for that would be RMInfo [1], like RTInfo but for modules instead of types.

[1] https://github.com/D-Programming-Language/dmd/pull/2271

-- 
/Jacob Carlborg

Am Thu, 01 May 2014 09:38:51 +0000
schrieb "Dicebot" <public@dicebot.lv>:

> On Thursday, 1 May 2014 at 08:51:47 UTC, Johannes Pfau wrote:
> 
> You only need to make sure all modules are transitively imported from initial one. Everything else can be done via recursive reflection with __traits(allMembers), module constructors are not needed either.
> 
> One can also generate special test entry module that imports all project modules using build system help.

Is there some boost licensed reflection code somewhere which shows how to do recursive reflection correctly? I know the basic idea but stuff like dealing with protection of members would take some time to figure that out.

Maybe someone familiar with the details could sketch up a quick std.test proof-of-concept which parses the following (with all necessary protection checks + detecting unittest blocks in classes/structs):
-----------
unittest
{

}

@_unittest void namedTest()
{

}
----------

@Andrei do you think having to explicitly import modules to be tested is an issue? Apart from this limitation such an approach sounds great to me. There are quite some possibilities how user frameworks could extend std.test so it sounds like an interesting idea. (And std.benchmark could work in the same way)

Am Wed, 30 Apr 2014 09:02:54 -0700
schrieb Andrei Alexandrescu <SeeWebsiteForEmail@erdani.org>:

> > To summarize: It provides a function pointer for every  unit test to druntime or user code. This is actually easy to do. Naming tests requires changes in the parser, but I guess that shouldn't be difficult either.
> 
> That's fantastic, would you be willing to reconsider that work?

Are you still interested in this?

I guess we could build a std.test phobos module to completely replace the current unittest implementation, see: http://forum.dlang.org/post/ljtbch$lg6$1@digitalmars.com

This seems to be a better solution, especially considering extension possibilities.

On 2014-04-30 22:41, Andrei Alexandrescu wrote:

> Yah I think that's possible but I'd like the name to be part of the
> function name as well e.g. unittest__%s.

Why is that necessary? To have the correct symbol name when debugging?

>> I'm using something quite similar to RSpec from the Ruby world:
>>
>> describe! "toMsec" in {
>>      it! "returns the time in milliseconds" in {
>>          assert(true);
>>      }
>> }
>>
>> This uses the old syntax, with UDA's it becomes something like this:
>>
>> @describe("toMsec")
>> {
>>      @it("returns the time in milliseconds") unittest
>>      {
>>          assert(true);
>>      }
>> }
>
> That looks... interesting.

The Ruby syntax looks like this:

describe "toMsec" do
  it "reruns the time in milliseconds" do
    assert true
  end

  context "when the time parameter is nil" do
    it "returns nil" do
      assert true
    end
  end
end

The interesting part about the Ruby implementation is that each it-block (the code between do/end) is executed in the context of an anonymous class instance. Each describe/context-block is turned in to a class, nested blocks inherit from the outer block. In D the implementation would look like this:

class __toMsec
{
    void __someUniqueName123 ()
    {
    }

    class __SomeUniqueName456 : __toMsec
    {
        void __someUniqueName789 ()
        {
        }
    }
}

Each it-block (unit test) will be executed in a new instance of the closest surrounding class. This means you can have helper methods and instance variables shared across multiple test, but they will each get a fresh copy of the data.

Since the describe-blocks are implemented with classes that inherit you can override helper methods in subclasses.

The unit test runner can also print out a documentation, basically all text in the "it" and "describe" parameters. Something like this: https://coderwall-assets-0.s3.amazonaws.com/uploads/picture/file/1949/rspec_html_screen.png

-- 
/Jacob Carlborg

On Thursday, 1 May 2014 at 11:05:55 UTC, Jacob Carlborg wrote:
> On 2014-04-30 23:35, Andrei Alexandrescu wrote:
>
>> Agreed. I think we should look into parallelizing all unittests. -- Andrei
>
> I recommend running the tests in random order as well.

This is a bad idea. Tests could fail only some of the time. Even if bugs are missed, I would prefer it if tests did exactly the same thing every time.

Am Thu, 01 May 2014 13:24:07 +0200
schrieb Jacob Carlborg <doob@me.com>:

> On 2014-05-01 11:37, Dicebot wrote:
> 
> > You only need to make sure all modules are transitively imported from initial one.
> 
> The solution for that would be RMInfo [1], like RTInfo but for modules instead of types.
> 
> [1] https://github.com/D-Programming-Language/dmd/pull/2271
> 

But on a quick look I don't understand how this (or DRT #775) completely
solves the issue.

Now you don't have to import the modules anymore, which is a step forward, but let's say I want to used this to find all SubClasses of a class.

Now I can inspect that code in CTFE, but how do I build a list of all subclasses? In the most generic way (with DLLs/shared libraries) this can only work if code can be executed at runtime, so we'd need a way to emit module constructors AFAICS. These should be able to avoid the usual constructor dependency rules though.

Le 01/05/2014 09:23, Dicebot a écrit :
> On Thursday, 1 May 2014 at 01:45:21 UTC, Xavier Bigand wrote:
>> Splitting all features at an absolute atomic level can be achieve for
>> open-source libraries, but it's pretty much impossible for an
>> industrial software. Why being so restrictive when it's possible to
>> support both vision by extending a little the language by something
>> already logical?
>
> You are pretty much saying here "writing good code is possible for
> open-source libraries but not for industrial software".
It's just a lot harder when you are under pressure.
I am working for a very small company and our dead lines clearly doesn't help us with that, and because I am in the video game industry it's not really critical to have small bugs.

Not every body have the capacity or resources (essentially time) to design his code in the pure conformance of unittests definition, and IMO isn't not an excuse to avoid tests completely.
If a language/standard library can help democratization of tests it's a good thing, so maybe writing tests have to stay relatively simple and straightforward.

My point is just when you are doing things only for you it's often simpler to them like they must be.