We're having a strong need for named unittests at Facebook for multiple reasons.

1. We have sophisticated tooling that verifies whether unittests are flaky. The automated monitor (for e.g. C++) figures whether a given unittest fails several times across several commits. Unittests are identified by name; relying on file/line is impossible because the line of a failure is not stable across changes.

2. Again for efficient automated testing and flakiness detection, one should be able to run only a subset of unittests by mentioning them by line in the command line. Note that this implies there's no interdependency between distinct unittests, which is fine because the new ability is opt-on; I'd say is pure style anyway.

3. Mentioning unittest names in failure messages helps human communication (e.g. "AddPeer is failing after your change"). This is impossible with file and line numbers.

I'd like to make a DIP for named unittests. Who can help me with that?


Andrei

On Monday, 30 March 2015 at 21:52:35 UTC, Andrei Alexandrescu wrote:
> We're having a strong need for named unittests at Facebook for multiple reasons.
>
> 1. We have sophisticated tooling that verifies whether unittests are flaky. The automated monitor (for e.g. C++) figures whether a given unittest fails several times across several commits. Unittests are identified by name; relying on file/line is impossible because the line of a failure is not stable across changes.
>
> 2. Again for efficient automated testing and flakiness detection, one should be able to run only a subset of unittests by mentioning them by line in the command line. Note that this implies there's no interdependency between distinct unittests, which is fine because the new ability is opt-on; I'd say is pure style anyway.
>
> 3. Mentioning unittest names in failure messages helps human communication (e.g. "AddPeer is failing after your change"). This is impossible with file and line numbers.
>
> I'd like to make a DIP for named unittests. Who can help me with that?
>
>
> Andrei

I've implemented this in a library and I'm sure others have as well. Are you sure, you want a language solution?

On 3/30/15 2:55 PM, Panke wrote:
> I've implemented this in a library and I'm sure others have as well. Are
> you sure, you want a language solution?

With attributes? That might be palatable but only as a standard solution. I'd want to add that to Phobos. -- Andrei

I'd prefer putting alternative test runner into Phobos instead which will support `@name("Something") unittest { }`

>
> I've implemented this in a library and I'm sure others have as well. Are you sure, you want a language solution?

Basic principle:

---
unittest {

    testCase("a testcase",
    {
        // actual test code here
    });
}
---

Testcase registers the structure and the runtime just runs the unittest as is does today and than calls all closures registered during the unittest. Makes this fully backward compatible.

Sounds like something that should be added as a library rather than the core language.

On Monday, 30 March 2015 at 21:57:33 UTC, Andrei Alexandrescu wrote:
> On 3/30/15 2:55 PM, Panke wrote:
>> I've implemented this in a library and I'm sure others have as well. Are
>> you sure, you want a language solution?
>
> With attributes? That might be palatable but only as a standard solution. I'd want to add that to Phobos. -- Andrei

In my case: no. I wrote it before UDA and wanted to be able to generate testcases at runtime. Used this to register a test case for a SAT solver for every instance in a subdirectory etc.

On 3/30/15 2:59 PM, Panke wrote:
>>
>> I've implemented this in a library and I'm sure others have as well.
>> Are you sure, you want a language solution?
>
> Basic principle:
>
> ---
> unittest {
>
>      testCase("a testcase",
>      {
>          // actual test code here
>      });
> }
> ---

Looks a bit too much work.

> Testcase registers the structure and the runtime just runs the unittest
> as is does today and than calls all closures registered during the
> unittest. Makes this fully backward compatible.

Backward compatibility is not an issue seeing this is a pure addition.


Andrei

>
> Backward compatibility is not an issue seeing this is a pure addition.
>
>
> Andrei

I mean that it lives peacefully alongside normal unittest blocks.

On Monday, 30 March 2015 at 21:58:13 UTC, Dicebot wrote:
> I'd prefer putting alternative test runner into Phobos instead which will support `@name("Something") unittest { }`

I agree that an annotation (Probably defined in object.d) is the best way to handle this.

Here's something I don't think many people know: You can't use `~` in deprecated() attributes. You can only put a string literal in the parenthesis. If you want to break messages over multiple lines you need to rely on implicit string concatenation.