Hi all,

D is cool language. It has a lot of good features.
One of such features is integrated unittests.

But, this feature is not convenient/usable without standard set of assert* methods, that could display human-readable output in case of error (what value expected, and what value we got).
Because of absense of such expected functionality, there are some set of third-party libraries that implement good and readable assertions. But, most popular assertion libraries depend on unit-threaded that is large test runner.

And it is strange, when you try to add small and simple library to dependencies of your project, but it brings whole unit-threaded package to your project. And i think, in significant amount of cases, this dependency is added just because standard set of asserts absent in standard library.

I think, for small libraries there is no sense to depend on large test runner. Ususally it is enough to use standard test runner.

Just take a look at python's assertions from standard library unittest

I think, it could be enough to have following asserts in stadard library:

• assertEqual
• assertNotEqual
• assertTrue
• assertFalse
• assertIn
• assertNotIn
• assertThrows
• assertGreater
• assertGreaterEqual
• assertLessEqual

I think, implementation of this list of asserts in the Phobos will allow users to start use unittest much easier.

Also, the question to authors of assert libraries (like unit-threaded:assertions, dshould, etc): what do you think about contributing assertions to Phobos?

Just example of why standard assert is not enough.

Assume, that we have simple app with unittests like below:

import std.stdio;

unittest {
    int x = 0, y = 5;
    assert(x == y, "X != Y");
}

unittest {
    import dshould;

    int x = 0, y = 5;
    x.should.equal(y);
}

unittest {
    import unit_threaded.assertions;

    int x = 0, y = 5;
    x.should == y;
}

void main()
{
	writeln("Edit source/app.d to start your project.");
}

And let's check output of assertions:

The first, standard assertion could look like below. As you can see, there is no info about values of x and y.

core.exception.AssertError@source/app.d(5): X != Y
----------------
??:? _d_unittest_msg [0x5596a817de88]
source/app.d:5 void app.__unittest_L3_C1() [0x5596a81578ef]
??:? void app.__modtest() [0x5596a8162ee0]
??:? int core.runtime.runModuleUnitTests().__foreachbody6(object.ModuleInfo*) [0x5596a8188ae6]
??:? int object.ModuleInfo.opApply(scope int delegate(object.ModuleInfo*)).__lambda2(immutable(object.ModuleInfo*)) [0x5596a8174c8b]
??:? int rt.minfo.moduleinfos_apply(scope int delegate(immutable(object.ModuleInfo*))).__foreachbody2(ref rt.sections_elf_shared.DSO) [0x5596a81827ff]
??:? int rt.sections_elf_shared.DSO.opApply(scope int delegate(ref rt.sections_elf_shared.DSO)) [0x5596a8182989]
??:? int rt.minfo.moduleinfos_apply(scope int delegate(immutable(object.ModuleInfo*))) [0x5596a818278d]
??:? int object.ModuleInfo.opApply(scope int delegate(object.ModuleInfo*)) [0x5596a8174c5d]
??:? runModuleUnitTests [0x5596a818891b]
??:? void rt.dmain2._d_run_main2(char[][], ulong, extern (C) int function(char[][])*).runAll() [0x5596a817f32c]
??:? void rt.dmain2._d_run_main2(char[][], ulong, extern (C) int function(char[][])*).tryExec(scope void delegate()) [0x5596a817f2b9]
??:? _d_run_main2 [0x5596a817f222]
??:? _d_run_main [0x5596a817f00b]
/usr/include/dmd/druntime/import/core/internal/entrypoint.d:29 main [0x5596a8157a11]
??:? [0x7fbc5aa2350f]
??:? __libc_start_main [0x7fbc5aa235c8]
??:? _start [0x5596a81577e4]
1/1 modules FAILED unittests

dshould will print following traceback, that has info about exact values of x and y.

dshould.ShouldType.FluentErrorImpl!(unit_threaded.exception.UnitTestError).FluentErrorImpl@source/app.d(12): Test failed: expected 5, but got 0
----------------
/home/katyukha/.dub/packages/dshould-1.7.1/dshould/src/dshould/ShouldType.d:127 pure nothrow @safe void dshould.ShouldType.ShouldType!(int delegate() pure @safe, ["equal"]).ShouldType.check(bool, lazy immutable(char)[], lazy immutable(char)[], immutable(char)[], ulong) [0x55b3b74d6fb5]
/home/katyukha/.dub/packages/dshould-1.7.1/dshould/src/dshould/basic.d:359 pure @safe void dshould.basic.numericCheck!(dshould.ShouldType.ShouldType!(int delegate() pure @safe, ["equal"]).ShouldType, int).numericCheck(dshould.ShouldType.ShouldType!(int delegate() pure @safe, ["equal"]).ShouldType, const(int), immutable(char)[], ulong) [0x55b3b74d7e70]
/home/katyukha/.dub/packages/dshould-1.7.1/dshould/src/dshould/basic.d:197 pure @safe void dshould.basic.equal!(dshould.ShouldType.ShouldType!(int delegate() pure @safe, []).ShouldType, int).equal(dshould.ShouldType.ShouldType!(int delegate() pure @safe, []).ShouldType, int, dshould.ShouldType.Fence, immutable(char)[], ulong) [0x55b3b74d6bc1]
/home/katyukha/.dub/packages/dshould-1.7.1/dshould/src/dshould/package.d:85 pure @safe void dshould.equal!(dshould.ShouldType.ShouldType!(int delegate() pure @safe, []).ShouldType, int).equal(dshould.ShouldType.ShouldType!(int delegate() pure @safe, []).ShouldType, int, dshould.ShouldType.Fence, immutable(char)[], ulong) [0x55b3b74d6963]
source/app.d:12 void app.__unittest_L8_C1() [0x55b3b74d4919]
??:? void app.__modtest() [0x55b3b74dfe98]
??:? int core.runtime.runModuleUnitTests().__foreachbody6(object.ModuleInfo*) [0x55b3b7505a06]
??:? int object.ModuleInfo.opApply(scope int delegate(object.ModuleInfo*)).__lambda2(immutable(object.ModuleInfo*)) [0x55b3b74f1c3b]
??:? int rt.minfo.moduleinfos_apply(scope int delegate(immutable(object.ModuleInfo*))).__foreachbody2(ref rt.sections_elf_shared.DSO) [0x55b3b74ff71f]
??:? int rt.sections_elf_shared.DSO.opApply(scope int delegate(ref rt.sections_elf_shared.DSO)) [0x55b3b74ff8a9]
??:? int rt.minfo.moduleinfos_apply(scope int delegate(immutable(object.ModuleInfo*))) [0x55b3b74ff6ad]
??:? int object.ModuleInfo.opApply(scope int delegate(object.ModuleInfo*)) [0x55b3b74f1c0d]
??:? runModuleUnitTests [0x55b3b750583b]
??:? void rt.dmain2._d_run_main2(char[][], ulong, extern (C) int function(char[][])*).runAll() [0x55b3b74fc24c]
??:? void rt.dmain2._d_run_main2(char[][], ulong, extern (C) int function(char[][])*).tryExec(scope void delegate()) [0x55b3b74fc1d9]
??:? _d_run_main2 [0x55b3b74fc142]
??:? _d_run_main [0x55b3b74fbf2b]
/usr/include/dmd/druntime/import/core/internal/entrypoint.d:29 main [0x55b3b74d49c9]
??:? [0x7fca6e82350f]
??:? __libc_start_main [0x7fca6e8235c8]
??:? _start [0x55b3b74d47e4]
1/1 modules FAILED unittests

And finally, unit-threaded:assertions will print info about exact values of variables compared in assertion:

unit_threaded.exception.UnitTestException@source/app.d(19): Expected: 5
     Got: 0
----------------
/home/katyukha/.dub/packages/unit-threaded-2.1.5/unit-threaded/subpackages/assertions/source/unit_threaded/assertions.d:56 pure @safe void unit_threaded.assertions.shouldEqual!(int, int).shouldEqual(ref int, ref int, immutable(char)[], ulong) [0x555c04e1ba7a]
/home/katyukha/.dub/packages/unit-threaded-2.1.5/unit-threaded/subpackages/assertions/source/unit_threaded/assertions.d:1074 pure @safe bool unit_threaded.assertions.should!(int).should(ref int).Should.opEquals!(int).opEquals(ref int, immutable(char)[], ulong) [0x555c04e1b9ac]
source/app.d:19 void app.__unittest_L15_C1() [0x555c04e198f6]
??:? void app.__modtest() [0x555c04e1c43c]
??:? int core.runtime.runModuleUnitTests().__foreachbody6(object.ModuleInfo*) [0x555c04e34822]
??:? int object.ModuleInfo.opApply(scope int delegate(object.ModuleInfo*)).__lambda2(immutable(object.ModuleInfo*)) [0x555c04e20cb3]
??:? int rt.minfo.moduleinfos_apply(scope int delegate(immutable(object.ModuleInfo*))).__foreachbody2(ref rt.sections_elf_shared.DSO) [0x555c04e2d403]
??:? int rt.sections_elf_shared.DSO.opApply(scope int delegate(ref rt.sections_elf_shared.DSO)) [0x555c04e2d58d]
??:? int rt.minfo.moduleinfos_apply(scope int delegate(immutable(object.ModuleInfo*))) [0x555c04e2d391]
??:? int object.ModuleInfo.opApply(scope int delegate(object.ModuleInfo*)) [0x555c04e20c85]
??:? runModuleUnitTests [0x555c04e34657]
??:? void rt.dmain2._d_run_main2(char[][], ulong, extern (C) int function(char[][])*).runAll() [0x555c04e29f30]
??:? void rt.dmain2._d_run_main2(char[][], ulong, extern (C) int function(char[][])*).tryExec(scope void delegate()) [0x555c04e29ebd]
??:? _d_run_main2 [0x555c04e29e26]
??:? _d_run_main [0x555c04e29c0f]
/usr/include/dmd/druntime/import/core/internal/entrypoint.d:29 main [0x555c04e19939]
??:? [0x7fa059e2350f]
??:? __libc_start_main [0x7fa059e235c8]
??:? _start [0x555c04e197e4]
1/1 modules FAILED unittests

I think, ideally, test assertions should display only the file and line with failed assertions, because all other part of traceback with D internals usually is not needed to fix the test, but the exact values of variable compared in assert are required.

It is possible, to get similar output with standard assert using format:

unittest {
    import std.format: format;
    int x = 0, y = 5;
    assert(x == y, "%s != %s".format(x, y));
}

But in case, if x and / or y is expression, the unittest becomes unreadable.

So, what do you think about this?

On Thursday, 30 March 2023 at 17:01:26 UTC, Dmytro Katyukha wrote:

Hi! Are you aware of the -checkaction=context compiler flag? Give it a shot:

unittest {
    int x = 0, y = 5;
    assert(x == y);
}

dmd -main -unittest -checkaction=context -run ./example.d

onlineapp.d(3): [unittest] 0 != 5
1/1 modules FAILED unittests

Test on run.dlang.io.

This is not an argument against a dedicated assertion module, but I think it's good to be aware what can be achieved at the compiler/runtime level vs library level.

On Thursday, 30 March 2023 at 17:22:39 UTC, Petar Kirov [ZombineDev] wrote:

unittest {
    int x = 0, y = 5;
    assert(x == y);
}

dmd -main -unittest -checkaction=context -run ./example.d

onlineapp.d(3): [unittest] 0 != 5
1/1 modules FAILED unittests

Thanks for information, i was not aware about this compiler switch.

Also, i would like to note that, there is no info about this compiler switch in unittests documentation

On Thursday, 30 March 2023 at 19:53:42 UTC, Dmytro Katyukha wrote:

All dmd compiler switches are documented on this page: dlang.org/dmd. This one specifically, here: dlang.org/dmd#switch-checkaction. However, I agree that the documentation doesn't provide sufficient information and that it should also be linked from the spec/unittest page.

Right now, probably the best documentation is on the dmd v2.085 changelog page, when it was first introduced: dlang.org/changelog/2.085.0#assert.

On Thursday, 30 March 2023 at 17:22:39 UTC, Petar Kirov [ZombineDev] wrote:

Be aware that this feature has linking issues.

https://issues.dlang.org/show_bug.cgi?id=19937
https://issues.dlang.org/show_bug.cgi?id=22374
https://issues.dlang.org/show_bug.cgi?id=22902

On Thursday, 30 March 2023 at 17:01:26 UTC, Dmytro Katyukha wrote:

I’d say most are superfluous:

• assertEqual(a, b) why not use assert(a == b)?
• assertNotEqual(a, b) why not use assert(a != b)?
• assertTrue(cond) why not use assert(cond)?
• assertFalse(cond) why not use assert(!cond)?
• assertGreater why not use assert(a > b)?
• assertGreaterEqual why not use assert(a >= b)?
• assertLessEqual why not use assert(a <= b)?

These have non-trivial, but straightforward lowerings:

• assertIn why not use canFind (or any) in std.algorithm.searching?
• assertNotIn why not use !canFind (or !any)?

This one is the only one that has value for DRY:

• assertThrows

Instead of

assertThrows!Exception(expression);

one could write

try { expression(); assert(0); } catch (Exception) {}

but I see that it is a non-trivial pattern and assertThrows just documents clearly what is expected. Even this isn’t that much.

Those functions can provide better error messages, but to be honest, the D compiler’s unittest failure messages should be improved instead. The “superfluous” list is just an enumeration of expressions commonly found in assert expressions that should have special treatment by the compiler. For assertThrows, maybe it could even be worth adding it as a primitive:

assert throw(ExceptionType, expression, message)

The message is optional, of course. A static assert version should exist as well.

On Tuesday, 4 April 2023 at 12:11:28 UTC, Quirin Schroll wrote:

They are redundant but they allow some slightly nicer failure messages upon failure (i.e. assert fail is not very helpful unless you are at one with all the code)

It may be of note that Atila has implemented most (all?) of this in unit_threaded.

On Tuesday, 4 April 2023 at 15:45:55 UTC, max haughton wrote:

I tend to use -checkaction=context together with -unitest...