Thread overview
Validate static asserts
Sep 09, 2022
Andrey Zherikov
Sep 09, 2022
Paul Backus
Sep 09, 2022
Ali Çehreli
Sep 09, 2022
Andrey Zherikov
Sep 09, 2022
Dennis
Sep 09, 2022
Ali Çehreli
Sep 09, 2022
user1234
September 09, 2022

I have bunch of static assert(<condition>, <message>) in my code and would like to validate that specific code triggers specific assert by checking what <message> is thrown.

Right now I do static assert(!__traits(compiles, { <my code> })); but since <my code> might not compile due to many different reasons, I might not be testing original static assert and might miss breaking change.

One way to do this is to extract <condition> and <message> into some function and test it outside of static assert:

auto check()
{
    return tuple(false, // check result  ('false' is just for example)
                 "message");
}

void f()
{
    enum result = check();
    static assert(result.condition, result.message);
}

unittest
{
    enum result = check();
    static assert(result.condition);
    static assert(result.message == "message");
}

But I don't like this approach because unit test doesn't really test f() (it tests duplicated code) so it can't guarantee that f() works as expected.

Is there a way to validate static asserts in unit tests?

September 09, 2022

On 9/9/22 10:35 AM, Andrey Zherikov wrote:

>

I have bunch of static assert(<condition>, <message>) in my code and would like to validate that specific code triggers specific assert by checking what <message> is thrown.

Right now I do static assert(!__traits(compiles, { <my code> })); but since <my code> might not compile due to many different reasons, I might not be testing original static assert and might miss breaking change.

One way to do this is to extract <condition> and <message> into some function and test it outside of static assert:

auto check()
{
     return tuple(false, // check result  ('false' is just for example)
                  "message");
}

void f()
{
     enum result = check();
     static assert(result.condition, result.message);
}

unittest
{
     enum result = check();
     static assert(result.condition);
     static assert(result.message == "message");
}

But I don't like this approach because unit test doesn't really test f() (it tests duplicated code) so it can't guarantee that f() works as expected.

Is there a way to validate static asserts in unit tests?

Even this doesn't validate that you get the right message for the expected failure.

You can just test the message generation (and use a function for that). That's easier than doing some weird tuple thing.

But validating that the correct message comes out of a failed compilation can only be done outside compilation.

-Steve

September 09, 2022

On Friday, 9 September 2022 at 14:35:33 UTC, Andrey Zherikov wrote:

>

I have bunch of static assert(<condition>, <message>) in my code and would like to validate that specific code triggers specific assert by checking what <message> is thrown.

It sounds like maybe your goal here is to test that attempting to compile a specific piece of code will result in a specific error message being shown to the user.

Unfortunately, the D compiler does not allow you to introspect on error messages, so it is impossible to write a unittest that covers this requirement. Instead, you will have to write an external script or program that attempts to compile a test program and checks the output for the expected error message.

September 09, 2022
On 9/9/22 07:35, Andrey Zherikov wrote:

> might not compile due to many different reasons

I faced a related situation recently: My error string generation was buggy, which taught me that the compiler does not even compile the string part of 'static assert' in the 'true' case.

The following program compiles! :)

void main() {
    static assert (true, "hello" / WAT);
}

> Is there a way to validate static asserts in unit tests?

I added and removed '&& false' to every 'static assert' condition manually one by one. :/

Perhaps a new compiler switch can compile every 'static assert' with an automatic 'false' and dump all their text to the output.

Ali

September 09, 2022

On Friday, 9 September 2022 at 15:22:30 UTC, Ali Çehreli wrote:

>

I added and removed '&& false' to every 'static assert' condition manually one by one. :/

It's not CI-friendly :(

>

Perhaps a new compiler switch can compile every 'static assert' with an automatic 'false' and dump all their text to the output.

What's about new compileOutput trait that returns compiler output?

static assert(__traits(compileOutput, { <my code> }) == "message");
September 09, 2022

On Friday, 9 September 2022 at 16:41:54 UTC, Andrey Zherikov wrote:

>

What's about new compileOutput trait that returns compiler output?

static assert(__traits(compileOutput, { <my code> }) == "message");

As a compiler dev, that sounds terrifying. It would make basically every change to dmd a breaking change.

September 09, 2022
On 9/9/22 10:35, Dennis wrote:
> On Friday, 9 September 2022 at 16:41:54 UTC, Andrey Zherikov wrote:
>> What's about new `compileOutput` trait that returns compiler output?
>> ```d
>> static assert(__traits(compileOutput, { <my code> }) == "message");
>> ```
>
> As a compiler dev, that sounds terrifying. It would make basically every
> change to dmd a breaking change.

For that very reason, I wrote the function 'assertErrorStringContains()' a couple of days ago to ensure *my* strings were in the output:

A precondition:

    void test_1(int i)
    in (i > 0, fooError("The value must be positive", i, 42))
    {
        // ...
    }

A unit test that ensures it fails and checks string pieces appear in the output:

    /*
        The .msg text of the error contains both the error string and the data
        that is included in the error.
    */
    assertErrorStringContains(() => test_1(-1), [ "The value must be positive",
                                                  "-1, 42" ]);
Here is assertErrorStringContains:

    // Assert that the expression throws an Error object and that its string
    // representation contains all expected strings.
    void assertErrorStringContains(void delegate() expr, string[] expected)
    {
        bool thrown = false;

        try
        {
            expr();

        }
        catch (Error err)
        {
            thrown = true;

            import std.algorithm : any, canFind, splitter;
            import std.conv : to;
            import std.format : format;

            auto lines = err.to!string.splitter('\n');
            foreach (exp; expected)
            {
                assert(lines.any!(line => line.canFind(exp)),
                       format!"Failed to find \"%s\" in the output: %-(\n  |%s%)"(
                           exp, lines));
            }
        }

        assert(thrown);
    }

Ali

September 09, 2022

On Friday, 9 September 2022 at 17:35:44 UTC, Dennis wrote:

>

On Friday, 9 September 2022 at 16:41:54 UTC, Andrey Zherikov wrote:

>

What's about new compileOutput trait that returns compiler output?

static assert(__traits(compileOutput, { <my code> }) == "message");

As a compiler dev, that sounds terrifying. It would make basically every change to dmd a breaking change.

Ah yes, it is so stupid that error message are part of the semantics