On Wednesday, 1 July 2015 at 09:08:22 UTC, Mike wrote:
> On Wednesday, 1 July 2015 at 08:40:24 UTC, Atila Neves wrote:
>>
>> I hate its verbosity and all I really want to write is `assert(a > b)` and get a decent error message.
>>
>
> I haven't been following this too closely as there is only so much I can pay attention to at any time, so I apologize in advance if the following comments are uninformed.  If so, just ignore them.
>
> I suspect from your comments that you want to use `assert`, but it's currently reserved by the language and https://github.com/D-Programming-Language/dmd/pull/1426 was not followed through on, and is currently owned by a ghost (literally).
>
> I therefore suggest using `assertThat(whatever)`.  If/when `assert` is improved upon to allow integration into this library, `assertThat` can be deprecated.

The problem there is `assertThat` gives no more information than `assert` currently does. This thread is all about how do we want to deal with custom assertions that give you useful information. My conclusion so far has been that what's there is probably the best we're going to get.

I could always change the shouldXXX functions to be called assertXXX instead, but it doesn't look as good with UFCS. That might get me Dicebot's vote though :)

Atila

Atila

>
> Sorry if I'm making noise,
> Mike

On Wednesday, 1 July 2015 at 08:21:28 UTC, Atila Neves wrote:
> Well, there's a PR for improving assertions here: https://github.com/D-Programming-Language/dmd/pull/1426

Sadly, this does not really fix the issue in the long term. Using assertions in tests prevents building non-fatal test runners because by spec AssertError is non-recoverable. That is quite a serious limitation.

To really fix it, this information needs to be available in library code.

On Wednesday, 1 July 2015 at 08:22:50 UTC, Atila Neves wrote:
> opEquals is easy. It's everything else that's hard.
>
> Atila

I thought that opNotEquals was also a challenge. The way I approached it should be usable by all operator overloads, but I'm not making the claim that the way I approached it was any good. Only saying its the direction I prefer.

On 01/07/15 15:45, Dicebot wrote:

> Sadly, this does not really fix the issue in the long term. Using
> assertions in tests prevents building non-fatal test runners because by
> spec AssertError is non-recoverable. That is quite a serious limitation.

There's an assertion handler that can be set in druntime [1]. Unfortunately the callback is declared as "nothrow". If this "nothrow" it could be used to throw an exception instead of an error.

[1] https://github.com/D-Programming-Language/druntime/blob/master/src/core/exception.d#L378

-- 
/Jacob Carlborg

On 01/07/15 10:40, Atila Neves wrote:

> So, despite the fact that I wrote `shouldBeGreaterThan`, I hate its
> verbosity

You could write "shouldBe.gt(value)".

> I don't buy that `should.` is more extensible. For two reasons, first
> because in all the test frameworks I've seen there aren't that many more
> types of assertions

In every project I have used RSpec I have added custom matchers/assertions. Just a couple of days ago I added a custom matcher to one of my projects:

code = code_to_file('void foo() {}')
code.should be_parsed_as('meta.definition.method.d')

The point of these custom matchers/assertions are that they should make the tests (or specs in the BDD case) more readable and provide better assertion failure messages. Of course, none of these assertions are necessary, we could all just use "assert" or "shouldEqual", in the end every assertion is just a boolean condition.

My test could instead look like this, without using a custom matcher:

file = code_to_file('void foo() {}')
result = `spec/bin/gtm < "#{file.path}" Syntaxes/D.tmLanguage 2>&1`
line = result.split("\n").first
scope = 'meta.definition.method.d'
assert line =~ /#{Regexp.escape(scope)}/

Which one of the two versions are more readable?

> and secondly because adding a member function to
> the struct returned by `should` and adding a new `shouldCamelCase`
> function is the same amount of work.

The idea would be that the part after "should" would be a free function and use UFCS to call it.

Should should()
{
    return Should();
}

void beCamelCase(Should s);

should.beCamelCase

That's what allows to create custom assertions. A user of the library can't extend the struct returned by "should".

-- 
/Jacob Carlborg

On Wednesday, 1 July 2015 at 19:38:20 UTC, Jacob Carlborg wrote:
> On 01/07/15 10:40, Atila Neves wrote:
>
>> [...]
>
> You could write "shouldBe.gt(value)".
>
>> [...]
>
> In every project I have used RSpec I have added custom matchers/assertions. Just a couple of days ago I added a custom matcher to one of my projects:
>
> [...]

Ah, makes sense. I think I'm convinced now. The UFCS as an extension mechanism could indeed be handy.

Atila

On Wednesday, 1 July 2015 at 19:38:20 UTC, Jacob Carlborg wrote:
> In every project I have used RSpec I have added custom matchers/assertions. Just a couple of days ago I added a custom matcher to one of my projects:
>
> code = code_to_file('void foo() {}')
> code.should be_parsed_as('meta.definition.method.d')
>
> The point of these custom matchers/assertions are that they should make the tests (or specs in the BDD case) more readable and provide better assertion failure messages. Of course, none of these assertions are necessary, we could all just use "assert" or "shouldEqual", in the end every assertion is just a boolean condition.
>
> My test could instead look like this, without using a custom matcher:
>
> file = code_to_file('void foo() {}')
> result = `spec/bin/gtm < "#{file.path}" Syntaxes/D.tmLanguage 2>&1`
> line = result.split("\n").first
> scope = 'meta.definition.method.d'
> assert line =~ /#{Regexp.escape(scope)}/
>
> Which one of the two versions are more readable?

Neither. But with the second one I at least have a chance to figure out what it actually tests. To understand the first one I'd need to read the code of that custom matcher.

This is exactly the problem with all that fancy - library/app author (who by definition knows the domain well) implicitly introduces custom DSL and expects everyone (who, by definition, have very vague understanding of domain) to be able to read it as comfortably as himself.

On 02/07/15 14:28, Dicebot wrote:

> Neither. But with the second one I at least have a chance to figure out
> what it actually tests. To understand the first one I'd need to read the
> code of that custom matcher.

Once you know what a matcher does or how it's used it's a lot more readable and easier to create new tests.

> This is exactly the problem with all that fancy - library/app author
> (who by definition knows the domain well) implicitly introduces custom
> DSL and expects everyone (who, by definition, have very vague
> understanding of domain) to be able to read it as comfortably as himself.

By that definition we should all write object oriented code in C. Heck, why don't we just use assembly and be done with it.

-- 
/Jacob Carlborg

On Thursday, 2 July 2015 at 19:06:22 UTC, Jacob Carlborg wrote:
> On 02/07/15 14:28, Dicebot wrote:
>
>> Neither. But with the second one I at least have a chance to figure out
>> what it actually tests. To understand the first one I'd need to read the
>> code of that custom matcher.
>
> Once you know what a matcher does or how it's used it's a lot more readable and easier to create new tests.

I am not going to investigate all custom matchers and stuff to submit one simple pull request to a project I don't care much about. Simple helper function may be slightly less "pretty" but much easier to grep for and reason about.

> By that definition we should all write object oriented code in C. Heck, why don't we just use assembly and be done with it.

That is exactly why language features are better than free form AST macros. Any abstraction has inherent learning costs. Any non-standard abstraction - even more so. For in-house project it tends to be still worth it because maintenance costs tend to be higher than learning costs and development team is relatively stable. For open-source ecosystem - quite the contrary. And here we speak about something that lays foundation to any open-source contributions - standard testing system.

1. 'assert' is the wrong thing

I do not need a stack trace for a failed expectation in a unit test.
But, usually, I need a stack trace for a contract violation from deep down in the unit under test:

https://github.com/linkrope/dunit#failures-vs-errors

Consequently, the failed expectation should throw some "failed expectation" 'Exception' instead of an 'AssertError'. So, std.exception.assertThrown cannot be used for expectations.

2. something like an 'assert' with better diagnostics could be nice

Catch for C++ introduced 'REQUIRE' with helpful failure messages:

    Example.cpp:9: FAILED:
      REQUIRE( Factorial(0) == 1 )
    with expansion:
      0 == 1

https://github.com/philsquared/Catch

3. 'should' seems to be obsolete

Now, 'expect' should be used for expectations:

http://rspec.info/blog/2012/06/rspecs-new-expectation-syntax/