On Tuesday, 30 June 2015 at 12:42:40 UTC, Sebastiaan Koppe wrote:
> On Tuesday, 30 June 2015 at 08:06:37 UTC, Atila Neves wrote:
>> [...]
>
>
> Much rather prefer the composable ones over the `shouldEquals`, simply for readability and easy extending.
>
> These days I am leaning towards BDD, but everybody has his favorite. Maybe just providing the low-level details in std.testing would enough; e.g. a test runner, UDA's and assertions.
>
> Then everyone can write his on version of given().when().then() on top of it. Or simply make a pull-request for std.testing.bdd

Yeah, I'm starting to think it might be better to delete `should.d` from my current PR, try to get the rest approved then work on where the community wants the fancy assertions to go. It's a shame though because I think it's a massively important piece of the whole thing. It's a night and day difference when a test fails.

Atila

On Tuesday, 30 June 2015 at 14:58:45 UTC, Atila Neves wrote:
> On Tuesday, 30 June 2015 at 12:42:40 UTC, Sebastiaan Koppe wrote:
>> On Tuesday, 30 June 2015 at 08:06:37 UTC, Atila Neves wrote:
>>> [...]
>>
>> These days I am leaning towards BDD, but everybody has his favorite. Maybe just providing the low-level details in std.testing would enough; e.g. a test runner, UDA's and assertions.
>>
>
> Yeah, I'm starting to think it might be better to delete `should.d` from my current PR, try to get the rest approved then work on where the community wants the fancy assertions to go. It's a shame though because I think it's a massively important piece of the whole thing. It's a night and day difference when a test fails.
>
> Atila

Makes sense. You could still keep the should's, just rename the whole lot to isEmpty / isNotEmpty / isGreaterThan and have it return a bool instead of calling fail internally. Then you would simply expect the callee to do that. As in: `assert(5.isEqual(6));`.

On 30/06/15 16:58, Atila Neves wrote:

> Yeah, I'm starting to think it might be better to delete `should.d` from
> my current PR, try to get the rest approved then work on where the
> community wants the fancy assertions to go. It's a shame though because
> I think it's a massively important piece of the whole thing. It's a
> night and day difference when a test fails.

Yeah, I agree, it would be a shame to not have these assertions.

-- 
/Jacob Carlborg

On 30/06/15 10:06, Atila Neves wrote:

> Well, the dream would be that `assert(foo ==
> bar)` did what part of this PR does, but that's another story and
> something that can't be done by a library unless we had AST macros,
> which we won't. Or Lisp's reader macros, but we won't get those either.

I was thinking the same. Both the test!"==" and shouldEqual are workarounds to get a nice message on an assertion failure. I'm wondering how hard it would be to have the compiler generate a string representing the failing expression.

-- 
/Jacob Carlborg

https://www.youtube.com/watch?v=pWS8Mg-JWSg&feature=player_detailpage#t=81

On Tuesday, 30 June 2015 at 11:43:36 UTC, Dicebot wrote:
> In absence of language changes, I don't see anything as clear and simple as operator mixins. Less magic in unittests -> better. Common misconception IMHO is that tests should look nice for library/app author, while, from the ecosystem PoV, they should look simple and predictable for contributors - and that is most important property.

I tend to agree with your position on testing frameworks. It seems really cool to utilize English to spell out an expectation, but it ends up more complicated. For one, it can never actually be English (I wouldn't want it to be) and for another, I've already got expressions from the language being used that mean the same thing.

    unittest {
        struct A {
            int m;
        }

        A a;
        A b;
        a.m = 5;
        b.m = 5;

        auto testA = a.test;
        with(testA) {
            verify(testA.lhs == b);
            b.m = 6;
            verify(testA.lhs != b);
            verify(testA.lhs == b);
        }
    }

It is kind of like what you're saying, but the condition isn't reportable. I'm actually less concerned about the condition as I am being able to report all the values utilized by the condition. I'd like to see assert be more descriptive or a framework that looks like assert and is more descriptive. Here is the crap code to run the above:

https://gist.github.com/JesseKPhillips/df79479cbf6a0e3c6b0d

On Tuesday, 30 June 2015 at 19:48:46 UTC, Jacob Carlborg wrote:
> On 30/06/15 10:06, Atila Neves wrote:
>
>> Well, the dream would be that `assert(foo ==
>> bar)` did what part of this PR does, but that's another story and
>> something that can't be done by a library unless we had AST macros,
>> which we won't. Or Lisp's reader macros, but we won't get those either.
>
> I was thinking the same. Both the test!"==" and shouldEqual are workarounds to get a nice message on an assertion failure. I'm wondering how hard it would be to have the compiler generate a string representing the failing expression.

Well, there's a PR for improving assertions here: https://github.com/D-Programming-Language/dmd/pull/1426

Since I have 0 experience in compilers in general and dmd in particular, I thought that'd be an easy way for me to get an in on the assert situation. It seems... more complicated than I can handle at the moment.

The 100% ideal situation is for assert to do what I'm doing with the functions in the `should` module. That module really shouldn't even exist.

Atila

On Wednesday, 1 July 2015 at 00:26:21 UTC, Jesse Phillips wrote:
> On Tuesday, 30 June 2015 at 11:43:36 UTC, Dicebot wrote:
>> [...]
>
> I tend to agree with your position on testing frameworks. It seems really cool to utilize English to spell out an expectation, but it ends up more complicated. For one, it can never actually be English (I wouldn't want it to be) and for another, I've already got expressions from the language being used that mean the same thing.
>
> [...]

opEquals is easy. It's everything else that's hard.

Atila

On Tuesday, 30 June 2015 at 08:06:37 UTC, Atila Neves wrote:
> In case you don't know what I'm talking about: https://github.com/D-Programming-Language/phobos/pull/3207
>
> Since this is an API issue it's import to get it right the first time. Personally I'm not sure what I prefer (well, I am, but what I actually want isn't syntactically valid D). I think the options so far are:
>
> 1) What's there already, namely `shouldEquals`, `shouldBeIn`, etc.
> 2a) Compile-time strings for operators: `should!"=="`, `should!"in"`
> 2b) Dicebot's `test!"=="`. `assert` is so much better, I wish we could use that.
> 3) Composable ones: should.equals, should.not.equals, or another word that isn't "should"
> 4) Anything else?
>
> I'm not convinced composability brings anything to the table except for editor dot-completion. I don't like the verbosity of what's there now, but my prefererred syntax doesn't work except for the ubiquitous  check for equality (`should ==`). Well, the dream would be that `assert(foo == bar)` did what part of this PR does, but that's another story and something that can't be done by a library unless we had AST macros, which we won't. Or Lisp's reader macros, but we won't get those either.
>
> Thoughts? Votes?
>
> Atila

After much thinking and some _incredibly_ hacky attempts to come up with a library solution that can use operators (it was really hacky, I was using `assert()` and making the expression always return true), I came to the conclusion that what I have there already is probably as good as it's going to get bar language changes.

The reason is this: D has very smartly side-stepped C++'s problems with operator overloading and boilerplate by making things like `a >=b` be translated to `a.opCmp(b) >=0`. This is one of them Good Things. The same with `!=`, it's `!(a.opEquals(b)`. Again, good.

But for test asserts, that means that by the time any possible library code gets to handle it there's been a loss of information. There's so way to know whether opEquals was called with or without negation, or if opCmp was called because the user typed `>`, `>=`, `<` or `<=`.

I can make this work:

3.timesTwo.should == 6;
3.timesTwo.should.not == 5;


But anything with opCmp is a no-go. The only other operator that would make sense (to me) to overload is opBinary!("in"), but the naming there is a problem. `should.in`? `should.beIn`? Oh wait, that's not the operator anymore. Which brings me to...

Naming. Any one word that isn't `assert` is... not as good. It always breaks down for one use case or another, and I think what's in the PR is the best I've heard so far on this forum or in the comments. A compile-time string is... ugly and error-prone especially since the most common assertion is for equality and `!"=="` looks too much like `!=`. So, despite the fact that I wrote `shouldBeGreaterThan`, I hate its verbosity and all I really want to write is `assert(a > b)` and get a decent error message.

I don't buy that `should.` is more extensible. For two reasons, first because in all the test frameworks I've seen there aren't that many more types of assertions, and secondly because adding a member function to the struct returned by `should` and adding a new `shouldCamelCase` function is the same amount of work. As for auto-completion, I also see no real difference between the two.

Other than assert being semi-magical, I don't see how it can be any better.

Atila

On Wednesday, 1 July 2015 at 08:40:24 UTC, Atila Neves wrote:
>
> I hate its verbosity and all I really want to write is `assert(a > b)` and get a decent error message.
>

I haven't been following this too closely as there is only so much I can pay attention to at any time, so I apologize in advance if the following comments are uninformed.  If so, just ignore them.

I suspect from your comments that you want to use `assert`, but it's currently reserved by the language and https://github.com/D-Programming-Language/dmd/pull/1426 was not followed through on, and is currently owned by a ghost (literally).

I therefore suggest using `assertThat(whatever)`.  If/when `assert` is improved upon to allow integration into this library, `assertThat` can be deprecated.

Sorry if I'm making noise,
Mike