LOL. This was too funny not to share. I had a unittest failure in a project that I'm working on, and dub ended up just printing out

Program exited with code 1

rather than printing out the actual failures, which was not exactly useful. And I thought that dub must have a bug, so I started trying to see if anyone else had seen a similar problem and which version of dub the problems started in (since I was sure that I'd see proper test failure output from dub previously), and finally I figured out that no, dub didn't have a bug. Rather, it looked like dmd or druntime did, because the problem happened even without dub (though without dub, _nothing_ was printed, since the "Progam exited with code 1" is a dub thing, whereas normally; the D program that failed the unit tests just exits with a code of 1 without printing anything), and it turned out that whether or not I saw the AssertErrors from a test failure depended on where in the file the failing test was, which definitely made it sound like a dmd bug.

So, in order to report the bug, I used dustmite to reduce the code and managed to reduce it quite a bit (though not as much as I might like), and then I manually reduced it further. And ultimately, this is basically what I ended up with

import core.sys.posix.unistd;

unittest
{
    close(2);
    assert("hello" == "7");
}

void main()
{
}

My faulty code had managed to close stderr, which made it impossible for druntime to print out the assertion failure. :)

So, I get to feel like a bit of an idiot over the whole thing, but it definitely made me laugh when I figured out what I'd managed to do.

However, it does highlight the fact that if you just look at the output of running the unit tests and not the exit code, you might think that your tests succeeded when they actually failed. But fortunately, that's not terribly likely (much as I managed to do it), and if you're using dub, it does check the exit code.

- Jonathan M Davis

On Sunday, 23 October 2016 at 21:52:26 UTC, Jonathan M Davis wrote:
> LOL. This was too funny not to share. I had a unittest failure in a project that I'm working on, and dub ended up just printing out

Nice story, thanks for sharing :)

You can change your shell prompt so it displays the exit code of the last command. It's something frequently seen in people's custom prompts. (Here's mine: https://github.com/CyberShadow/cyberzshadow)

On Sunday, October 23, 2016 21:58:30 Vladimir Panteleev via Digitalmars-d wrote:
> You can change your shell prompt so it displays the exit code of the last command. It's something frequently seen in people's custom prompts. (Here's mine: https://github.com/CyberShadow/cyberzshadow)

Wow, that's fancy. I've been using zsh for the past couple of years, but I don't have it doing anything like that. The fanciest I got was getting the git branch in the command prompt, and the way I did that causes repaint problems whenever a command wraps to the next line. I'll definitely have to check out what you've got.

- Jonathan M Davis

On 10/23/2016 05:52 PM, Jonathan M Davis via Digitalmars-d wrote:
>
> import core.sys.posix.unistd;
>
> unittest
> {
>      close(2);
>      assert("hello" == "7");
> }
>

Ouch! I like the punchline here :)

> However, it does highlight the fact that if you just look at the output of
> running the unit tests and not the exit code, you might think that your
> tests succeeded when they actually failed. But fortunately, that's not
> terribly likely (much as I managed to do it), and if you're using dub, it
> does check the exit code.
>

Honestly, it's gets worse than that IMO:

I've had a bunch of time over the course of using D, with any build tool from bud/xfbuild (remember those?), to rdmd, to dub, where a slip-up in my buildscript, project configuration, or a missing import, etc would result in one or all of my unittests *silently!* failing to run.

I even hit it again just this week - where most tests in mysql-native were running...except for one, because of a missing import. The only reason I caught it was because I had learned to be pedantic and temporarily tossed in a writeln "just to be ABSOLUTELY sure...". Good thing I did.

This example is not only easier to hit than yours, but it's also worse because even the exit code will report "A-ok!".

These are all examples of why I was always opposed from the beginning to D's default of "no output unless a test fails": Because it makes "success" indistinguishable from certain types of failures.

That's why I always tried to add some "Testing blah blah whatever..." output at the start of every test in most of my projects: so I *KNOW* that what merely looks like unittest success actually *IS* unittest success.

That's also one of the biggest things I like about unit-threaded: It does that automatically, AND (IIRC), it shows a count of how many tests were run - so when I have any project with more than like...five...tests, I can look at that number and *know* that I'm not overlooking a "Hey, wait, this list of tests run is missing 'Testing fizzbar'!!!"

On 10/23/2016 05:58 PM, Vladimir Panteleev wrote:
> On Sunday, 23 October 2016 at 21:52:26 UTC, Jonathan M Davis wrote:
>> LOL. This was too funny not to share. I had a unittest failure in a
>> project that I'm working on, and dub ended up just printing out
>
> Nice story, thanks for sharing :)
>
> You can change your shell prompt so it displays the exit code of the
> last command. It's something frequently seen in people's custom prompts.
> (Here's mine: https://github.com/CyberShadow/cyberzshadow)

Cool, mine is quite similar (I don't use mc so I use RPS1). One extra thing I found useful is to display the running time of the last command if longer than 1 second. -- Andrei

On 2016-10-24 14:37, Andrei Alexandrescu wrote:

> Cool, mine is quite similar (I don't use mc so I use RPS1). One extra
> thing I found useful is to display the running time of the last command
> if longer than 1 second. -- Andrei

What do you do if it takes longer than 1 second to build the prompt ;)

I use iTerm on macOS [1]. With shell integration enabled it will put a small icon in front prompt to indicate if the last command was successful or not [2].

[1] http://iterm2.com/
[2] http://iterm2.com/img/screenshots/v3-screen-shots/iterm2-shell-integration-navigation-demo.mov

-- 
/Jacob Carlborg

On Monday, 24 October 2016 at 18:45:36 UTC, Jacob Carlborg wrote:
> On 2016-10-24 14:37, Andrei Alexandrescu wrote:
>
>> Cool, mine is quite similar (I don't use mc so I use RPS1). One extra
>> thing I found useful is to display the running time of the last command
>> if longer than 1 second. -- Andrei
>
> What do you do if it takes longer than 1 second to build the prompt ;)

Simply try to not take too long for prompt creation, e.g. by doing it async ;)
Google for "zsh async prompt" or get inspired by pure (which I use):
https://github.com/sindresorhus/pure