On Thursday, 22 October 2020 at 20:54:52 UTC, H. S. Teoh wrote:
>
> I agree tests are watchers.  But it starts a bit ridiculous when you need to test your tests...
>

It might not be.

In my opinion, the answer to "who watches the watchmen?" is "a manual test". If you automate a test, you should test the test yourself. If you automate testing the test, you should test the test-tester and so on.

What is the point then in unit tests? I believe it is in reducing the amount of manual testing. You will not have to change a good unit test nearly as often as you have to change the code it is testing. If it does not change, it does not need rechecking.

The assertions within the function usually don't share this property. They may rely on the internal logic of the function, and thus get changed with it. Manually testing them sucks just as much as relying on manual tests in general. Contracts might be better in this regard though, as long as the function is either public or widely used internally, so that it's interface and intended functionality don't change often.

On Thursday, 22 October 2020 at 20:54:52 UTC, H. S. Teoh wrote:
> I agree tests are watchers.  But it starts a bit ridiculous when you need to test your tests...

They have tests: https://github.com/atilaneves/unit-threaded/blob/master/tests/unit_threaded/ut/should.d

On Friday, 23 October 2020 at 09:03:43 UTC, Jacob Carlborg wrote:
> Regardless of what's correct or not, I think there must be a way to have unit tests keep running after a failed test. Otherwise D will have very poor experience for unit test frameworks. It's already bad enough with the default unit test runner that aborts the rest of the unit tests inside a module after a failed test (it will continue with other modules).

FWIW ldc performs unwinding optimizations only in release compilation mode and not in others.

On 10/22/20 1:38 PM, Adam D. Ruppe wrote:

> On Thursday, 22 October 2020 at 20:23:49 UTC, Ali Çehreli wrote:
>> 1) Thread entry functions should catch and report any Throwable.
>> Otherwise, main thread will see only a timeout and stack trace of the
>> worker will lost.
>
> Worth remembering catching actually loses valuable information when
> running in a debugger and/or with core dumps enabled.
>
> If you did want to do this, at a minimum it would have to honor the
> DRT-trapExceptions value.

Once again, thank you. But I tried this and it seems to be effective only on the main thread.

So, in case of multiple threads, assuming that a thread threw Error, I don't know any better solution than the following.

The options are progressively more wishful. In theory, options beyond 'a' should not be attempted but option 'a' is not practical nor safe.

a) Don't do anything. I think this choice is horrible because

- The rest of the threads would still be continuing with presumably invalid program state.

- Nobody would know that a thread has died.

- Nobody would know the stack trace when Error was thrown.

b) Catch Error in the worker thread and call abort() immediately. Note: In the strictest sense, even calling abort() should not be attempted but this thinking is madness because otherwise the rest of the program would be continuing with "invalid state."

c) Same as 'b', but do stderr.writeln(err) before abort(). This way, we *may* have an idea of what happened but in a safety-critical application, we may be killing a human instead of printing anything.

d) Catch Error and send it as a message to the main thread and let the main thread do one of the above. Even though this feels the cleanest, I don't see any benefit in doing this over the others.

Over the years, I've been a part of many error management strategy discussions on these forums and elsewhere I still don't know what to do. For example, there are safety-critical programs out there that can't even abort. The thinking is that instead of leaving a machine in its "invalid state", which may kill humans, there is always a better way of degrading the machine's operations to do something safer. For example, in the case of autonomous driving, slowing down the vehicle slowly and parking on the side of the road may be better than aborting at 120kph on the highway.

Always a fascinating discussion...

Ali

On Friday, 23 October 2020 at 16:30:53 UTC, Ali Çehreli wrote:
> c) Same as 'b', but do stderr.writeln(err)

Note that stderr doesn't necessarily exist. A Win32 GUI application is likely to have stderr set to an invalid file descriptor even in the best conditions.

but idk.

On Fri, Oct 23, 2020 at 09:30:53AM -0700, Ali Çehreli via Digitalmars-d wrote: [...]
> Over the years, I've been a part of many error management strategy discussions on these forums and elsewhere I still don't know what to do. For example, there are safety-critical programs out there that can't even abort.  The thinking is that instead of leaving a machine in its "invalid state", which may kill humans, there is always a better way of degrading the machine's operations to do something safer. For example, in the case of autonomous driving, slowing down the vehicle slowly and parking on the side of the road may be better than aborting at 120kph on the highway.

I think Walter's go-to answer on that one is, have a monitor process that detects when the main process has crashed, and immediately take over to initiate a safe shutdown procedure.  Basically, the main process has entered an invalid state, so we can no longer assume it's able to correctly carry out the safe shutdown procedure (what if the bug that triggered the assert has also corrupted the program's data so that when it tries to slow down it accelerates instead?). So we use a redundant, independent component (the monitor process) that's known to be in a consistent state to do the safe shutdown.

I'm not sure how to apply this principle (or if it's even applicable) to all situations, but there you have it.


T

-- 
Маленькие детки - маленькие бедки.

On Friday, 23 October 2020 at 16:30:53 UTC, Ali Çehreli wrote:
> c) Same as 'b', but do stderr.writeln(err) before abort().

This is most appropriate, basically the same as what druntime does.

On 10/23/20 10:04 AM, H. S. Teoh wrote:

> I think Walter's go-to answer on that one is, have a monitor process
> that detects when the main process has crashed, and immediately take
> over to initiate a safe shutdown procedure.

And "immediately" must be a very short time.

> Basically, the main process
> has entered an invalid state, so we can no longer assume it's able to
> correctly carry out the safe shutdown procedure (what if the bug that
> triggered the assert has also corrupted the program's data so that when
> it tries to slow down it accelerates instead?). So we use a redundant,
> independent component (the monitor process) that's known to be in a
> consistent state to do the safe shutdown.

Yes, that's what's being done in the industry. But how to debug what happened? Perhaps leave a trace that explains what was being attempted, instead of what went wrong? I guess...

Or perhaps, if possible, decouple the faulty program and let it attempt to give more information in a sand box.

> I'm not sure how to apply this principle (or if it's even applicable) to
> all situations, but there you have it.

Yeah, this kind of discussion better be attached to how safety-critical the system is. Yes, in theory even a format() will not work but it's ingrained in our assert() expressions and we use it all over the place and it's very practical:

  assert(c, format!"Can this work? %s"(i));

In practice it works and is extremely useful.

Ali

A related issue is:

https://github.com/dlang/dmd/pull/6896

where `scope` should catch Exceptions, not Throwables.

On Thursday, 22 October 2020 at 16:34:14 UTC, Adam D. Ruppe wrote:
> and encourages ugly, unusable examples because they are run as automatic tests instead of you know, actually being written as examples.

Sorry but that's not a problem with documented unittests. They should be written like so:

///
struct S { }

///
unittest
{
    // show example of *simple* usage that is generated in the documentation
}

// NOT generated in the documentation, and can be heavy-duty and have contexts
unittest
{
}

The responsibility of writing good documentation unittests lies solely on the person introducing the new functionality (and the reviewers!). If people write terrible examples in `///` unittests then they write terrible documentation regardless. You can add the requirement for  the need to import into the module if you want (and this is a good idea, maybe we could abuse static for this), but in the end those same people will still abuse the feature. They'll add the necessary import, and still write terrible documentation examples. Can't fix bad documentation unless there's someone willing to review and point it out.