On 04/28/2010 03:24 PM, Bernard Helyer wrote:
> On 29/04/10 08:06, Andrei Alexandrescu wrote:
>> On 04/28/2010 02:53 PM, Walter Bright wrote:
>>>> I'm not sure to what extent segfaulting is detectable, but we definitely must find good ways to address that too.
>>>>
>>>
>>> Debuggers are the standard tool for that.
>>
>> I hear you but don't have one, and I swore to never use gdb. Ideally we should find a solution within the confines of the compiler.
>>
>> Andrei
>> __________
>
> But surely once a process has been sent SIGSEGV, we can't really recover from a generic handler? The process at that point is in an unknown state. The correct behaviour on SIGSEGV is to die.

I agree. But couldn't the process printf() a string before dying?

Andrei

On 28/04/10 21:54, Andrei Alexandrescu wrote:
>> But surely once a process has been sent SIGSEGV, we can't really recover from a generic handler? The process at that point is in an unknown state. The correct behaviour on SIGSEGV is to die.
>
> I agree. But couldn't the process printf() a string before dying?

There's a GNU library for handling this, libsigsegv ( http://libsigsegv.sourceforge.net/ ). I took the liberty of downloading and inspecting this, it's 5659 LoC to handle segfaults (So say, 3000 if we rewrote it in D :P). That's a lot of extra code to handle a segfault for unittests. Add to this not all platforms support it, and it needs reimplementing on every platform. If you want this code in druntime/phobos then you'll have to write it from scratch due to licensing issues and be unable to look at this well written/tested library as a reference... It seems like a lot of extra effort for something I maintain should be done with a debugger.

Not to mention the fact that printing a string before dying is as good as useless. You know the order the unittests are being run, as you wrote the code... From this you can figure out which case is failing (unless it's the first test of an unknown file, but again, debugger). You just need to look what the last test case was to run before the segfault. Alternatively you could get the unittester to do something along the lines of:
----
writef( "%s:%d: ", file, line );
fflush(stdout);
writefln( "%s", runTest() == true ? "success" : "fail" );
----
Which would allow you to see the test that was being run before it runs.

On 04/28/2010 04:17 PM, Robert Clipsham wrote:
> There's a GNU library for handling this, libsigsegv ( http://libsigsegv.sourceforge.net/ ). I took the liberty of downloading and inspecting this, it's 5659 LoC to handle segfaults (So say, 3000 if we rewrote it in D :P). That's a lot of extra code to handle a segfault for unittests. Add to this not all platforms support it, and it needs reimplementing on every platform. If you want this code in druntime/phobos then you'll have to write it from scratch due to licensing issues and be unable to look at this well written/tested library as a reference... It seems like a lot of extra effort for something I maintain should be done with a debugger.

Ouch.

> Not to mention the fact that printing a string before dying is as good as useless. You know the order the unittests are being run, as you wrote the code... From this you can figure out which case is failing (unless it's the first test of an unknown file, but again, debugger). You just need to look what the last test case was to run before the segfault.

The problem is scale. Phobos is pretty large already. I did make changes in unittests as you suggest above, but there are a lot of those to make. Besides, when compiling Phobos as one unit and unittesting everything, all unittests are run in one shot so you don't know about modules and stuff.

I improved that by painstakingly changing the build process to print and unittest one Phobos module at a time, while unittests are disabled in all other modules. That does help but is still far from perfect.

(Perfect: when it fails tell me where.)

> Alternatively you could get the unittester to do something along the lines of:
> ----
> writef( "%s:%d: ", file, line );
> fflush(stdout);
> writefln( "%s", runTest() == true ? "success" : "fail" );
> ----
> Which would allow you to see the test that was being run before it runs.

That would be too noisy.


Andrei

On Apr 28, 2010, at 4:47 PM, Robert Clipsham <robert at octarineparrot.com> wrote:

> On 28/04/10 21:36, Walter Bright wrote:
>> Yes, and you've been a great help in pushing these things along!
>
> Just doing my bit :) Since I started playing with D2 I've been using dmd, and the state of debugging wasn't any way near what I was used to... So I fixed it. I figured I may as well push for a couple of other debugging things to get sorted while my head was in debugging mode :) Of course having played with dmd I now find myself wanting to fix more issues ;)

As one who attempted to do quick surgical fixes and failed, I appreciate your efforts! I'm curious, how long did you spend researching before you could start making useful changes?

Every time I try peeking under the hood of dmd, I wish there was some kind of basic overview document. Even a few sentences at the top of a file indicating its purpose would be a great help. It's probably 10x better still if I stop acting so lazy and dedicate a few hours to getting a basic sense of what the code is doing!

Andrei Alexandrescu wrote:
>
> That's a bug. An assertion failure stops the current unittest and proceeds to the next one. Walter, I meant to mention this to you when you first described how you plan to change unittests, but I forgot.
>

It's debatable. Let's try this for a while and see how people like it.

On 28/04/10 22:53, Jason House wrote:
> As one who attempted to do quick surgical fixes and failed, I appreciate

That doesn't surprise me, I don't think there's such thing as a quick surgical fix in dmd, there's so many things you don't expect to break but do with even trivial changes, unless you know the bit of code you're working with well.

> your efforts! I'm curious, how long did you spend researching before you could start making useful changes?

It took me about 3 or 4 days to reverse engineer various parts of the DWARF info[1] to figure out how it worked, figure out how dmd wrote the DWARF info, and track down/fix the bug in total. Subsequent DWARF bugs took next to no time to track down and fix, as I'm not pretty familiar with that part of the codebase.

> Every time I try peeking under the hood of dmd, I wish there was some kind of basic overview document. Even a few sentences at the top of a file indicating its purpose would be a great help. It's probably 10x better still if I stop acting so lazy and dedicate a few hours to getting a basic sense of what the code is doing!

An overview document would be nice for a lot of people, it needs someone to write it though. I've spent a lot of time in the dmd front end/other compiler front ends over the past year or so, so I'm pretty familiar with the layout and how things work now, that said, most bugs would still take me a few days. I think the best way to figure out what's happening is to head to mars.c and read through main(), seeing how it works. You can skip the first half that parses options etc. This should give you a rough idea of how it all fits together, it will take some time though.

--
[1] It was mostly reverse engineered, I used the spec, the gdb source and dmd source as a reference... I tried reading the spec, it was going in one eye and out the other though... I found it far easier to reverse engineer it and use various sources as a reference.

On 28/04/10 22:45, Andrei Alexandrescu wrote:
> Ouch.

Ouch indeed.

> The problem is scale. Phobos is pretty large already. I did make changes in unittests as you suggest above, but there are a lot of those to make. Besides, when compiling Phobos as one unit and unittesting everything, all unittests are run in one shot so you don't know about modules and stuff.

This could be solved by named unit tests, or even getting druntime to output the name of the module before each test (although for projects with a lot of modules this could get spammy, as you've mentioned below).

> I improved that by painstakingly changing the build process to print and unittest one Phobos module at a time, while unittests are disabled in all other modules. That does help but is still far from perfect.

This is far from ideal, there should be a way to do this automatically, I agree.

> (Perfect: when it fails tell me where.)

What fails to tell you where?

> That would be too noisy.

I agree, it's far from ideal, there's not much of an option otherwise (as far as I can see) though without implementing a few thousand lines of code to handle a SIGSEGV or using a debugger though.

> Andrei

On 04/28/2010 05:16 PM, Robert Clipsham wrote:
>> (Perfect: when it fails tell me where.)
>
> What fails to tell you where?

I meant "tell" imperatively.

Perfect: when it fails, please tell me where.


Andrei

On 28/04/10 23:18, Andrei Alexandrescu wrote:
> On 04/28/2010 05:16 PM, Robert Clipsham wrote:
>>> (Perfect: when it fails tell me where.)
>>
>> What fails to tell you where?
>
> I meant "tell" imperatively.
>
> Perfect: when it fails, please tell me where.
>
>
> Andrei

My bad. It does this already for cases where it doesn't segfault, correct? Otherwise, we've already been through the options: use a debugger; add ~3000LoC for druntime; lots of spammy output.

On Apr 28, 2010, at 5:45 PM, Andrei Alexandrescu <andrei at erdani.com> wrote:

> The problem is scale. Phobos is pretty large already. I did make changes in unittests as you suggest above, but there are a lot of those to make. Besides, when compiling Phobos as one unit and unittesting everything, all unittests are run in one shot so you don't know about modules and stuff.
>
> I improved that by painstakingly changing the build process to print and unittest one Phobos module at a time, while unittests are disabled in all other modules. That does help but is still far from perfect.

I think there will be a lot of opinions on what should be done:
  ? print failing test(s)
  ? Module-level status like what you did
  ? Test-level status (test name + pass/fail) like someone else
suggested
  ? Test with other log messages
  ? Level based logging for customizable post-mortem review

As a reference, cruise control with nunit outputs a set of web pages. One is a top level showing each time the test suite was run and an overall success/failure indicator. Clicking on one brings up a summary page with number of tests, number passing tests, number of failed tests, number of skipped tests.  Further down is a summary of tests by category. Each one is colored based on %success, %failure, and %skipped. Clicking on one shows individual test results with failures first and skips second. Clicking on one of those tests shows the logs generated while running the test.

NUnit has a concept of test fixture setup, test setup, test teardown, and test fixture teardown. A test fixture with N tests would run test fixture and teardown once, but run test setup and teardown either 0 or N times. Test fixture setup failure skips all N tests. Test setup failure skips that one test.

I think providing overidable hooks for "starting test" and "finished test" is a 90% solution to letting people do whatever they deem best.