On Mon, 23 May 2011 07:41:22 +0300, Robert Jacques <sandford@jhu.edu> wrote:

> Well, all I can say is that it doesn't throw on my install. (Windows, DMD 2.052) for either the patched nor un-patched appender implementation. What version are you using?

2.053.

-- 
Best regards,
 Vladimir                            mailto:vladimir@thecybershadow.net

On Mon, 23 May 2011 00:48:07 -0400, Vladimir Panteleev <vladimir@thecybershadow.net> wrote:

> On Mon, 23 May 2011 07:41:22 +0300, Robert Jacques <sandford@jhu.edu> wrote:
>
>> Well, all I can say is that it doesn't throw on my install. (Windows, DMD 2.052) for either the patched nor un-patched appender implementation. What version are you using?
>
> 2.053.
>

Ah. I think I've tracked down the problem. The term is(Range == Unqual!(T)[]) used to be true for a string and char[], and now it's properly being evaluated to false, which in turn causes the wrong static if branch to compile. So yes, this is a regression bug. You can fix by replacing the conditional (at line 1594) with:

        // note, we disable this branch for appending one type of char to
        // another because we can't trust the length portion.
        static if (is(typeof(_data.arr[0..1] = items[0..1])) &&
                   is(typeof(items.length) == size_t))

Or you can try out my patch to appender:
http://d.puremagic.com/issues/show_bug.cgi?id=5813

This seems very cool. It's churning away at a DMD64 bug that's manifested in SDC. If it produces something usable, I'll be very impressed.

(I'm really only posting to see my name on the IRC DSpamBot :P)

On 20/05/2011 23:01, Vladimir Panteleev wrote:
> Inspired by Tigris Delta and the "Want to help DMD bugfixing? Write a
> simple utility" thread from digitalmars.D.learn. I hope the DMD
> development team will find this useful.
>
> Advantages over Tigris delta:
>
> * Easy to use (takes only two arguments, no need to fiddle with levels)
> * Readable output (comments and indentation are preserved)
> * Native support for multiple files (accepts a path to an entire
> directory for input)
> * Written for D
> * Written in D
> * Not written in Perl
> * Can recognize constructs such as try/catch, function invariants
> (in/out/body)
> * Only 440 lines of source code
>
> If you've never used delta: this is a tool which attempts to shrink
> files by deleting fragments iteratively, as long as the file satisfies a
> user-specified condition (for example, a specific error message when
> passed through the compiler).
>
> Usage:
>
> 1. Formulate a condition command, which should exit with a status code
> of 0 when DustMite is on the right track, and anything else otherwise.
> Example: dmd test.d 2>&1 | grep -qF "Assertion failed"
> 2. Place all the files that dustmite is to minimize in a new directory.
> 3. If you'd like to test your condition command, don't forget to clean
> up temporary files afterwards.
> 4. Run: dustmite path/to/directory test-command
> 5. After a while, dustmite will finish working and create
> path/to/directory.reduced
>
> I've tested it with a self-induced "bug" in std.datetime, it seems to
> work great. If you find that it breaks on something, let me know.
>
> https://github.com/CyberShadow/DustMite
>

Just attempted to use it, I get the following when trying to compile:

$ dmd -O -release -inline dustmite.d dsplit.d
dsplit.d(245): Error: undefined identifier module dsplit.replaceInPlace
dsplit.d(253): Error: undefined identifier module dsplit.replaceInPlace
dsplit.d(259): Error: undefined identifier module dsplit.replaceInPlace

With git revision 0699cca148a59beb4d4def44bea0e7b49f558d58.

-- 
Robert
http://octarineparrot.com/

On Mon, 23 May 2011 22:48:52 +0300, Robert Clipsham <robert@octarineparrot.com> wrote:

> Just attempted to use it, I get the following when trying to compile:
>
> $ dmd -O -release -inline dustmite.d dsplit.d
> dsplit.d(245): Error: undefined identifier module dsplit.replaceInPlace
> dsplit.d(253): Error: undefined identifier module dsplit.replaceInPlace
> dsplit.d(259): Error: undefined identifier module dsplit.replaceInPlace
>
> With git revision 0699cca148a59beb4d4def44bea0e7b49f558d58.

Please update your compiler and standard library?

-- 
Best regards,
 Vladimir                            mailto:vladimir@thecybershadow.net

On 23/05/2011 20:50, Vladimir Panteleev wrote:
> On Mon, 23 May 2011 22:48:52 +0300, Robert Clipsham
> <robert@octarineparrot.com> wrote:
>
>> Just attempted to use it, I get the following when trying to compile:
>>
>> $ dmd -O -release -inline dustmite.d dsplit.d
>> dsplit.d(245): Error: undefined identifier module dsplit.replaceInPlace
>> dsplit.d(253): Error: undefined identifier module dsplit.replaceInPlace
>> dsplit.d(259): Error: undefined identifier module dsplit.replaceInPlace
>>
>> With git revision 0699cca148a59beb4d4def44bea0e7b49f558d58.
>
> Please update your compiler and standard library?
>

Ah, that sorted it, thanks.

-- 
Robert
http://octarineparrot.com/

On 20/05/2011 23:01, Vladimir Panteleev wrote:
> I've tested it with a self-induced "bug" in std.datetime, it seems to
> work great. If you find that it breaks on something, let me know.

Wow, this tool works fantastically! Narrowed down a 5000 line test case to about 80 lines (I've managed to get it down to 26 lines by removing comments/make semantic changes there's no way it would be able to do without some sort of semantic analysis). I'd note that the code wasn't the most simple either, involving templates, mixins, ctfe, objects, etc. A couple of notes that we've already discussed, but I'll paste here for others:

 * The tool spent about 10 minutes figuring out if it was ok to delete files in .git, then deleting them - this pretty much doubled the time taken to run the tool!
 * It took me a while to figure out, the test command is run from the directory you pass on the command line.

You also score bonus points for being far simpler to set up than delta, and working *a lot* better than it (for D at least, I haven't tried it with anything else). Well done.

-- 
Robert
http://octarineparrot.com/

On Tue, 24 May 2011 00:04:39 +0300, Robert Clipsham <robert@octarineparrot.com> wrote:

> On 20/05/2011 23:01, Vladimir Panteleev wrote:
>> I've tested it with a self-induced "bug" in std.datetime, it seems to
>> work great. If you find that it breaks on something, let me know.
>
> Wow, this tool works fantastically! Narrowed down a 5000 line test case to about 80 lines (I've managed to get it down to 26 lines by removing comments/make semantic changes there's no way it would be able to do without some sort of semantic analysis). I'd note that the code wasn't the most simple either, involving templates, mixins, ctfe, objects, etc. A couple of notes that we've already discussed, but I'll paste here for others:
>
>   * The tool spent about 10 minutes figuring out if it was ok to delete files in .git, then deleting them - this pretty much doubled the time taken to run the tool!
>   * It took me a while to figure out, the test command is run from the directory you pass on the command line.
>
> You also score bonus points for being far simpler to set up than delta, and working *a lot* better than it (for D at least, I haven't tried it with anything else). Well done.

Thanks! I've improved it based on your and others' feedback, and wrote some documentation:

https://github.com/CyberShadow/DustMite/wiki


-- 
Best regards,
 Vladimir                            mailto:vladimir@thecybershadow.net

Vladimir Panteleev wrote:
> Inspired by Tigris Delta and the "Want to help DMD bugfixing? Write a simple utility" thread from digitalmars.D.learn. I hope the DMD development team will find this useful.
> 
> Advantages over Tigris delta:
> 
> * Easy to use (takes only two arguments, no need to fiddle with levels)
> * Readable output (comments and indentation are preserved)
> * Native support for multiple files (accepts a path to an entire directory for input)
> * Written for D
> * Written in D
> * Not written in Perl
> * Can recognize constructs such as try/catch, function invariants (in/out/body)
> * Only 440 lines of source code
> 
> If you've never used delta: this is a tool which attempts to shrink files by deleting fragments iteratively, as long as the file satisfies a user-specified condition (for example, a specific error message when passed through the compiler).
> 
> Usage:
> 
> 1. Formulate a condition command, which should exit with a status code of 0 when DustMite is on the right track, and anything else otherwise.
>    Example: dmd test.d 2>&1 | grep -qF "Assertion failed"
> 2. Place all the files that dustmite is to minimize in a new directory.
> 3. If you'd like to test your condition command, don't forget to clean up temporary files afterwards.
> 4. Run: dustmite path/to/directory test-command
> 5. After a while, dustmite will finish working and create path/to/directory.reduced
> 
> I've tested it with a self-induced "bug" in std.datetime, it seems to work great. If you find that it breaks on something, let me know.
> 
> https://github.com/CyberShadow/DustMite
> 

This is fantastic for ICE bugs. But it doesn't work well for certain types of bugs, such as the one I tracked down recently, which are of the form:
assert(foo()==30);
int foo()
{
   if (func1())
     return 0;
   return func2();
}
It reduces foo() to: int foo() { return 0; }
which causes the test to still fail, but the bug is gone.
The --noremove option doesn't help much because you have to name all of the functions which have that behaviour, and you don't know them in advance.

I was pretty surprised to find that reduced test cases still had comments in them. They should be the first things to be stripped out.

----

Something which has occured to me is that nearly always when I'm reducing a bug, the code which is actually getting run is close to minimal. So running with -cov and then removing any line with 0000 hits
would probably be pretty effective. Of course, you need to make sure it still compiles, which could be quite difficult. You could run -cov in the first run, and then when you slice up the file, you could mark sections as 'hit' (count>0), 'missed'(count==0), or 'void' (no count, could be a declaration, or a comment).
Generally any section with a 'hit' shouldn't be removed, but the test case isn't minimal until all 'missed' lines are gone. Most of the 'void' lines are probably removable too (it just won't compile, if they are necessary).

Don wrote:
> Vladimir Panteleev wrote:
>> Inspired by Tigris Delta and the "Want to help DMD bugfixing? Write a simple utility" thread from digitalmars.D.learn. I hope the DMD development team will find this useful.
>>
>> Advantages over Tigris delta:
>>
>> * Easy to use (takes only two arguments, no need to fiddle with levels)
>> * Readable output (comments and indentation are preserved)
>> * Native support for multiple files (accepts a path to an entire directory for input)
>> * Written for D
>> * Written in D
>> * Not written in Perl
>> * Can recognize constructs such as try/catch, function invariants (in/out/body)
>> * Only 440 lines of source code
>>
>> If you've never used delta: this is a tool which attempts to shrink files by deleting fragments iteratively, as long as the file satisfies a user-specified condition (for example, a specific error message when passed through the compiler).
>>
>> Usage:
>>
>> 1. Formulate a condition command, which should exit with a status code of 0 when DustMite is on the right track, and anything else otherwise.
>>    Example: dmd test.d 2>&1 | grep -qF "Assertion failed"
>> 2. Place all the files that dustmite is to minimize in a new directory.
>> 3. If you'd like to test your condition command, don't forget to clean up temporary files afterwards.
>> 4. Run: dustmite path/to/directory test-command
>> 5. After a while, dustmite will finish working and create path/to/directory.reduced
>>
>> I've tested it with a self-induced "bug" in std.datetime, it seems to work great. If you find that it breaks on something, let me know.
>>
>> https://github.com/CyberShadow/DustMite
>>
> 
> This is fantastic for ICE bugs. But it doesn't work well for certain types of bugs, such as the one I tracked down recently, which are of the form:
> assert(foo()==30);
> int foo()
> {
>    if (func1())
>      return 0;
>    return func2();
> }
> It reduces foo() to: int foo() { return 0; }
> which causes the test to still fail, but the bug is gone.
> The --noremove option doesn't help much because you have to name all of the functions which have that behaviour, and you don't know them in advance.
> 
> I was pretty surprised to find that reduced test cases still had comments in them. They should be the first things to be stripped out.
> 
> ----
> 
> Something which has occured to me is that nearly always when I'm reducing a bug, the code which is actually getting run is close to minimal. So running with -cov and then removing any line with 0000 hits
> would probably be pretty effective. Of course, you need to make sure it still compiles, which could be quite difficult. You could run -cov in the first run, and then when you slice up the file, you could mark sections as 'hit' (count>0), 'missed'(count==0), or 'void' (no count, could be a declaration, or a comment).
> Generally any section with a 'hit' shouldn't be removed, but the test case isn't minimal until all 'missed' lines are gone. Most of the 'void' lines are probably removable too (it just won't compile, if they are necessary).

Specific suggestion:
When dustmite does the initial slicing of the file, it should look for a matching .lst file. If it exists, then for any line in the .lst file where there is a '1'..'9' before the first non-numeric character on that line, that corresponding line in the .d file should be ineligible for deletion. (This is recursive; the function it's in shouldn't be deletable in its entirity).
Everything else should be unchanged.
Then, for these unstable bugs, the user just needs to compile with -cov first, and retain the .lst files in the directory.