Hi guys,

I just started a D fuzzer. The goal is to randomly generate input source files in an attempt to crash the compiler. It's in very early stage at this point (less than 300 loc), but it already can crash gdc.

https://github.com/Ace17/defuzzed

Please let me know if you think this could be a valuable tool.
( I already know about dsmith (a fork of csmith), but I wasn't able to crash neither gdc nor dmd with it. I also aim for a simpler tool, and for a one written in D )

On 2 Apr 2016 7:30 pm, "Sebastien Alaiwan via D.gnu" <d.gnu@puremagic.com> wrote:
>
> Hi guys,
>
> I just started a D fuzzer. The goal is to randomly generate input source
files in an attempt to crash the compiler. It's in very early stage at this point (less than 300 loc), but it already can crash gdc.
>
> https://github.com/Ace17/defuzzed
>
> Please let me know if you think this could be a valuable tool.
> ( I already know about dsmith (a fork of csmith), but I wasn't able to
crash neither gdc nor dmd with it. I also aim for a simpler tool, and for a one written in D )
>

I expect many bug reports.  :-)

On Saturday, 2 April 2016 at 17:29:53 UTC, Sebastien Alaiwan wrote:
> Hi guys,
>
> I just started a D fuzzer. The goal is to randomly generate input source files in an attempt to crash the compiler. It's in very early stage at this point (less than 300 loc), but it already can crash gdc.
>
> https://github.com/Ace17/defuzzed
>
> Please let me know if you think this could be a valuable tool.
> ( I already know about dsmith (a fork of csmith), but I wasn't able to crash neither gdc nor dmd with it. I also aim for a simpler tool, and for a one written in D )

there is also generated from Brain Schott aka. Hackerpilot.
It may be useful to take a peek there.

On Sunday, 3 April 2016 at 12:08:32 UTC, Stefan Koch wrote:
> there is also generated from Brain Schott aka. Hackerpilot.
> It may be useful to take a peek there.

Thanks, this definitely is going to be a source of inspiration!
(I'm also looking at AFL (American Fuzzy Lop) - lots of great stuff in this tool).

Unlike 'generated', I aim to produce semantically valid random programs.

I might want to allow some degree of invalidity, as the same error could manifest itself as both 'ice-on-valid' and 'ice-on-invalid', but might be more easily revealed through 'ice-on-invalid'.

My long-term goal is the complete eradication of "ice-on-valid" errors from gdc :-)
(I plan to keep the tool compiler-agnostic).

Your ideas/contributions are welcome!

On Saturday, 2 April 2016 at 17:29:53 UTC, Sebastien Alaiwan wrote:
> Please let me know if you think this could be a valuable tool.

I am really thrilled to see effort being put towards more testing tools, and I'm sure that most other LDC devs are as well.

There are so many opportunities which remain unexplored simply because nobody took the time to actively try and break the current D compilers yet. ICE fuzz-testing like this is a first step, and I'm pretty sure that csmith-style comparison of executable results between the three compilers will yield lots of low-hanging fruit as well.

Please do keep the LDC community posted about your progress too!

 — David

On Sunday, 3 April 2016 at 17:05:56 UTC, David Nadlinger wrote:
> On Saturday, 2 April 2016 at 17:29:53 UTC, Sebastien Alaiwan wrote:
>> Please let me know if you think this could be a valuable tool.
>
> I am really thrilled to see effort being put towards more testing tools, and I'm sure that most other LDC devs are as well.
>
> There are so many opportunities which remain unexplored simply because nobody took the time to actively try and break the current D compilers yet. ICE fuzz-testing like this is a first step, and I'm pretty sure that csmith-style comparison of executable results between the three compilers will yield lots of low-hanging fruit as well.
>
> Please do keep the LDC community posted about your progress too!

+1 !

There is a "dsmith" tool out there?! Didn't know about it, that's great news!

On 4/3/2016 5:48 AM, Sebastien Alaiwan via D.gnu wrote:
> On Sunday, 3 April 2016 at 12:08:32 UTC, Stefan Koch wrote:
>> there is also generated from Brain Schott aka. Hackerpilot.
>> It may be useful to take a peek there.
>
> Thanks, this definitely is going to be a source of inspiration!
> (I'm also looking at AFL (American Fuzzy Lop) - lots of great stuff in
> this tool).
>
> Unlike 'generated', I aim to produce semantically valid random programs.
>
> I might want to allow some degree of invalidity, as the same error could
> manifest itself as both 'ice-on-valid' and 'ice-on-invalid', but might
> be more easily revealed through 'ice-on-invalid'.
>
> My long-term goal is the complete eradication of "ice-on-valid" errors
> from gdc :-)
> (I plan to keep the tool compiler-agnostic).
>
> Your ideas/contributions are welcome!

Also, given the shared frontend among dmd, gdc, and ldc, most of the bugs will likely be in that shared code and should result in bugs going to issues.dlang.org.

I wouldn't worry too much about valid vs invalid.  ICE's in general tend to be fairly easy to fix and have a pretty big negative impact on user experience.  So, both are very worth discovering and reporting.

Later,
Brad