It's been a work-in-progress for half a year, but finished now:

http://johanengelen.github.io/ldc/2018/01/14/Fuzzing-with-LDC.html

"A not-so-well-written article about the fuzzing capability recently added to LDC, using LLVM’s libFuzzer. Compiling code with -fsanitize=fuzzer adds control-flow instrumentation used to guide the fuzzing and links-in the libFuzzer library that drives the fuzz testing (same as Clang). -fsanitize=fuzzer is available from LDC 1.4.0, not on Windows. LDC 1.6.0 was used for the examples in this article."

Hope the article gives you enough information to start fuzz testing your own projects.

cheers,
  Johan

(per Ali's request, the article has a nice fuzzy font for some of you :P)

On Saturday, 13 January 2018 at 23:59:52 UTC, Johan Engelen wrote:
> It's been a work-in-progress for half a year, but finished now:
>
> http://johanengelen.github.io/ldc/2018/01/14/Fuzzing-with-LDC.html
>
> "A not-so-well-written article about the fuzzing capability recently added to LDC, using LLVM’s libFuzzer. Compiling code with -fsanitize=fuzzer adds control-flow instrumentation used to guide the fuzzing and links-in the libFuzzer library that drives the fuzz testing (same as Clang). -fsanitize=fuzzer is available from LDC 1.4.0, not on Windows. LDC 1.6.0 was used for the examples in this article."
>
> Hope the article gives you enough information to start fuzz testing your own projects.
>
> cheers,
>   Johan
>
> (per Ali's request, the article has a nice fuzzy font for some of you :P)

Nice post, enjoyed reading it.  I need to look into trying out the shift sanitizer sometime.

On Sunday, 14 January 2018 at 17:33:45 UTC, Joakim wrote:
>
> Nice post, enjoyed reading it.  I need to look into trying out the shift sanitizer sometime.

Definitely would be *great* to have `-fsanitize=undefined` !

- Johan

On Saturday, 13 January 2018 at 23:59:52 UTC, Johan Engelen wrote:
> It's been a work-in-progress for half a year, but finished now:
>
> http://johanengelen.github.io/ldc/2018/01/14/Fuzzing-with-LDC.html
>
> "A not-so-well-written article about the fuzzing capability recently added to LDC

Argh, and just now I find out about:
- https://www.youtube.com/watch?v=k-Cv8Q3zWNQ
- https://github.com/google/oss-fuzz

Anybody interested in looking at OSS Fuzz?
"OSS-Fuzz - Continuous Fuzzing for Open Source Software
Status: Beta. We are now accepting applications from widely-used open source projects."

Cheers,
  Johan

On 1/13/2018 3:59 PM, Johan Engelen wrote:
> It's been a work-in-progress for half a year, but finished now:


It's now on Reddit and Hackernews:

  https://www.reddit.com/r/programming/comments/7qbpa3/fuzzing_d_code_with_ldc/

  https://news.ycombinator.com/item?id=16144860

(Don't click on the specific ycombinator link, or your upvotes will not be counted. Click on:

  https://news.ycombinator.com/news

instead and look for it.)

Also, such postings do a LOT better when they:

1. Include a synopsis of what the article is about. (I've added them.)

2. Include an "Ask me anything" posting from the author.

Not doing these things means the impact of the article is severely blunted from what it could be.

On Saturday, 13 January 2018 at 23:59:52 UTC, Johan Engelen wrote:
> It's been a work-in-progress for half a year, but finished now:
>
> http://johanengelen.github.io/ldc/2018/01/14/Fuzzing-with-LDC.html
>
> "A not-so-well-written article about the fuzzing capability recently added to LDC, using LLVM’s libFuzzer. Compiling code with -fsanitize=fuzzer adds control-flow instrumentation used to guide the fuzzing and links-in the libFuzzer library that drives the fuzz testing (same as Clang). -fsanitize=fuzzer is available from LDC 1.4.0, not on Windows. LDC 1.6.0 was used for the examples in this article."
>
> Hope the article gives you enough information to start fuzz testing your own projects.
>
> cheers,
>   Johan
>
> (per Ali's request, the article has a nice fuzzy font for some of you :P)

// `Throwable`s thrown are not bugs (in contrast to `Errors`).

They _can_ be bugs.
class Error : Throwable
class Exception : Throwable

On Monday, January 15, 2018 09:49:24 Temtaime via Digitalmars-d-announce wrote:
> On Saturday, 13 January 2018 at 23:59:52 UTC, Johan Engelen wrote:
> > It's been a work-in-progress for half a year, but finished now:
> >
> > http://johanengelen.github.io/ldc/2018/01/14/Fuzzing-with-LDC.html
> >
> > "A not-so-well-written article about the fuzzing capability recently added to LDC, using LLVM’s libFuzzer. Compiling code with -fsanitize=fuzzer adds control-flow instrumentation used to guide the fuzzing and links-in the libFuzzer library that drives the fuzz testing (same as Clang). -fsanitize=fuzzer is available from LDC 1.4.0, not on Windows. LDC 1.6.0 was used for the examples in this article."
> >
> > Hope the article gives you enough information to start fuzz testing your own projects.
> >
> > cheers,
> >
> >   Johan
> >
> > (per Ali's request, the article has a nice fuzzy font for some
> > of you :P)
>
> // `Throwable`s thrown are not bugs (in contrast to `Errors`).
>
> They _can_ be bugs.
> class Error : Throwable
> class Exception : Throwable

Talking about Throwables always gets a bit weird. Exceptions aren't bugs, whereas Errors are, and we usually talk about Exceptions and Errors rather than Throwables, and talking about Throwables implies both, but you _can_ also throw Throwables that aren't derived from Exception or Error (be it a naked Throwable or a class derived from Throwable). You're asking for it if you do, since there really isn't much difference between a Throwable and an Error in terms of how the compiler and runtime deal with clean-up code and nothrow, so trying to use a Throwable that isn't an Exception or Error can work, but it's basically going to be treated like an Error. Really, as far as the language and compiler are concerned, you have Exceptions, and then you have all other Throwables. nothrow has to do with Exceptions, and whether clean-up code is guaranteed to run or not has to do with Exceptions. Error really doesn't come into it except that we derive from Error rather than Throwable to more easily distinguish them from Exception. Ultimately, Errors and other non-Exception Throwables are really treated the same.

So, all-in-all, things just get weird when you try and talk about Throwables, and I'd generally advise against anyone trying to use any Throwables that aren't Exceptions or Errors. Certainly, if you do, you had better know what you're doing and be _very_ careful about it.

- Jonathan M Davis

On Monday, 15 January 2018 at 09:49:24 UTC, Temtaime wrote:
>
> // `Throwable`s thrown are not bugs (in contrast to `Errors`).
>
> They _can_ be bugs.
> class Error : Throwable
> class Exception : Throwable

Good catch ( ;-) ) thanks.

-Johan

On Sat, Jan 13, 2018 at 11:59:52PM +0000, Johan Engelen via Digitalmars-d-announce wrote:
> It's been a work-in-progress for half a year, but finished now:
> 
> http://johanengelen.github.io/ldc/2018/01/14/Fuzzing-with-LDC.html
[...]

It would be good to program custom fuzzers for Phobos modules, especially the ones with generic code like std.algorithm and std.range, a lot of which I suspect have unhandled edge cases and other such overlooked flaws.  Fuzzing at this level will be harder than merely fuzzing with random data; as input we'd need higher-level objects like ranges of various types and by-reference / by-value semantics.  And detecting problem cases may not be so easy (how to detect if, say, reduce() returned the wrong answer when the input is randomized?).


T

-- 
Beware of bugs in the above code; I have only proved it correct, not tried it. -- Donald Knuth