Hi all,

in case you have an afternoon or two to spare, here are two ideas how you could help with D compiler development (DMD/GDC/LDC):


 1) Provide an Open Source clean-room implementation of response_expand, the function the DMD frontend uses to parse response files. Unfortunately, it is under the copyright belongs (partly?) to Symantec, so Walter can't simply re-license it for use in GDC/LDC, where it is needed to provide DMD compatibility (gdmd/ldmd). For the full discussion, see: http://forum.dlang.org/thread/kdce69%24303k%241@digitalmars.com


 2) Write an ABI fuzzer: The C ABI can be quite complex to implement on some systems, notably x86_64 Posixen (i.e. the AMD64 System V ABI). There has been a number of issues in the past (see e.g. the infamous http://d.puremagic.com/issues/show_bug.cgi?id=5570), but while most of the cases frequently occurring in common C APIs are handled correctly now, there are still a number of issues remaining.

What makes working on the related compiler code annoying, at least for me, is that test cases are few - bugs comings from ABI issues are hard to track down for most non-compiler people - and cumbersome to write/build manually, as you need to integrate the D part with another one compiler by the host C compiler.

Thus, the idea is to write a tool which randomly generates function prototypes and creates both C and D 'caller' and 'callee' functions with that signature. The caller supplies a defined value for each parameter, and the callee in the respective other language checks if it was passed correctly. Same in the other direction for the return value.

The tool then compiles the files, links them together, and executes the test case. It will be beneficial for throughput to batch multiple function pairs together into one pair of source files. If any check fails, the test executable returns with an error code (or a segfault, ...), causing the fuzzer to save away the offending test case. If not, the next set of tests will be generated, and so on.

It might make sense to bias the parameter type generator towards "complex" types containing unions, packed structs, varargs, etc. In any case, it should be able to find issues on x86_64 in the latest released versions of both DMD (3-byte structs) and LDC (small structs passed as varargs when there are still registers available).


David

>  1) Provide an Open Source clean-room implementation of response_expand, the function the DMD frontend uses to parse response files. Unfortunately, it is under the copyright belongs (partly?) to Symantec, so Walter can't simply re-license it for use in GDC/LDC, where it is needed to provide DMD compatibility (gdmd/ldmd). For the full discussion, see: http://forum.dlang.org/thread/kdce69%24303k%241@digitalmars.com

Hey David,

This looks like an interesting little project,
however I'm unsure on copyright matters etc etc.

I read the source code in:
https://github.com/D-Programming-Language/dmd/blob/master/src/root/response.c

Could you explain what needs to be done exactly ?
e.g.:
  How should such a clean room implementation look like ??
  Does the structure need to change ??
  Is it enough to rewrite the code in D ??
  Should the code be in C if so how much change ??

Well just some thoughts, but the code looks pretty straight forward..

Gr,
Danny Arends

>   How should such a clean room implementation look like ??
>   Does the structure need to change ??
>   Is it enough to rewrite the code in D ??
>   Should the code be in C if so how much change ??

AFAIK "clean room implementation" means reimplementing it without looking at the source code of the original implementation.

How is anyone supposed to know what it does then ??

Also any new code needs to satisfy/adhere to the current interface,
so it would be pretty useless not knowing what that is
Furthermore to replace you need to know what to return on error / success

Gr,


On Sunday, 3 February 2013 at 14:17:11 UTC, jerro wrote:
>>  How should such a clean room implementation look like ??
>>  Does the structure need to change ??
>>  Is it enough to rewrite the code in D ??
>>  Should the code be in C if so how much change ??
>
> AFAIK "clean room implementation" means reimplementing it without looking at the source code of the original implementation.

Perhaps I should create an requirements outline after reading the source:
From the post David links to:

"2) Could somebody read the source and document the quirks of the
parser in painstaking detail, so that somebody else can do a
clean room implementation?"

Gr,
Danny

On Sunday, 3 February 2013 at 14:51:59 UTC, Danny Arends wrote:
> Perhaps I should create an requirements outline after reading the source:
> From the post David links to:
>
> "2) Could somebody read the source and document the quirks of the
> parser in painstaking detail, so that somebody else can do a
> clean room implementation?"
>
> Gr,
> Danny

Yes, this is a common practice for avoiding copyright issues - one man reads the source and provides detailed spec, second re-implements needed stuff based on the spec without having a single look into original sources.

On 02/03/2013 03:17 PM, jerro wrote:
> AFAIK "clean room implementation" means reimplementing it without looking at the
> source code of the original implementation.

So, is there spec for what it should do?

Am 03.02.2013 16:33, schrieb Joseph Rushton Wakeling:
> On 02/03/2013 03:17 PM, jerro wrote:
>> AFAIK "clean room implementation" means reimplementing it without
>> looking at the
>> source code of the original implementation.
>
> So, is there spec for what it should do?
>

Someone needs to write it, hence David's request.

On 02/03/2013 05:33 PM, Paulo Pinto wrote:
> Someone needs to write it, hence David's request.

I thought David's request was for an implementation, not a spec!  There's little point for those of us who haven't seen the original code to go and look at it to write a spec, when it merely takes away our ability to write something clean-room.

Am 03.02.2013 17:53, schrieb Joseph Rushton Wakeling:
> On 02/03/2013 05:33 PM, Paulo Pinto wrote:
>> Someone needs to write it, hence David's request.
>
> I thought David's request was for an implementation, not a spec!
> There's little point for those of us who haven't seen the original code
> to go and look at it to write a spec, when it merely takes away our
> ability to write something clean-room.

Why?

You just need one person to go through the trouble of reading the code
and writing the corresponding spec on the Wiki for example.

Another person in the community would then do a clean implementation from the Wiki page.

--
Paulo