What is the process for developing the dmd compiler?

Years ago, when I worked on the IBM PLX/PLAS compiler, we had a
large collection of test case files.

We would create two executable versions of the compiler.
One executable was the "good" version and the other version was
the 'GOOD' version with the developers changes.

We would run a script.

      1) first compiler runs the test cases.
      2) second compiler runs the test cases.
      3) each compiler generated its own unique directory structure
of results.
      4) the results can be compiler lists, assembly listings, etc
      5) the two directory structures are compared with a compare
utility
      6) the developer looks through the compare program output
      7) if the differences make sense, the changes are added to
the
code base.

pros
       helps reduce breakage of existing code when adding new
features

cons
       can slow down development looking through a bunch of compare
listings.

Our script was very complex, it allowed us to change compiler
settings and change what we wanted to compare and what was
generated by the compiler.
Also the compare utility had settings that allowed it to only put
out differences or differences with a couple of lines before and
after to help locate the cause of the differences. or the
developer could just go back and rerun the particular test case
if it needed further analysis.

Note:The dmd compiler is written in C++ not D, so you do not have
the luxury of builtin unit tests.

On Sunday, 1 April 2012 at 21:26:44 UTC, Tim Krimm wrote:
> What is the process for developing the dmd compiler?
>
> Years ago, when I worked on the IBM PLX/PLAS compiler, we had a
> large collection of test case files.
>
> We would create two executable versions of the compiler.
> One executable was the "good" version and the other version was
> the 'GOOD' version with the developers changes.
>
> We would run a script.
>
>       1) first compiler runs the test cases.
>       2) second compiler runs the test cases.
>       3) each compiler generated its own unique directory structure
> of results.
>       4) the results can be compiler lists, assembly listings, etc
>       5) the two directory structures are compared with a compare
> utility
>       6) the developer looks through the compare program output
>       7) if the differences make sense, the changes are added to
> the
> code base.
>
> pros
>        helps reduce breakage of existing code when adding new
> features
>
> cons
>        can slow down development looking through a bunch of compare
> listings.
>
> Our script was very complex, it allowed us to change compiler
> settings and change what we wanted to compare and what was
> generated by the compiler.
> Also the compare utility had settings that allowed it to only put
> out differences or differences with a couple of lines before and
> after to help locate the cause of the differences. or the
> developer could just go back and rerun the particular test case
> if it needed further analysis.
>
> Note:The dmd compiler is written in C++ not D, so you do not have
> the luxury of builtin unit tests.

Or in other words you did regression testing.

Regarding DMD - Walter has a test suite for it for the same purpose. I don't know the specifics of it as I never looked at it and not even sure whether it's open source or not. The core devs would be able to provide more specifics.

Please note that while Walter writes in c++ he has a unique style of it - some claim it's very D like.

On 2 April 2012 10:02, foobar <foo@bar.com> wrote:
> Or in other words you did regression testing.
>
> Regarding DMD - Walter has a test suite for it for the same purpose. I don't know the specifics of it as I never looked at it and not even sure whether it's open source or not. The core devs would be able to provide more specifics.
>
> Please note that while Walter writes in c++ he has a unique style of it - some claim it's very D like.

There are unit tests in D for almost all features and bug fixes. Sometimes there is a regression, if it's found a test for it is added when the fix is pushed, to make sure that it doesn't happen again. If the tests don't compile, that's very bad and all the tests have to pass before a release.

D's development is pretty solid, there are very, very, few breaking changes that aren't explicit and most people don't have the compiler suddenly pull the rug from underneath them (unless they were relying on the buggy behaviour).

In general, the actual output of the compiler doesn't matter that much, its the behaviour of the code that matters, so that is why the tests are in D, it doesn't matter if the codegen changes, as long as the software still functions the same. That is why we have several thousand tests for the compiler that check various behaviours.

--
James Miller

On Mon, Apr 02, 2012 at 12:02:56AM +0200, foobar wrote: [...]
> Regarding DMD - Walter has a test suite for it for the same purpose. I don't know the specifics of it as I never looked at it and not even sure whether it's open source or not. The core devs would be able to provide more specifics.

The source code is right here:

	https://github.com/D-Programming-Language/dmd

It comes with an extensive set of unit tests, which can be run by specifying the 'unittest' target for the makefile. The tests range from sources that should be compilable, sources that should fail compilation, sources that can be run, and win32-specific tests. I've run the tests before; they do take quite some time to complete because they're quite exhaustive.


> Please note that while Walter writes in c++ he has a unique style of it - some claim it's very D like.

Indeed.


T

-- 
People walk. Computers run.

James Miller:

> and most people don't have the compiler
> suddenly pull the rug from underneath them (unless they were relying on the buggy behaviour).

Relying on "buggy behavior" (or on deprecated features) is not rare. An example:
http://d.puremagic.com/issues/show_bug.cgi?id=7736

Fixing this will cause some problems, and the more we wait, the more problems it will cause (and this stuff _must_ be fixed):
http://d.puremagic.com/issues/show_bug.cgi?id=3934

There are several other similar examples. Like enforcing properties and requiring override, and generally using deprecated features like complex numbers. The more we wait now, the more pain will be felt later.

Walter now likes to design D features strict at first and then "loosen" them later, this has worked quite well (this has happened with inout and pure). But Walter has not applied this strategy since the beginning on "tags" as shown in issue 3934, that are managed in an very very sloppy way now, and will need to be tightened later to produce a decent enough language.

Bye,
bearophile

I was not being critical of D development, I was just curious.

BTW - I am the guy that started the discussion on creating a subset of D called "D-" or "embedded D", i.e. D without garbage collection.
D for the low resource, 32 bit microcontroller world.

I guess I was just trying to put "my 2 cents" in the discussions.


Note 1: I am a D novice but I have been watching the language for several years.

NOTE 2: I have been showing my support of the effort by buying Andrei's book and Walter's C++ CD

Oops, I think I had a "brain fart".

The compiler is written in C++, but the executable is DMD which compiles D code, which allows unit testing.

http://d.puremagic.com/test-results/