For those doing compiler patches, I know it's a tough job, and thanks!

But please consider running the dmd test suite (and try to build/run phobos unittests) before issuing a pull request. There are a lot of interdependencies in the compiler, and the test suite is designed to flush them out. The cool thing is the test suite consists of cases that are already minimized! The suite also is designed to not take that long to run.

I run it constantly when I do dev on the compiler, and I constantly get dinged by it for something I overlooked. I consider the D test suite to be a major asset for us.

Please use it. If you've got problems using it, please post here and we'll try to help.

On 29 maj 2011, at 20:13, Walter Bright wrote:

> For those doing compiler patches, I know it's a tough job, and thanks!
> 
> But please consider running the dmd test suite (and try to build/run phobos unittests) before issuing a pull request. There are a lot of interdependencies in the compiler, and the test suite is designed to flush them out. The cool thing is the test suite consists of cases that are already minimized! The suite also is designed to not take that long to run.
> 
> I run it constantly when I do dev on the compiler, and I constantly get dinged by it for something I overlooked. I consider the D test suite to be a major asset for us.
> 
> Please use it. If you've got problems using it, please post here and we'll try to help.
> _______________________________________________
> dmd-internals mailing list
> dmd-internals at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/dmd-internals


I don't know how to run it. I've tried this, in the "test" directory:

* Running ./do_test.sh, results in:

/do_test.sh: line 106: /combinations: No such file or directory

The path to "combinations" seems to be wrong, why is it searching in "/"?


* Running "make", results in that dmd can't find druntime or phobos. I actually don't know how this is supposed to work with the different repositories. There isn't a dmd.conf file in the repository?

-- 
/Jacob Carlborg

My personal experience:
I run the test suite with only two combinations: no flags, and with -O
-inline -release.  It's only very obscure bugs that aren't detected
with one of those two (personally I've never encountered one, although
I believe they do exist; but failures which are OS-specific are more
common by at least a factor of ten). It takes three minutes to run,
and Phobos takes about two minutes. So it really should not be painful
to run the tests.

If people are not running the tests because they are finding it difficult or unpleasant, we need to work out why, and fix the problem. Please speak up!

-Don.

On 29 May 2011 20:13, Walter Bright <walter at digitalmars.com> wrote:
> For those doing compiler patches, I know it's a tough job, and thanks!
>
> But please consider running the dmd test suite (and try to build/run phobos unittests) before issuing a pull request. There are a lot of interdependencies in the compiler, and the test suite is designed to flush them out. The cool thing is the test suite consists of cases that are already minimized! The suite also is designed to not take that long to run.
>
> I run it constantly when I do dev on the compiler, and I constantly get dinged by it for something I overlooked. I consider the D test suite to be a major asset for us.
>
> Please use it. If you've got problems using it, please post here and we'll
> try to help.
> _______________________________________________
> dmd-internals mailing list
> dmd-internals at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/dmd-internals
>

On 5/29/11 8:57 PM, Jacob Carlborg wrote:
> * Running "make", results in that dmd can't find druntime or phobos. I actually don't know how this is supposed to work with the different repositories. There isn't a dmd.conf file in the repository?

Same here, on OS X 10.6. If I set the DMD variable to my system installation (which has a dmd.conf), I get:

---
Running runnable tests
make --no-print-directory run_runnable_tests
make[1]: *** No rule to make target `test_results/runnable/A16.d.out',
needed by `run_runnable_tests'.  Stop.
---

The problem is probably easily solvable, but I didn't have time to look into it yet.

David

On 5/29/2011 1:24 PM, David Nadlinger wrote:
> On 5/29/11 8:57 PM, Jacob Carlborg wrote:
>> * Running "make", results in that dmd can't find druntime or phobos. I actually don't know how this is supposed to work with the different repositories. There isn't a dmd.conf file in the repository?
> 
> Same here, on OS X 10.6. If I set the DMD variable to my system installation (which has a dmd.conf), I get:
> 
> ---
> Running runnable tests
> make --no-print-directory run_runnable_tests
> make[1]: *** No rule to make target `test_results/runnable/A16.d.out', needed by `run_runnable_tests'.  Stop.
> ---
> 
> The problem is probably easily solvable, but I didn't have time to look into it yet.
> 
> David

The auto-tester code documents how I've always built and run things.  The code can be found here:

https://github.com/braddr/d-tester/tree/master/client/src

On 29 May 2011 19:13, Walter Bright <walter at digitalmars.com> wrote:

> For those doing compiler patches, I know it's a tough job, and thanks!
>
> But please consider running the dmd test suite (and try to build/run phobos unittests) before issuing a pull request. There are a lot of interdependencies in the compiler, and the test suite is designed to flush them out. The cool thing is the test suite consists of cases that are already minimized! The suite also is designed to not take that long to run.
>
> I run it constantly when I do dev on the compiler, and I constantly get dinged by it for something I overlooked. I consider the D test suite to be a major asset for us.
>
> Please use it. If you've got problems using it, please post here and we'll try to help.
>

I advise you add a README.md to the root of the dmd repository with contents looking something like:
----
# DigitalMars D Compiler and Test Suite

Compiler and test suite for the D programming language - http://d-programming-language.org/

## Building

Replace $OS with windows, linux, osx, freebsd, solaris or openbsd depending on your operating system.

    cd src
    make -f $OS.mak

## Pull Requests

When creating pull requests, please make sure to run the test suite before! Even if it's a small change and you think it's not needed - you'd be surprised what a small change can break!

## Running the test suite

I actually don't know how to do this, but some instructions should go here. Also take a look at http://github.github.com/github-flavored-markdown/ for information on formatting in this file.
----

I could clean this up and make a pull request if you're interested? ;)

-- 
Robert
http://octarineparrot.com/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.puremagic.com/pipermail/dmd-internals/attachments/20110529/da709a4c/attachment.html>

On 29 May 2011 19:13, Walter Bright <walter at digitalmars.com> wrote:

> For those doing compiler patches, I know it's a tough job, and thanks!
>
> But please consider running the dmd test suite (and try to build/run phobos unittests) before issuing a pull request. There are a lot of interdependencies in the compiler, and the test suite is designed to flush them out. The cool thing is the test suite consists of cases that are already minimized! The suite also is designed to not take that long to run.
>
> I run it constantly when I do dev on the compiler, and I constantly get dinged by it for something I overlooked. I consider the D test suite to be a major asset for us.
>
> Please use it. If you've got problems using it, please post here and we'll try to help.
>

An idea relayed from Vladimir Panteleev: It could be a good idea to integrate pull requests with the automatic test runner - github has APIs for pull requests, the test runner could pull out the commits, test them, then post a comment on the pull request.

I see a few problems with this however:
 - Added load for the test runner (although, they could be run with low
priority and only when there aren't commits to master to test)
 - Security - if you're testing all pull requests some sort of sandbox would
be needed so the patch isn't running rm -rf or something more malicious
 - More commits can be added to pull requests later on, it would need
re-running in this case
 - The test suite could be altered to make it look like the pull request is
fine - although I guess anything that gets integrated will be looked over
anyway, so this isn't much of an issue.

-- 
Robert
http://octarineparrot.com/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.puremagic.com/pipermail/dmd-internals/attachments/20110530/4ed12f9b/attachment.html>

On 5/29/2011 4:07 PM, Robert Clipsham wrote:
> On 29 May 2011 19:13, Walter Bright <walter at digitalmars.com <mailto:walter at digitalmars.com>> wrote:
> 
>     For those doing compiler patches, I know it's a tough job, and thanks!
> 
>     But please consider running the dmd test suite (and try to build/run phobos unittests) before issuing a pull
>     request. There are a lot of interdependencies in the compiler, and the test suite is designed to flush them out. The
>     cool thing is the test suite consists of cases that are already minimized! The suite also is designed to not take
>     that long to run.
> 
>     I run it constantly when I do dev on the compiler, and I constantly get dinged by it for something I overlooked. I
>     consider the D test suite to be a major asset for us.
> 
>     Please use it. If you've got problems using it, please post here and we'll try to help.
> 
> 
> An idea relayed from Vladimir Panteleev: It could be a good idea to integrate pull requests with the automatic test runner - github has APIs for pull requests, the test runner could pull out the commits, test them, then post a comment on the pull request.
> 
> I see a few problems with this however:
>  - Added load for the test runner (although, they could be run with low priority and only when there aren't commits to
> master to test)
>  - Security - if you're testing all pull requests some sort of sandbox would be needed so the patch isn't running rm -rf
> or something more malicious
>  - More commits can be added to pull requests later on, it would need re-running in this case
>  - The test suite could be altered to make it look like the pull request is fine - although I guess anything that gets
> integrated will be looked over anyway, so this isn't much of an issue.
> 
> -- 
> Robert
> http://octarineparrot.com/

All of that could be dealt with, probably, but it still doesn't fix the underlying issue: developers should be running the tests on the code they ask to be pulled themselves, BEFORE the pull request is filed.  Not doing so puts potentially buggy code in the face of others to soon.

Automated testing could help with finding conflicts between pull requests or something ahead of it being merged.  But using it as a crutch to not test yourself isn't something I want to encourage.

Le 2011-05-29 ? 16:24, David Nadlinger a ?crit :

> On 5/29/11 8:57 PM, Jacob Carlborg wrote:
>> * Running "make", results in that dmd can't find druntime or phobos. I actually don't know how this is supposed to work with the different repositories. There isn't a dmd.conf file in the repository?
> 
> Same here, on OS X 10.6. If I set the DMD variable to my system installation (which has a dmd.conf), I get:
> 
> ---
> Running runnable tests
> make --no-print-directory run_runnable_tests
> make[1]: *** No rule to make target `test_results/runnable/A16.d.out', needed by `run_runnable_tests'.  Stop.
> ---
> 
> The problem is probably easily solvable, but I didn't have time to look into it yet.

I think the root of the problem is that it's not easy to setup things so that DMD, druntime, Phobos, and DMD's test suite are compiled together.

For my own use, I made an Xcode project that does all that (with some scripting that invokes the makefiles), but it took me some time to setup things. Perhaps what we need is a repository which would contain all those repositories (linked as submodules) along with the necessary code to make everything build. That way you can just clone that integration repository and compile everything in one go (and run all the tests) with no hassle. Perhaps I should create one...

-- 
Michel Fortin
michel.fortin at michelf.com
http://michelf.com/

On 2011-05-29 16:33, Michel Fortin wrote:
> Le 2011-05-29 ? 16:24, David Nadlinger a ?crit :
> > On 5/29/11 8:57 PM, Jacob Carlborg wrote:
> >> * Running "make", results in that dmd can't find druntime or phobos. I actually don't know how this is supposed to work with the different repositories. There isn't a dmd.conf file in the repository?
> > 
> > Same here, on OS X 10.6. If I set the DMD variable to my system installation (which has a dmd.conf), I get:
> > 
> > ---
> > Running runnable tests
> > make --no-print-directory run_runnable_tests
> > make[1]: *** No rule to make target `test_results/runnable/A16.d.out',
> > needed by `run_runnable_tests'.  Stop. ---
> > 
> > The problem is probably easily solvable, but I didn't have time to look into it yet.
> 
> I think the root of the problem is that it's not easy to setup things so that DMD, druntime, Phobos, and DMD's test suite are compiled together.
> 
> For my own use, I made an Xcode project that does all that (with some scripting that invokes the makefiles), but it took me some time to setup things. Perhaps what we need is a repository which would contain all those repositories (linked as submodules) along with the necessary code to make everything build. That way you can just clone that integration repository and compile everything in one go (and run all the tests) with no hassle. Perhaps I should create one...

I don't think that git can have sub-repositories like that. Certainly, git-svn can't deal with repositories that an svn repository references. You have to pull them all in individually and manage each of them individually. So, I don't think that having a repository which holds the others is going to work unless it's literally copying the code from the other repositories. It can't link them in.

- Jonathan M Davis