On Fri, Aug 23, 2013 at 11:07 AM, Walter Bright <newshound2@digitalmars.com>wrote:

> That's exactly the problem. If these large projects are incorporated into the autotester, who is going to isolate/fix problems arising with them?
>
> The test suite is designed to be a collection of already-isolated issues, so understanding what went wrong shouldn't be too difficult. Note that already it is noticeably much harder to debug a phobos unit test gone awry than the other tests. A full blown project that nobody understands would fare far worse.
>

AFAIR both Clang and GCC have entire third-party projects in their test suite. I know that at least SQLite is part of both, and that's a pretty big project. If I recall correctly, GCC releases are blocked on successfully compiling the Linux kernel, all of Firefox and I possibly Qt. The third-party project tests need to finish without failures as well.

My recollection is a bit vague here though.

Now, do they compile and run all the tests for these projects on every commit or make sure nothing has broken just before making a new release, I don't know. But I do know that it's the latter at the very least.

On 23/08/13 20:38, H. S. Teoh wrote:
> One idea that occurred to me is to put large external projects under a
> separate tester, not bound to the core dmd/druntime/phobos autotesting,
> but an independent tester that regularly checks out git HEAD and
> compiles & tests said large projects.

I proposed something along these lines shortly after DConf:
http://forum.dlang.org/thread/mailman.47.1369319426.13711.digitalmars-d@puremagic.com

I thought it could be useful both as a stability tester _and_ as a means to evaluate the prospective impact of deliberately breaking changes.

It was quite well received as an idea but is probably a big job to take on ... :-(

On Saturday, 24 August 2013 at 07:50:00 UTC, Joseph Rushton Wakeling wrote:
> I proposed something along these lines shortly after DConf:
> http://forum.dlang.org/thread/mailman.47.1369319426.13711.digitalmars-d@puremagic.com
>
> I thought it could be useful both as a stability tester _and_ as a means to evaluate the prospective impact of deliberately breaking changes.
>
> It was quite well received as an idea but is probably a big job to take on ... :-(

I do want to contribute one once I decide what I want to do about a more powerful server (such suite is a bit too hard for my small VPS). Pretty sure something can be done this year, but will take some time (months+).

On 25/08/13 21:57, Dicebot wrote:
> I do want to contribute one once I decide what I want to do about a more
> powerful server (such suite is a bit too hard for my small VPS). Pretty sure
> something can be done this year, but will take some time (months+).

Is this something where we can do some crowdfunding?  Might be worth trying to invest in some server/cloud infrastructure for D projects like this that have collective benefit.

On Sunday, 25 August 2013 at 20:02:49 UTC, Joseph Rushton Wakeling wrote:
> On 25/08/13 21:57, Dicebot wrote:
>> I do want to contribute one once I decide what I want to do about a more
>> powerful server (such suite is a bit too hard for my small VPS). Pretty sure
>> something can be done this year, but will take some time (months+).
>
> Is this something where we can do some crowdfunding?  Might be worth trying to invest in some server/cloud infrastructure for D projects like this that have collective benefit.

Well, I wanted to do the move anyway for my own needs so it is just a planned side effect. It is not that costly, I simply don't have time to do the move properly.