Just to let you guys know - and to be sure no one is doing the same - I decided to go ahead and *start* writing an autotester that will fetch dmd nightly and unittest each dub package.

It will be using a classic master-worker architecture and will leverage docker containers.

I am aiming really low at first, but will eventually add things like memory usage, history, notifications, etc.

On 17/07/2016 8:34 AM, Sebastiaan Koppe wrote:
> Just to let you guys know - and to be sure no one is doing the same - I
> decided to go ahead and *start* writing an autotester that will fetch
> dmd nightly and unittest each dub package.
>
> It will be using a classic master-worker architecture and will leverage
> docker containers.
>
> I am aiming really low at first, but will eventually add things like
> memory usage, history, notifications, etc.

If you add nightly can you add x last major releases?

Also how about adding a 'button' for each one that says weather it passed or not and for which version of dmd.

On Saturday, 16 July 2016 at 20:34:49 UTC, Sebastiaan Koppe wrote:
> Just to let you guys know - and to be sure no one is doing the same - I decided to go ahead and *start* writing an autotester that will fetch dmd nightly and unittest each dub package.
>
> It will be using a classic master-worker architecture and will leverage docker containers.
>
> I am aiming really low at first, but will eventually add things like memory usage, history, notifications, etc.

I think that everybody will agree that's an excellent ideas to discover regressions. How do you plan to handle libraries that are not purely written in D (i.e requiring -L-lClib linker option) ? There are probably other cases where a build failure won't be significant.

On Saturday, 16 July 2016 at 20:34:49 UTC, Sebastiaan Koppe wrote:
> Just to let you guys know - and to be sure no one is doing the same - I decided to go ahead and *start* writing an autotester that will fetch dmd nightly and unittest each dub package.
>
> It will be using a classic master-worker architecture and will leverage docker containers.
>
> I am aiming really low at first, but will eventually add things like memory usage, history, notifications, etc.

Perhaps this code could also be used to find dub packages which are not currently compiling and mark them on code.dlang.org? Which is a feature which people have been asking for for a while.

On Sunday, 17 July 2016 at 04:28:54 UTC, rikki cattermole wrote:
> On 17/07/2016 8:34 AM, Sebastiaan Koppe wrote:
> If you add nightly can you add x last major releases?

Yeah, specially for dub, nightly is not that important.

> Also how about adding a 'button' for each one that says weather it passed or not and for which version of dmd.

You mean like a badge? That is possible. Of course dub shows all packages at once, so would have to coordinate to avoid a flood of requests.

On Sunday, 17 July 2016 at 04:47:40 UTC, Basile B. wrote:
> I think that everybody will agree that's an excellent ideas to discover regressions. How do you plan to handle libraries that are not purely written in D (i.e requiring -L-lClib linker option) ? There are probably other cases where a build failure won't be significant.

Besides installing often used dependencies in the build image, I don't know. Lets see how many there are and go from there.

On 17/07/2016 6:15 PM, Sebastiaan Koppe wrote:
> On Sunday, 17 July 2016 at 04:28:54 UTC, rikki cattermole wrote:
>> On 17/07/2016 8:34 AM, Sebastiaan Koppe wrote:
>> If you add nightly can you add x last major releases?
>
> Yeah, specially for dub, nightly is not that important.
>
>> Also how about adding a 'button' for each one that says weather it
>> passed or not and for which version of dmd.
>
> You mean like a badge? That is possible. Of course dub shows all
> packages at once, so would have to coordinate to avoid a flood of requests.

Yeah badges. With caching that shouldn't be too much of an issue.
If you use redirection to the actual badge, it shouldn't eat too much of bandwidth or cpu time. After all, the badge should be the same for dmd version + pass/fail.

On Sunday, 17 July 2016 at 06:19:16 UTC, Sebastiaan Koppe wrote:
> On Sunday, 17 July 2016 at 04:47:40 UTC, Basile B. wrote:
>> I think that everybody will agree that's an excellent ideas to discover regressions. How do you plan to handle libraries that are not purely written in D (i.e requiring -L-lClib linker option) ? There are probably other cases where a build failure won't be significant.
>
> Besides installing often used dependencies in the build image, I don't know. Lets see how many there are and go from there.

a new DUB property (in the package description) could solve this. This highly anticipated but in case your project becomes somewhat "official" there is this solution.

On Saturday, 16 July 2016 at 20:34:49 UTC, Sebastiaan Koppe wrote:
> Just to let you guys know - and to be sure no one is doing the same - I decided to go ahead and *start* writing an autotester that will fetch dmd nightly and unittest each dub package.
>
> It will be using a classic master-worker architecture and will leverage docker containers.
>
> I am aiming really low at first, but will eventually add things like memory usage, history, notifications, etc.

That would be really really great, especially if the unittests are also run in "release" build type.

On 2016-07-16 22:34, Sebastiaan Koppe wrote:
> Just to let you guys know - and to be sure no one is doing the same - I
> decided to go ahead and *start* writing an autotester that will fetch
> dmd nightly and unittest each dub package.
>
> It will be using a classic master-worker architecture and will leverage
> docker containers.
>
> I am aiming really low at first, but will eventually add things like
> memory usage, history, notifications, etc.

Why not using something existing, like GitLab? Although GitLab is a source code hosting system its CI is excellent. It uses a master-worker architecture as well, GitLab being the master and one or more runners. Runners are available for all major operating systems: macOS, Linux and Windows. For Linux a Docker runner is available. This way it could run tests on multiple platforms. The installation is straight forward with native packages.

-- 
/Jacob Carlborg