So learning from the recent Vibe.d regression fiasco (we temporarily disabled a subconfiguration in Vibe.d and promptly got a regression in 2.081), I think we should try to add more projects to the Project Tester.

The current list is here:
https://github.com/dlang/ci/blob/master/vars/runPipeline.groovy#L443

Any suggestions?

Why should I add my project to the Project Tester?
--------------------------------------------------

Once a project is added to the Project Tester, DMD can't regress on it anymore as for every PR at dmd, druntime, phobos, tools and dub the testsuite of the added projects are run.

How does the Project Tester work?
---------------------------------

- By default, it will run the same commands as Travis would do. Although, if necessary, custom commands can be used too.
- It will checkout the latest, stable git tag

Requirements
------------

- moderately popular or was prone to regressions in the past
- rather easy to build (i.e. you don't need to download and recompile clang)
- no flaky testsuite (random errors in the testsuite due to network connectivity shouldn't happen. Though there's `DETERMINISTIC_HINT=1` set, s.t. you could disable such parts of your testsuite)
- reachable author or development (if there's ever a case where we need to push changes via a trivial PR to the repo, it shouldn't sit in the queue for weeks)

On Friday, 6 July 2018 at 03:19:44 UTC, Seb wrote:
> So learning from the recent Vibe.d regression fiasco (we temporarily disabled a subconfiguration in Vibe.d and promptly got a regression in 2.081), I think we should try to add more projects to the Project Tester.
>
> The current list is here:
> https://github.com/dlang/ci/blob/master/vars/runPipeline.groovy#L443
>
> Any suggestions?
>
> Why should I add my project to the Project Tester?
> --------------------------------------------------
>
> Once a project is added to the Project Tester, DMD can't regress on it anymore as for every PR at dmd, druntime, phobos, tools and dub the testsuite of the added projects are run.
>
> How does the Project Tester work?
> ---------------------------------
>
> - By default, it will run the same commands as Travis would do. Although, if necessary, custom commands can be used too.
> - It will checkout the latest, stable git tag
>
> Requirements
> ------------
>
> - moderately popular or was prone to regressions in the past
> - rather easy to build (i.e. you don't need to download and recompile clang)
> - no flaky testsuite (random errors in the testsuite due to network connectivity shouldn't happen. Though there's `DETERMINISTIC_HINT=1` set, s.t. you could disable such parts of your testsuite)
> - reachable author or development (if there's ever a case where we need to push changes via a trivial PR to the repo, it shouldn't sit in the queue for weeks)

The LDC compiler? kinke recently had an issue because of all the C++ integration changes upstream:

https://github.com/ldc-developers/ldc/pull/2752#issuecomment-398897813

As perhaps the largest consumer of extern(C++), it may make sense to add it for all the C++ work being done. It would require the llvm package be pre-installed in the test environ.

List of current projects looks great, was tough to think of anything to add.

On Friday, 6 July 2018 at 05:02:56 UTC, Joakim wrote:
> The LDC compiler? kinke recently had an issue because of all the C++ integration changes upstream:
>
> https://github.com/ldc-developers/ldc/pull/2752#issuecomment-398897813
>
> As perhaps the largest consumer of extern(C++), it may make sense to add it for all the C++ work being done. It would require the llvm package be pre-installed in the test environ.
>
> List of current projects looks great, was tough to think of anything to add.

Thats a great idea! Especially since there is a whole lot more extern(C++) stuff coming in the next DMD release.

On Friday, 6 July 2018 at 03:19:44 UTC, Seb wrote:
> So learning from the recent Vibe.d regression fiasco (we temporarily disabled a subconfiguration in Vibe.d and promptly got a regression in 2.081), I think we should try to add more projects to the Project Tester.
>
> The current list is here:
> https://github.com/dlang/ci/blob/master/vars/runPipeline.groovy#L443
>
> Any suggestions?
>
> Why should I add my project to the Project Tester?
> --------------------------------------------------
>
> Once a project is added to the Project Tester, DMD can't regress on it anymore as for every PR at dmd, druntime, phobos, tools and dub the testsuite of the added projects are run.
>
> How does the Project Tester work?
> ---------------------------------
>
> - By default, it will run the same commands as Travis would do. Although, if necessary, custom commands can be used too.
> - It will checkout the latest, stable git tag
>
> Requirements
> ------------
>
> - moderately popular or was prone to regressions in the past
> - rather easy to build (i.e. you don't need to download and recompile clang)
> - no flaky testsuite (random errors in the testsuite due to network connectivity shouldn't happen. Though there's `DETERMINISTIC_HINT=1` set, s.t. you could disable such parts of your testsuite)
> - reachable author or development (if there's ever a case where we need to push changes via a trivial PR to the repo, it shouldn't sit in the queue for weeks)

Include Windows as part of the testing done.

On Friday, 6 July 2018 at 03:19:44 UTC, Seb wrote:
> Why should I add my project to the Project Tester?
> --------------------------------------------------
>
> Once a project is added to the Project Tester, DMD can't regress on it anymore as for every PR at dmd, druntime, phobos, tools and dub the testsuite of the added projects are run.

By the way, is there any policy for outdated dub packages? There are some packages in dub registry, which don't build with the latest D version anymore. Should such packages stay in the registry? It gives a bit inflated view of packages, seems like there are many, but 50% (for example) aren't even usable without additional effort.

On Friday, 6 July 2018 at 21:47:34 UTC, JN wrote:
> On Friday, 6 July 2018 at 03:19:44 UTC, Seb wrote:
>> Why should I add my project to the Project Tester?
>> --------------------------------------------------
>>
>> Once a project is added to the Project Tester, DMD can't regress on it anymore as for every PR at dmd, druntime, phobos, tools and dub the testsuite of the added projects are run.
>
> By the way, is there any policy for outdated dub packages?

You just found an idea for the score algorithm.

On Friday, 6 July 2018 at 23:56:01 UTC, Basile B. wrote:
> On Friday, 6 July 2018 at 21:47:34 UTC, JN wrote:
>>
>> By the way, is there any policy for outdated dub packages?
>
> You just found an idea for the score algorithm.

Why isn't there something like "compiler compatibility" in a dub config file? E.g. currently the vibe.d project lists the compiler versions the code is compatible with [1].
Wouldn't a field in dub like

"compilerCompatibility":
{
    "dmd": "2.080.0",
    "ldc": "..."
}

be nice? This could also help code.dlang.org to show compatibility of libraries/packages.

The only pain I guess would be to keep such a list up to date.

[1]: https://github.com/vibe-d/vibe.d#support

On Friday, 6 July 2018 at 03:19:44 UTC, Seb wrote:
>
> Requirements
> ------------
>
> - moderately popular or was prone to regressions in the past
> - rather easy to build (i.e. you don't need to download and recompile clang)
> - no flaky testsuite (random errors in the testsuite due to network connectivity shouldn't happen. Though there's `DETERMINISTIC_HINT=1` set, s.t. you could disable such parts of your testsuite)
> - reachable author or development (if there's ever a case where we need to push changes via a trivial PR to the repo, it shouldn't sit in the queue for weeks)

Please include the intel-intrinsics package, its test expose a regression on 2.081 that I haven't looked after yet. The goal is to provide a _stable and usable_ SIMD interface.

On Friday, 6 July 2018 at 03:19:44 UTC, Seb wrote:
> So learning from the recent Vibe.d regression fiasco (we temporarily disabled a subconfiguration in Vibe.d and promptly got a regression in 2.081), I think we should try to add more projects to the Project Tester.
>

A meta-problem is that many package unittest (at least on typical CI) don't exercise the DMD optimizer since they don't run with -O or -release. Dub has no default options to dub test with optimizations so ti's very rarely tested.

On Saturday, 7 July 2018 at 16:29:15 UTC, Guillaume Piolat wrote:
> Please include the intel-intrinsics package, its test expose a regression on 2.081 that I haven't looked after yet. The goal is to provide a _stable and usable_ SIMD interface.

https://github.com/dlang/ci/pull/224