On Saturday, 7 July 2018 at 08:04:47 UTC, Timoses wrote:
> On Friday, 6 July 2018 at 23:56:01 UTC, Basile B. wrote:
>> On Friday, 6 July 2018 at 21:47:34 UTC, JN wrote:
>>>
>>> By the way, is there any policy for outdated dub packages?
>>
>> You just found an idea for the score algorithm.
>
> Why isn't there something like "compiler compatibility" in a dub config file? E.g. currently the vibe.d project lists the compiler versions the code is compatible with [1].
> Wouldn't a field in dub like
>
> "compilerCompatibility":
> {
>     "dmd": "2.080.0",
>     "ldc": "..."
> }
>
> be nice? This could also help code.dlang.org to show compatibility of libraries/packages.
>
> The only pain I guess would be to keep such a list up to date.
>
> [1]: https://github.com/vibe-d/vibe.d#support

I'm not sure it's a good idea to manually hard-code such a list into the dub.json.
However, for every release or so, the dub registry could crawl all its packages and run `dub test` on them. Of course, there's the chance that a build still fails (e.g. due to missing dependencies) even though it would pass locally, but each package that passes with the latest registry could receive additional points.

On Friday, 6 July 2018 at 21:25:28 UTC, tide wrote:
> On Friday, 6 July 2018 at 03:19:44 UTC, Seb wrote:
>> So learning from the recent Vibe.d regression fiasco (we temporarily disabled a subconfiguration in Vibe.d and promptly got a regression in 2.081), I think we should try to add more projects to the Project Tester.
>>
>> The current list is here:
>> https://github.com/dlang/ci/blob/master/vars/runPipeline.groovy#L443
>>
>> Any suggestions?
>>
>> Why should I add my project to the Project Tester?
>> --------------------------------------------------
>>
>> Once a project is added to the Project Tester, DMD can't regress on it anymore as for every PR at dmd, druntime, phobos, tools and dub the testsuite of the added projects are run.
>>
>> How does the Project Tester work?
>> ---------------------------------
>>
>> - By default, it will run the same commands as Travis would do. Although, if necessary, custom commands can be used too.
>> - It will checkout the latest, stable git tag
>>
>> Requirements
>> ------------
>>
>> - moderately popular or was prone to regressions in the past
>> - rather easy to build (i.e. you don't need to download and recompile clang)
>> - no flaky testsuite (random errors in the testsuite due to network connectivity shouldn't happen. Though there's `DETERMINISTIC_HINT=1` set, s.t. you could disable such parts of your testsuite)
>> - reachable author or development (if there's ever a case where we need to push changes via a trivial PR to the repo, it shouldn't sit in the queue for weeks)
>
> Include Windows as part of the testing done.

Just to avoid confusion, testing of DMD/Druntime/Phobos/etc. is done on Windows:

https://auto-tester.puremagic.com/platform-history.ghtml?projectid=1&os=Win_32
https://auto-tester.puremagic.com/platform-history.ghtml?projectid=1&os=Win_32_64
https://ci.appveyor.com/project/greenify/dmd
...

In my experience are the huge majority of all regressions originating in the D frontend where architecture actually doesn't matter so much.

That being said, we are currently investigating to move the Project Tester from Jenkins to Buildkite (https://github.com/dlang/ci/pull/225) and one advantage of this move is that it'll be a lot easier to add new machines to the build cloud (so you could then hook up that Windows machine in your garage with the Project Tester).
However, AFAICT many of the dub projects do have dependencies that are harder to install on Windows, so this might not happen in the near future.

On Friday, 6 July 2018 at 05:02:56 UTC, Joakim wrote:
> The LDC compiler? kinke recently had an issue because of all the C++ integration changes upstream:
>
> https://github.com/ldc-developers/ldc/pull/2752#issuecomment-398897813
>
> As perhaps the largest consumer of extern(C++), it may make sense to add it for all the C++ work being done. It would require the llvm package be pre-installed in the test environ.
>
> List of current projects looks great, was tough to think of anything to add.

Good idea. Not sure how easy/hard it is, but let's give it a try: https://github.com/dlang/ci/pull/228

On Thursday, 5 July 2018 21:19:44 MDT Seb via Digitalmars-d wrote:
> So learning from the recent Vibe.d regression fiasco (we temporarily disabled a subconfiguration in Vibe.d and promptly got a regression in 2.081), I think we should try to add more projects to the Project Tester.
>
> The current list is here: https://github.com/dlang/ci/blob/master/vars/runPipeline.groovy#L443
>
> Any suggestions?
>
> Why should I add my project to the Project Tester?
> --------------------------------------------------
>
> Once a project is added to the Project Tester, DMD can't regress on it anymore as for every PR at dmd, druntime, phobos, tools and dub the testsuite of the added projects are run.
>
> How does the Project Tester work?
> ---------------------------------
>
> - By default, it will run the same commands as Travis would do.
> Although, if necessary, custom commands can be used too.
> - It will checkout the latest, stable git tag
>
> Requirements
> ------------
>
> - moderately popular or was prone to regressions in the past
> - rather easy to build (i.e. you don't need to download and
> recompile clang)
> - no flaky testsuite (random errors in the testsuite due to
> network connectivity shouldn't happen. Though there's
> `DETERMINISTIC_HINT=1` set, s.t. you could disable such parts of
> your testsuite)
> - reachable author or development (if there's ever a case where
> we need to push changes via a trivial PR to the repo, it
> shouldn't sit in the queue for weeks)

dxml should probably be on the list. Assuming that this is only on *nix and not Windows (I have yet to write a test script for Windows), then ./test.sh will run the tests without -release, and ./testRelease.sh will run the tests with -release (simply running dub test won't do anything useful, because all of the unittest blocks are versioned to avoid problems when projects depending on dxml compile their tests).

- Jonathan M Davis

On Monday, 9 July 2018 at 05:58:17 UTC, Jonathan M Davis wrote:
> On Thursday, 5 July 2018 21:19:44 MDT Seb via Digitalmars-d wrote:
>> [...]
>
> dxml should probably be on the list.

Yes!
https://github.com/dlang/ci/pull/230

> Assuming that this is only on *nix and not Windows

Yes.

> (I have yet to write a test script for Windows)

This might help: https://github.com/Abscissa/AppVeyor-D

> then ./test.sh will run the tests without -release, and ./testRelease.sh will run the tests with -release (simply running dub test won't do anything useful, because all of the unittest blocks are versioned to avoid problems when projects depending on dxml compile their tests).

The Project Tester will by default look at (if existent) the .travis.yml and run the same 'script' commands, so it should do the right thing, but I will keep this in mind. Thanks for the explanation!

On Monday, 9 July 2018 02:44:31 MDT Seb via Digitalmars-d wrote:
> On Monday, 9 July 2018 at 05:58:17 UTC, Jonathan M Davis wrote:
> > then ./test.sh will run the tests without -release, and ./testRelease.sh will run the tests with -release (simply running dub test won't do anything useful, because all of the unittest blocks are versioned to avoid problems when projects depending on dxml compile their tests).
>
> The Project Tester will by default look at (if existent) the .travis.yml and run the same 'script' commands, so it should do the right thing, but I will keep this in mind. Thanks for the explanation!


Ah. Then I will have to figure out what needs to go in .travis.yml and add it.

- Jonathan M Davis