I've been battling DMD's Travis CI for the past couple of weeks with a few of my pull requests.  It often fails for various reasons that have nothing to do with my PRs (e.g. "The job exceeded the maximum time limit for jobs, and has been terminated.")  It's also takes almost half a day to complete.

What is the purpose of Travis CI?  What is it testing for that the much more reliable AutoTester isn't already testing for?  Can anything be done to make it more reliable and efficient?

Thanks,
Mike

On Sunday, 10 September 2017 at 01:25:29 UTC, Michael V. Franklin wrote:
> I've been battling DMD's Travis CI for the past couple of weeks with a few of my pull requests.  It often fails for various reasons that have nothing to do with my PRs (e.g. "The job exceeded the maximum time limit for jobs, and has been terminated.")  It's also takes almost half a day to complete.
>
> What is the purpose of Travis CI?  What is it testing for that the much more reliable AutoTester isn't already testing for?  Can anything be done to make it more reliable and efficient?
>
> Thanks,
> Mike

Travis CI is just a service that runs every-time that you submit a commit. The testing that's done is whatever the maintainers set up for it to run. There seem to be a few options, but Travis CI is the most popular. I suspect your issue is more with the tests that are running.

On balance, I think these autotesters are a positive. I might test code I submit on my machine, but these allow testing on a wide array of architectures and machines, which is a positive for everyone.

Of course, it's not always easy. Sometimes it's a little annoying to get the formatting just right. I certainly have had that issue recently, but I think I have more of a hang of what is required now.

More generally, I would acknowledge that they are not perfect and there is certainly room for improvement. For instance, I have a phobos PR with an X from codecov, which seems to be suggesting that I'm not testing all the functions. However, it seems to be counting member functions defined in the unittests as part of this and I only included those so that I could analyze them at compile-time.

On Sunday, 10 September 2017 at 01:25:29 UTC, Michael V. Franklin wrote:
> I've been battling DMD's Travis CI for the past couple of weeks with a few of my pull requests.  It often fails for various reasons that have nothing to do with my PRs (e.g. "The job exceeded the maximum time limit for jobs, and has been terminated.")  It's also takes almost half a day to complete.
>
> What is the purpose of Travis CI?

Testing DMD bootstrapped with other and older compilers.

> What is it testing for that the much more reliable AutoTester isn't already testing for?

The reason for using Travis is that
1) it's really hard to modify the configuration of the auto-tester (example: https://github.com/braddr/at-client/pull/3)
2) resources/machines at the auto-tester are limited and paid by Brad personally - Travis allows five jobs in parallel for free

> Can anything be done to make it more reliable and efficient?

I agree that these random failures are really annoying and actually there has been work to improve this. The latest effort are the dub registry mirrors which will be added by default to the upcoming dub 1.6, e.g.:

https://github.com/dlang/dub/pull/1190

The maximum time limit issue could be solved by reducing the amount of permutations tested even further.

On Sunday, 10 September 2017 at 01:25:29 UTC, Michael V. Franklin wrote:
> I've been battling DMD's Travis CI for the past couple of weeks with a few of my pull requests.  It often fails for various reasons that have nothing to do with my PRs (e.g. "The job exceeded the maximum time limit for jobs, and has been terminated.")  It's also takes almost half a day to complete.

The command "source "$(CURL_USER_AGENT="$CURL_USER_AGENT" bash install.sh dmd --activate)"" failed and exited with 1 during .

Yet another error.