On Wednesday, 9 September 2020 at 12:13:32 UTC, kinke wrote:
> On Wednesday, 9 September 2020 at 11:33:22 UTC, wjoe wrote:
>> I suspect that maybe the compiler wasn't properly installed in the container ?
>
> Maybe just a typo in your Dockerfile? You're installing `g++9`, but the package name is `g++-9`.

Ahh good catch! It works now! Thanks :)

On Wednesday, 9 September 2020 at 12:37:37 UTC, wjoe wrote:
> On Wednesday, 9 September 2020 at 12:13:32 UTC, kinke wrote:
>> On Wednesday, 9 September 2020 at 11:33:22 UTC, wjoe wrote:
>>> I suspect that maybe the compiler wasn't properly installed in the container ?
>>
>> Maybe just a typo in your Dockerfile? You're installing `g++9`, but the package name is `g++-9`.
>
> Ahh good catch! It works now! Thanks :)

The build as well as the unittests finished successfully.
The entire run took close to 70  minutes. This was a linux container with 4 CPUs and 10G RAM.

Which files should be kept once the task completed and what should happen with them ?
On success I could add a package task.

Next up is to test if Cirrus CI can handle the remaining platforms since the limits are lower than for linux containers. Unit tests almost hit the 9G RAM mark. Mac is a single core VM on community cluster.

Cron jobs aren't specified in the configuration file but in the Cirrus app setting on GitHub.

On Wednesday, 9 September 2020 at 18:32:07 UTC, wjoe wrote:
> On Wednesday, 9 September 2020 at 12:37:37 UTC, wjoe wrote:
>> On Wednesday, 9 September 2020 at 12:13:32 UTC, kinke wrote:
>>> On Wednesday, 9 September 2020 at 11:33:22 UTC, wjoe wrote:
>>>> I suspect that maybe the compiler wasn't properly installed in the container ?
>>>
>>> Maybe just a typo in your Dockerfile? You're installing `g++9`, but the package name is `g++-9`.
>>
>> Ahh good catch! It works now! Thanks :)
>
> The build as well as the unittests finished successfully.
> The entire run took close to 70  minutes. This was a linux container with 4 CPUs and 10G RAM.
>

Sounds about right.  There are a couple heavy modules that instantiate tens of thousands of functions when building phobos unittests.

> Which files should be kept once the task completed and what should happen with them ?
> On success I could add a package task.
>

There's 'make install'.  I probably wouldn't prune anything copied during that recipe, as you'll lose integration with C, C++ and LTO compilers if any of those components are missing.

> Next up is to test if Cirrus CI can handle the remaining platforms since the limits are lower than for linux containers. Unit tests almost hit the 9G RAM mark. Mac is a single core VM on community cluster.
>

You'll need to implement DSO handling on Darwin, there's a little bit of compiler support code, and the rest is in the library.  I've got a patch somewhere with maybe 90% of the work done.  From what I recall there was some weirdness with how dynamic loading works.

On Wednesday, 9 September 2020 at 18:32:07 UTC, wjoe wrote:
> Mac is a single core VM on community cluster.

Nope, it's a dual-core VM with hyperthreading and so 4 logical cores.

On Thursday, 10 September 2020 at 09:30:15 UTC, kinke wrote:
> On Wednesday, 9 September 2020 at 18:32:07 UTC, wjoe wrote:
>> Mac is a single core VM on community cluster.
>
> Nope, it's a dual-core VM with hyperthreading and so 4 logical cores.

All the better. For whatever reason the way I got it it's a VM with 1 physical core and hyperthreading = 2 logical CPUs :)
It's still sort of a meager setup. But there's hoping that it will be enough to complete the task within the 2 hours time limit.

On Friday, 11 September 2020 at 12:19:52 UTC, wjoe wrote:
> On Thursday, 10 September 2020 at 09:30:15 UTC, kinke wrote:
>> On Wednesday, 9 September 2020 at 18:32:07 UTC, wjoe wrote:
>>> Mac is a single core VM on community cluster.
>>
>> Nope, it's a dual-core VM with hyperthreading and so 4 logical cores.
>
> All the better. For whatever reason the way I got it it's a VM with 1 physical core and hyperthreading = 2 logical CPUs :)
> It's still sort of a meager setup. But there's hoping that it will be enough to complete the task within the 2 hours time limit.

You can set build_bootstrap="disable" within the buildci script so it only builds the compiler in one step.

On Wednesday, 9 September 2020 at 23:13:38 UTC, Iain Buclaw wrote:
> On Wednesday, 9 September 2020 at 18:32:07 UTC, wjoe wrote:
>> On Wednesday, 9 September 2020 at 12:37:37 UTC, wjoe wrote:
>>> On Wednesday, 9 September 2020 at 12:13:32 UTC, kinke wrote:
>>>> On Wednesday, 9 September 2020 at 11:33:22 UTC, wjoe wrote:
>>>>> I suspect that maybe the compiler wasn't properly installed in the container ?
>>>>
>>>> Maybe just a typo in your Dockerfile? You're installing `g++9`, but the package name is `g++-9`.
>>>
>>> Ahh good catch! It works now! Thanks :)
>>
>> The build as well as the unittests finished successfully.
>> The entire run took close to 70  minutes. This was a linux container with 4 CPUs and 10G RAM.
>>
>
> Sounds about right.  There are a couple heavy modules that instantiate tens of thousands of functions when building phobos unittests.
>
>> Which files should be kept once the task completed and what should happen with them ?
>> On success I could add a package task.
>>
>
> There's 'make install'.  I probably wouldn't prune anything copied during that recipe, as you'll lose integration with C, C++ and LTO compilers if any of those components are missing.

I would create a prefix for make install, install to that location, tar that folder and keep the tarball.

Also, on failure it would probably be a good idea to preserve the logs ?

The docs mention that it's possible to define GitHub actions, e.g. to email stuff. Would that be useful ?

On Friday, 11 September 2020 at 12:30:11 UTC, Iain Buclaw wrote:
> On Friday, 11 September 2020 at 12:19:52 UTC, wjoe wrote:
>> On Thursday, 10 September 2020 at 09:30:15 UTC, kinke wrote:
>>> On Wednesday, 9 September 2020 at 18:32:07 UTC, wjoe wrote:
>>>> Mac is a single core VM on community cluster.
>>>
>>> Nope, it's a dual-core VM with hyperthreading and so 4 logical cores.
>>
>> All the better. For whatever reason the way I got it it's a VM with 1 physical core and hyperthreading = 2 logical CPUs :)
>> It's still sort of a meager setup. But there's hoping that it will be enough to complete the task within the 2 hours time limit.
>
> You can set build_bootstrap="disable" within the buildci script so it only builds the compiler in one step.

I'll keep that in mind. Thanks.

On Friday, 11 September 2020 at 12:33:02 UTC, wjoe wrote:
> On Wednesday, 9 September 2020 at 23:13:38 UTC, Iain Buclaw wrote:
>> On Wednesday, 9 September 2020 at 18:32:07 UTC, wjoe wrote:
>>> On Wednesday, 9 September 2020 at 12:37:37 UTC, wjoe wrote:
>>>> On Wednesday, 9 September 2020 at 12:13:32 UTC, kinke wrote:
>>>>> On Wednesday, 9 September 2020 at 11:33:22 UTC, wjoe wrote:
>>>>>> I suspect that maybe the compiler wasn't properly installed in the container ?
>>>>>
>>>>> Maybe just a typo in your Dockerfile? You're installing `g++9`, but the package name is `g++-9`.
>>>>
>>>> Ahh good catch! It works now! Thanks :)
>>>
>>> The build as well as the unittests finished successfully.
>>> The entire run took close to 70  minutes. This was a linux container with 4 CPUs and 10G RAM.
>>>
>>
>> Sounds about right.  There are a couple heavy modules that instantiate tens of thousands of functions when building phobos unittests.
>>
>>> Which files should be kept once the task completed and what should happen with them ?
>>> On success I could add a package task.
>>>
>>
>> There's 'make install'.  I probably wouldn't prune anything copied during that recipe, as you'll lose integration with C, C++ and LTO compilers if any of those components are missing.
>
> I would create a prefix for make install, install to that location, tar that folder and keep the tarball.
>

Yes, that works, IIRC the same was done for the historical binary tarballs.

> Also, on failure it would probably be a good idea to preserve the logs ?
>

If stdout/stderr is accessible after build, then the relevant logs can just be cat'd.  For the testsuite, these be ./gcc/testsuite/gdc/gdc.log and ./x86_64-pc-linux-gnu/libphobos/testsuite/libphobos.sum (substitute the target where necessary).

> The docs mention that it's possible to define GitHub actions, e.g. to email stuff. Would that be useful ?

In the event of failed builds, or builds where the success/fail status changed, perhaps.  Having emails for every build would just be noise.

I've added the above tasks and they are reported to have been completed successfully in the Cirrus summary.
However, on a closer look I can spot multiple failures or files/directories which can't be found.
- Some 12 failed tests as well as 10 unresolved test cases in the unittest step.
- The build package step failed with: cd ./libcc1: no such file or directory.