On Tuesday, 8 September 2020 at 13:50:26 UTC, wjoe wrote:
> On Monday, 7 September 2020 at 10:41:50 UTC, wjoe wrote:
>>
>> Options I can think of are:
>> A) A Dockerfile for each case in (1.,) 2. and 3., or
>> B) A docker container which provides the environment to build GCC and a (Cirrus) CI config which defines the tasks to cover (1.,) 2. and 3.
>
>
> Small update.
> Option A) isn't possible with Cirrus CI because the check-out phase takes about 54 minutes on average and building the container about 6 minutes.
> Then the task is terminated on the 60 minutes mark (the default timeout) before it even starts the GCC configuration phase.
>
> The environment is 4GB RAM and dual CPU with 16 threads.

It is just doing a plain git clone?  Can you control whether checkout is done using --single-branch or --depth?

On Tuesday, 8 September 2020 at 14:18:10 UTC, Iain Buclaw wrote:
> On Tuesday, 8 September 2020 at 13:50:26 UTC, wjoe wrote:
>> On Monday, 7 September 2020 at 10:41:50 UTC, wjoe wrote:
>>>
>>> Options I can think of are:
>>> A) A Dockerfile for each case in (1.,) 2. and 3., or
>>> B) A docker container which provides the environment to build GCC and a (Cirrus) CI config which defines the tasks to cover (1.,) 2. and 3.
>>
>>
>> Small update.
>> Option A) isn't possible with Cirrus CI because the check-out phase takes about 54 minutes on average and building the container about 6 minutes.
>> Then the task is terminated on the 60 minutes mark (the default timeout) before it even starts the GCC configuration phase.
>>
>> The environment is 4GB RAM and dual CPU with 16 threads.
>
> It is just doing a plain git clone?  Can you control whether checkout is done using --single-branch or --depth?

This update was about a build via the zero configuration feature by just providing a Dockerfile.
Just to see if it works but it didn't.
I didn't find a git clone command in the log so I couldn't really tell.

A lot of things, such as RAM, timeout, custom git checkouts, custom command to build a docker container, environment variables, etc., can be configured in a cirrus configuration file.
So what I'm doing now is making a .cirrus.yml with a custom checkout, a shallow clone with --depth=1 of the master-ci branch should do the trick, using the container I made for building gcc.
Then I'll adapt the build-ci script for use with Cirrus-CI.
E.g. the dependency installation for instance isn't necessary by using a container that already provides all those.

I'm not sure if Johannes was referring to that script being a bad idea in hindsight. If so it's not a problem to define the necessary steps in the cirrus config.

On Tuesday, 8 September 2020 at 16:44:39 UTC, wjoe wrote:
> On Tuesday, 8 September 2020 at 14:18:10 UTC, Iain Buclaw wrote:
>> On Tuesday, 8 September 2020 at 13:50:26 UTC, wjoe wrote:
>>> On Monday, 7 September 2020 at 10:41:50 UTC, wjoe wrote:
>>>>
>>>> Options I can think of are:
>>>> A) A Dockerfile for each case in (1.,) 2. and 3., or
>>>> B) A docker container which provides the environment to build GCC and a (Cirrus) CI config which defines the tasks to cover (1.,) 2. and 3.
>>>
>>>
>>> Small update.
>>> Option A) isn't possible with Cirrus CI because the check-out phase takes about 54 minutes on average and building the container about 6 minutes.
>>> Then the task is terminated on the 60 minutes mark (the default timeout) before it even starts the GCC configuration phase.
>>>
>>> The environment is 4GB RAM and dual CPU with 16 threads.
>>
>> It is just doing a plain git clone?  Can you control whether checkout is done using --single-branch or --depth?
>
> This update was about a build via the zero configuration feature by just providing a Dockerfile.
> Just to see if it works but it didn't.
> I didn't find a git clone command in the log so I couldn't really tell.
>
> A lot of things, such as RAM, timeout, custom git checkouts, custom command to build a docker container, environment variables, etc., can be configured in a cirrus configuration file.
> So what I'm doing now is making a .cirrus.yml with a custom checkout, a shallow clone with --depth=1 of the master-ci branch should do the trick, using the container I made for building gcc.
> Then I'll adapt the build-ci script for use with Cirrus-CI.
> E.g. the dependency installation for instance isn't necessary by using a container that already provides all those.
>
> I'm not sure if Johannes was referring to that script being a bad idea in hindsight. If so it's not a problem to define the necessary steps in the cirrus config.

Well the ci script in the repo [1] should be used as a baseline, if not in its entirety as it's been used on enough to work in many environments, whether building a native or cross compiler.

At the very least, you just need to set-up the environment() using whatever variables Cirrus provides and it'll just go and run.

I think I do have a copy somewhere that adds support for running it locally without any dependencies on a specific CI.

[1] https://github.com/D-Programming-GDC/gcc/blob/master-ci/buildci.sh

On Tuesday, 8 September 2020 at 20:03:03 UTC, Iain Buclaw wrote:
> On Tuesday, 8 September 2020 at 16:44:39 UTC, wjoe wrote:
>> [...]
>
> Well the ci script in the repo [1] should be used as a baseline, if not in its entirety as it's been used on enough to work in many environments, whether building a native or cross compiler.
>
> At the very least, you just need to set-up the environment() using whatever variables Cirrus provides and it'll just go and run.
>
> I think I do have a copy somewhere that adds support for running it locally without any dependencies on a specific CI.
>
> [1] https://github.com/D-Programming-GDC/gcc/blob/master-ci/buildci.sh

Yes, that's the one I was talking about. I'll use that. Thanks.

On Tuesday, 8 September 2020 at 20:12:47 UTC, wjoe wrote:
> On Tuesday, 8 September 2020 at 20:03:03 UTC, Iain Buclaw wrote:
>> On Tuesday, 8 September 2020 at 16:44:39 UTC, wjoe wrote:
>>> [...]
>>
>> Well the ci script in the repo [1] should be used as a baseline, if not in its entirety as it's been used on enough to work in many environments, whether building a native or cross compiler.
>>
>> At the very least, you just need to set-up the environment() using whatever variables Cirrus provides and it'll just go and run.
>>
>> [...]

Except there's a problem with the installation of gcc-9. Apparently there's no Debian release providing a gcc-9 package.

The build of the container fails in step 4 on apt-get update with:

> Step 3/5 : RUN add-apt-repository -y ppa:ubuntu-toolchain-r/test
>  ---> Running in dedb85ee0ddd
> gpg: keybox '/tmp/tmpcn_fbb85/pubring.gpg' created
> gpg: /tmp/tmpcn_fbb85/trustdb.gpg: trustdb created
> gpg: key 1E9377A2BA9EF27F: public key "Launchpad Toolchain builds" imported
> gpg: Total number processed: 1
> gpg:               imported: 1
> Warning: apt-key output should not be parsed (stdout is not a terminal)
> gpg: no valid OpenPGP data found.
> Removing intermediate container dedb85ee0ddd
>  ---> 290665f0cc40
> Step 4/5 : RUN apt-get update     && apt-get install -y git autogen autoconf automake bison dejagnu     flex libcurl4-gnutls-dev libgmp-dev libisl-dev libmpc-dev     libmpfr-dev make patch tzdata xz-utils binutils libc6-dev gcc-9 g++-9     sudo curl     && rm -rf /var/lib/apt/lists/*
>  ---> Running in fcc6340c33b0
> Hit:1 http://deb.debian.org/debian buster InRelease
> Hit:2 http://deb.debian.org/debian buster-updates InRelease
> Ign:3 http://ppa.launchpad.net/ubuntu-toolchain-r/test/ubuntu groovy InRelease
> Hit:4 http://security.debian.org/debian-security buster/updates InRelease
> Err:5 http://ppa.launchpad.net/ubuntu-toolchain-r/test/ubuntu groovy Release
>   404  Not Found [IP: 91.189.95.83 80]
> Reading package lists...
> E: The repository 'http://ppa.launchpad.net/ubuntu-toolchain-r/test/ubuntu groovy Release' does not have a Release file.
> The command '/bin/sh -c apt-get update     && apt-get install -y git autogen autoconf automake bison dejagnu     flex libcurl4-gnutls-dev libgmp-dev libisl-dev libmpc-dev     libmpfr-dev make patch tzdata xz-utils binutils libc6-dev gcc-9 g++-9     sudo curl     && rm -rf /var/lib/apt/lists/*' returned a non-zero code: 100

When I let buildci.sh install the repositories instead it fails like so:

> ./buildci.sh setup
> gpg: keybox '/tmp/tmpc4p6pqmr/pubring.gpg' created
> gpg: /tmp/tmpc4p6pqmr/trustdb.gpg: trustdb created
> gpg: key 1E9377A2BA9EF27F: public key "Launchpad Toolchain builds" imported
> gpg: Total number processed: 1
> gpg:               imported: 1
> Warning: apt-key output should not be parsed (stdout is not a terminal)
> gpg: no valid OpenPGP data found.
> E: The repository 'http://ppa.launchpad.net/ubuntu-toolchain-r/test/ubuntu groovy Release' does not have a Release file.
> E: Unable to locate package gcc-9
> E: Unable to locate package g++-9> E: Couldn't find any package by regex 'g++-9'> E: Unable to locate package gdc-9> E: Unable to locate package pxz

This bug seems to be related https://bugs.launchpad.net/ubuntu/+source/wget/+bug/994097

Is there a strict dependency on Debian?

On Tuesday, 8 September 2020 at 23:48:58 UTC, wjoe wrote:
> On Tuesday, 8 September 2020 at 20:12:47 UTC, wjoe wrote:
>> On Tuesday, 8 September 2020 at 20:03:03 UTC, Iain Buclaw wrote:
>>> On Tuesday, 8 September 2020 at 16:44:39 UTC, wjoe wrote:
>>>> [...]
>>>
>>> Well the ci script in the repo [1] should be used as a baseline, if not in its entirety as it's been used on enough to work in many environments, whether building a native or cross compiler.
>>>
>>> At the very least, you just need to set-up the environment() using whatever variables Cirrus provides and it'll just go and run.
>>>
>>> [...]
>
> Except there's a problem with the installation of gcc-9. Apparently there's no Debian release providing a gcc-9 package.
>

This issue is resolved.

Small update.

I managed to install gcc-9 and g++-9 from the ubuntu toolchain ppa in the Docker container [1] and it's successfully built.
I instructed Cirrus CI to clone the repository like so
> git clone --branch=master-ci --depth=1 https://github.com/W-joe/gcc /tmp/cirrus-ci-build
and it reaches the configure stage in the buildci [2] script, however it fails like so:

./buildci.sh setup
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100 2327k  100 2327k    0     0  1961k      0  0:00:01  0:00:01 --:--:-- 1959k
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  1 1249k    1 13880    0     0  24139      0  0:00:52 --:--:--  0:00:52 24097
100 1249k  100 1249k    0     0  1620k      0 --:--:-- --:--:-- --:--:-- 1618k
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
 37  654k   37  242k    0     0   263k      0  0:00:02 --:--:--  0:00:02  263k
100  654k  100  654k    0     0   642k      0  0:00:01  0:00:01 --:--:--  642k
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100 1619k  100 1619k    0     0  2221k      0 --:--:-- --:--:-- --:--:-- 2221k
checking build system type... x86_64-pc-linux-gnu
checking host system type... x86_64-pc-linux-gnu
checking target system type... x86_64-pc-linux-gnu
checking for a BSD-compatible install... /usr/bin/install -c
checking whether ln works... yes
checking whether ln -s works... yes
checking for a sed that does not truncate output... /bin/sed
checking for gawk... no
checking for mawk... mawk
checking for libatomic support... yes
checking for libvtv support... yes
checking for libhsail-rt support... yes
checking for libphobos support... yes
checking for x86_64-linux-gnu-gcc... gcc-9
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc-9 accepts -g... yes
checking for gcc-9 option to accept ISO C89... none needed
> checking whether we are using the GNU C++ compiler... no
checking whether g++-9 accepts -g... no
checking whether g++ accepts -static-libstdc++ -static-libgcc... no
checking for x86_64-linux-gnu-gnatbind... no
checking for gnatbind... no
checking for x86_64-linux-gnu-gnatmake... no
checking for gnatmake... no
checking whether compiler driver understands Ada... no
checking how to compare bootstrapped objects... cmp --ignore-initial=16 $$f1 $$f2
checking whether g++-9 supports C++11 features by default... no
checking whether g++-9 supports C++11 features with -std=gnu++11... no
checking whether g++-9 supports C++11 features with -std=gnu++0x... no
checking whether g++-9 supports C++11 features with -std=c++11... no
checking whether g++-9 supports C++11 features with +std=c++11... no
checking whether g++-9 supports C++11 features with -h std=c++11... no
checking whether g++-9 supports C++11 features with -std=c++0x... no
checking whether g++-9 supports C++11 features with +std=c++0x... no
checking whether g++-9 supports C++11 features with -h std=c++0x... no
> configure: error: *** A compiler with support for C++11 language features is required.

It says that it's not using the GNU C++ compiler, why is that ?
Also, shouldn't g++-9 support C++11 ?

I suspect that maybe the compiler wasn't properly installed in the container ?

[1] https://github.com/w-joe/gcc/blob/master-ci/Dockerfile
[2] https://github.com/w-joe/gcc/blob/master-ci/buildci.sh

On Tuesday, 8 September 2020 at 23:48:58 UTC, wjoe wrote:
> On Tuesday, 8 September 2020 at 20:12:47 UTC, wjoe wrote:
>>> [...]
>
> Except there's a problem with the installation of gcc-9. Apparently there's no Debian release providing a gcc-9 package.
>
> The build of the container fails in step 4 on apt-get update with:
>
>> [...]
>
> When I let buildci.sh install the repositories instead it fails like so:
>
>> [...]
>
> This bug seems to be related https://bugs.launchpad.net/ubuntu/+source/wget/+bug/994097
>
> Is there a strict dependency on Debian?

Doesn't seem related, though the PPA is meant for Ubuntu distributions, not Debian.  The problem might be in incompatibilities between the two.

On Wednesday, 9 September 2020 at 11:22:14 UTC, Iain Buclaw wrote:
> On Tuesday, 8 September 2020 at 23:48:58 UTC, wjoe wrote:
>> On Tuesday, 8 September 2020 at 20:12:47 UTC, wjoe wrote:
>>>> [...]
>>
>> Except there's a problem with the installation of gcc-9. Apparently there's no Debian release providing a gcc-9 package.
>>
>> The build of the container fails in step 4 on apt-get update with:
>>
>>> [...]
>>
>> When I let buildci.sh install the repositories instead it fails like so:
>>
>>> [...]
>>
>> This bug seems to be related https://bugs.launchpad.net/ubuntu/+source/wget/+bug/994097
>>
>> Is there a strict dependency on Debian?
>
> Doesn't seem related, though the PPA is meant for Ubuntu distributions, not Debian.  The problem might be in incompatibilities between the two.

No, it's not. The issue seems to be that add-apt-repository adds a source for the current Ubuntu version which is groovy which doesn't provide a Release file. Some such. It works when I take the ppa from Ubuntu bionic.

Anyways, I managed to install gcc-9 and g++-9 from the ubuntu toolchain ppa in the Docker container [1] and it builds successfully.
I instructed Cirrus CI to clone the repository like so
> git clone --branch=master-ci --depth=1 https://github.com/W-joe/gcc /tmp/cirrus-ci-build
and it reaches the configure stage in the buildci [2] script, however it fails.

It says that it's not using the GNU C++ compiler, why is that ?
Also, shouldn't g++-9 support C++11 ?

I suspect that maybe the compiler wasn't properly installed in the container ?

This is the log:

./buildci.sh setup
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100 2327k  100 2327k    0     0  1961k      0  0:00:01  0:00:01 --:--:-- 1959k
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  1 1249k    1 13880    0     0  24139      0  0:00:52 --:--:--  0:00:52 24097
100 1249k  100 1249k    0     0  1620k      0 --:--:-- --:--:-- --:--:-- 1618k
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
 37  654k   37  242k    0     0   263k      0  0:00:02 --:--:--  0:00:02  263k
100  654k  100  654k    0     0   642k      0  0:00:01  0:00:01 --:--:--  642k
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100 1619k  100 1619k    0     0  2221k      0 --:--:-- --:--:-- --:--:-- 2221k
checking build system type... x86_64-pc-linux-gnu
checking host system type... x86_64-pc-linux-gnu
checking target system type... x86_64-pc-linux-gnu
checking for a BSD-compatible install... /usr/bin/install -c
checking whether ln works... yes
checking whether ln -s works... yes
checking for a sed that does not truncate output... /bin/sed
checking for gawk... no
checking for mawk... mawk
checking for libatomic support... yes
checking for libvtv support... yes
checking for libhsail-rt support... yes
checking for libphobos support... yes
checking for x86_64-linux-gnu-gcc... gcc-9
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc-9 accepts -g... yes
checking for gcc-9 option to accept ISO C89... none needed
checking whether we are using the GNU C++ compiler... no
checking whether g++-9 accepts -g... no
checking whether g++ accepts -static-libstdc++ -static-libgcc... no
checking for x86_64-linux-gnu-gnatbind... no
checking for gnatbind... no
checking for x86_64-linux-gnu-gnatmake... no
checking for gnatmake... no
checking whether compiler driver understands Ada... no
checking how to compare bootstrapped objects... cmp --ignore-initial=16 $$f1 $$f2
checking whether g++-9 supports C++11 features by default... no
checking whether g++-9 supports C++11 features with -std=gnu++11... no
checking whether g++-9 supports C++11 features with -std=gnu++0x... no
checking whether g++-9 supports C++11 features with -std=c++11... no
checking whether g++-9 supports C++11 features with +std=c++11... no
checking whether g++-9 supports C++11 features with -h std=c++11... no
checking whether g++-9 supports C++11 features with -std=c++0x... no
checking whether g++-9 supports C++11 features with +std=c++0x... no
checking whether g++-9 supports C++11 features with -h std=c++0x... no
configure: error: *** A compiler with support for C++11 language features is required.

[1] https://github.com/w-joe/gcc/blob/master-ci/Dockerfile
[2] https://github.com/w-joe/gcc/blob/master-ci/buildci.sh

On Wednesday, 9 September 2020 at 11:33:22 UTC, wjoe wrote:
> I suspect that maybe the compiler wasn't properly installed in the container ?

Maybe just a typo in your Dockerfile? You're installing `g++9`, but the package name is `g++-9`.