I'm working on the CI setup for iOS using Bitrise [1]. I'm currently hitting the 45 min build time limit. It takes around 11 minutes for building LDC itself and then running `ldc-build-runtime` times out, so I'm guessing it takes more than 34 minutes. The last completed object file is at the progress: 847/896, so it's almost complete.

Is there a way to reduce the build times? It looks like it compiles all files separately. Is it possible to compile multiple files at once? Perhaps that could reduce the build times. It also seems that there are four versions built: regular, debug, unittest and unittest-debug. Are all necessary? Can we avoid building some version(s)? Perhaps skip unittest and only build unittest-debug?

Since this is done using cross-compiling it might be possible to build the binaries on a different CI service and then trigger a build on Bitrise using their API to run the actual tests. But that would make it more complicated.

[1] https://bitrise.io

-- 
/Jacob Carlborg

On Tuesday, 17 March 2020 at 20:09:54 UTC, Jacob Carlborg wrote:
> Is there a way to reduce the build times? It looks like it compiles all files separately.

The regular libs are built all-at-once (by default), but the unittests require separate compilation (otherwise you need something like 24 GB of RAM). With ninja, you're using parallelization by default, but I'd still recommend to set the number of parallel jobs explicitly in the ldc-build-runtime cmdline. Each job should have about 2 GB of memory. What are the hardware specs for bitrise?

> Can we avoid building some version(s)? Perhaps skip unittest and only build unittest-debug?

Sure, if the release unittests take too long to compile, then go ahead and exclude them for now. A simple variant is to invoke ldc-build-runtime as usual, but without `--testrunners`, and instead invoke `ninja -j<N> druntime-test-runner-debug phobos2-test-runner-debug` manually in the ldc-build-runtime.tmp directory.

> Since this is done using cross-compiling it might be possible to build the binaries on a different CI service and then trigger a build on Bitrise using their API to run the actual tests. But that would make it more complicated.

Yeah, 45 minutes is pretty short (and 11 minutes to compile LDC very slow). An x86 simulator for Azure CI or so would be pretty much unrestricted (6h time-out...).

On 2020-03-17 22:33, kinke wrote:

> The regular libs are built all-at-once (by default), but the unittests require separate compilation (otherwise you need something like 24 GB of RAM). With ninja, you're using parallelization by default, but I'd still recommend to set the number of parallel jobs explicitly in the ldc-build-runtime cmdline. Each job should have about 2 GB of memory. What are the hardware specs for bitrise?

Not great:

Processor Speed: 2.58 GHz
Number of Processors: 1
Total Number of Cores: 2
L2 Cache (per Core): 256 KB
L3 Cache: 30 MB
Memory: 4 GB

> Sure, if the release unittests take too long to compile, then go ahead and exclude them for now. A simple variant is to invoke ldc-build-runtime as usual, but without `--testrunners`, and instead invoke `ninja -j<N> druntime-test-runner-debug phobos2-test-runner-debug` manually in the ldc-build-runtime.tmp directory.

I'll give that a try. Thanks.

> Yeah, 45 minutes is pretty short (and 11 minutes to compile LDC very slow). An x86 simulator for Azure CI or so would be pretty much unrestricted (6h time-out...).

Yeah. This is what you get in the free open source plan.

-- 
/Jacob Carlborg

On 2020-03-17 22:33, kinke wrote:

> Yeah, 45 minutes is pretty short (and 11 minutes to compile LDC very slow). An x86 simulator for Azure CI or so would be pretty much unrestricted (6h time-out...).

With the free plan for closed source projects you only get 10 minutes for each build. That won't get you far.

https://www.bitrise.io/pricing/teams

-- 
/Jacob Carlborg

On Wednesday, 18 March 2020 at 16:25:56 UTC, Jacob Carlborg wrote:
> On 2020-03-17 22:33, kinke wrote:
>
>> Yeah, 45 minutes is pretty short (and 11 minutes to compile LDC very slow). An x86 simulator for Azure CI or so would be pretty much unrestricted (6h time-out...).
>
> With the free plan for closed source projects you only get 10 minutes for each build. That won't get you far.
>
> https://www.bitrise.io/pricing/teams

A bit off-topic, but how does it compare to other services like Azure Pipelines, GitHub Actions, BuildKite, etc. that support self-hosted runners? (Also the first two have free macOS runners and Windows runners.) From a quick look through the page they indeed offer many features tailored to mobile dev, but at least for work, I wouldn't choose a CI solution that doesn't offer support for self-hosting runners.

On Thursday, 19 March 2020 at 10:22:31 UTC, Petar Kirov [ZombineDev] wrote:
> On Wednesday, 18 March 2020 at 16:25:56 UTC, Jacob Carlborg wrote:
>> On 2020-03-17 22:33, kinke wrote:
>>
>>> Yeah, 45 minutes is pretty short (and 11 minutes to compile LDC very slow). An x86 simulator for Azure CI or so would be pretty much unrestricted (6h time-out...).
>>
>> With the free plan for closed source projects you only get 10 minutes for each build. That won't get you far.
>>
>> https://www.bitrise.io/pricing/teams
>
> A bit off-topic, but how does it compare to other services like Azure Pipelines, GitHub Actions, BuildKite, etc. that support self-hosted runners? (Also the first two have free macOS runners and Windows runners.) From a quick look through the page they indeed offer many features tailored to mobile dev,

> but at least for work, I wouldn't choose a CI solution that doesn't offer support for self-hosting runners.

For work, I wouldn't choose any cloud solution at all. E.g., Azure has been a major pain in the ass the last week - python.exe not in PATH for some hours/days, afterwards PRs and pushes not triggering builds automatically for 2-3 days, and after that, I've had to create a new 'service connection' via GitHub PAT in order to successfully upload artifacts to the GitHub releases again. In the meantime, the MS apt servers like to fail from time to time due to checksum failures (yes, always the MS ones), so that `apt-get update` fails etc.

So if possible, I'd prefer to have an existing CI service be extended by a new iOS job, for a reduced maintenance burden. Azure's Mac VMs are quad-cores (unlike GitHub Actions for example, which only offer dual-cores; LDC compiles in under 4 mins) and ship with Xcode + simulators, see https://github.com/microsoft/azure-pipelines-image-generation/blob/master/images/macos/macos-10.14-Readme.md.

CircleCI (also with quad-core Mac VMs) would be another option; they have some test-related docs: https://circleci.com/docs/2.0/testing-ios/

On 2020-03-19 11:22, Petar Kirov [ZombineDev] wrote:

> A bit off-topic, but how does it compare to other services like Azure Pipelines, GitHub Actions, BuildKite, etc. that support self-hosted runners? (Also the first two have free macOS runners and Windows runners.) From a quick look through the page they indeed offer many features tailored to mobile dev,

The only reason I picked Bitrise is because it's the only CI service, that I could find at least, that offers testing on physical devices for free.

> but at least for work, I wouldn't choose a CI solution that doesn't offer support for self-hosting runners.

In this case I would say it's an advantage to not have to deal with self-hosting runners. It's a PITA to automate testing on physical devices. For example, I have no idea how the CI service (if they do) reset the devices after a test.

The only way I've been able to get an app on an device (without having to upload it somewhere) is through Xcode. The only way I've been able to run something on the device using the command line (which is required for automation) is to create an XCTest test (the testing framework used by Xcode). It's extremely unfriendly if you're not following the exact workflows that Apple have designed. Apple have proprietary protocols and private frameworks to deploy something to a device.

-- 
/Jacob Carlborg

On 2020-03-19 15:28, kinke wrote:

> So if possible, I'd prefer to have an existing CI service be extended by a new iOS job, for a reduced maintenance burden. Azure's Mac VMs are quad-cores (unlike GitHub Actions for example, which only offer dual-cores; LDC compiles in under 4 mins) and ship with Xcode + simulators, see https://github.com/microsoft/azure-pipelines-image-generation/blob/master/images/macos/macos-10.14-Readme.md. 

I had plan to setup a CI pipeline for testing on physical devices. See my reply to Petar [1]. I have actually not tested on the simulator yet.

[1] https://forum.dlang.org/post/r50det$aah$1@digitalmars.com

-- 
/Jacob Carlborg

You mean that CI compiles LDC and druntime right on iphone? I thought CPU clock looks like arm, but bitrise docs only say they have only macOS with simulator stack, that can't run on arm, can it?