Jump to page: 1 2 3
Thread overview
D support for the Meson build system
Aug 21, 2016
Matthias Klumpp
Aug 22, 2016
jkpl
Aug 22, 2016
Matthias Klumpp
Aug 22, 2016
jkpl
Apr 07, 2017
kinke
Apr 08, 2017
Martin Nowak
Apr 10, 2017
Atila Neves
Apr 10, 2017
Russel Winder
Apr 10, 2017
Matthias Klumpp
Apr 10, 2017
Russel Winder
Apr 10, 2017
Matthias Klumpp
Apr 10, 2017
Russel Winder
Apr 10, 2017
Russel Winder
Apr 10, 2017
Matthias Klumpp
Apr 10, 2017
H. S. Teoh
Apr 10, 2017
Martin Nowak
Apr 11, 2017
Russel Winder
Apr 11, 2017
Atila Neves
Apr 11, 2017
Martin Nowak
Apr 07, 2017
Russel Winder
Apr 07, 2017
aberba
Apr 09, 2017
Martin Nowak
May 07, 2017
Andrew Godfrey
May 08, 2017
Matthias Klumpp
August 21, 2016
Hi!
Last week I was at this year's GUADEC conference and listened to a very interesting talk on the Meson build system[2] which is designed for very fast builds and as a much more modern replacement for Automake with a simple syntax.
In the past few days I added support for D (all three major compilers) to the Meson build system, with some great results-

While dub is awesome for simple and small projects, you usually want a bit more for bigger projects, especially if you are mixing D code with C bits or are linking against libraries written in C (which is what pretty much all the D code I write does excessively).
You also make the life of Linux distribution packagers much easier with this, since dub isn't really working well for us in that usecase (only one D package in Debian uses it at time, and only for an experiment I made to check whether using dub is viable. At time, dub is pretty high up on my D complaints list[3]).

So, should you consider adding Meson support for your project? That depends.
A few highlights of Meson include:
 * Very fast builds using the Ninja[4] build tool (which is also used by projects like Google Chrome) - for a project like Terminix, dub with LDC builds in 8.6s, while Meson and ninja take only 6s here. The change is even more dramatic with a lot more D source files.

 * Support for installing files, and generally good support for Linux distributions.

 * Mixing in parts written in C or C++ is very simple.

 * Depending on system libraries (Cairo, Libarchive, GTK+) is really easy, and you can also version the dependencies.

 * You can configure files at build time and also run arbitrary commands to e.g. generate additional code to compile in or to compile additional resources, which is needed for a lot of applications.

If you want to see a real world example on how to use Meson, check out my example patch for Terminix:
https://github.com/gnunn1/terminix/commit/01d11dcff7da4626a91cf0344897c4ff1783a69e

For simpler examples, check out the D unit tests for Meson:
https://github.com/mesonbuild/meson/tree/master/test%20cases/d

Meson supports building files for Visual Studio and is supposed to work on Windows as well. I developed the D pieces solely on Linux and couldn't test anything on Windows yet (but it would be interesting to know if this works).

D support will be available in Meson with the upcoming 0.34 release, which is when it will show up in Linux distributions like Ubuntu and Debian.
Until then, it's really easy to try Meson from Git master[5].

I hope you find this as useful as I do :-)

Cheers,
    Matthias

[1]: https://en.wikipedia.org/wiki/GNOME_Users_And_Developers_European_Conference
[2]: http://mesonbuild.com/
[3]: https://gist.github.com/ximion/77dda83a9926f892c9a4fa0074d6bf2b
[4]: https://ninja-build.org/
[5]: https://github.com/mesonbuild/meson

August 22, 2016
On Sunday, 21 August 2016 at 19:08:59 UTC, Matthias Klumpp wrote:
> for a project like Terminix, dub with LDC builds in 8.6s, while Meson and ninja take only 6s here.

Did you try to build with DUB but with WIFI or ethernet interface toggled off ?

August 22, 2016
On Monday, 22 August 2016 at 01:34:36 UTC, jkpl wrote:
> On Sunday, 21 August 2016 at 19:08:59 UTC, Matthias Klumpp wrote:
>> for a project like Terminix, dub with LDC builds in 8.6s, while Meson and ninja take only 6s here.
>
> Did you try to build with DUB but with WIFI or ethernet interface toggled off ?

No, but I obviously excluded any download times, and also dependency build-times to be fair to dub.

But, with network access toggled off:

Terminix:
  debug build:
    dub: 0m8.60s
    ninja: 0m6.36s
  release build (standard args):
    dub: 0m12.923s
    ninja: 0m11.740s

asgen:
 debug build:
    dub: 0m13.00s
    ninja: 0m13.04s
 release build:
    dub: 0m25.72s
    ninja: 0m19.19s

I ran these a few times, and the results were comparable, although I think if I would perform proper statistics the variance would be relatively high. Compiler was LDC 1.0.0 (on a Xeon E3-1231 v3).

One important thing about Ninja is that it will perform split-builds by default, so if you change something, only the changed piece needs to be recompiled. I *think* dub can do that too, but for some reason it never does it (even when using --parallel).
That behavior drastically reduces compile times when working on a project.
Also, Ninja knows about all the source files and generated files in advance, while dub needs to search for them on every run. That of course also has the drawback that one needs to specify the source files prior to building in the meson.build file.
August 22, 2016
On Monday, 22 August 2016 at 11:45:37 UTC, Matthias Klumpp wrote:
> On Monday, 22 August 2016 at 01:34:36 UTC, jkpl wrote:
>> On Sunday, 21 August 2016 at 19:08:59 UTC, Matthias Klumpp wrote:
>>> for a project like Terminix, dub with LDC builds in 8.6s, while Meson and ninja take only 6s here.
>>
>> Did you try to build with DUB but with WIFI or ethernet interface toggled off ?
> I ran these a few times, and the results were comparable, although I think if I would perform proper statistics the variance would be relatively high. Compiler was LDC 1.0.0 (on a Xeon E3-1231 v3).

Sorry I was thinking to something, that could have slowed down the DUB build, but this was obviouslt wrong.



April 07, 2017
On Sunday, 21 August 2016 at 19:08:59 UTC, Matthias Klumpp wrote:
[…]
> In the past few days I added support for D (all three major compilers) to the Meson build system, with some great results-
[…]
> I hope you find this as useful as I do :-)
[…]

Yes, I do , it's great.

Currently though there is apparently no integration between Meson and the Dub repository, i.e. it is not possible to get dependencies via Dub as purely a dependency manager and nothing to do with actual build.

<Should this thread switch to a list that is not Announce?>
April 07, 2017
On Monday, 22 August 2016 at 11:45:37 UTC, Matthias Klumpp wrote:
> One important thing about Ninja is that it will perform split-builds by default, so if you change something, only the changed piece needs to be recompiled. I *think* dub can do that too, but for some reason it never does it (even when using --parallel).
> That behavior drastically reduces compile times when working on a project.

So if I understand correctly, Meson/Ninja compile each module separately, while dub builds them all in a single command line.
This makes quite a big difference. Compiling all at once requires a full rebuild every time a single file is modified and implies full single-threadedness. The advantages are that each file is only loaded and parsed once, but much more importantly, everything is emitted into a single object file, allowing for implicit full cross-module inlining (across the compiled modules).
So while compiling each file separately in parallel is potentially much much faster, the produced release binary may be slower due to less/no cross-module inlining (e.g., LDC's option is still experimental and known to have issues).
April 07, 2017
On Sunday, 21 August 2016 at 19:08:59 UTC, Matthias Klumpp wrote:
> Hi!
> Last week I was at this year's GUADEC conference and listened to a very interesting talk on the Meson build system[2] which is designed for very fast builds and as a much more modern replacement for Automake with a simple syntax.
> In the past few days I added support for D (all three major compilers) to the Meson build system, with some great results-
>
> [...]

Seems like good news for us developing Linux apps.
April 08, 2017
On Friday, 7 April 2017 at 07:57:02 UTC, kinke wrote:
> So while compiling each file separately in parallel is potentially much much faster, the produced release binary may be slower due to less/no cross-module inlining (e.g., LDC's option is still experimental and known to have issues).

In fact single-module compilation is slower than package compilation most of the time due to the redundant parsing&semantic of common imports. As an extreme example, last time I tried, single-module compilation for gtkd was ~10x slower than compiling the library at once. Savings on recompilation hardly make up for this huge overhead.
Overall I'd recommend organizing and building subpackages when a project becomes too big.

April 09, 2017
On Sunday, 21 August 2016 at 19:08:59 UTC, Matthias Klumpp wrote:
> I hope you find this as useful as I do :-)

Yes, mature build systems for bigger projects are great.
Any opinion about/experience with [Bazel](https://bazel.build/)?
https://github.com/mesonbuild/meson/wiki/Comparisons#bazel
April 10, 2017
On Saturday, 8 April 2017 at 19:11:35 UTC, Martin Nowak wrote:
> On Friday, 7 April 2017 at 07:57:02 UTC, kinke wrote:
>> So while compiling each file separately in parallel is potentially much much faster, the produced release binary may be slower due to less/no cross-module inlining (e.g., LDC's option is still experimental and known to have issues).
>
> In fact single-module compilation is slower than package compilation most of the time due to the redundant parsing&semantic of common imports. As an extreme example, last time I tried, single-module compilation for gtkd was ~10x slower than compiling the library at once. Savings on recompilation hardly make up for this huge overhead.
> Overall I'd recommend organizing and building subpackages when a project becomes too big.

Per package is significantly faster.

As far as I know the only build system that does this by default for D is reggae.

Atila
« First   ‹ Prev
1 2 3