Hello,

Dub support was finally merged to the Meson's upstream.
For the ones that don't know, Meson[1] is a fast build system that uses ninja[2] as a backend. Until now it was impossible to use dependencies via the Dub and many many packages only supported Dub. Now you can import dependencies that already exist in Dub's local repo. You still need to fetch and build them though.

First, you need to fetch and build the dependency.

  dub fetch vibe-d
  dub build vibe-d

Then you just need to specify Dub as the dependency method.

  vibed_dep = dependency('vibe-d', method: 'dub')

Together with this, a 'dlang' module was also merged. You can use it to automatically generate a dub.json file.

Here's an example.

  dlang = import('dlang') # Import the module
  # dlang.generate_dub_file(project-name, path/to/where/to/install/the/dub/file,
  #                         arguments... )
  dlang.generate_dub_file(meson.project_name().to_lower(), meson.source_root(),
                          authors: 'Foo',
                          description: 'Bar',
                          copyright: 'Copyright © 2018, Foo',
                          license: 'MIT',
                          sourceFiles: 'foobar.d',
                          targetType: 'executable',
                          dependencies: vibed_dep
  )

If you have any issue with this, contact me at lains@archlinux.org.

Thank you,
Filipe Laíns

[1] https://github.com/mesonbuild/meson
[2] https://github.com/ninja-build/ninja

Great stuff.

I have failed to date finishing Dub support in SCons, I shall have to put more effort in now. :-)

The single biggest problem with D and Meson is that unit-threaded doesn't work with the way Meson works. Which is why I introduced the ProgramAllAtOnce builder into SCons.

If there is a plan to try and make D/unit-threaded/Meson all work together, count me in as a tester at the very least.

-- 
Russel.
===========================================
Dr Russel Winder      t: +44 20 7585 2200
41 Buckmaster Road    m: +44 7770 465 077
London SW11 1EN, UK   w: www.russel.org.uk

On Saturday, 4 August 2018 at 19:01:50 UTC, Russel Winder wrote:
> Great stuff.
>
> I have failed to date finishing Dub support in SCons, I shall have to put more effort in now. :-)
>
> The single biggest problem with D and Meson is that unit-threaded doesn't work with the way Meson works. Which is why I introduced the ProgramAllAtOnce builder into SCons.
>
> If there is a plan to try and make D/unit-threaded/Meson all work together, count me in as a tester at the very least.

I'll keep an eye out for that. Maybe I'll try to implement something if my free time allows it :)

PS.: Sorry for the grammar errors in the original post (mainly the "the Dub" :facepalm:), I rewrote some of the sentences and I didn't properly read the whole post again.

Thanks,
Filipe Laíns

On Saturday, 4 August 2018 at 16:07:45 UTC, Filipe Laíns wrote:
> Hello,
>
> Dub support was finally merged to the Meson's upstream.
> For the ones that don't know, Meson[1] is a fast build system that uses ninja[2] as a backend. Until now it was impossible to use dependencies via the Dub and many many packages only supported Dub. Now you can import dependencies that already exist in Dub's local repo. You still need to fetch and build them though.
>
> [...]

How does it track dependencies given that ninja needs functionality akin to gcc's to do that? Or does it always compile everything if any file changes?

reggae has to wrap dmd to be able to use ninja for anything. I can't see any other way but to add the necessary dependencies generation as you compile to dmd itself.

On 07-08-18 22:33, Atila Neves wrote:
> How does it track dependencies given that ninja needs functionality akin to gcc's to do that? Or does it always compile everything if any file changes?

It currently only tracks dependencies when using gdc, for dmd and ldc dmd pull 6961[1] would have to be merged so that support can be extended to the other compilers.

So when using ldc or dmd you will currently have to call 'ninja clean && ninja' to compile everything.

[1] https://github.com/dlang/dmd/pull/6961

-- 
Mike Wey

On Wednesday, 8 August 2018 at 17:26:39 UTC, Mike Wey wrote:
> On 07-08-18 22:33, Atila Neves wrote:
>> How does it track dependencies given that ninja needs functionality akin to gcc's to do that? Or does it always compile everything if any file changes?
>
> It currently only tracks dependencies when using gdc, for dmd and ldc dmd pull 6961[1] would have to be merged so that support can be extended to the other compilers.
>
> So when using ldc or dmd you will currently have to call 'ninja clean && ninja' to compile everything.
>
> [1] https://github.com/dlang/dmd/pull/6961

Makes sense, thanks for the info.

On Sat, 2018-08-04 at 16:07 +0000, Filipe Laíns via Digitalmars-d-announce wrote:
> Hello,
> 
> Dub support was finally merged to the Meson's upstream.
> For the ones that don't know, Meson[1] is a fast build system
> that uses ninja[2] as a backend. Until now it was impossible to
> use dependencies via the Dub and many many packages only
> supported Dub. Now you can import dependencies that already exist
> in Dub's local repo. You still need to fetch and build them
> though.

Sadly not in Meson 0.47.1 is a new release due soon?

> First, you need to fetch and build the dependency.
> 
>    dub fetch vibe-d
>    dub build vibe-d

This feels a little unsatisfactory. Dub (and soon SCons I hope) builds handle getting the dependency as well as building it and thence linking the project to it. Meson both depends on the dub executable and yet isn't doing the getting, it is assuming already got.

Of course Meson relies on all dependencies being present at build specification time, so nothing inconsistent, it just seems a wee bit incomplete.

I haven't finished reading the Meson source to find out which Dub built archive is used in the Meson build but if it is the one in ~/.dub/packages/<package>-<version>/<package> then stuff can go wrong since it is the result of the last build. In order to get the build with the specific compiler on the specific platform you have to delve into ~/.dub/packages/<package>-<version>/<package>/.dub/ and know the Dub naming strategy. For SCons I am having problems with this because of the extra stuff in the directory name.

> Then you just need to specify Dub as the dependency method.
> 
>    vibed_dep = dependency('vibe-d', method: 'dub')

I have been getting projects such as Unit-Threaded, DInotify, TinyEndian, and D_YAML to have Meson builds so as to create shared libraries prior to using pkg-config to handle D project being dependent on these libraries. It's working nicely (for me anyway) except Dub uses GitHub tags for versioning and Meson requires version numbers in the meson.build file. This duplication is getting quite annoying for all concerned. As is the Meson requirement to list all source files, rather than have a convention over configuration approach as Dub does – SCons has a different approach but still avoids explicit source lists. I can get round it with Meson, but I am informed it is naughty and should not be done.

These builds create shared objects and allow for dynamic linking. Using Dub built archives means static linking. Unless Meson somehow manages to get shared objects out of the Dub build.

Unless Dub can generate shared objects, having the dependencies build locally and creating pkg-config files actually seems a better way that accessing Dub builds.

But I may be missing something!

[…]
-- 
Russel.
===========================================
Dr Russel Winder      t: +44 20 7585 2200
41 Buckmaster Road    m: +44 7770 465 077
London SW11 1EN, UK   w: www.russel.org.uk

On Friday, 10 August 2018 at 14:38:10 UTC, Russel Winder wrote:
> On Sat, 2018-08-04 at 16:07 +0000, Filipe Laíns via Digitalmars-d-announce wrote:
>> Hello,
>> 
>> Dub support was finally merged to the Meson's upstream.
>> For the ones that don't know, Meson[1] is a fast build system
>> that uses ninja[2] as a backend. Until now it was impossible to
>> use dependencies via the Dub and many many packages only
>> supported Dub. Now you can import dependencies that already exist
>> in Dub's local repo. You still need to fetch and build them
>> though.
>
> Sadly not in Meson 0.47.1 is a new release due soon?

There is no fixed release schedule, but from experience, this might take a few months.

>> First, you need to fetch and build the dependency.
>> 
>>    dub fetch vibe-d
>>    dub build vibe-d
>
> This feels a little unsatisfactory. Dub (and soon SCons I hope) builds handle getting the dependency as well as building it and thence linking the project to it. Meson both depends on the dub executable and yet isn't doing the getting, it is assuming already got.
>
> Of course Meson relies on all dependencies being present at build specification time, so nothing inconsistent, it just seems a wee bit incomplete.

This is intentional, see this comment from Jussi: https://github.com/mesonbuild/meson/pull/3592#issuecomment-390421754

Unfortunately, this makes the dub feature almost useless for Linux distro integration.
However, I hope we might be able to implement reading dub.json files that are locally installed and incorporate D sources from the dependency into the regular Meson build process.

This would mean that Meson's dlang plugin would have to implement quite a bit of dub and dub.json parsing, which is some effort.
The benefit would be the creation of a near-seamless bridge between the dub and Meson worlds (you'd only have to locally install a dub package), that is also useful for Linux distribution packaging.

> [...]
> I have been getting projects such as Unit-Threaded, DInotify, TinyEndian, and D_YAML to have Meson builds so as to create shared libraries prior to using pkg-config to handle D project being dependent on these libraries. It's working nicely (for me anyway) except Dub uses GitHub tags for versioning and Meson requires version numbers in the meson.build file. This duplication is getting quite annoying for all concerned. As is the Meson requirement to list all source files, rather than have a convention over configuration approach as Dub does – SCons has a different approach but still avoids explicit source lists. I can get round it with Meson, but I am informed it is naughty and should not be done.

Jup, deterministic source lists are useful for split-building stuff efficiently. And globbing sources is possible with Meson, but highly discouraged (and even looks ugly).

> These builds create shared objects and allow for dynamic linking. Using Dub built archives means static linking. Unless Meson somehow manages to get shared objects out of the Dub build.
>
> Unless Dub can generate shared objects, having the dependencies build locally and creating pkg-config files actually seems a better way that accessing Dub builds.
> [...]

At the moment, I think for a final project, adding Meson files to dependencies still is the better way. However, for quickly testing out things and for building something locally, the new dub support is quite valuable.
Also, we can expand it in future, this is a first step :-)

On Mon, 2018-08-13 at 14:33 +0000, Matthias Klumpp via Digitalmars-d-announce
wrote:
[…]
> This is intentional, see this comment from Jussi: https://github.com/mesonbuild/meson/pull/3592#issuecomment-390421754

I am not entirely sure I agree with everything there, but he is BDFL of the project so his view wins.

> Unfortunately, this makes the dub feature almost useless for
> Linux distro integration.
> However, I hope we might be able to implement reading dub.json
> files that are locally installed and incorporate D sources from
> the dependency into the regular Meson build process.

I am rapidly heading this route for SCons: instead of using Dub to create the libraries just use it to get the sources and then compile the sources as a sub-project. However I still prefer having a Meson build for the library and building that way. If however there is a way of using dub.sdl or dub.json that would be interesting.

> This would mean that Meson's dlang plugin would have to implement quite a bit of dub and dub.json parsing, which is some effort. The benefit would be the creation of a near-seamless bridge between the dub and Meson worlds (you'd only have to locally install a dub package), that is also useful for Linux distribution packaging.

Perhaps there is a route whereby there can be a separate Python package to do this that both Meson and SCons can use? Otherwise it risks there being implementation of the same code for Meson and SCons separately.

[…]
> Jup, deterministic source lists are useful for split-building stuff efficiently. And globbing sources is possible with Meson, but highly discouraged (and even looks ugly).

I am not entirely convinced explicit manual lists are better, nor are they required for consistent builds. If the source files structure changes you have to create a new build anyway.

What is the ugly globbing way you know – I'm wondering if it better than the hack I have.

[…]
> At the moment, I think for a final project, adding Meson files to
> dependencies still is the better way. However, for quickly
> testing out things and for building something locally, the new
> dub support is quite valuable.
> Also, we can expand it in future, this is a first step :-)

I get the feeling Meson Rust/Cargo support is easier and better than D/Dub support, even though D seems to get more publicity in the Meson documentation.

I think a really good way forward for Meson (and analogously SCons) would be
to have the Dub dub.{json,sdl} file usable to create (and deploy if using
shared objects) a Meson (Scons) build of the dependency and then building the
project based on the created dependencies.

-- 
Russel.
===========================================
Dr Russel Winder      t: +44 20 7585 2200
41 Buckmaster Road    m: +44 7770 465 077
London SW11 1EN, UK   w: www.russel.org.uk

On Friday, 10 August 2018 at 14:38:10 UTC, Russel Winder wrote:
>> First, you need to fetch and build the dependency.
>> 
>>    dub fetch vibe-d
>>    dub build vibe-d
>
> This feels a little unsatisfactory. Dub (and soon SCons I hope) builds handle getting the dependency as well as building it and thence linking the project to it. Meson both depends on the dub executable and yet isn't doing the getting, it is assuming already got.

Yeah, Meson is a build system, not a package manager. This problem can be solved by simply creating a script that fetches and builds de dependencies. When creating a script, you might also want to compile the dependencies against the system's default D compiler. This way, you won't have any issues.

> I haven't finished reading the Meson source to find out which Dub built archive is used in the Meson build but if it is the one in ~/.dub/packages/<package>-<version>/<package> then stuff can go wrong since it is the result of the last build. In order to get the build with the specific compiler on the specific platform you have to delve into ~/.dub/packages/<package>-<version>/<package>/.dub/ and know the Dub naming strategy. For SCons I am having problems with this because of the extra stuff in the directory name.

It uses 'dub describe' to find the files, this also allows us to check the compiler. Meson won't even let you try to compile the project if there's an compiler mismatch.

> I have been getting projects such as Unit-Threaded, DInotify, TinyEndian, and D_YAML to have Meson builds so as to create shared libraries prior to using pkg-config to handle D project being dependent on these libraries. It's working nicely (for me anyway) except Dub uses GitHub tags for versioning and Meson requires version numbers in the meson.build file. This duplication is getting quite annoying for all concerned. As is the Meson requirement to list all source files, rather than have a convention over configuration approach as Dub does – SCons has a different approach but still avoids explicit source lists. I can get round it with Meson, but I am informed it is naughty and should not be done.

I mean, you can do this but as you said, it's not recommended.

> These builds create shared objects and allow for dynamic linking. Using Dub built archives means static linking. Unless Meson somehow manages to get shared objects out of the Dub build.
>
> Unless Dub can generate shared objects, having the dependencies build locally and creating pkg-config files actually seems a better way that accessing Dub builds.

Dub is implemented this way for a reason. We really *shouldn't* use dynamic libraries, at least for now.