On Tuesday, 16 April 2019 at 01:45:26 UTC, 9il wrote:
> Hi all,
>
> 'DUB - call to arms' post by Seb
> https://forum.dlang.org/post/unimbbqxosxcmjdhtzch@forum.dlang.org
>
> Hope we will move other direction and extend Meson build system with better D support and make a new Dlang registry.

Why would we need a new registry? Even if we did, how would we convince the existing users that switching would be in their best interests?

>
> Reasons to move forward with Meson:
>
> 0. Progressive and large community
> 1. Has clear python source code: faster to extend, easier to maintain.

This wouldn't be too important since said extension should really only be done once.

> 2. D+Meson already used in production: last year it is a major build system for all of Mir and its commercial applications.

My usage of meson with D revealed how bad its dub integration was.

> 3. Meson language is super cool. It is much better than dub for commercial projects, for large projects, for library distribution.

Anything is better than dub as a build system, including raw make.

> 4. Ability to compile and configure D code with other languages like C/C++.

AFAIK, reggae, CMake and Premake can all do this as well.

> 5. With Meson we don't need to catch up Rust, we can do better than Rust with fewer resources.

I don't know what this means. They seem to be doing fine with cargo.

On Tue, 2019-04-16 at 12:19 +0000, Atila Neves via Digitalmars-d wrote: […]
> 
> Anything is better than dub as a build system, including raw make.

That's going a bit far, and is fighting talk. But let's not even think about that argument here. :-)

[…]
> 
> AFAIK, reggae, CMake and Premake can all do this as well.

SCons, and I believe Meson, are also able to do mixed language builds very well.

I started a Dub add-in to SCons to handle dependencies but I got disheartened by the naming strategy of Dub's build of dependencies and gave up. I'd be happy to restart work on D support for SCons if it had any chance of being used, and SCons Dub support if there was a deterministic algorithm for finding Dub builds.

[…]

-- 
Russel.
===========================================
Dr Russel Winder      t: +44 20 7585 2200
41 Buckmaster Road    m: +44 7770 465 077
London SW11 1EN, UK   w: www.russel.org.uk

On Tue, Apr 16, 2019 at 06:13:30AM +0000, JN via Digitalmars-d wrote: [...]
> The problem with using anything else is that we're back to square one. If you push Meson as an official build tool, soon people will want to bring CMake, SCons, Gradle, makefiles etc. into the mix. Which will be problematic, especially on Windows. Right now you can download dub and hope that most of stuff will build, without having to chase down dependencies and additional build tools. And after a while someone will propose, why do we use all these tools, why not create a single build tool, written in D, that can support these usecases. And then we'll reinvent dub again :)

The problem is that while dub fills a very important role in the D ecosystem, it also has some unfortunate design decisions that has since become major limitations as the ecosystem grows. It originally targeted a very specific, and narrow use case, and the design was fine in that context.  But as the D ecosystem grows to encompass wider use cases, the inherent limitations in that design has since become a major issue, esp. with those of us who need to do stuff beyond just "pull in dependencies, build everything". For example, for me, supporting D codegen via external tools (themselves needing to be compiled and run from the same source tree) and mixed-language codebases (requiring multiple different toolchains) is absolutely essential.  Since dub has no way of doing this, it simply isn't even a viable option for me.  Performance is also a big showstopper -- although it has improved since I last checked, the code-compile-test turnaround time is still far too slow for everyday usage.

What we really need is a dub replacement that has a different, more flexible design that can accomodate things that people want to do, that the current dub simply cannot due to its design limitations.  There are some promising ideas in the other thread about improving dub, and I hope that discussion will eventually lead to a redesign / revamp of dub that makes it more viable for my use case.


T

-- 
The problem with the world is that everybody else is stupid.

On Tuesday, 16 April 2019 at 16:42:24 UTC, H. S. Teoh wrote:
> What we really need is a dub replacement that has a different, more flexible design that can accomodate things that people want to do, that the current dub simply cannot due to its design limitations.

I absolutely agree. Trying keep backward compatibility is a good intention but sometimes you just have to redo everything from scratch using accumulated experience. It can be painful but this is evolution. Otherwise it can lead to stagnation.

On Tuesday, 16 April 2019 at 16:42:24 UTC, H. S. Teoh wrote:
> On Tue, Apr 16, 2019 at 06:13:30AM +0000, JN via Digitalmars-d wrote: [...]
> What we really need is a dub replacement that has a different, more flexible design that can accomodate things that people want to do, that the current dub simply cannot due to its design limitations.  There are some promising ideas in the other thread about improving dub, and I hope that discussion will eventually lead to a redesign / revamp of dub that makes it more viable for my use case.
>
>
> T

Dub replacement doesn't sound good. D ecosystem is fragmented enough, no need to add additional split dub/dub replacement. I think it'd be best to just start with dub and try to improve on it. Cargo could be used as an inspiration, looks to me like in Cargo, CMake and Meson and other build systems are added as dependencies (https://docs.rs/meson/1.0.0/meson/ https://crates.io/crates/cmake ). Perhaps something like this could be adapted for dub. Some kind of plugin system, with optional cmake and meson plugins.

On Tuesday, 16 April 2019 at 16:42:24 UTC, H. S. Teoh wrote:
> On Tue, Apr 16, 2019 at 06:13:30AM +0000, JN via Digitalmars-d wrote: [...]
>
> What we really need is a dub replacement that has a different, more flexible design that can accomodate things that people want to do, that the current dub simply cannot due to its design limitations.  There are some promising ideas in the other thread about improving dub, and I hope that discussion will eventually lead to a redesign / revamp of dub that makes it more viable for my use case.
>
>
> T

IMO, the best course of action is:
1.0. Create a generic foundation based on DAG for describing targets and their relations.
This can start as a completely independent project with zero dependencies.

1.1. On top of this core functionality we can add an interoperability layer that allows exporting this abstract representation to other build system formats.

1.2. We can reuse the existing build systems written in D as a build backend.

2.0. Integrate the new build system foundation in Dub 2.0:

2.1 Replace dub's current execution code with this new core while keeping backwards compatibility with existing dub projects

2.2 `dub build` consists of two general steps: 1) fetching dependencies 2) building each project in the set { dependencies..., top-level project } in topological order. (Of course there are many other (sub)steps, which I'm mentioning here for simplicity.) Now that in 2.1 we replaced Dub's existing execution code with the new build foundation we have the option to not execute those steps directly, but instead use Dub as a meta build-system which can generate build description for other build systems to execute.

2.3 Allow describing complex builds based on the foundation layer. This can be done either declaratively, directly in dub.json / dub.sdl, or dub can call to a D script that generates a JSON describing the build dependencies that the core described in 1.1 can read.

What do you think? Are you willing to help with the design and/or implemention on this initiative?

On 17/04/2019 6:43 PM, Petar Kirov [ZombineDev] wrote:
> On Tuesday, 16 April 2019 at 16:42:24 UTC, H. S. Teoh wrote:
>> On Tue, Apr 16, 2019 at 06:13:30AM +0000, JN via Digitalmars-d wrote: [...]
>>
>> What we really need is a dub replacement that has a different, more flexible design that can accomodate things that people want to do, that the current dub simply cannot due to its design limitations.  There are some promising ideas in the other thread about improving dub, and I hope that discussion will eventually lead to a redesign / revamp of dub that makes it more viable for my use case.
>>
>>
>> T
> 
> IMO, the best course of action is:
> 1.0. Create a generic foundation based on DAG for describing targets and their relations.
> This can start as a completely independent project with zero dependencies.
> 
> 1.1. On top of this core functionality we can add an interoperability layer that allows exporting this abstract representation to other build system formats.
> 
> 1.2. We can reuse the existing build systems written in D as a build backend.
> 
> 2.0. Integrate the new build system foundation in Dub 2.0:
> 
> 2.1 Replace dub's current execution code with this new core while keeping backwards compatibility with existing dub projects
> 
> 2.2 `dub build` consists of two general steps: 1) fetching dependencies 2) building each project in the set { dependencies..., top-level project } in topological order. (Of course there are many other (sub)steps, which I'm mentioning here for simplicity.) Now that in 2.1 we replaced Dub's existing execution code with the new build foundation we have the option to not execute those steps directly, but instead use Dub as a meta build-system which can generate build description for other build systems to execute.
> 
> 2.3 Allow describing complex builds based on the foundation layer. This can be done either declaratively, directly in dub.json / dub.sdl, or dub can call to a D script that generates a JSON describing the build dependencies that the core described in 1.1 can read.

That's going to require a ton of work and refactoring.
But it would be well worth funding after "1.0".

On Wednesday, 17 April 2019 at 06:43:35 UTC, Petar Kirov [ZombineDev] wrote:
> On Tuesday, 16 April 2019 at 16:42:24 UTC, H. S. Teoh wrote:
>> On Tue, Apr 16, 2019 at 06:13:30AM +0000, JN via Digitalmars-d wrote: [...]
>>
>> What we really need is a dub replacement that has a different, more flexible design that can accomodate things that people want to do, that the current dub simply cannot due to its design limitations.  There are some promising ideas in the other thread about improving dub, and I hope that discussion will eventually lead to a redesign / revamp of dub that makes it more viable for my use case.
>>
>>
>> T
>
> IMO, the best course of action is:
> 1.0. Create a generic foundation based on DAG for describing targets and their relations.
> This can start as a completely independent project with zero dependencies.

I considered this; I even named the project. However, after I read this:

https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&cad=rja&uact=8&ved=2ahUKEwjNgZLl3dbhAhXN66YKHXbEBkEQFjAAegQIBBAC&url=https%3A%2F%2Fwww.microsoft.com%2Fen-us%2Fresearch%2Fuploads%2Fprod%2F2018%2F03%2Fbuild-systems-final.pdf&usg=AOvVaw2j71hXjOoEQLNNjvEOp_RQ

I'm not sure just describing a DAG is enough. I've started playing about with the same concepts in D here:

https://github.com/atilaneves/rocksteady

> 1.1. On top of this core functionality we can add an interoperability layer that allows exporting this abstract representation to other build system formats.

This might not always be possible. First, because of what I said above, and secondly, the feature sets sometimes don't match. I had to give up trying to express some concepts in the reggae tup backend because they're not doable there. And ninja requires modifications to the compiler that I solved by writing a compiler wrapper.

> 2.1 Replace dub's current execution code with this new core while keeping backwards compatibility with existing dub projects

I'm already doing that.

> 2.2 `dub build` consists of two general steps: 1) fetching dependencies

Which has nothing to do with building and should be done as a separate step.

> 2) building each project in the set { dependencies..., top-level project } in topological order.

The builds are independent, they should be done in parallel.

> 2.3 Allow describing complex builds based on the foundation layer. This can be done either declaratively, directly in dub.json / dub.sdl, or dub can call to a D script that generates a JSON describing the build dependencies that the core described in 1.1 can read.

This would work for many build requirements, but not all.

> What do you think? Are you willing to help with the design and/or implemention on this initiative?

Already doing it.

If reimplementing build systems is being investigated, the jam(perforce)/bjam(b2,boost)/boost.build/rypple searches should dig up a lot of well thought through issues for very generic and cross platform considerations.