On 16/06/2017 7:30 AM, Russel Winder via Digitalmars-d wrote:
> A direct question to Walter and Andrei really.
> 
> If someone, let us say Russel Winder, create a CMake/Ninja and/or
> Meson/Ninja build for DMD, is there any chance of it being allowed to
> replace the Make system?
> 
> If the answer is no, then Russel will obviously not waste his time
> doing something that will not be accepted.

Just an FYI:
Eventually dmd-fe will be split up to be able to work as a library (on-going work). At which point in time dub will need to be available for use. Now dub should be able to generate any other descriptor that you need.

This is where I would put my energy, but it is up to you on where you see things going and helps you out the most.

On Mon, 2017-06-19 at 13:44 +0000, Atila Neves via Digitalmars-d wrote:
> 
[…]
> I replicated the _entirety_ [1] of Phobos's posix.mak (that Makefile does a _lot_) with reggae and that never happened, so I guess the answer is "vanishingly small".
> 

In the end as long as the D front end, druntime, and phobos get to the LDC folk so as to create ldc packages for Debian, Fedora, and Homebrew (*), I don't actually care what sort of pain they have with their workflow. It does seem though that the barrier has now been raised, it's not just dmd, it all of dmsd, druntime, phobos and dlang.org as a big bang addition before even getting to review. I really can't see me doing that. One repository at a time maybe, but it sounds as though your experience of Reggae and Phobos is "DMD developers use Make and deal with the pain". Sad.


(*) I've had to switch from MacPorts to Homebrew on my MBP (**), which is
both sad, and interesting.

(**) OSX really is crap compared to Linux/GNOME, how do people stand it? <rhetorical-question-i-think/>

-- 
Russel.
=============================================================================
Dr Russel Winder     t:+44 20 7585 2200   voip:sip:
russel.winder@ekiga.net
41 Buckmaster Road   m:+44 7770 465 077   xmpp:russel@winder.org.uk
London SW11 1EN, UK  w: www.russel.org.uk skype:russel_winder

On Monday, 19 June 2017 at 14:15:22 UTC, Russel Winder wrote:
> On Mon, 2017-06-19 at 13:44 +0000, Vladimir Panteleev via Digitalmars-d wrote:
>> […]
>> 
>> DMD is insufficient, and is not the hardest makefile to port. Any serious proposal would need to cover all core repositories - dmd, druntime, phobos, tools, and dlang.org. We would need to evaluate any proposals thoroughly, and it will likely involve a trial period during which both will be maintained in parallel before any switch-over occurs.
>
> One doesn't port Makefiles, one writes a new build that achieves the same final outcomes.

We will likely need to continue providing shim makefiles which invoke the replacement build system for the foreseeable future. The big problem here is that all make targets are essentially part of their public interface, so if we are to avoid breaking anyone's scripts or workflow, they must be kept working. Otherwise, I don't disagree.

> If those 5 are so interconnected why are they in separate repositories?

Legacy reasons; prohibitively high cost of migrating to a monorepo; higher granularity of specialization for maintainers and reviewers of various codebase components, as well as assigning permissions to them.

> It should be entirely feasible to replace, and evaluate the replacement of, the builds of each of the repositories independently.

The current build system has a number of problems caused by the components being in separate repositories. For example, if you change a file in Phobos and want to rebuild the documentation (dlang.org), the latter won't know that only a single file changed in the former, because the dlang.org makefile is not aware of the rules governing the building of Phobos files. Furthermore, both the dlang.org makefile and the Phobos makefile have a target for building Phobos documentation, but they work in different ways and produce different results. A replacement build system that can achieve correct interoperability as described above would score a lot of points towards being accepted.

More importantly, if we accept a proposal for three repos and on the fourth one we run into a blocker (or even the replacement simply being very unsatisfying), then that would put us in an unpleasant situation. So, I think that requiring working replacements for all components makes sense as a prerequisite for any single component being switched over to the new build system.

> To demand it is an all or nothing, is to set a bar to high for volunteer labour.

My labour is just as volunteer as yours. I'm not demanding that someone presents a complete solution; but neither am I going to write one from scratch myself. Which is why I said I'd be interested in further discussion and proposals.

Will you have some time to chat about Reggae on IRC tomorrow? I should be around about 12 hours from now for about 12 hours since then.

> Interestingly I bet the Make-based build didn't have to undergo such review. ;-)

It certainly did not. However, migrating from the status quo to the status quo is a no-op and has a cost of zero. Here, we need to balance the cost of the migration against the benefits of the proposal.

> That the Makefiles are tangled does not imply a proper build should be.

There are many moving parts, and we will keep wanting to add more.

On Mon, 2017-06-19 at 14:52 +0000, Vladimir Panteleev via Digitalmars-d wrote:
> 
[…]
> We will likely need to continue providing shim makefiles which invoke the replacement build system for the foreseeable future. The big problem here is that all make targets are essentially part of their public interface, so if we are to avoid breaking anyone's scripts or workflow, they must be kept working. Otherwise, I don't disagree.

The usual approach is to leave the old system in place, put the new system in place, and run in parallel. Then people can remove their dependency on the old system over time. Then you deprecate the old system, and then remove it. The new system does not have to replicate any part of the old system, it just has to produce the end products.

[…]
> The current build system has a number of problems caused by the components being in separate repositories. For example, if you change a file in Phobos and want to rebuild the documentation (dlang.org), the latter won't know that only a single file changed in the former, because the dlang.org makefile is not aware of the rules governing the building of Phobos files. Furthermore, both the dlang.org makefile and the Phobos makefile have a target for building Phobos documentation, but they work in different ways and produce different results. A replacement build system that can achieve correct interoperability as described above would score a lot of points towards being accepted.
> 
> More importantly, if we accept a proposal for three repos and on the fourth one we run into a blocker (or even the replacement simply being very unsatisfying), then that would put us in an unpleasant situation. So, I think that requiring working replacements for all components makes sense as a prerequisite for any single component being switched over to the new build system.

GStreamer seems like a good model here as they have dealt with this exact same situation (well not exact, they do not have the dlang.org problem in the same way). As part of their trialling Meson to replace Autotools, they created a build repository that pulls in all the other repositories such that the whole structure has a well-defined architecture, all paths are known. Thus you get a nice simplification that enables a global build as well as individual repository builds.

Of course they have made the decision to trial on the real repositories, with the option to delete it all and return to using Autotools. This has a major benefit, it is the real repositories being worked on and anyone can chip in at any time.

This approach also leads to earlier "buy in" from more people. Though in GStreamer case Meson is the natural step on from Autotools, so there is perhaps less argumentation over options, before heading to a trial.

[…]
> Will you have some time to chat about Reggae on IRC tomorrow? I should be around about 12 hours from now for about 12 hours since then.

Possibly, what time are you thinking of? I haven't used IRC, well not in over 20 years anyway, so I would need data on connecting.

[…]
-- 
Russel. ============================================================================= Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder@ekiga.net 41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel@winder.org.uk London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder

On Tuesday, 20 June 2017 at 07:24:26 UTC, Russel Winder wrote:
> The usual approach is to leave the old system in place, put the new system in place, and run in parallel. Then people can remove their dependency on the old system over time. Then you deprecate the old system, and then remove it. The new system does not have to replicate any part of the old system, it just has to produce the end products.

Yep; same as what was done with ddmd.

> Possibly, what time are you thinking of? I haven't used IRC, well not in over 20 years anyway, so I would need data on connecting.

Sorry Russell, I thought I was replying to Atila :) But you are of course welcome on IRC. The D channel is on Freenode (#d). If you don't want to install an IRC client, there is a web portal at http://webchat.freenode.net/.

On Tuesday, 20 June 2017 at 09:08:32 UTC, Vladimir Panteleev wrote:
> On Tuesday, 20 June 2017 at 07:24:26 UTC, Russel Winder wrote:
>> [...]
>
> Yep; same as what was done with ddmd.
>
>> [...]
>
> Sorry Russell, I thought I was replying to Atila :) But you are of course welcome on IRC. The D channel is on Freenode (#d). If you don't want to install an IRC client, there is a web portal at http://webchat.freenode.net/.

In that case, what were you replying? :P

I'm serious, I'm lost now.

Atila

On 06/19/2017 04:06 AM, Russel Winder via Digitalmars-d wrote:
> 
> Reggae is D's pitch in the CMake and Meson class of meta-build tools.
> Why aren't all the D compiler and tool developments using it?

I'm convinced a big part of that is because DUB is ubiquitous and incredibly helpful in the D world for package management, but plays very poorly with any build system that isn't DUB's internal one.

I've blown enormous amounts of time trying to get my projects (that need DUB for dependencies) to use alternate buildsystems without DUB getting in the way, and never really succeeded.

A. Figuring out how to obtain all the necessary import paths, link paths, etc to USE a DUB-repo-provided package (especially vibe) on the compiler command-line, and be sure that I'm actualy getting everything I need and that it won't be broken when the dependency is updated...

B. Figuring out how to tell dub "quit trying to compile/link my files yourself" without screwing up part "A" above in the process...

...it's just a mess, and despite the rediculous amounts of effort I've put into trying to get that all working sanely, even I mostly just gave up, just use DUB to build everything (even though it's slow), and shy away from attempting anything that's part of the supposedly "0.1%" of use-cases (such as including any libs or components that aren't D) which DUB, by design, doesn't attempt to address.

On Tuesday, 20 June 2017 at 19:06:05 UTC, Nick Sabalausky (Abscissa) wrote:
> On 06/19/2017 04:06 AM, Russel Winder via Digitalmars-d wrote:
>> 
>> Reggae is D's pitch in the CMake and Meson class of meta-build tools.
>> Why aren't all the D compiler and tool developments using it?
>
> I'm convinced a big part of that is because DUB is ubiquitous and incredibly helpful in the D world for package management, but plays very poorly with any build system that isn't DUB's internal one.
>

While the dub documentation is not always the best, it seems to me to be in a better state than reggae's. I've heard about reggae a bit on the forum, but I never really made any attempt to try to use it. dub seems a lot simpler for small projects.

Maybe Atila could do a blog post with some simple examples and compare/contrast with dub?

On Monday, June 19, 2017 1:45:27 PM MDT meppl via Digitalmars-d wrote:
> On Monday, 19 June 2017 at 09:19:32 UTC, Dejan Lekic wrote:
> > On Friday, 16 June 2017 at 06:30:01 UTC, Russel Winder wrote:
> >> A direct question to Walter and Andrei really.
> >>
> >> If someone, let us say Russel Winder, create a CMake/Ninja and/or Meson/Ninja build for DMD, is there any chance of it being allowed to replace the Make system?
> >>
> >> If the answer is no, then Russel will obviously not waste his time doing something that will not be accepted.
> >
> > Why?
> >
> > Why replacing a rock-stable Make with build-system-X that most likely adds another dependency. I am with Walter on this one. - We should continue using Make unless there is a real need for something else. DMD's makefiles are really simple!
>
> is there a point in disallowing several alternate build systems
> residing in the dmd repository?
> If it is just allowed to upload README-files and make-files of
> alternate build systems etc, it would not be necessary to waste
> time with this discussion here.

Having alternate build systems means maintaining more than one build system. The main reason that a number of us would like to see make replaced is to _reduce_ the maintenance requirements, not increase them.

- Jonathan M Davis

On Tuesday, 20 June 2017 at 21:26:02 UTC, Jonathan M Davis wrote:
> On Monday, June 19, 2017 1:45:27 PM MDT meppl via Digitalmars-d wrote:
>> On Monday, 19 June 2017 at 09:19:32 UTC, Dejan Lekic wrote:
>> > On Friday, 16 June 2017 at 06:30:01 UTC, Russel Winder wrote:
>> >> A direct question to Walter and Andrei really.
>> >>
>> >> If someone, let us say Russel Winder, create a CMake/Ninja and/or Meson/Ninja build for DMD, is there any chance of it being allowed to replace the Make system?
>> >>
>> >> If the answer is no, then Russel will obviously not waste his time doing something that will not be accepted.
>> >
>> > Why?
>> >
>> > Why replacing a rock-stable Make with build-system-X that most likely adds another dependency. I am with Walter on this one. - We should continue using Make unless there is a real need for something else. DMD's makefiles are really simple!
>>
>> is there a point in disallowing several alternate build systems
>> residing in the dmd repository?
>> If it is just allowed to upload README-files and make-files of
>> alternate build systems etc, it would not be necessary to waste
>> time with this discussion here.
>
> Having alternate build systems means maintaining more than one build system. The main reason that a number of us would like to see make replaced is to _reduce_ the maintenance requirements, not increase them.
>
> - Jonathan M Davis

No, as I noted earlier in this thread, all you have to do is keep the Makefiles up to date, and dump the maintenance burden for Meson/reggae build scripts on their proponents, who in turn don't have to keep them up to date.  Once the project contributors have solid experience with the Make alternatives, you consider making a switch.