I am pleased to finally announce the build system I've been slowly working on for over a year in my spare time:

    Docs:   http://jasonwhite.github.io/button/
    Source: https://github.com/jasonwhite/button

Features:

- Correct incremental builds.
- Automatic dependency detection (for any build task, even shell scripts).
- Build graph visualization using GraphViz.
- Language-independent. It can build anything.
- Can automatically build when an input file is modified (using inotify).
- Recursive: It can build the build description as part of the build.
- Lua is the primary build description language.

A ton of design work went into this. Over the past few years, I went through many different designs and architectures. I finally settled on this one about a year ago and then went to work on implementing it. I am very happy with how it turned out.

Note that this is still a ways off from being production-ready. It needs some polishing. Feedback would be most appreciated (file some issues!). I really want to make this one of the best build systems out there.

Here is an example build description for DMD:

    https://github.com/jasonwhite/dmd/blob/button/src/BUILD.lua

I'd say that's a lot easier to read than this crusty thing:

    https://github.com/dlang/dmd/blob/master/src/posix.mak

In fact, there is some experimental support for automatic conversion of Makefiles to Button's build description format using a fork of GNU Make itself: https://github.com/jasonwhite/button-make

Finally, a few notes:

- I was hoping to give a talk on this at DConf, but sadly my submission was turned down. :'(

- I am aware of Reggae, another build system written in D. Although, I admit I haven't looked at it very closely. I am curious how it compares.

- You might also be interested in the two other libraries I wrote specifically for this project:

  - https://github.com/jasonwhite/darg (A command-line parser)
  - https://github.com/jasonwhite/io (An IO streams library)

On Monday, 30 May 2016 at 19:16:50 UTC, Jason White wrote:
> I am pleased to finally announce the build system I've been slowly working on for over a year in my spare time:
>
>     Docs:   http://jasonwhite.github.io/button/
>     Source: https://github.com/jasonwhite/button
>

Great news! Love to see innovation in this area.

> - Lua is the primary build description language.

Why not D?

On Monday, 30 May 2016 at 20:58:51 UTC, poliklosio wrote:
>> - Lua is the primary build description language.
>
> Why not D?

Generating the JSON build description should entirely deterministic. With Lua, this can be guaranteed. You can create a sandbox where only certain operations are permitted. For example, reading files is permitted, but writing to them is not. I can also intercept all file reads and mark the files that get read as dependencies.

It certainly could be done in D, or any other language for that matter. All that needs to be done is to write a program that can output the fundamental JSON build description.

On Monday, 30 May 2016 at 19:16:50 UTC, Jason White wrote:
> I am pleased to finally announce the build system I've been slowly working on for over a year in my spare time:
>
> [...]

[snip]
Button:
>  - https://github.com/jasonwhite/darg (A command-line parser)
>  - https://github.com/jasonwhite/io (An IO streams library)

That's great, sharing your D tools.

On 31/05/2016 7:16 AM, Jason White wrote:
> I am pleased to finally announce the build system I've been slowly
> working on for over a year in my spare time:
>
>     Docs:   http://jasonwhite.github.io/button/
>     Source: https://github.com/jasonwhite/button
>
> Features:
>
> - Correct incremental builds.
> - Automatic dependency detection (for any build task, even shell scripts).
> - Build graph visualization using GraphViz.
> - Language-independent. It can build anything.
> - Can automatically build when an input file is modified (using inotify).
> - Recursive: It can build the build description as part of the build.
> - Lua is the primary build description language.
>
> A ton of design work went into this. Over the past few years, I went
> through many different designs and architectures. I finally settled on
> this one about a year ago and then went to work on implementing it. I am
> very happy with how it turned out.
>
> Note that this is still a ways off from being production-ready. It needs
> some polishing. Feedback would be most appreciated (file some issues!).
> I really want to make this one of the best build systems out there.
>
> Here is an example build description for DMD:
>
>     https://github.com/jasonwhite/dmd/blob/button/src/BUILD.lua
>
> I'd say that's a lot easier to read than this crusty thing:
>
>     https://github.com/dlang/dmd/blob/master/src/posix.mak
>
> In fact, there is some experimental support for automatic conversion of
> Makefiles to Button's build description format using a fork of GNU Make
> itself: https://github.com/jasonwhite/button-make
>
> Finally, a few notes:
>
> - I was hoping to give a talk on this at DConf, but sadly my submission
> was turned down. :'(
>
> - I am aware of Reggae, another build system written in D. Although, I
> admit I haven't looked at it very closely. I am curious how it compares.
>
> - You might also be interested in the two other libraries I wrote
> specifically for this project:
>
>   - https://github.com/jasonwhite/darg (A command-line parser)
>   - https://github.com/jasonwhite/io (An IO streams library)

Are you on Freenode (no nick to name right now)?
I would like to talk to you about a few ideas relating to lua and D.

On Monday, 30 May 2016 at 19:16:50 UTC, Jason White wrote:
> I am pleased to finally announce the build system I've been slowly working on for over a year in my spare time:
>
> snip
> In fact, there is some experimental support for automatic conversion of Makefiles to Button's build description format using a fork of GNU Make itself: https://github.com/jasonwhite/button-make

I'm going to take a look at that!

> - I am aware of Reggae, another build system written in D. Although, I admit I haven't looked at it very closely. I am curious how it compares.

Since I wrote reggae, let me compare ;)

> - Correct incremental builds.

Yep.

> - Automatic dependency detection (for any build task, even shell scripts).

Yes for C/C++/D, no for random tasks in general, but yes if you use the tup backend.

> - Build graph visualization using GraphViz.

Use the ninja backend, get it for free.

> - Language-independent. It can build anything.

So can reggae, but the built-in high-level rules only do C/C++/D right now.

> - Can automatically build when an input file is modified (using inotify).

Nope, I never found that interesting. Possibly because I keep saving after every edit in OCD style and I really don't want things running automatically.

> - Recursive: It can build the build description as part of the build.

I'm not sure what that means. reggae copies CMake here and runs itself when the build description changes, if that's what you mean.

> - Lua is the primary build description language.

In reggae you can pick from D, Python, Ruby, Javascript and Lua.

Atila

Can it be built from just plain dmd/phobos install available? One of major concernc behind discussion that resulted in Atila reggae effort is that propagating additional third-party dependencies is very damaging for build systems. Right now Button seems to fail rather hard on this front (i.e. Lua for build description + uncertain amount of build dependencies for Button itself).

On Tuesday, 31 May 2016 at 03:40:32 UTC, rikki cattermole wrote:
> Are you on Freenode (no nick to name right now)?
> I would like to talk to you about a few ideas relating to lua and D.

No, I'm not on IRC. I'll see if I can find the time to hop on this weekend.

On Tuesday, 31 May 2016 at 10:15:14 UTC, Atila Neves wrote:
> On Monday, 30 May 2016 at 19:16:50 UTC, Jason White wrote:
>> I am pleased to finally announce the build system I've been slowly working on for over a year in my spare time:
>>
>> snip
>> In fact, there is some experimental support for automatic conversion of Makefiles to Button's build description format using a fork of GNU Make itself: https://github.com/jasonwhite/button-make
>
> I'm going to take a look at that!

I think the Makefile converter is probably the coolest thing about this build system. I don't know of any other build system that has done this. The only problem is that it doesn't do well with Makefiles that invoke make recursively. I tried compiling Git using it, but Git does some funky stuff with recursive make like grepping the output of the sub-make.

>> - Can automatically build when an input file is modified (using inotify).
>
> Nope, I never found that interesting. Possibly because I keep saving after every edit in OCD style and I really don't want things running automatically.

I constantly save like a madman too. If an incremental build is sufficiently fast, it doesn't really matter. You can also specify a delay so it accumulates changes and then after X milliseconds it runs a build.

>> - Recursive: It can build the build description as part of the build.
>
> I'm not sure what that means. reggae copies CMake here and runs itself when the build description changes, if that's what you mean.

It means that Button can run Button as a build task (and it does it correctly). A child Button process reports its dependencies to the parent Button process via a pipe. This is the same mechanism that detects dependencies for ordinary tasks. Thus, there is no danger of doing incorrect incremental builds when recursively running Button like there is with Make.

>> - Lua is the primary build description language.
>
> In reggae you can pick from D, Python, Ruby, Javascript and Lua.

That's pretty cool. It is possible for Button to do the same, but I don't really want to support that many languages. In fact, the Make and Lua build descriptions both work the same exact way - they output a JSON build description for Button to use. So long as someone can write a program to do this, they can write their build description in it.

On Tuesday, 31 May 2016 at 14:28:02 UTC, Dicebot wrote:
> Can it be built from just plain dmd/phobos install available? One of major concernc behind discussion that resulted in Atila reggae effort is that propagating additional third-party dependencies is very damaging for build systems. Right now Button seems to fail rather hard on this front (i.e. Lua for build description + uncertain amount of build dependencies for Button itself).

Building it only requires dmd+phobos+dub.

Why is having dependencies so damaging for build systems? Does it really matter with a package manager like Dub? If there is another thread that answers these questions, please point me to it.

The two dependencies Button itself has could easily be moved into the same project. I kept them separate because they can be useful for others. These are the command-line parser and IO stream libraries.

As for the dependency on Lua, it is statically linked into a separate executable (called "button-lua") and building it is dead-simple (just run make). Using the Lua build description generator is actually optional, it's just that writing build descriptions in JSON would be horribly tedious.