On Tuesday, 22 September 2015 at 12:39:48 UTC, Per Nordlöw wrote:
> On Wednesday, 16 September 2015 at 14:07:17 UTC, Atila Neves wrote:
>> http://code.dlang.org/my_packages/reggae
>>
>> What's new:
>> Atila
>
> If you want to build a really revolutionary *new* build system you should turn reggae into a client-server-architecture that listens to file moves/copies/notifications via inotify on Linux. I'm not sure how the develop/builder interface would look like then, however. A web-client would be one way.

That idea is in the TODO.txt of the project. There's a 99.9% chance I won't bother.

I just benchmarked a toy auto-generated project of 30'000 D files, because I wanted to profile and optimise the binary backend against ninja and tup. I'm doing this by measuring the overhead of checking all of these files for modification timestamps, since actually compiling and linking take much longer. The overhead for ninja is 0.18s. Tup does it even faster. My current binary backend spends all of 0.3s, and that's what I was trying to optimize. Make, however... 22s I think.

It would seem obvious that using something like inotify would be incredibly fast, but in practice it's irrelevant. Well, it's irrelevant for pretty much every project out there. I'm incredibly sensible to how long my feedback loops are, but if a 30k file project takes under a second to check, I really don't care.

Now, you may say that my toy project doesn't accurately represent real life. But I've used ninja on a real C codebase of 40M lines spread over ~14k files. This is much larger than most people will ever deal with. A no-op build took 1s.

Make is slow. Just don't use make. The ninja, tup and binary backends of reggae are fast enough. Incidentally, this is why ninja didn't try to get faster. It doesn't need to.

I might consider a system where you don't even have to tell the computer to build anything. I'm not even sure that's that useful. Maybe it's just the way I work, I don't know.

> This however does not rebuild the dependency tree between invokations which is a serious limitation. I believe reggae could be even smarter here.

It already is ;)

> I've spent the last two years designing and implementing a SCons-based build system at work. Involving, among others, adding support for Ada and auto-detection of include-paths as you have done in reggae.

Feel free to issue a PR for Ada auto-detection for reggae. :)

> I you want any advice on this matter please contact me. I'd be glad to be of service.

Send me an email, I'm more than happy to waffle away about build systems.

On 2015-09-24 16:46, Atila Neves wrote:

> That's not been my experience at all using reggae. I only do incremental
> builds now and have never run into a problem. Can you give an example?

Here's one old post [1] that describes the problem. I'm not sure how much of it still applies today.

[1] http://forum.dlang.org/thread/h8ddc5$1h67$1@digitalmars.com

-- 
/Jacob Carlborg

On Friday, 25 September 2015 at 12:09:01 UTC, Jacob Carlborg wrote:
> On 2015-09-24 16:46, Atila Neves wrote:
>
>> That's not been my experience at all using reggae. I only do incremental
>> builds now and have never run into a problem. Can you give an example?
>
> Here's one old post [1] that describes the problem. I'm not sure how much of it still applies today.
>
> [1] http://forum.dlang.org/thread/h8ddc5$1h67$1@digitalmars.com

I wasn't around in 2009 so I don't how dmd was then.

How does one compile 3 files "at the same time" and generate 3 object files? There was a reference to a -multiobj option in that post but that's not even in the man page.

Even if it exists and works as I assume it does, it seems to me to be a silly way to compile files. As Andrei mentioned at DConf this year, the D idiom is to group source files per package when compiling. That's what reggae does by default for D files. It just works.

Atila

On Thursday, 24 September 2015 at 15:04:49 UTC, Atila Neves wrote:
>> I you want any advice on this matter please contact me. I'd be glad to be of service.
>
> Send me an email, I'm more than happy to waffle away about build systems.

BTW. I'm planning on visiting Berlin for DConf 2016. We could have a chat there.

On Friday 25 September 2015 23:27, Atila Neves wrote:

> How does one compile 3 files "at the same time" and generate 3 object files? There was a reference to a -multiobj option in that post but that's not even in the man page.

dmd -c foo.d bar.d baz.d

rdmd would probably do this by now, if it weren't for the template instantiation issue. There's a nice, simple example of the issue in the PR discussion: https://github.com/D-Programming-Language/tools/pull/170#issuecomment-112526734

> Even if it exists and works as I assume it does, it seems to me to be a silly way to compile files. As Andrei mentioned at DConf this year, the D idiom is to group source files per package when compiling. That's what reggae does by default for D files. It just works.

Aside from the template instantiation issue, I don't see how doing a package at a time is better than compiling exactly those files that need to be compiled.

On Friday, 25 September 2015 at 22:12:49 UTC, anonymous wrote:
> On Friday 25 September 2015 23:27, Atila Neves wrote:
>
>> How does one compile 3 files "at the same time" and generate 3 object files? There was a reference to a -multiobj option in that post but that's not even in the man page.
>
> dmd -c foo.d bar.d baz.d

Huh, I didn't know that. I guess I've always used the `-of` option. Unsurprisingly, so does reggae. Which means that incremental compilation is no longer an issue.

> Aside from the template instantiation issue, I don't see how doing a package at a time is better than compiling exactly those files that need to be compiled.

There have been threads about this before. It turns out that compiling per file is usually slower than compiling the whole package/app at once. It's not intuitive, but it's true (I measured it myself). reggae has an option to compile per-file but I haven't used it since switching to per-package once.

Atila

On Saturday 26 September 2015 01:24, Atila Neves wrote:

> There have been threads about this before. It turns out that compiling per file is usually slower than compiling the whole package/app at once. It's not intuitive, but it's true (I measured it myself). reggae has an option to compile per-file but I haven't used it since switching to per-package once.

Compiling one file at a time is yet another thing.

1) Compile everything into one object file:
dmd -c -ofresult.o foo.d bar.d
2) Compile to multiple object files in one run:
dmd -c foo.d bar.d
3) Compile to multiple object files in multiple runs:
dmd -c foo.d; dmd -c bar.d

Since you didn't know about 2, I take it that you measured 1 vs 3. 3 is bound to be slower when the modules share dependencies, because they have to be parsed again and again.

2 may be faster or slower than 1, I don't know, a quick check didn't show a difference. When the template instantiation issue ever gets fixed, I wouldn't be surprised if 2 got slower than 1. Of course, any slowness of 2 must then be weighed against 1 compiling more than necessary.

I rarely visit the D forums and even more rarely make a post, but this thread caught my eye.

I've been writing a build system in D too: https://github.com/jasonwhite/brilliant-build (I'm not very fond of the name. Naming is hard!)

It is a general build system with an emphasis on correctness.

It is a work in progress at this point, but I'm very happy with how it is turning out. I'm interested in your guys' thoughts on it.

HI Atilla,

Dub's looking interesting! Some of the links are broken when browsing from code.dlang.org though.

From http://code.dlang.org/packages/reggae

click on, say, 'detailed documentation'. ( Where I wanted to go!).

Get ..

404 - Not Found

Not Found

Internal error information:
No routes match path '/packages/doc/index.md'


Not sure if that's problem with reggae's layout of the site....

Cheers,

A.

On Saturday, 26 September 2015 at 08:23:46 UTC, Andy Smith wrote:
> HI Atilla,
>
> Dub's looking interesting! Some of the links are broken when browsing from code.dlang.org though.
>
> From http://code.dlang.org/packages/reggae
>
> click on, say, 'detailed documentation'. ( Where I wanted to go!).
>
> Get ..
>
> 404 - Not Found
>
> Not Found
>
> Internal error information:
> No routes match path '/packages/doc/index.md'
>
>
> Not sure if that's problem with reggae's layout of the site....
>
> Cheers,
>
> A.

^dub^reggae^ :-)

Doh! - morning coffee hasn't kicked in yet!

Cheers,

A.