On 4/6/13, bearophile <bearophileHUGS@lycos.com> wrote:
> I think the recursive scan is mostly meant for small projects. I think large projects will usually use some kind of build scripts.

A gtkD benchmark:

$ C:\dev\projects\GtkD\demos\gtk>timeit rdmd --build-only --force -IC:\dev\projects\GtkD\src HelloWorld.d
> Done in 26_247_732 usecs.

$ C:\dev\projects\GtkD\demos\gtk>timeit dmd -rb -IC:\dev\projects\GtkD\src HelloWorld.d
> Done in 22_826_820 usecs.

4 seconds shaved off. Now let's try with a prebuilt static library:

timeit rdmd C:\dev\projects\GtkD\src\GtkD.lib --exclude=atk --exclude=cairo --exclude=gthread --exclude=gobject --exclude=glib --exclude=gio --exclude=gdk --exclude=gdkpixbuf --exclude=gtk --exclude=gtkc --exclude=gtkD --exclude=pango --build-only --force -IC:\dev\projects\GtkD\src HelloWorld.d

Done in 3_100_329 usecs.

timeit dmd -rb C:\dev\projects\GtkD\src\GtkD.lib -rxatk.* -rxcairo.* -rxgthread.* -rxgobject.* -rxglib.* -rxgio.* -rxgdk.* -rxgdkpixbuf.* -rxgtk.* -rxgtkc.* -rxgtkD.* -rxpango.* -IC:\dev\projects\GtkD\src HelloWorld.d

Done in 1_663_442 usecs.

It definitely helps when you're building something from scratch.

But RDMD can track changes to dependencies, which DMD still can't do. I want to try implementing this feature in DMD and see the speed difference for incremental builds.

On 4/6/13 1:14 PM, Andrej Mitrovic wrote:
> On 4/5/13, Vladimir Panteleev<vladimir@thecybershadow.net>  wrote:
>> I noticed that compiling D programs in the usual manner (rdmd) is
>> as much as 40% slower than it can be.
>
> I've implemented the -rb and -rx switchex (the -r switch itself is
> taken, it's an undocumented switch). These switches are used to enable
> recursive builds and to exclude modules/packages, respectively[1]. The
> -rx switch can take a module name or a wildcard option, e.g. -rxfoo.*
> to exclude all modules from the package "foo".
>
> I'm seeing about 30% faster clean builds when using DMD with this
> feature compared to using RDMD. But I've only tested this on smaller
> projects, I wonder what the impact is on larger projects.
>
> The std and core packages, and the object module are implicitly
> excluded from building.
>
> [1] : https://github.com/AndrejMitrovic/dmd/tree/BuildRecurse

This is a dramatic improvement, and better build times are always well received. Do you plan to convert your work into a pull request?

For larger projects I wonder if it's possible to restrict scanning only to specific directories. That way building becomes a modularity enforcer.


Andrei

On Saturday, 6 April 2013 at 17:52:40 UTC, Andrej Mitrovic wrote:
> It definitely helps when you're building something from scratch.

Great stuff!

> But RDMD can track changes to dependencies, which DMD still can't do.
> I want to try implementing this feature in DMD and see the speed
> difference for incremental builds.

Why not adapt rdmd so that it takes advantage of the new script? Is there any benefit of having dependency change checks in dmd instead of rdmd?

On 4/6/13 9:38 PM, Vladimir Panteleev wrote:
> On Saturday, 6 April 2013 at 17:52:40 UTC, Andrej Mitrovic wrote:
>> It definitely helps when you're building something from scratch.
>
> Great stuff!
>
>> But RDMD can track changes to dependencies, which DMD still can't do.
>> I want to try implementing this feature in DMD and see the speed
>> difference for incremental builds.
>
> Why not adapt rdmd so that it takes advantage of the new script? Is
> there any benefit of having dependency change checks in dmd instead of
> rdmd?

Yah, I think it would be great to dedicate rdmd to the dependency/caching part and leave the build to the compiler. One possibility would be to run the build and the dependency saving in parallel (!).

Andrei

On Sunday, 7 April 2013 at 02:04:54 UTC, Andrei Alexandrescu wrote:
> Yah, I think it would be great to dedicate rdmd to the dependency/caching part and leave the build to the compiler. One possibility would be to run the build and the dependency saving in parallel (!).

Why in parallel and not in one go? (-v with -rb and without -o-)

On 4/6/13 10:12 PM, Vladimir Panteleev wrote:
> On Sunday, 7 April 2013 at 02:04:54 UTC, Andrei Alexandrescu wrote:
>> Yah, I think it would be great to dedicate rdmd to the
>> dependency/caching part and leave the build to the compiler. One
>> possibility would be to run the build and the dependency saving in
>> parallel (!).
>
> Why in parallel and not in one go? (-v with -rb and without -o-)

I'm not sure how rdmd would distingish dmd's output from the output of the running program. Oh wait, the run will be a distinct step - awesome.

Andrei

On Sunday, 7 April 2013 at 02:16:10 UTC, Andrei Alexandrescu wrote:
>> Why in parallel and not in one go? (-v with -rb and without -o-)
>
> I'm not sure how rdmd would distingish dmd's output from the output of the running program. Oh wait, the run will be a distinct step - awesome.

I'll have a go at the rdmd side of this.

Any reason not to use the deps file as the build witness?

On 4/7/13 12:07 AM, Vladimir Panteleev wrote:
> On Sunday, 7 April 2013 at 02:16:10 UTC, Andrei Alexandrescu wrote:
>>> Why in parallel and not in one go? (-v with -rb and without -o-)
>>
>> I'm not sure how rdmd would distingish dmd's output from the output of
>> the running program. Oh wait, the run will be a distinct step - awesome.
>
> I'll have a go at the rdmd side of this.
>
> Any reason not to use the deps file as the build witness?

There was a reason, but darn I forgot it...

Andrei

On Sunday, 7 April 2013 at 04:25:19 UTC, Andrei Alexandrescu wrote:
> On 4/7/13 12:07 AM, Vladimir Panteleev wrote:
>> On Sunday, 7 April 2013 at 02:16:10 UTC, Andrei Alexandrescu wrote:
>>>> Why in parallel and not in one go? (-v with -rb and without -o-)
>>>
>>> I'm not sure how rdmd would distingish dmd's output from the output of
>>> the running program. Oh wait, the run will be a distinct step - awesome.
>>
>> I'll have a go at the rdmd side of this.
>>
>> Any reason not to use the deps file as the build witness?
>
> There was a reason, but darn I forgot it...

I think I figured it out - --makedepend will update the deps file, but not the executable.

On Sunday, 7 April 2013 at 04:07:55 UTC, Vladimir Panteleev wrote:
> I'll have a go at the rdmd side of this.

https://github.com/CyberShadow/tools/compare/BuildRecurse