Is there a way to tell DUB to compile all its sources as separate objects which are then fed to ld.

My project has grown beyound 10k lines of code.

And I don't want DMD to recompile every line in my project if I change only of them...and yes most of my cross-module interfaces are un-templatized.

I'm assuming caching a la SCons is supported by this logic aswell.

If not are there any plans to add this feature to DUB?

If not could somebody point out where I should add it ;)

On Monday, 26 January 2015 at 22:33:21 UTC, Nordlöw wrote:
> My project has grown beyound 10k lines of code.

Eh, that's nothing, especially if they aren't templates. dmd will compile that in a fraction of a second (and even then, if you used phobos, reading the stdlib into memory will dominate the compile time)

idk if dub has an option for this or not, but it probably won't make much of a difference - if anything, it'll slow things down because then the compiler will likely look at the same thing several times. (when you compile them all at once, the modules' representation are cached in memory after the first import, whereas doing them separately, it needs to be done each time the compiler runs)

On Monday, 26 January 2015 at 22:41:09 UTC, Adam D. Ruppe wrote:
> On Monday, 26 January 2015 at 22:33:21 UTC, Nordlöw wrote:
>> My project has grown beyound 10k lines of code.
>
> Eh, that's nothing, especially if they aren't templates. dmd will compile that in a fraction of a second (and even then, if you used phobos, reading the stdlib into memory will dominate the compile time)

Ok, then it's more than that. It takes 7 seconds:

time dub build

Building knetquery ~master configuration "application", build type debug.
Compiling using dmd...
../sort_ex.d(143): Warning: use std.algorithm.sort instead of .sort property
../sort_ex.d(168): Warning: use std.algorithm.sort instead of .sort property
Linking...

real	0m7.338s
user	0m6.121s
sys	0m1.137s

On Monday, 26 January 2015 at 22:44:26 UTC, Nordlöw wrote:
> Ok, then it's more than that. It takes 7 seconds:
>
> time dub build

time dub build --combined --force

is in the same order - around 7 seconds.

On Monday, 26 January 2015 at 22:46:46 UTC, Nordlöw wrote:
> On Monday, 26 January 2015 at 22:44:26 UTC, Nordlöw wrote:
>> Ok, then it's more than that. It takes 7 seconds:
>>
>> time dub build
>
> time dub build --combined --force
>
> is in the same order - around 7 seconds.

Why is DUB compiling it all in one shot when I do *not* feed it the --combine flag?

Entering

    time dub build --force --verbose

gives

Using dub registry url 'http://code.dlang.org/'
Refreshing local packages (refresh existing: true)...
Looking for local package map at /var/lib/dub/packages/local-packages.json
Looking for local package map at /home/per/.dub/packages/local-packages.json
Ignoring version specification (~master) for path based dependency ../../
Ignoring version specification (~master) for path based dependency ../../
Ignoring version specification (~master) for path based dependency ../../
Note: Failed to determine version of package knetquery at .. Assuming ~master.
Refreshing local packages (refresh existing: false)...
Looking for local package map at /var/lib/dub/packages/local-packages.json
Looking for local package map at /home/per/.dub/packages/local-packages.json
Ignoring version specification (~master) for path based dependency ../../
Ignoring version specification (~master) for path based dependency ../../
Ignoring version specification (~master) for path based dependency ../../
Checking for missing dependencies.
Refreshing local packages (refresh existing: false)...
Looking for local package map at /var/lib/dub/packages/local-packages.json
Looking for local package map at /home/per/.dub/packages/local-packages.json
Ignoring version specification (~master) for path based dependency ../../
Ignoring version specification (~master) for path based dependency ../../
Ignoring version specification (~master) for path based dependency ../../
Checking for upgrades.
Using cached upgrade results...
Generating using build
Generate target knetquery (executable /home/per/Work/justd/knetquery knetquery)
Building knetquery ~master configuration "application", build type debug.
Compiling using dmd...
dmd -c -of.dub/build/application-debug-linux.posix-x86_64-dmd_2067-5C25CF5E0907389BD865D47A5E97D174/knetquery.o -debug -g -wi -version=Have_knetquery -I.. ../algorithm_ex.d ../arsd/characterencodings.d ../arsd/dom.d ../assert_ex.d ../backtrace/backtrace.d ../combinations.d ../dbg.d ../grammars.d ../ixes.d ../knet/base.d ../knet/cn5.d ../knet/decodings.d ../knet/folklex.d ../knet/languages.d ../knet/lectures/all.d ../knet/lemmas.d ../knet/moby.d ../knet/nell.d ../knet/origins.d ../knet/relations.d ../knet/roles.d ../knet/senses.d ../knet/swesaurus.d ../knet/synlex.d ../knet/thematics.d ../knet/wordnet.d ../mmfile_ex.d ../msgpack.d ../permutations.d ../predicates.d ../range_ex.d ../rational.d ../rcstring.d ../skip_ex.d ../sort_ex.d ../stemming.d ../traits_ex.d ../wordnet.d source/app.d

On Monday, 26 January 2015 at 22:33:21 UTC, Nordlöw wrote:
> Is there a way to tell DUB to compile all its sources as separate objects which are then fed to ld.

For reference here's the DUB build file for my project:

https://github.com/nordlow/justd/blob/master/knetquery/dub.json

On Monday, 26 January 2015 at 22:51:44 UTC, Nordlöw wrote:
> On Monday, 26 January 2015 at 22:33:21 UTC, Nordlöw wrote:
>> Is there a way to tell DUB to compile all its sources as separate objects which are then fed to ld.

And, yes, I'm using ld.gold as my default linker ;)

On Monday, 26 January 2015 at 22:46:46 UTC, Nordlöw wrote:
> On Monday, 26 January 2015 at 22:44:26 UTC, Nordlöw wrote:
>> Ok, then it's more than that. It takes 7 seconds:
>>
>> time dub build
>
> time dub build --combined --force
>
> is in the same order - around 7 seconds.

The only way I managed to change behaviour of DUB was through the --build-mode flag as

    time dub build --build-mode=singleFile

which makes it compile each .d file separately:

Building knetquery ~master configuration "application", build type debug.
Compiling using dmd...
Compiling ../algorithm_ex.d...
Compiling ../arsd/characterencodings.d...
Compiling ../arsd/dom.d...
Compiling ../assert_ex.d...
Compiling ../backtrace/backtrace.d...
Compiling ../combinations.d...
Compiling ../dbg.d...
Compiling ../grammars.d...
Compiling ../ixes.d...
Compiling ../knet/base.d...
../sort_ex.d(143): Warning: use std.algorithm.sort instead of .sort property
../sort_ex.d(168): Warning: use std.algorithm.sort instead of .sort property
Compiling ../knet/cn5.d...
Compiling ../knet/decodings.d...
Compiling ../knet/folklex.d...
Compiling ../knet/languages.d...
Compiling ../knet/lectures/all.d...
Compiling ../knet/lemmas.d...
Compiling ../knet/moby.d...
Compiling ../knet/nell.d...
Compiling ../knet/origins.d...
Compiling ../knet/relations.d...
Compiling ../knet/roles.d...
Compiling ../knet/senses.d...
Compiling ../knet/swesaurus.d...
Compiling ../knet/synlex.d...
Compiling ../knet/thematics.d...
Compiling ../knet/wordnet.d...
Compiling ../mmfile_ex.d...
Compiling ../msgpack.d...
Compiling ../permutations.d...
Compiling ../predicates.d...
Compiling ../range_ex.d...
Compiling ../rational.d...
Compiling ../skip_ex.d...
Compiling ../sort_ex.d...
Compiling ../stemming.d...
Compiling ../traits_ex.d...
Compiling ../wordnet.d...
Compiling source/app.d...
Linking...

This takes about twice as long...

- It doesn't seem to support parallelism here. Where's the -j --jobs flag?!
- And I can't see any caching in action either.

What's wrong?

Do I have to switch to SCons to get what I want?

I'm using DUB master HEAD.

On Monday, 26 January 2015 at 22:41:09 UTC, Adam D. Ruppe wrote:
> On Monday, 26 January 2015 at 22:33:21 UTC, Nordlöw wrote:
>> My project has grown beyound 10k lines of code.
>
> Eh, that's nothing, especially if they aren't templates. dmd will compile that in a fraction of a second (and even then, if you used phobos, reading the stdlib into memory will dominate the compile time)

Correction: I checked again. It's around 30k lines, including arsd/dom.d ;)

I'm using Phobos higher-order ranges whereever I can.

I meant that I try to make the inter-object interfaces in knet/*.d un-templatized whenever I can.

On Monday, 26 January 2015 at 22:33:21 UTC, Nordlöw wrote:
> Is there a way to tell DUB to compile all its sources as separate objects which are then fed to ld.
>
> And I don't want DMD to recompile every line in my project if I change only of them...and yes most of my cross-module interfaces are un-templatized.

Ping. Why no answers? Should i post this as a DUB issue on github instead?