I have a huge code base... it seems D likes to recompile everything every time from scratch. I realize there are ways to fix this but can't D go a step further?

Much of the code does not change. D has to compile a file per file basis.

Could D not just has the file in such a way as to reuse parts of previous compilations?

e.g., each function in the file could be hashed and if no changes are made then previous code generation is used.

There should be absolutely no reason a function, specially pure functions have to be regenerated every compilation.

By proper hashing design one should be able to almost limit compilation to only the changes that matter.

If a file, module, package does not change it should not be recompiled but reused. If a file's internals do not change in specific ways then they too generally can be used.

This would probably not be too complex of a problem if integrated in with D's parser. Each node in the AST is associated with a hash of the text(or it could be the AST node itself in some way but the text should do).

All code blocks need to determine what escapes and the object code should express those escapes in some way so they can be fixed up(which is basically global references and potential meta changes). If the compiler monitors and records this stuff proper then most of the work is already done.

Then it is just a matter of reading in the object code at the right time to replace the old code.

While we can compile to individual object files this requires user interaction. I am thinking of D having one large file that contains all the info that it can use to reduce compilation times by reusing information rather than regenerating. Files that do not change text and have do not escape references do not have to be recompiled. The compiler itself should handle this, not the user. For individual files that do change, a more granular approach is used. This requires every object file to also include the granularity though for maximum speed.

On Saturday, 19 October 2019 at 21:56:44 UTC, Brett wrote:
> I have a huge code base... it seems D likes to recompile everything every time from scratch. I realize there are ways to fix this but can't D go a step further?
>
> [...]

I think this might answer some of your questions:

https://forum.dlang.org/post/mailman.484.1570820437.8294.dlang-internal@puremagic.com

tl;dr: there's a PR ...

On Sat, Oct 19, 2019 at 09:56:44PM +0000, Brett via Digitalmars-d wrote:
> I have a huge code base... it seems D likes to recompile everything every time from scratch. I realize there are ways to fix this but can't D go a step further?

Are you using dub?  If you are, you might want to consider using a different build tool instead. One of the reasons I gave up dub long ago was because of its compulsion to rebuild everything from scratch every single time, which is unbearably slow. Now that I have my own build system setup with SCons, I'm much happier.

Running dmd without dub is actually incredibly fast (compared to, say, C++) that you generally only need separate compilation for packages. Compiling all modules in a single package at once is generally fast enough that there's no need to go to the per-file level of granularity.


T

-- 
"I suspect the best way to deal with procrastination is to put off the procrastination itself until later. I've been meaning to try this, but haven't gotten around to it yet. " -- swr

On Sunday, 20 October 2019 at 05:44:14 UTC, H. S. Teoh wrote:
> On Sat, Oct 19, 2019 at 09:56:44PM +0000, Brett via Digitalmars-d wrote:
>> I have a huge code base... it seems D likes to recompile everything every time from scratch. I realize there are ways to fix this but can't D go a step further?
>
> Are you using dub?  If you are, you might want to consider using a different build tool instead. One of the reasons I gave up dub long ago was because of its compulsion to rebuild everything from scratch every single time, which is unbearably slow. Now that I have my own build system setup with SCons, I'm much happier.
>
> Running dmd without dub is actually incredibly fast (compared to, say, C++) that you generally only need separate compilation for packages. Compiling all modules in a single package at once is generally fast enough that there's no need to go to the per-file level of granularity.
>
>
> T

In a dub package you can also organize your modules into sub packages. These sub packages are only compiled when a source module of it is touched.

This might already solve the problem. But yes, there are a lot of potential in Dub to make it faster.

Kind regards
Andre