On Wednesday, 21 November 2018 at 08:07:52 UTC, Vladimir Panteleev wrote:
> https://blog.thecybershadow.net/2018/11/18/d-compilation-is-too-slow-and-i-am-forking-the-compiler/

Very interesting.

I'm also currently working on a project to save my bloodstream from the cortisol drip that happens when anything a computer does takes over a second, which these days means waiting for dmd to compile my code so I can see the result of the tests. I'll share more details when I have time.

But: one of the things I want to do is a version of this / precompiled headers. I've complained before that compiling std.path with -unittest takes forever (0.5s or so, and most of it due to std.uni). That's a price I pay every time I make a one line change to any file, and the linker hasn't even been invoked yet. Here's the thing: Phobos only changes from one release to the next. Why am I waiting to recompile a read-only file that won't change unless I update the compiler, over and over again?

I'd love it if I could precompile Phobos and just use the digested information every time I'm iterating.

On Wednesday, 21 November 2018 at 11:35:02 UTC, Atila Neves wrote:
> I'm also currently working on a project to save my bloodstream from the cortisol drip that happens when anything a computer does takes over a second, which these days means waiting for dmd to compile my code so I can see the result of the tests. I'll share more details when I have time.

Looking forward to it!

> But: one of the things I want to do is a version of this / precompiled headers. I've complained before that compiling std.path with -unittest takes forever (0.5s or so, and most of it due to std.uni). That's a price I pay every time I make a one line change to any file, and the linker hasn't even been invoked yet. Here's the thing: Phobos only changes from one release to the next. Why am I waiting to recompile a read-only file that won't change unless I update the compiler, over and over again?

That particular problem is in large part due to that the -unittest switch is not namespaced. I ran into the same issue with -allinst - with std.path, it breaks only if -unittest is also specified. I don't want to compile std.path unit tests, just my own!

Have we tried disabling -unittest for modules that aren't on the compiler's command line yet (or, in case of -i, not excluded)?

> I'd love it if I could precompile Phobos and just use the digested information every time I'm iterating.

I agree, it would be nice if we could ship some "precompiled module" files along with Phobos .lib / .so files, but it looks like implementing this feature correctly might be non-trivial.

Maybe this hack could be developed further into a more generic "compiler server" idea.

On Wednesday, 21 November 2018 at 11:58:25 UTC, Vladimir Panteleev wrote:
> Have we tried disabling -unittest for modules that aren't on the compiler's command line yet (or, in case of -i, not excluded)?

Not that I know of, thats a great idea!

> Maybe this hack could be developed further into a more generic "compiler server" idea.

Wasn't that Robert's Masters thesis (Dconf 2013(?) presentation)? ;)

On Wednesday, 21 November 2018 at 13:05:27 UTC, Nicholas Wilson wrote:
> On Wednesday, 21 November 2018 at 11:58:25 UTC, Vladimir Panteleev wrote:
>> Have we tried disabling -unittest for modules that aren't on the compiler's command line yet (or, in case of -i, not excluded)?
>
> Not that I know of, thats a great idea!
>
>> Maybe this hack could be developed further into a more generic "compiler server" idea.
>
> Wasn't that Robert's Masters thesis (Dconf 2013(?) presentation)? ;)

The main problem with this, is the amount of context a compilers needs.
And the current design of DMD does not lend itself splitting out the context.
If you wanted you could consider the semantic pass of the compiler as a database, which answers queries such as:

 - which size does this type have.
 - which arguments does this function have
 - can the type A be casted to type B
 - which conversion function should be invoked for (B)A ?
 - is this function known to be pure?

The data-base containing this information needs to be maintained on the compile-nodes, and that possibly leads to many data-dependencies.
Which may degrade the performance of the "compiler server" to the point where it is quicker to do it locally.

I am currently working (albeit very slowly due to lack of time and focus) to enable programmers to circumvent slow parts in compiler. When completed this should make a compiler-server unnecessary for some time to come.

On Wednesday, 21 November 2018 at 13:05:27 UTC, Nicholas Wilson wrote:
> On Wednesday, 21 November 2018 at 11:58:25 UTC, Vladimir Panteleev wrote:
>> Have we tried disabling -unittest for modules that aren't on the compiler's command line yet (or, in case of -i, not excluded)?
>
> Not that I know of, thats a great idea!


Yes it's sadly a well-known problem e.g. https://github.com/dlang/dmd/pull/8124

On Wednesday, 21 November 2018 at 11:18:22 UTC, Nicholas Wilson wrote:
> On Wednesday, 21 November 2018 at 08:07:52 UTC, Vladimir Panteleev wrote:
>> https://blog.thecybershadow.net/2018/11/18/d-compilation-is-too-slow-and-i-am-forking-the-compiler/
>
> This is #2 on HN at the moment.

I would be wary of such titles.

Whatever will remain in minds will be something like "D compilation is slow" which isn't accurate when compared to the competition.

The article is clever, the title is clever, but most people will only read the title.

On Wednesday, 21 November 2018 at 11:58:25 UTC, Vladimir Panteleev wrote:
> On Wednesday, 21 November 2018 at 11:35:02 UTC, Atila Neves wrote:
>> [...]
>
> Looking forward to it!
>
>> [...]
>
> That particular problem is in large part due to that the -unittest switch is not namespaced. I ran into the same issue with -allinst - with std.path, it breaks only if -unittest is also specified. I don't want to compile std.path unit tests, just my own!
>
> Have we tried disabling -unittest for modules that aren't on the compiler's command line yet (or, in case of -i, not excluded)?

I'm not sure what the real issue is here or what the solution would be. There was a PR to fix it that was closed without merging.

On Wednesday, 21 November 2018 at 11:18:22 UTC, Nicholas Wilson wrote:
> This is #2 on HN at the moment.

Also on reddit:

https://www.reddit.com/r/programming/comments/9z36xg/d_compilation_is_too_slow_and_i_am_forking_the/

On 11/21/2018 3:19 AM, Iain Buclaw wrote:
> On Wednesday, 21 November 2018 at 10:56:02 UTC, Walter Bright wrote:
>>
>> Wouldn't it be awesome to have the lexing/parsing of the imports all done in parallel? The main difficulty in getting that to work is dealing with the shared string table.
> 
> What about creating a new Fiber for each module needing lexing/parsing/semantic to be ran?  Compilation of one module would then get as far as it can until it needs to defer, then calls yield() to continue compilation of the next module.  This in hope that when the round trip returns, the AST will be sufficiently complete enough for compilation to continue.

Since the lexing/parsing does not do any blocking calls, fibers won't help. It has to be another hardware thread.

Trying to parallelize semantic over multiple modules will be very hard to do.

On 11/21/2018 5:55 AM, Guillaume Piolat wrote:
> On Wednesday, 21 November 2018 at 11:18:22 UTC, Nicholas Wilson wrote:
>> On Wednesday, 21 November 2018 at 08:07:52 UTC, Vladimir Panteleev wrote:
>>> https://blog.thecybershadow.net/2018/11/18/d-compilation-is-too-slow-and-i-am-forking-the-compiler/ 
>>>
>>
>> This is #2 on HN at the moment.
> 
> I would be wary of such titles.
> 
> Whatever will remain in minds will be something like "D compilation is slow" which isn't accurate when compared to the competition.
> 
> The article is clever, the title is clever, but most people will only read the title.

Unfortunately, you're right. The title will leave the impression "D is slow at compiling". You have to carefully read the article to see otherwise, and few will do that.