On Tuesday, April 03, 2018 21:08:48 Andrei Alexandrescu via Digitalmars-d wrote:
> On 04/03/2018 05:53 PM, bachmeier wrote:
> > On Tuesday, 3 April 2018 at 21:17:35 UTC, Rubn wrote:
> >> I feel that's probably the case for any comparisons across two languages, you are going to have a person that is more knowledgeable in one language than another. Mistakes are going to be made, but I think it should be blatantly obvious that one language is going to compiler slower if it is compiling all the unittests for a library compared to one that isn't. That's just blatant bias against D, not a mistake from misunderstanding Go.
> >
> > Yeah, I don't understand that either. Unit tests can be arbitrarily large, so no matter how fast the compiler, it would always be possible to make it take longer than any other language.
>
> Exactly, which is why I'm insisting this - and not compiler benchmarking, let alone idle chattaroo in the forums - is where we need to hit. What we have here, ladies and gentlemen, is a high-impact preapproved item of great general interest. Shall we start the auction?

Steven already said somewhere in here that he's planning to work on this during the dconf hackathon. So, at least one person has stepped forward and said that they'll look into it.

- Jonathan M Davis

On Wednesday, 4 April 2018 at 01:08:48 UTC, Andrei Alexandrescu wrote:
> [ ... ]
>
> Exactly, which is why I'm insisting this - and not compiler benchmarking, let alone idle chattaroo in the forums - is where we need to hit. What we have here, ladies and gentlemen, is a high-impact preapproved item of great general interest. Shall we start the auction?

I have analyzed the problem;
And in this case it points to _std.uni_ as the main time-taker.

I have also created and uploaded a video where I show how to do this kind of profiling.
It got a bit carried away at the beginning so you can skip to 10:45 :)
https://www.youtube.com/watch?v=28bOtu64CCM

For those are not up to watching me talk slowly:
Phobos is in dire need of having dependencies broken.

And having dedicated compile-time profiling tools is pretty useful.
Even if they are very rudimentary it's better then semi-randomly commenting out code.

--
Stefan

On Wednesday, 4 April 2018 at 19:25:43 UTC, Stefan Koch wrote:
> On Wednesday, 4 April 2018 at 01:08:48 UTC, Andrei Alexandrescu wrote:
>> [ ... ]
>>
>> Exactly, which is why I'm insisting this - and not compiler benchmarking, let alone idle chattaroo in the forums - is where we need to hit. What we have here, ladies and gentlemen, is a high-impact preapproved item of great general interest. Shall we start the auction?
>
> I have analyzed the problem;
> And in this case it points to _std.uni_ as the main time-taker.

Could be because of huge tables. Does it point specifically to std.uni or its dependencies might count as well?

Anyway to import a binary file as an array? Turbo Pascal had a feature like that, it was awesome ;)

On other thought - maybe just .di + separate compilation is it.

On Wednesday, 4 April 2018 at 01:08:48 UTC, Andrei Alexandrescu wrote:
> Exactly, which is why I'm insisting this - and not compiler benchmarking, let alone idle chattaroo in the forums - is where we need to hit. What we have here, ladies and gentlemen, is a high-impact preapproved item of great general interest. Shall we start the auction?

Are you aware of this PR? https://github.com/dlang/dmd/pull/8124

On Wednesday, 4 April 2018 at 20:04:04 UTC, Jack Stouffer wrote:
> On Wednesday, 4 April 2018 at 01:08:48 UTC, Andrei Alexandrescu wrote:
>> Exactly, which is why I'm insisting this - and not compiler benchmarking, let alone idle chattaroo in the forums - is where we need to hit. What we have here, ladies and gentlemen, is a high-impact preapproved item of great general interest. Shall we start the auction?
>
> Are you aware of this PR? https://github.com/dlang/dmd/pull/8124

This is but a layer of paint over the real problem.
Unneeded Dependencies. Programming should not be a game of jenga.
Piling things on top of other things rarely works out.

On Wednesday, 4 April 2018 at 20:29:19 UTC, Stefan Koch wrote:
> On Wednesday, 4 April 2018 at 20:04:04 UTC, Jack Stouffer wrote:
>> On Wednesday, 4 April 2018 at 01:08:48 UTC, Andrei Alexandrescu wrote:
>>> Exactly, which is why I'm insisting this - and not compiler benchmarking, let alone idle chattaroo in the forums - is where we need to hit. What we have here, ladies and gentlemen, is a high-impact preapproved item of great general interest. Shall we start the auction?
>>
>> Are you aware of this PR? https://github.com/dlang/dmd/pull/8124
>
> This is but a layer of paint over the real problem.
> Unneeded Dependencies. Programming should not be a game of jenga.
> Piling things on top of other things rarely works out.

I encourage you to leave a review on the PR then with corrective measures.

On Wednesday, 4 April 2018 at 20:29:19 UTC, Stefan Koch wrote:
> On Wednesday, 4 April 2018 at 20:04:04 UTC, Jack Stouffer wrote:
>> On Wednesday, 4 April 2018 at 01:08:48 UTC, Andrei Alexandrescu wrote:
>>> Exactly, which is why I'm insisting this - and not compiler benchmarking, let alone idle chattaroo in the forums - is where we need to hit. What we have here, ladies and gentlemen, is a high-impact preapproved item of great general interest. Shall we start the auction?
>>
>> Are you aware of this PR? https://github.com/dlang/dmd/pull/8124
>
> This is but a layer of paint over the real problem.
> Unneeded Dependencies. Programming should not be a game of jenga.
> Piling things on top of other things rarely works out.

Having unittests included from precompiled libraries is a problem in and of itself.  This is causing many templates to be instantiated that will never be used by the application, killing compilation time.

There are also other problems...here's a link to my description of "Lazy Imports" that I think would help other issues we currently have.

https://github.com/marler8997/dlangfeatures#lazy-imports

On Wed, Apr 04, 2018 at 08:29:19PM +0000, Stefan Koch via Digitalmars-d wrote:
> On Wednesday, 4 April 2018 at 20:04:04 UTC, Jack Stouffer wrote:
> > On Wednesday, 4 April 2018 at 01:08:48 UTC, Andrei Alexandrescu wrote:
> > > Exactly, which is why I'm insisting this - and not compiler benchmarking, let alone idle chattaroo in the forums - is where we need to hit. What we have here, ladies and gentlemen, is a high-impact preapproved item of great general interest. Shall we start the auction?
> > 
> > Are you aware of this PR? https://github.com/dlang/dmd/pull/8124
> 
> This is but a layer of paint over the real problem.  Unneeded Dependencies. Programming should not be a game of jenga.  Piling things on top of other things rarely works out.

Unneeded dependencies is (partly) addressed by this PR, by isolating
user code from the unittests (and by extension, their dependencies) of
external libraries that said user code has no interest in.

True, this does not solve the entire problem, which is that too much of Phobos is involved in a hairball of inextricable dependencies.  The situation actually has already improved since 3-4 years ago, when it was much, much worse.  It's just that we have still some ways to go.

On the flip side, though, why *shouldn't* unittests in a Phobos module make use of other Phobos modules?  It's one thing to say that Phobos code proper (i.e., outside of unittests) should be as independent as possible, and that I agree with, strongly.  But unittests should be free to import other stuff in order to make testing easier, and also ddoc'd unittest examples more relevant to the reader.  By separating the import of a module from the importing of its unittests (which doesn't even make sense from an API perspective -- why should downstream user code need to pull in upstream unittests, which are supposed to be used only for upstream development?), we provide freedom for upstream library authors to make use of external components in order to test their code, without penalizing downstream user code with dependencies they don't care about.

I'd say this PR strikes at the heart of this problem, it's not just papering over the issue.  It's not a full solution, but it's an important step towards a full solution.


T

-- 
English is useful because it is a mess. Since English is a mess, it maps well onto the problem space, which is also a mess, which we call reality. Similarly, Perl was designed to be a mess, though in the nicest of all possible ways. -- Larry Wall

On Wednesday, 4 April 2018 at 20:02:56 UTC, Dmitry Olshansky wrote:
> On Wednesday, 4 April 2018 at 19:25:43 UTC, Stefan Koch wrote:
>> On Wednesday, 4 April 2018 at 01:08:48 UTC, Andrei Alexandrescu wrote:
>>> [ ... ]
>>>
>>> Exactly, which is why I'm insisting this - and not compiler benchmarking, let alone idle chattaroo in the forums - is where we need to hit. What we have here, ladies and gentlemen, is a high-impact preapproved item of great general interest. Shall we start the auction?
>>
>> I have analyzed the problem;
>> And in this case it points to _std.uni_ as the main time-taker.
>
> Could be because of huge tables. Does it point specifically to std.uni or its dependencies might count as well?
>
> Anyway to import a binary file as an array? Turbo Pascal had a feature like that, it was awesome ;)
>
> On other thought - maybe just .di + separate compilation is it.

There is the string import feature, which can give you a char array which you can then cast to ubyte slice and from there to a ubyte pointer you cast that to a pointer of the target type and slice it. This requires the target type to have no pointers inside of it.