So I was looking at this overly beefy compilation with an eye on what can cause it take so much memory and time.

Was using -vtemplates and was surprised to see tens of thousands (I mean literally: over 45,000 of them) of distinct instantiations of the obscure template AliasThisTypeOf.

What the heck is that? Turns out it's a private symbol (that for some reason turns out to be duplicated in druntime and phobos). That private symbol, in turn, is used in a suite of desperate templates of which drabness is only overcome by their mind-numbing repetitiveness:

BooleanTypeOf
IntegralTypeOf
FloatingPointTypeOf
CharTypeOf
StaticArrayTypeOf
DynamicArrayTypeOf
AssocArrayTypeOf

What are these used for? Glad you asked! All these support "alias this" for the std.format.format function.

In turn, format turns out to be used, often gratuitously, in very, very many places during compilation, particularly in std.traits itself (as an aside: format("%s[]")? Really? What happened with good old string concatenation?). And that creates an ungodly amount of template instantiations. See for yourselves: https://run.dlang.io/is/6j9WI2

But the main problem is the Saturnism that Phobos seems to be suffering from. A slow descent into madness.

I do understand the boiling frog slipping on the slippery slope toward the road to hell paved with whatever. I do. Heck, I even added a slab here and there. But if we look today at the code at https://github.com/dlang/phobos/blob/master/std/traits.d#L5819 with the sobriety and clearheadedness of a priest on a Sunday morning, the only reaction we should have is: "This madness has gotten too far and we must stop and turn around RIGHT NOW."

The code in the standard library represents the language, and how we'd like people to do things in the language. Sure, we may err on the side of optimizing things and throwing in some low-level code for efficiency purposes. But this code is not optimization; not algorithms; nothing of interest, really. It's mad scaffolding around a temple that should not have been built in the first place.

The madness has to stop.

On Friday, 7 May 2021 at 20:47:57 UTC, Andrei Alexandrescu wrote:
> In turn, format turns out to be used, often gratuitously, in very, very many places during compilation, particularly in std.traits itself (as an aside: format("%s[]")? Really? What happened with good old string concatenation?).

Except that string concatenation doesn't work with callback / output range based `toString` implementations.

On Friday, 7 May 2021 at 22:36:09 UTC, MoonlightSentinel wrote:
> On Friday, 7 May 2021 at 20:47:57 UTC, Andrei Alexandrescu wrote:
>> In turn, format turns out to be used, often gratuitously, in very, very many places during compilation, particularly in std.traits itself (as an aside: format("%s[]")? Really? What happened with good old string concatenation?).
>
> Except that string concatenation doesn't work with callback / output range based `toString` implementations.

Point taken, but I was referring to compile-time uses where that is practically never. There are zero instances of that case here: https://github.com/dlang/phobos/pull/8052. Almost in all cases the code that replaces calls to format() is shorter and simpler.

On 5/7/2021 4:58 PM, Andrei Alexandrescu wrote:

> This is a nice idea, thanks. I think the gating factor here is Phobos versioning, which requires language additions. Then it would be interesting to put together a white paper.
> 

That's a great idea. Doing a white paper, though, is not gated on language improvements. We know what those improvements need to accomplish, and shouldn't affect the redesign.

On Fri, May 07, 2021 at 11:58:06PM +0000, Andrei Alexandrescu via Digitalmars-d wrote:
> On Friday, 7 May 2021 at 23:19:30 UTC, SealabJaster wrote:
[...]
> > Having it all separated over multiple forum posts may be ok for discussions, but to see the overall view and direction, perhaps it is best to consolidate it all together somehow?
[...]
> This is a nice idea, thanks. I think the gating factor here is Phobos versioning, which requires language additions. Then it would be interesting to put together a white paper.

I think his point is to collect the current criticisms of Phobos into a single point of reference, perhaps on the wiki or something, that could serve as a list of improvements to be made.

Stuff posted in the forum will inevitably get forgotten in the dusts of time.


T

-- 
Beware of bugs in the above code; I have only proved it correct, not tried it. -- Donald Knuth

On Saturday, 8 May 2021 at 01:20:34 UTC, H. S. Teoh wrote:
> On Fri, May 07, 2021 at 11:58:06PM +0000, Andrei Alexandrescu via Digitalmars-d wrote:
>> On Friday, 7 May 2021 at 23:19:30 UTC, SealabJaster wrote:
> [...]
>> > Having it all separated over multiple forum posts may be ok for discussions, but to see the overall view and direction, perhaps it is best to consolidate it all together somehow?
> [...]
>> This is a nice idea, thanks. I think the gating factor here is Phobos versioning, which requires language additions. Then it would be interesting to put together a white paper.
>
> I think his point is to collect the current criticisms of Phobos into a single point of reference, perhaps on the wiki or something, that could serve as a list of improvements to be made.
>
> Stuff posted in the forum will inevitably get forgotten in the dusts of time.
>
>
> T

+1

It's the same thing at home, you need to pause and take a look. Then you see "omg I need to clean", and when you're done you never regret it.

But, you gotta be careful...
https://ibb.co/sH4VZcc

On 5/8/21 6:01 AM, Berni44 wrote:
> 
> - Let Phobos 1.0 be as is (maybe even remove the deprecation cycles).
> - Move over to Phobos 2.0 (e. g.) until summer 2023 (with a list of goals at hand, e.g. no auto-*coughing*, a better range/string concept, etc.). This will include several breaking changes.
> - End of 2022 Phobos 2.0 will be frozen: Nothing new is accepted, only bugfixes.
> - From summer 2023 on, Phobos 2.0 is stable. We guarantee backward compatibility forever, but things may be deprecated.
> - We may then start working on Phobos 2.1, which will be released in summer 2025 and frozen at end of 2024. And so on.

Couple of things missing here: (a) the rule of incrementalism and (b) the rule of nonduplication.

The rule of incrementalism is that code can use arbitrary parts of Phobos 2.0 in conjunction with arbitrary parts of Phobos 1.0. The migration path would be as simple as progressively replacing std.somemodule with std.v2.somemodule and get code to compile.

The rule of nonduplication is there should be no code in Phobos 2.0 that is identical to code in Phobos 1.0. Whenever Phobos 2.0 needs functionality of Phobos 1.0, it must be able to do so with ease.

These are much more difficult to achieve than it seems. However, they are existential - we must satisfy them if we want to survive.