On Monday, 24 February 2020 at 15:41:16 UTC, Steven Schveighoffer wrote:
> On 2/24/20 4:38 AM, Atila Neves wrote:
>> There's also the practical question of template instantiations and compile times even if the DIP that was being discussed were to be modified in the way suggested.
>
> I want to actually explore this.

<snip>

Thanks for the detailed write-up, there are a lot of good things here.

On Monday, 24 February 2020 at 16:22:22 UTC, Atila Neves wrote:
> Thanks for the detailed write-up, there are a lot of good things here.

We talked about this in the other thread, too.

I doubt the one template here will be a big deal. My experience is templates get bad when we use them in loops or recursion. One line of user code can easily expand a dozen separate instantiations.

Like in my jni thing, this one line: `DTypesToJni!(typeof(args))` expands to (args.length * 3) different templates (the impl is basically `alias it = AliasSeq!(convert!T[0], convert![T .. $])`). There's certainly the possibility it will be less, like foo(int, int) will be able to reuse that int one, but it is still work for the compiler to do.

Various std.typecons type things and std.algorithm builders can easily grow hugely - a range pipeline has templates returning templates based on templated input types.

But back to mine, since its intent is to be used in a __traits(allMembers) loop with different things each time, it can easily be hundreds of templates in two lines of user code.

Now, contrast that to a tuple builder literal, which has no internal recursion at all. It is just one instantiation. Even in a loop we might use in code generation:

static foreach(member; __traits(allMembers, T))
  mixin(i"void $member() {}".idup);


Well, each unique i"" only gives one template. That call to idup is a second one. So we are at two instantiations... no matter what the input is.


It is certainly possible to write code that does a bad job with it. But it is harder than you think. Even doing what I so often tell people not to do:


static foreach(member; __traits(allMembers, T))
  mixin(i"$(__traits(getMember, T, member).stringof) $member() {}".idup);


stuff like that with stringof... is still only two instantiations for any T. The string is still a literal and the types of the arguments are still the same.

I think you'd have to actively try to make more than a constant two template instantiations out of the i"" string. Natural uses will tend to give just one for the string itself and one for the function you call with it (which btw is already a cost we're paying - writeln's variadic template list does the same as it is! So it is really an increase of one template per use, constant. No multiplicative growth like with typical recursive templates, or worse as we see in the really pathological cases.)

On Monday, 24 February 2020 at 09:51:16 UTC, Walter Bright wrote:
> On 2/24/2020 12:43 AM, Robert M. Münch wrote:
>> I mean, people spend a lot of time thinking, making suggestions, etc. and the end result is: we now have nothing. Which, IMO is the worst result for all.
>
> Not totally. I was unable to convince people that printf format validation was an orthogonal feature. So I went ahead and implemented that to show it had nothing to do with the proposal:
>
> https://github.com/dlang/dmd/pull/10812

With the way that the DIP you wrote up, it really isn't orthogonal. How people *expect* string interpolation to work is like this:

   float a;
   int b;

   printf(i"$a $b"); // should just work


What your DIP did is this:


   float a;
   int b;

   // printf(i"$a $b"); becomes:
   printf("%s %s", a, b);

   printf(i"${%d}a ${%f}b"); // need to do formatting on your own anyways

The only way you wouldn't want printf format validation for such a feature is if you are fine with buggy code. You literally still require the user to format the strings themselves, this is no different than simply using printf. And guess what:

> It turned out kinda sweet, and found a huge number of latent bugs in our own code, proving we shoulda done that long ago :-) Bad printfs turned out to be much more common than I'd anticipated.

No shit.

On Mon, Feb 24, 2020 at 10:41:16AM -0500, Steven Schveighoffer via Digitalmars-d-announce wrote: [...]
> I will also note that we did something very similar with switch(string), where prior to this the compiler did all the "heavy lifting" of generating the code to convert strings into integers. Now it's done via a template which accepts all the strings as parameters. I don't think it was a bad change, and I'm sure it increased compile times and compiler memory usage.

No kidding, at one point it was making Phobos uncompilable except for machines with huge gobs of RAM, because std.datetime had this gigantic switch statement over, like, *every* locale name that exists in the world, and it was causing a gigantic explosion of template bloat. I think the solution that was eventually adopted was to replace that switch with something else, which is not altogether a bad thing I suppose, but still, it does put to question the practicality of having a template generate code for a switch statement with >1000 string items.


> We shouldn't shy away from this type of stuff, but rather invest in making it better.
[...]

Agreed.  I think after newCTFE is finally merged, the next major item to tackle is how to improve the performance of template-heavy code.


T

-- 
Don't modify spaghetti code unless you can eat the consequences.

On 2020-02-24 10:02:26 +0000, Mike Parker said:

> The DIP review process is outlined here:
> 
> https://github.com/dlang/DIPs
> 
> The final decision rests with the language maintainers. Now, that means Walter and Atila.

Thanks, and sorry for my ignorance... How about adding a note who the "language maintainers" are?

> ... If that proposal and its review manage to convince Walter and Atila, then the feature gets in.

Ok. I understand.

> The DIP process is one way to approach that. It allows everyone to provide feedback and influence the drafting of the DIP. That requires convincing the DIP author to revise the DIP and ultimately convincing the language maintainers to accept it. It means there's a high barrier for acceptance, but in my own opinion that's how it should be.

I agree. My impression and point was (after following the topic loosly) that the DIP might be close to find a good solution and that it was suddenly withdrawn. But anyway...


-- 
Robert M. Münch
http://www.saphirion.com
smarter | better | faster

On 2020-02-24 09:51:16 +0000, Walter Bright said:

> I talk it over with Atila after the review threads are done, then decide. Voting is a terrible way to do engineering design, for many reasons I can list if you like.

You don't need to, I'm not a fan of such a voting approach too. I just added a bunch of variants as I didn't know how the process works nor who is involved at what step.

> Not totally. I was unable to convince people that printf format validation was an orthogonal feature. So I went ahead and implemented that to show it had nothing to do with the proposal:
> 
> https://github.com/dlang/dmd/pull/10812
> 
> It turned out kinda sweet, and found a huge number of latent bugs in our own code, proving we shoulda done that long ago :-) Bad printfs turned out to be much more common than I'd anticipated.

Proving by showing the results... Ok, looking forward how things develop.

-- 
Robert M. Münch
http://www.saphirion.com
smarter | better | faster

On 2/24/20 12:31 PM, H. S. Teoh wrote:
> On Mon, Feb 24, 2020 at 10:41:16AM -0500, Steven Schveighoffer via Digitalmars-d-announce wrote:
> [...]
>> I will also note that we did something very similar with
>> switch(string), where prior to this the compiler did all the "heavy
>> lifting" of generating the code to convert strings into integers. Now
>> it's done via a template which accepts all the strings as parameters.
>> I don't think it was a bad change, and I'm sure it increased compile
>> times and compiler memory usage.
> 
> No kidding, at one point it was making Phobos uncompilable except for
> machines with huge gobs of RAM, because std.datetime had this gigantic
> switch statement over, like, *every* locale name that exists in the
> world, and it was causing a gigantic explosion of template bloat. I
> think the solution that was eventually adopted was to replace that
> switch with something else, which is not altogether a bad thing I
> suppose, but still, it does put to question the practicality of having a
> template generate code for a switch statement with >1000 string items.

The solution was to forward templates over a certain number of string elements to a single runtime function with an array created from the tuple list (which is a minimal wrapper) instead of a binary search via unrolled templates: https://github.com/dlang/druntime/blob/87bf88ba88d2926d7a470b5e126e0a5b1a1e266f/src/core/internal/switch_.d#L60-L73. It's unfortunate we even have to use CTFE to generate the immutable strings.

I wouldn't imagine a much different approach for this proposal as well -- convert to something useful at runtime and then call a runtime-only implementation.

As I said before, if we can make it so such "wrappers" are just forwarders, it would make the resulting code less bloated, and probably save on memory as well.

-Steve

Having the compiler lower string interpolation to some hidden template is - AST macros. We're not doing AST macros.

Hidden user-defined semantics are not for D. Every language I'm familiar with that supports it wound up with users creating their own completely undocumented personal language that nobody else can use or has the remotest interest in using.

Including me.

Exhibit A: https://www.boost.org/doc/libs/1_72_0/libs/preprocessor/doc/index.html

Everyone agrees, but says "but I know how to do AST macros in a sane way." They're wrong.

If someone sees an i"abc" in the source code, they should be able to look in the language spec and know exactly what it does. They should not have to go trolling through imports looking for the definition of some template they never heard of.

On Monday, 24 February 2020 at 19:35:16 UTC, Walter Bright wrote:
> Having the compiler lower string interpolation to some hidden template is - AST macros. We're not doing AST macros.

This is untrue.

> Hidden user-defined semantics are not for D.

We are NOT calling for this.


What, exactly, do you think we're proposing?

On Monday, 24 February 2020 at 19:45:49 UTC, Adam D. Ruppe wrote:
> On Monday, 24 February 2020 at 19:35:16 UTC, Walter Bright wrote:
>> Having the compiler lower string interpolation to some hidden template is - AST macros. We're not doing AST macros.
>
> This is untrue.
>
>> Hidden user-defined semantics are not for D.
>
> We are NOT calling for this.
>
>
> What, exactly, do you think we're proposing?

I knew it. You needs an actual implementation here, to show walter what you are talking about. This reminds me of the whole mangle only discussion a while back. It took an actual PR for Walter to accept it instead of days arguing with him.

Let get to work on this.

-Alex