On Wed, Apr 22, 2020 at 11:35 PM Manu <turkeyman@gmail.com> wrote:

> On Wed, Apr 22, 2020 at 11:20 PM Steven Schveighoffer via Digitalmars-d < digitalmars-d@puremagic.com> wrote:
>
>> On 4/22/20 8:04 AM, Manu wrote:
>> > We have a compile time problem, and this is basically the cure. Intuitively, people imagine CTFE is expensive (and it kinda is), but really, the reason our compile times are bad is template instantiation.
>> >
>> > This DIP single-handedly fixes compile-time issues in programs I've written by reducing template instantiations by near-100%, in
>> particular,
>> > the expensive ones; recursive instantiations, usually implementing some form of static map.
>> >
>> > https://github.com/dlang/DIPs/pull/188
>> >
>> > This is an RFC on a draft, but I'd like to submit it with a reference implementation soon.
>> >
>> > Stefan Koch has helped me with a reference implementation, which has so far gone surprisingly smoothly, and has shown 50x improvement in
>> compile
>> > times in some artificial tests.
>>
>> Yes please! Where is the reference implementation? I want to try some things out.
>>
>
> The link is in the DIP.
> Most tests we've done are working, except template instantiation expansion
> is not yet implemented: ie, Thing!(Tuple, x, y)...
> That's got a lot of implementation baggage attached to it in DMD.
>

Stefan worked out how to make the TemplateInstance case work. The branch is
updated, and it's working now!!
If you wanna do some tests, it'd be really interesting to see how much it
helps your code, and also help find edge cases and bugs.

>

On Wednesday, 22 April 2020 at 12:04:30 UTC, Manu wrote:
> [...]
>
> https://github.com/dlang/DIPs/pull/188
>
> [...]

looking at it again, I actually quite start to like it because when I first looked at it I thought it would start to be a C++ mess like allowing (Tuple + )... to basically work like AST macros and sum together a tuple. But actually thinking about ... as a unary operator like described in the DIP makes it quite clear.

But how exactly would it handle parentheses? For the most simple implementation I would expect (Tup + 10)[2]... to error because it would first try to evaluate (Tup + 10) like in current D, and thus error, and after that try to expand the Result[2]... - For correct syntax I would think of (Tup + 10)...[2] here

If both (Tup + 10)[2]... and (Tup + 10)...[2] however are going to perform the same thing, I can see multiple potential pitfalls because the compiler tries to be too smart.

It also says it goes through the expr tree but recursing through the entire expression tree might introduce unexpected issues and would be a non-intuitive experience to the programmer. Take a call like `X!(T!Tup)...` for example: if normally `T!Tup` expects a Tuple as argument but if the ... would go through the entire expression tree, it would change it to: `X!(T!(Tup[0]), T!(Tup[1]), ..., T!(Tup[$-1]))` - performing unwanted expansion of T!Tup when even though I wanted `X!(T!(Tup)[0]), X!(T!(Tup)[1]), ..., X!(T!(Tup)[$-1])`

About the part "f(Items.MemberTup...)" I sent in another reply I think it would very much make sense that it just expands to f(Items[0].MemberTup, Items[1].MemberTup, ...) because when doing an access to a non-tuple member it's also not actually existing on the whole tuple list but only on individual items.

I could actually much rather see a use-case of accessing all members of MemberTup inside the Items tuple, which could probably be realized using __traits(getMember, Items, MemberTup)...

On Wednesday, 22 April 2020 at 12:04:30 UTC, Manu wrote:
> This DIP single-handedly fixes compile-time issues in programs I've written by reducing template instantiations by near-100%, in particular, the expensive ones; recursive instantiations, usually implementing some form of static map.
>
> We should have done this a long time ago.

This is beautiful and awesome (syntax and all).

I was wondering if there's any way to to do a cross product with this, like fun(Xs, Ys)... expand to fun(Xs[0], Ys[0]), fun(Xs[0], Ys[1]), fun(Xs[1], Ys[0]), fun(Xs[1], Ys[1]), but that might very well be rare enough to not warrant special consideration.

The other thing that worries me a little is the difference between Foo!(AliasSeq!(1,2))... and Foo!(Numbers)... - would Foo!(AliasSeq!(1,2)...)... do the same as Foo!(Numbers)...?

--
  Simen

On 4/23/20 8:36 AM, WebFreak001 wrote:
> On Wednesday, 22 April 2020 at 12:04:30 UTC, Manu wrote:
>> [...]
>>
>> https://github.com/dlang/DIPs/pull/188
>>
>> [...]
> 
> looking at it again, I actually quite start to like it because when I first looked at it I thought it would start to be a C++ mess like allowing (Tuple + )... to basically work like AST macros and sum together a tuple. But actually thinking about ... as a unary operator like described in the DIP makes it quite clear.
> 
> But how exactly would it handle parentheses? For the most simple implementation I would expect (Tup + 10)[2]... to error because it would first try to evaluate (Tup + 10) like in current D, and thus error, and after that try to expand the Result[2]... - For correct syntax I would think of (Tup + 10)...[2] here

If are looking for the second item out of the tuple, your last expression is correct. However, it's kind of silly not to just do:

Tup[2] + 10

The first instance is not invalid, it's going to be:

(Tup[0] + 10)[2], (Tup[1] + 10)[2], ...

which could be valid for some types (they could support opBinary!"+" and opIndex)

> 
> If both (Tup + 10)[2]... and (Tup + 10)...[2] however are going to perform the same thing, I can see multiple potential pitfalls because the compiler tries to be too smart.

I think they will be different because of where the ... operator is applied.

> 
> It also says it goes through the expr tree but recursing through the entire expression tree might introduce unexpected issues and would be a non-intuitive experience to the programmer. Take a call like `X!(T!Tup)...` for example: if normally `T!Tup` expects a Tuple as argument but if the ... would go through the entire expression tree, it would change it to: `X!(T!(Tup[0]), T!(Tup[1]), ..., T!(Tup[$-1]))` - performing unwanted expansion of T!Tup when even though I wanted `X!(T!(Tup)[0]), X!(T!(Tup)[1]), ..., X!(T!(Tup)[$-1])`
> 

I had a discussion with Manu on this elsewhere in the thread -- I believe he is going to update the DIP to clarify this situation. In essence, you have to first evaluate the T!Tup expression into its tuple, and then use the result inside the expanding expression for it to work properly.

> About the part "f(Items.MemberTup...)" I sent in another reply I think it would very much make sense that it just expands to f(Items[0].MemberTup, Items[1].MemberTup, ...) because when doing an access to a non-tuple member it's also not actually existing on the whole tuple list but only on individual items.

Yep.

> 
> I could actually much rather see a use-case of accessing all members of MemberTup inside the Items tuple, which could probably be realized using __traits(getMember, Items, MemberTup)...
> 

That would be cool. Of course, MemberTup has to be a list of names, but a very nice feature -> convert a list of names into a list of members without messy templates!

Manu -- another clarification in the DIP is needed here -- __traits is not a template, but I think it should be treated similarly. That is, __traits(allMembers, Foo)... should be equivalent to __traits(allMembers, Foo)

-Steve

On Thursday, 23 April 2020 at 12:43:59 UTC, Simen Kjærås wrote:
> On Wednesday, 22 April 2020 at 12:04:30 UTC, Manu wrote:
>> This DIP single-handedly fixes compile-time issues in programs I've written by reducing template instantiations by near-100%, in particular, the expensive ones; recursive instantiations, usually implementing some form of static map.
>>
>> We should have done this a long time ago.
>
> This is beautiful and awesome (syntax and all).
>
> I was wondering if there's any way to to do a cross product with this, like fun(Xs, Ys)... expand to fun(Xs[0], Ys[0]), fun(Xs[0], Ys[1]), fun(Xs[1], Ys[0]), fun(Xs[1], Ys[1]), but that might very well be rare enough to not warrant special consideration.
>

doing the cartesian product of the tuples was in my first implementation.
It's not too useful in the general case though.

On Thursday, 23 April 2020 at 13:38:10 UTC, Steven Schveighoffer wrote:
>
> Manu -- another clarification in the DIP is needed here -- __traits is not a template, but I think it should be treated similarly. That is, __traits(allMembers, Foo)... should be equivalent to __traits(allMembers, Foo)
>
> -Steve

It already is. Tuples without operations are just expanded verbatim same as the regular expansion.
I've started collaborating on the DIP since Manu and me are in opposite places of the earth that means 24/7 support!

On Thursday, 23 April 2020 at 13:38:10 UTC, Steven Schveighoffer wrote:
> On 4/23/20 8:36 AM, WebFreak001 wrote:
>> On Wednesday, 22 April 2020 at 12:04:30 UTC, Manu wrote:
>>> [...]
>>>
>>> https://github.com/dlang/DIPs/pull/188
>>>
>>> [...]
>> 
>> looking at it again, I actually quite start to like it because when I first looked at it I thought it would start to be a C++ mess like allowing (Tuple + )... to basically work like AST macros and sum together a tuple. But actually thinking about ... as a unary operator like described in the DIP makes it quite clear.
>> 
>> But how exactly would it handle parentheses? For the most simple implementation I would expect (Tup + 10)[2]... to error because it would first try to evaluate (Tup + 10) like in current D, and thus error, and after that try to expand the Result[2]... - For correct syntax I would think of (Tup + 10)...[2] here
>
> If are looking for the second item out of the tuple, your last expression is correct. However, it's kind of silly not to just do:
>
> Tup[2] + 10

sure it's silly but it might be something else than [2] like .foobar()

>
> The first instance is not invalid, it's going to be:
>
> (Tup[0] + 10)[2], (Tup[1] + 10)[2], ...
>

I don't quite see how this is? Isn't the ... working on the arary index like in the examples? If you had a (Tup + 10)[OtherTup + 8]... wouldn't it then extend to (Tup + 10)[OtherTup[0] + 8], (Tup + 10)[OtherTup[1] + 8], ..., (Tup + 10)[OtherTup[$ - 1] + 8] ?

If the ... works depending on if the identifier in there is a tuple or not it would become extremely random and no longer be representable using a context free grammar

On 4/23/20 8:43 AM, Simen Kjærås wrote:
> On Wednesday, 22 April 2020 at 12:04:30 UTC, Manu wrote:
>> This DIP single-handedly fixes compile-time issues in programs I've written by reducing template instantiations by near-100%, in particular, the expensive ones; recursive instantiations, usually implementing some form of static map.
>>
>> We should have done this a long time ago.
> 
> This is beautiful and awesome (syntax and all).
> 
> I was wondering if there's any way to to do a cross product with this, like fun(Xs, Ys)... expand to fun(Xs[0], Ys[0]), fun(Xs[0], Ys[1]), fun(Xs[1], Ys[0]), fun(Xs[1], Ys[1]), but that might very well be rare enough to not warrant special consideration.

You need to do part of it with templates I think. But it should be more pleasant. I was writing something, but I haven't finished it. Will see if I can come up with it.

These kinds of puzzles are fun ;)

> The other thing that worries me a little is the difference between Foo!(AliasSeq!(1,2))... and Foo!(Numbers)... - would Foo!(AliasSeq!(1,2)...)... do the same as Foo!(Numbers)...?

No, Foo!(AliasSeq!(1, 2)...)... is equivalent to Foo!(AliasSeq!(1, 2))

Whereas if Numbers is a tuple, Foo!(Numbers)... is equivalent to Foo!(Numbers[0]), Foo!(Numbers[1]), ...

-Steve

On Thu, Apr 23, 2020 at 10:45 PM Simen Kjærås via Digitalmars-d < digitalmars-d@puremagic.com> wrote:

> On Wednesday, 22 April 2020 at 12:04:30 UTC, Manu wrote:
> > This DIP single-handedly fixes compile-time issues in programs I've written by reducing template instantiations by near-100%, in particular, the expensive ones; recursive instantiations, usually implementing some form of static map.
> >
> > We should have done this a long time ago.
>
> This is beautiful and awesome (syntax and all).
>
> I was wondering if there's any way to to do a cross product with
> this, like fun(Xs, Ys)... expand to fun(Xs[0], Ys[0]), fun(Xs[0],
> Ys[1]), fun(Xs[1], Ys[0]), fun(Xs[1], Ys[1]), but that might very
> well be rare enough to not warrant special consideration.
>

You can do this by expanding tuples with the appropriate indices:
  fun(Xs[CrossIndexX], Ys[CrossIndexY])...
Where CrossIndexX is (0, 0, 1, 1) and  CrossIndexY is (0, 1, 0, 1).

The other thing that worries me a little is the difference
> between Foo!(AliasSeq!(1,2))... and Foo!(Numbers)... - would
> Foo!(AliasSeq!(1,2)...)... do the same as Foo!(Numbers)...?
>

This needs to be clarified in the DIP; the AliasSeq!() instantiation there
is NOT actually a tuple (yet). It's just a template instantiation
expression.

So, Foo!(AliasSeq!(1,2)...)... does nothing; there's no tuple in the
expression to expand.
If you want to expand that AliasSeq, you need the second expression you
wrote:
  alias Numbers =  AliasSeq!(1,2);
  Foo!(Numbers)...

So the answer is no, they are not the same.

On Thursday, 23 April 2020 at 13:48:54 UTC, Steven Schveighoffer wrote:
>
> These kinds of puzzles are fun ;)
>

yeah no.

It's not fun if you have to make sure the rewrite is what people expect.

Foo!(AliasSeq(1,2)...)... will expand to Compiler_Tuple(Foo!(1), Foo!(2))