On Wed, Apr 22, 2020 at 11:50 PM WebFreak001 via Digitalmars-d < digitalmars-d@puremagic.com> wrote:

> On Wednesday, 22 April 2020 at 12:19:25 UTC, rikki cattermole wrote:
> > Change it to something like .unpackSequence instead of ... and I would be happy.
>
> I'm for some other thing than ... too, in C++ it's a quite bad syntax


Negative, it's the single best thing that happened to C++ in almost 2 decades.

and it can become very ugly to maintain with all the
> different combinations you could use it with from a compiler viewpoint.


Actually, the implementation turned out to be 100x more simple and sane than I expected going in.

Additionally if you do miss some combinations or allow
> more complex operators it will quickly become a mess of "why does this compile but why doesn't this compile" or "why does this do something different"
>

If you're asking the question "why does this do something different", then
you exposed an edge case that should be considered a bug.
This has uniform application.

I'm especially not a fan of allowing all of
> (Tup*10)...  -->  ( Tup[0]*10, Tup[1]*10, ... , Tup[$-1]*10 )
> and
> (Tup1*Tup2)...
> and
> myArr[Tup + 1]... -> myArr[Tup[0] + 1], myArr[Tup[1] + 1],
> myArr[Tup[2] + 1]
>

I don't understand your criticism... this is exactly the point of the DIP.

would this be valid:
> f!T...(x)
> or rather this?
> f!T(x)...
>

Well they're both syntactically possible, but the first one is a nonsense
expression; you're trying to call a TupleExp, which I expect would be a
compile error.
The second is a tremendously useful pattern.

what does (cast(Tup)x)... evaluate to?
>

  (cast(Tup[0])x, cast(Tup[1])x,  cast(Tup[2])x, ...)

is f(Tup) and f(Tup...) now doing the same thing?
>

Yes, the identity expansion is a no-op. This is perfectly reasonable and a very nice property of the DIP.

if
> f(Items.x...)
> works then what about
> f(Items.MemberTup...) ?
>

We have encountered this, and defining this is the only WIP detail I'm
aware of.
If we follow normal D rules where tuples in tuples flatten, then the
natural result with no special-case intervention is:
  f( Items[0].MemberTup[0],  Items[0].MemberTup[1],
Items[1].MemberTup[0],  Items[1].MemberTup[1], ... )

I'm not sure that's particularly useful, but that's what naturally falls
from D's tuple auto-expansion rules.
We're discussing this case, and if breaking from the natural semantics to
implement a special case is worth the complexity in the spec. My feeling is
that it is not worth a special-case in the spec, and the (probably not
useful) expansion I show above would be the natural language rule.

On Wednesday, 22 April 2020 at 12:04:30 UTC, Manu wrote:
> We have a compile time problem, and this is basically the cure. Intuitively, people imagine CTFE is expensive (and it kinda is), but really, the reason our compile times are bad is template instantiation.


YES PLEASE!

I'm actively avoiding ranges and concept-like things in order to avoid template instantiation. It doesn't make sense in the context of D.

With new CTFE and this the sky is the limit :)

On Wednesday, 22 April 2020 at 12:04:30 UTC, Manu wrote:
> [...]
> We should have done this a long time ago.
>
> - Manu

Love it. Syntax and all.

On Thu, Apr 23, 2020 at 12:05 AM Steven Schveighoffer via Digitalmars-d < digitalmars-d@puremagic.com> wrote:

> On 4/22/20 9:35 AM, Manu wrote:
> > On Wed, Apr 22, 2020 at 11:20 PM Steven Schveighoffer via Digitalmars-d <digitalmars-d@puremagic.com <mailto:digitalmars-d@puremagic.com>>
> wrote:
> >
> >     On 4/22/20 8:04 AM, Manu wrote:
> >      > We have a compile time problem, and this is basically the cure.
> >      > Intuitively, people imagine CTFE is expensive (and it kinda is),
> but
> >      > really, the reason our compile times are bad is
> >     template instantiation.
> >      >
> >      > This DIP single-handedly fixes compile-time issues in programs
> I've
> >      > written by reducing template instantiations by near-100%, in
> >     particular,
> >      > the expensive ones; recursive instantiations, usually
> >     implementing some
> >      > form of static map.
> >      >
> >      > https://github.com/dlang/DIPs/pull/188
> >      >
> >      > This is an RFC on a draft, but I'd like to submit it with a
> >     reference
> >      > implementation soon.
> >      >
> >      > Stefan Koch has helped me with a reference implementation, which
> >     has so
> >      > far gone surprisingly smoothly, and has shown 50x improvement in
> >     compile
> >      > times in some artificial tests.
> >
> >     Yes please! Where is the reference implementation? I want to try some
> >     things out.
> >
> >
> > The link is in the DIP.
>
> Oops, it's at the top, I missed that. Thanks.
>
> > Most tests we've done are working, except template instantiation expansion is not yet implemented: ie, Thing!(Tuple, x, y)... That's got a lot of implementation baggage attached to it in DMD.
>
> Ugh, I think I won't be able to test then, most of this stuff works with lists of types, not values.
>

Types work now, it just depends what you do with them.
For instance, `cast(TypeTup)ValueTup` works right now.

And come to think of it, staticMap works, but I need Filter as well. I
> suppose it can help a bit, but I still am going to have tons of "temporary" templates that are going to clog up the memory.
>

I'd like to see some of your use cases. I do expect you'll need some shim templates in some cases (ie, filter), but the major difference is that they won't tend to be recursive expansions. Recursive expansions have quadratic cost... without those, your compilation should return to a linear cost world.

Fold/reduce operations can be proposed similarly to this as a follow up. It might be possible to specify a filter combining a map + fold.

Stefan, where's that implementation of first class types for CTFE-only
> functions you promised? ;)


We've been talking about this, and I actually think this lays some
foundation for some of that work.
Type functions is a pretty big spec challenge. This will solve a huge
amount of perf issues, simplify code, and it's a very simple addition
relatively speaking.

Another development that would benefit us hugely would be first-class tuples which Timon (I think?) has been working on for some time. That would eliminate the awkward AliasSeq hack, and simplify the implementation. In a first-class tuple world, we may be able to nest tuples, whereas today, nested AliasSeq flatten.

>
> > I expect you won't be able to run practical tests on real code yet
> > without TemplateInstance expansion.
> > The problem is that existing code is factored exclusively into
> > template instantiations, so a trivial experiment will apply to existing
> > code in that form.
>
> The trivial experiment to test is to take a list of types with possible duplicates and remove all duplicates. It doesn't have to be in any order.
>

Probably not possible with nothing but a map operation. There will be additional logic, but an efficient map should improve the perf substantially.

I'm hoping something like this might work:
>
> template NewFilter(pred, T...)
> {
>     template process(U) {
>       static if(pred!U) alias process = U;
>       else alias process = AliasSeq!();
>     }
>     alias NewFilter = AliasSeq!(process!(T)...); // do I need the
> AliasSeq here?
> }
>

This should work. No, you don't need the AliasSeq.

template RemoveDuplicates(T...)
> {
>     template keepIt(U) {
>        enum isSame(V) = __traits(isSame, U, V);
>        enum keepIt = NewFilter!(isSame, T).length  == 1;
>     }
>     alias RemoveDuplicates = NewFilter!(keepIt, T);
> }
>

I think efficient implementation here would depend on a static fold, which
I plan for a follow-up, and it's a very trivial expansion from this DIP.
Static reduce would allow `...` as an argument to a BinOp
Ie; `Tup + ...` would expand `Tup[0] + Tup[1] + Tup[2] + ...`
You could do `is(FindType == Tup) || ...`, and it would evaluate true if
FindType exists in Tup, with no junk template instantiations!

Will this work better? You will still have a bunch of NewFilter,
> process, keepIt, and isSame instantiations, all with horribly long
> symbol names. But of note is there is no recursive instantiation patterns.
>

Using fold as above, you can eliminate the boilerplate.
I don't want to propose that here though. It's a relatively trivial
expansion from this initial DIP.

One thing I noticed, in order to use a property on a static map
> expansion (i.e. call a function with the resulting sequence, or access .length), you will need extra parentheses to distinguish the ... token from the . token.
>

>  call a function with the resulting sequence

Not sure what you mean exactly?

Calling a function that receives the sequence as var args:
  f(TupExpr...)

Calling a function for each element in the sequence?
  f(TupExpr)...

Is there an easier/better way to do this with the new feature?
>

Can you show an example? Maybe the precedence is wrong?

On 4/22/20 9:45 AM, WebFreak001 wrote:
> On Wednesday, 22 April 2020 at 12:19:25 UTC, rikki cattermole wrote:
>> Change it to something like .unpackSequence instead of ... and I would be happy.
> 
> I'm for some other thing than ... too, in C++ it's a quite bad syntax and it can become very ugly to maintain with all the different combinations you could use it with from a compiler viewpoint. Additionally if you do miss some combinations or allow more complex operators it will quickly become a mess of "why does this compile but why doesn't this compile" or "why does this do something different"
> 
> I'm especially not a fan of allowing all of
> (Tup*10)...  -->  ( Tup[0]*10, Tup[1]*10, ... , Tup[$-1]*10 )
> and
> (Tup1*Tup2)...
> and
> myArr[Tup + 1]... -> myArr[Tup[0] + 1], myArr[Tup[1] + 1], myArr[Tup[2] + 1]
> 
> would this be valid:
> f!T...(x)

It would expand to:

(f!T[0], f!(T[1]), ... , f!(T[n]))(x)

I don't think this would compile.

> or rather this?
> f!T(x)...

That would work, you would have:

(f!(T[0])(x), f!(T[1])(x), ..., f!(T[n])(x))

> what does (cast(Tup)x)... evaluate to?

(cast(Tup[0])x, cast(Tup[1])x, ..., cast(Tup[n])x);

> is f(Tup) and f(Tup...) now doing the same thing?

Yes, using Tup... is the same as Tup. Only inside expressions that use Tup would the ... make sense.

> if
> f(Items.x...)
> works then what about
> f(Items.MemberTup...) ?

I would expect an expansion as Manu described, where the natural expansion of Items becomes Items[0].MemberTup, Items[1].MemberTup, ...

However, there are some ambiguities that I'm not sure have been solved. What about templates that expand to tuples? Are the results of those tuples part of the ... expansion?

What about:

alias F!(T...) = AliasSeq!(T, T);

Consider this expansion:

alias t = AliasSeq!(int, char);

F!(F!t))...

Does it mean:

F!(F!int), F!(F!char)) => int, int, int, int, char, char, char, char

or does it mean:

F!(AliasSeq!(int, char, int, char))... => int, char, int, char, int, char, int, char

In other words, what if part of the expression *creates* a tuple. Is that the thing that gets expanded? I would have to say no, right? Otherwise, the whole thing might be expanded and the ... trivially applied.

So that means things like (assuming G does something similar but not identical to F:

F!(G!int)... would have to be the same as F!(G!int). This might be very confusing to someone expecting the inner tuple to be done before the expansion is considered.

I don't know how to DIP-ify this idea. But it definitely needs to be talked about.

-Steve

On 4/22/20 10:37 AM, Manu wrote:
>     One thing I noticed, in order to use a property on a static map
>     expansion (i.e. call a function with the resulting sequence, or access
>     .length), you will need extra parentheses to distinguish the ... token
>     from the . token.
> 
> 
>  > call a function with the resulting sequence
> 
> Not sure what you mean exactly?

Like this works today:

AliasSeq!(1, 2, 3, 4).text;

Similarly, if you wanted to use some tuple expansion:

(transform!T...).text;

you need the parentheses, or else you have transform!T....text.

Or else if you want to get the length, you have:

(transform!T...).length

>     Is there an easier/better way to do this with the new feature?
> 
> 
> Can you show an example? Maybe the precedence is wrong?

I think that question was left over from something else that I deleted before posting, the .prop just needs clarification.

-Steve

On Thu, Apr 23, 2020 at 12:55 AM Steven Schveighoffer via Digitalmars-d < digitalmars-d@puremagic.com> wrote:

> On 4/22/20 9:45 AM, WebFreak001 wrote:
> > if
> > f(Items.x...)
> > works then what about
> > f(Items.MemberTup...) ?
>
> I would expect an expansion as Manu described, where the natural expansion of Items becomes Items[0].MemberTup, Items[1].MemberTup, ...
>

And of course, tuples in tuples auto-flatten according to current D semantics.

However, there are some ambiguities that I'm not sure have been solved.
> What about templates that expand to tuples? Are the results of those tuples part of the ... expansion?
>
> What about:
>
> alias F!(T...) = AliasSeq!(T, T);
>
> Consider this expansion:
>
> alias t = AliasSeq!(int, char);
>
> F!(F!t))...
>

The expansion is evaluated from the leaf upwards... your question is valid,
and this is the case we've discussed quite a lot.
I'm basically just interested to run it and see what happens naturally!

Does it mean:
>
> F!(F!int), F!(F!char)) => int, int, int, int, char, char, char, char
>
> or does it mean:
>
> F!(AliasSeq!(int, char, int, char))... => int, char, int, char, int,
> char, int, char
>
> In other words, what if part of the expression *creates* a tuple. Is
> that the thing that gets expanded? I would have to say no, right?
> Otherwise, the whole thing might be expanded and the ... trivially applied.
>
> So that means things like (assuming G does something similar but not identical to F:
>
> F!(G!int)... would have to be the same as F!(G!int). This might be very
> confusing to someone expecting the inner tuple to be done before the
> expansion is considered.
>
> I don't know how to DIP-ify this idea. But it definitely needs to be talked about.


I have thought about how to discuss this in the DIP; I describe the
semantic, and what happens is what happens.
This will work, and something will happen... when we implement
TemplateInstance, we'll find out exactly what it is :P
What I will do is show what such a nested tuple does when code works in the
DIP to instantiate TemplateInstances.

It's basically the same thing as what you showed above with:
Items[0].MemberTup, Items[1].MemberTup, ...
In general, in D currently, nested tuples flatten. Evaluate from the leaf
upwards. Your answer will materialise.

On 4/22/20 11:17 AM, Manu wrote:
> 
> I have thought about how to discuss this in the DIP; I describe the semantic, and what happens is what happens.
> This will work, and something will happen... when we implement TemplateInstance, we'll find out exactly what it is :P
> What I will do is show what such a nested tuple does when code works in the DIP to instantiate TemplateInstances.
> 
> It's basically the same thing as what you showed above with: Items[0].MemberTup, Items[1].MemberTup, ...
> In general, in D currently, nested tuples flatten. Evaluate from the leaf upwards. Your answer will materialise.

I think this is more complicated than the MemberTup thing. By inference of the name, MemberTup is a tuple, but only defined in the context of the expanded items. There aren't any tuples for the compiler to expand in there.

A template that returns a tuple based on it's parameters is a tuple with or without expansion.

F!(F!t)) is valid. It's going to return int, char, int, char, int, char, int, char

F!(F!t))... what does this do?

Saying "let's see what happens" is not a good way to create a DIP. We have enough of "design by implementation" in D.

-Steve

On 4/22/20 10:37 AM, Manu wrote:
> I think efficient implementation here would depend on a static fold, which I plan for a follow-up, and it's a very trivial expansion from this DIP.
> Static reduce would allow `...` as an argument to a BinOp
> Ie; `Tup + ...` would expand `Tup[0] + Tup[1] + Tup[2] + ...`
> You could do `is(FindType == Tup) || ...`, and it would evaluate true if FindType exists in Tup, with no junk template instantiations!

This is awesome, and I'm not seeing why you would save it for later.

In general, couldn't this DIP be done just strictly on binary operators and do what this DIP does with commas?

i.e.

foo(T) , ... expands to foo(T[0]), foo(T[1]), ..., foo(T[n])

-Steve

On Wednesday, 22 April 2020 at 14:02:45 UTC, Steven Schveighoffer wrote:
>
> Stefan, where's that implementation of first class types for CTFE-only functions you promised? ;)
>

As you can see I have been busy with something similar.
ctfe-only type-access or type-functions as I call them, coming closer through the work that Manu and I are doing.

The type-functions I have in mind will essentially supersede  ... expressions.
However ... expressions can be there in the next weeks whereas type functions need much heavier machinery; and therefore will take some more invasive changes to dmd which are unlikely to be finished soon.

Cheers,
Stefan