January 19, 2023
On Thursday, 19 January 2023 at 15:19:06 UTC, Steven Schveighoffer wrote:
> On 1/18/23 5:20 PM, Walter Bright wrote:
>> On 1/17/2023 1:58 PM, H. S. Teoh wrote:

> I think we can all agree though that it is less than ideal to have to worry about the internal details of how templates are implemented.
>
> -Steve

i disagree with this.
- it makes the library harder to maintain.
- it hides the actual problem.
- people writing their own code will still be left with the problem.
January 19, 2023

On 1/19/23 11:22 AM, Commander Zot wrote:

>

On Thursday, 19 January 2023 at 15:19:06 UTC, Steven Schveighoffer wrote:

>

On 1/18/23 5:20 PM, Walter Bright wrote:

>

On 1/17/2023 1:58 PM, H. S. Teoh wrote:

>

I think we can all agree though that it is less than ideal to have to worry about the internal details of how templates are implemented.

i disagree with this.

  • it makes the library harder to maintain.
  • it hides the actual problem.
  • people writing their own code will still be left with the problem.

I'm not sure I understand your disagreement with the quote.

I think we are saying the same thing?

-Steve

January 19, 2023

On Thursday, 19 January 2023 at 16:39:17 UTC, Steven Schveighoffer wrote:

>

On 1/19/23 11:22 AM, Commander Zot wrote:

>

On Thursday, 19 January 2023 at 15:19:06 UTC, Steven Schveighoffer wrote:

>

On 1/18/23 5:20 PM, Walter Bright wrote:

>

On 1/17/2023 1:58 PM, H. S. Teoh wrote:

>

I think we can all agree though that it is less than ideal to have to worry about the internal details of how templates are implemented.

i disagree with this.

  • it makes the library harder to maintain.
  • it hides the actual problem.
  • people writing their own code will still be left with the problem.

I'm not sure I understand your disagreement with the quote.

I think we are saying the same thing?

-Steve
maybe i misunderstand your point, but let me express my thoughts a bit more:

we should worry about the internal details of how templates are implemented. they should be implemented in the most idiomatic way.
manually optimising the code and making it harder to understand is suboptimal.
and doing it won't fix the same problem in user code. it really just hides it.

January 19, 2023

On 1/19/23 11:56 AM, Commander Zot wrote:

>

On Thursday, 19 January 2023 at 16:39:17 UTC, Steven Schveighoffer wrote:

>

On 1/19/23 11:22 AM, Commander Zot wrote:

>

On Thursday, 19 January 2023 at 15:19:06 UTC, Steven Schveighoffer wrote:

>

I think we can all agree though that it is less than ideal to have to worry about the internal details of how templates are implemented.

i disagree with this.

  • it makes the library harder to maintain.
  • it hides the actual problem.
  • people writing their own code will still be left with the problem.

I'm not sure I understand your disagreement with the quote.

I think we are saying the same thing?

maybe i misunderstand your point, but let me express my thoughts a bit more:

we should worry about the internal details of how templates are implemented. they should be implemented in the most idiomatic way.

Yes, this is what I said "less than ideal to have to worry about the internal details". Ideally, we should be able to use as much templates as we want, and pay a minimal cost.

Now, to be sure, we are going to pay a cost. It just shouldn't be so expensive that it causes abandonment of D for it (some recent examples noted on these forums).

Making Phobos constraint templates less costly is a worthy goal, whether it's done via reducing the cost of templates overall, or finding less expensive ways to implement them. Worth noting that the compiler can't just be completely magic, sometimes you do have to rearrange things to get it to work better.

-Steve

February 18, 2023

On Monday, 16 January 2023 at 15:13:04 UTC, Steven Schveighoffer wrote:

>

In general, Phobos templates should try to avoid using simple wrappers for internal things. One thing I didn't discuss in the post is that the ReturnType instances here are only ever going to be instantiated once, and on something that is never used (the lambda function).

If each test used lvalueOf from std.traits then the instantiations would be reused (there would be 2 just in isInputRange), and likely elsewhere too. There could be a convention to use lvalueOf too and avoid instantiating rvalueOf for instantiation reuse.

is(typeof(() { return (*cast(R*)null).empty; }()) == bool)
is(typeof(() { return lvalueOf!R.empty; }()) == bool)

That looks a bit nicer as it says what its intent is. Hopefully it wouldn't affect memory/performance much.

Another idea is to factor out all the r parameter declarations into one, and use that for the return type tests too:

// now
enum isInputRange(R) =
    is(typeof(R.init) == R)
    && is(typeof(() { return (*cast(R*)null).empty; }()) == bool)
    && (is(typeof((return ref R r) => r.front)) ||
        is(typeof(ref (return ref R r) => r.front)))
    && !is(typeof(() { return (*cast(R*)null).front; }()) == void)
    && is(typeof((R r) => r.popFront));

// factored
enum isInputRange(R) =
    __traits(compiles, (R r) {
        static assert(
            is(typeof(R.init) == R) &&
            is(typeof({ return r.empty; }()) == bool) &&
            is(typeof(() return => r.front)) &&
            is(typeof(ref () return => r.front)) &&
            !is(typeof({ return r.front; }()) == void) &&
            is(typeof({ r.popFront; }))
        );
    });

The factored version looks much easier to read for me. I don't know how they compare for memory/performance though.

February 18, 2023

On 2/18/23 7:04 AM, Nick Treleaven wrote:

>

On Monday, 16 January 2023 at 15:13:04 UTC, Steven Schveighoffer wrote:

>

In general, Phobos templates should try to avoid using simple wrappers for internal things. One thing I didn't discuss in the post is that the ReturnType instances here are only ever going to be instantiated once, and on something that is never used (the lambda function).

If each test used lvalueOf from std.traits then the instantiations would be reused (there would be 2 just in isInputRange), and likely elsewhere too. There could be a convention to use lvalueOf too and avoid instantiating rvalueOf for instantiation reuse.

is(typeof(() { return (*cast(R*)null).empty; }()) == bool)
is(typeof(() { return lvalueOf!R.empty; }()) == bool)

That looks a bit nicer as it says what its intent is. Hopefully it wouldn't affect memory/performance much.

Another idea is to factor out all the r parameter declarations into one, and use that for the return type tests too:

// now
enum isInputRange(R) =
     is(typeof(R.init) == R)
     && is(typeof(() { return (*cast(R*)null).empty; }()) == bool)
     && (is(typeof((return ref R r) => r.front)) ||
         is(typeof(ref (return ref R r) => r.front)))
     && !is(typeof(() { return (*cast(R*)null).front; }()) == void)
     && is(typeof((R r) => r.popFront));

// factored
enum isInputRange(R) =
     __traits(compiles, (R r) {
         static assert(
             is(typeof(R.init) == R) &&
             is(typeof({ return r.empty; }()) == bool) &&
             is(typeof(() return => r.front)) &&
             is(typeof(ref () return => r.front)) &&
             !is(typeof({ return r.front; }()) == void) &&
             is(typeof({ r.popFront; }))
         );
     });

The factored version looks much easier to read for me. I don't know how they compare for memory/performance though.

This is how isInputRange used to look. The reason it was changed is because the compiler now "sees" the different clauses separated by &&, and will tell you which one failed. When you wrap it like this, it just sees one big constraint.

I actually did get a PR merged. It wasn't as simple as I had written in that blog post, due to the stupid inout requirement that inout data can only be used inside a function with an inout parameter.

I did start with using lvalueOf!R, but then realized that since isInputRange validates that typeof(R.init) == R, I just used R.init as the parameter.

See the merged PR here: https://github.com/dlang/phobos/pull/8682

-Steve

February 18, 2023

On Saturday, 18 February 2023 at 17:16:18 UTC, Steven Schveighoffer wrote:

>

This is how isInputRange used to look. The reason it was changed is because the compiler now "sees" the different clauses separated by &&, and will tell you which one failed. When you wrap it like this, it just sees one big constraint.

Makes sense, although I don't get that on my machine with dmd v2.101.0:

import std.range;
int f(R)() if (isInputRange!R) => 2;
int i = f!int;
isinputrange.d(68): Error: template instance `isinputrange.f!int` does not match template declaration `f(R)()`
  with `R = int`
  must satisfy the following constraint:
`       isInputRange!R`

That's it, nothing about why isInputRange failed. Maybe I'm doing something wrong.

Anyway, assuming dmd can use that I wonder if this would work (aside from the inout issue):

template isInputRange(R) {
    extern R r; // dummy
    enum isInputRange =
        is(typeof(R.init) == R) &&
        is(typeof({ return r.empty; }()) == bool) &&
        (is(typeof(() return => r.front)) ||
            is(typeof(ref () return => r.front))) &&
        !is(typeof({ return r.front; }()) == void) &&
        is(typeof({ r.popFront; }));
}

dmd could still see through the eponymous template to the expression, in theory.

>

I actually did get a PR merged. It wasn't as simple as I had written in that blog post, due to the stupid inout requirement that inout data can only be used inside a function with an inout parameter.

OK, so typeof((R r) { return r.empty; } (R.init)) works with inout.

Thanks for the blog post BTW.

>

I did start with using lvalueOf!R, but then realized that since isInputRange validates that typeof(R.init) == R, I just used R.init as the parameter.

Is that validation to detect types with redefined init? I didn't realize Phobos needs to care about that.

February 18, 2023

On Saturday, 18 February 2023 at 19:25:01 UTC, Nick Treleaven wrote:

> >

I did start with using lvalueOf!R, but then realized that since isInputRange validates that typeof(R.init) == R, I just used R.init as the parameter.

Is that validation to detect types with redefined init? I didn't realize Phobos needs to care about that.

There is at least one project in the project tester that uses redefined init, so yes, Phobos has to care about it (at least until the deprecation [1] goes through).

[1] https://github.com/dlang/dmd/pull/12512

February 19, 2023

On Saturday, 18 February 2023 at 20:04:27 UTC, Paul Backus wrote:

>

On Saturday, 18 February 2023 at 19:25:01 UTC, Nick Treleaven wrote:

> >

I did start with using lvalueOf!R, but then realized that since isInputRange validates that typeof(R.init) == R, I just used R.init as the parameter.

Is that validation to detect types with redefined init? I didn't realize Phobos needs to care about that.

There is at least one project in the project tester that uses redefined init, so yes, Phobos has to care about it (at least until the deprecation [1] goes through).

[1] https://github.com/dlang/dmd/pull/12512

>

Declaring tupleof, stringof, min, max and maybe others as members should also be deprecated in the future.

min, max? why??? i hope that won't happen anytime soon

init/deinit can't be used

create/destroy can't be used

Maybe it's time to introduce something, a token for builtin features?

February 19, 2023
On 2/18/23 2:25 PM, Nick Treleaven wrote:
> On Saturday, 18 February 2023 at 17:16:18 UTC, Steven Schveighoffer wrote:
>> This is how isInputRange used to look. The reason it was changed is because the compiler now "sees" the different clauses separated by &&, and will tell you which one failed. When you wrap it like this, it just sees one big constraint.
> 
> Makes sense, although I don't get that on my machine with dmd v2.101.0:
> ```d
> import std.range;
> int f(R)() if (isInputRange!R) => 2;
> int i = f!int;
> ```
> ```
> isinputrange.d(68): Error: template instance `isinputrange.f!int` does not match template declaration `f(R)()`
>    with `R = int`
>    must satisfy the following constraint:
> `       isInputRange!R`
> ```
> That's it, nothing about why isInputRange failed. Maybe I'm doing something wrong.

uhh... I think I am wrong on the reasoning then. I thought it would keep going down into the constraint, but it doesn't? Maybe it was planned? I do know that && in the constraint itself does do that. e.g.:

```d
void foo(R)(R r) if (isInputRange!R && !is(R == int[])) {}

void main()
{
   foo(1);
   foo([1]);
}
```

```
onlineapp.d(8): Error: none of the overloads of template `onlineapp.foo` are callable using argument types `!()(int)`
onlineapp.d(2):        Candidate is: `foo(R)(R r)`
  with `R = int`
  must satisfy the following constraint:
`       isInputRange!R`
onlineapp.d(9): Error: none of the overloads of template `onlineapp.foo` are callable using argument types `!()(int[])`
onlineapp.d(2):        Candidate is: `foo(R)(R r)`
  with `R = int[]`
  must satisfy the following constraint:
`       !is(R == int[])`
```

and I also do know that the original `isInputRange` code was like you proposed -- one lambda with all the things.

> Anyway, assuming dmd can use that I wonder if this would work (aside from the inout issue):
> ```d
> template isInputRange(R) {
>      extern R r; // dummy
>      enum isInputRange =
>          is(typeof(R.init) == R) &&
>          is(typeof({ return r.empty; }()) == bool) &&
>          (is(typeof(() return => r.front)) ||
>              is(typeof(ref () return => r.front))) &&
>          !is(typeof({ return r.front; }()) == void) &&
>          is(typeof({ r.popFront; }));
> }
> ```
> dmd could still see through the eponymous template to the expression, in theory.

I don't know if that works, would it fail to link? It's an interesting idea.

> 
>> I actually did get a PR merged. It wasn't as simple as I had written in that blog post, due to the stupid `inout` requirement that `inout` data can only be used inside a function with an `inout` parameter.
> 
> OK, so `typeof((R r) { return r.empty; } (R.init))` works with inout.

Yes, that's what I found that worked. You need the lambda parameter to mimic the mutability of the type. Which kind of sucks, but it's what we have right now.

`lvalueOf!R` would work too, I just figured it isn't needed.

I wonder what the cost of using `T.init` is vs. a cached lookup of a template. It's probably pretty small, but these are the kinds of things that are unintuitive.

> 
> Thanks for the blog post BTW.

You're welcome! I write blog posts too infrequently, but I do have a bunch of half-started ones. I'm working on another right now...

>> I did start with using `lvalueOf!R`, but then realized that since `isInputRange` validates that `typeof(R.init) == R`, I just used `R.init` as the parameter.
> 
> Is that validation to detect types with redefined `init`? I didn't realize Phobos needs to care about that.

I believe so. I'm not 100% sure why we are checking for redefined init here, but apparently it's needed somewhere inside phobos.

-Steve