April 29

On Thursday, 25 April 2024 at 17:56:59 UTC, Quirin Schroll wrote:

>

https://github.com/Bolpat/DIPs/blob/2bfef0b05e99c5448b416078da2e743482e3193d/DIPs/1NNN-QFS.md

Yet another non-orthogonal language feature that all generic code will have to go out of its way to account for until the end of time. ref is already bad enough; the last thing we need is another feature that repeats the exact same design mistakes.

For example: currently, in order to forward an argument correctly in generic code, we must write the following:

void forwardTo(alias fun, T)(auto ref T arg)
{
    import core.lifetime: forward;
    fun(forward!arg);
}

Notice how much overhead is required just to deal with the ref storage class.

If this proposal were accepted, we would have to replace all instances of the above with the following:

void forwardTo(alias fun, T)(auto enum auto ref T arg)
{
    import core.lifetime: forward;
    static if (__traits(isEnum, arg))
        fun(arg);
    else
        fun(fporward!arg);
}

Needless to say, the vast majority of D programmers are not going to bother with this, which means that we will forever be plagued by buggy handling of enum parameters in our library code (just as we are already plagued by buggy handling of ref parameters).

I do not think it is a good idea to introduce language features that set future D programmers up for failure like this.

April 29
On 4/29/24 18:32, Timon Gehr wrote:
> The (logistical) issue with this is that e.g., the following is disallowed:
> 
> ```d
> void foo(T...,int x)(){}
> ```
> 
> This gets back to the issue of name mangling, because I think there is no way to distinguish an instantiation where the last template parameter goes to `T...` and an instantiation where the last template parameter goes to `x`. 

Sorry, my bad. This point did not fully make sense, because `x` is not optional in this example. The problem with ambiguity only occurs if you have multiple variadic parameters (which are also disallowed, but you do not need them in your proposal).

It is still true that D currently disallows adding any template parameters after the variadic parameter though.
May 05
On 4/29/24 21:17, Paul Backus wrote:
> 
> If this proposal were accepted, we would have to replace all instances of the above with the following:
> 
> ```d
> void forwardTo(alias fun, T)(auto enum auto ref T arg)
> {
>      import core.lifetime: forward;
>      static if (__traits(isEnum, arg))
>          fun(arg);
>      else
>          fun(fporward!arg);
> }

If the `__traits(isEnum, arg)` is required at all (which I strongly doubt), it would be put into `forward`.

The `auto enum` point stands, however, and another issue it causes is CTFE and template instantiation overhead even if the final function does not even accept `enum` parameters. I don't think this is even perfect forwarding, as it may cause CTFE that would not otherwise happen, which currently may actually change the observable behavior of the code due to e.g. floating-point shenanigans.
May 07

On Monday, 29 April 2024 at 19:17:47 UTC, Paul Backus wrote:

>

On Thursday, 25 April 2024 at 17:56:59 UTC, Quirin Schroll wrote:

>

https://github.com/Bolpat/DIPs/blob/2bfef0b05e99c5448b416078da2e743482e3193d/DIPs/1NNN-QFS.md

Yet another non-orthogonal language feature that all generic code will have to go out of its way to account for until the end of time. ref is already bad enough; the last thing we need is another feature that repeats the exact same design mistakes.

TL;DR: While auto enum is similar to auto ref in that it infers a storage class based on the argument passed, forwarding is a non-issue for auto enum and even auto enum auto ref. Analysis is below.

>

For example: currently, in order to forward an argument correctly in generic code, we must write the following:

void forwardTo(alias fun, T)(auto ref T arg)
{
    import core.lifetime: forward;
    fun(forward!arg);
}

Notice how much overhead is required just to deal with the ref storage class.

I agree that forwarding in its current form is indeed a nuisance.

>

If this proposal were accepted, we would have to replace all instances of the above with the following:

void forwardTo(alias fun, T)(auto enum auto ref T arg)
{
    import core.lifetime: forward;
    static if (__traits(isEnum, arg))
        fun(arg);
    else
        fun(forward!arg);
}

Needless to say, the vast majority of D programmers are not going to bother with this, which means that we will forever be plagued by buggy handling of enum parameters in our library code (just as we are already plagued by buggy handling of ref parameters).

I do not think it is a good idea to introduce language features that set future D programmers up for failure like this.

You’re right in that forward better be mentioned in the DIP. Writing an answer to this post, I found out that not only could forward easily be adapted to recognize enum parameters and handle them properly by leaving them alone, forward actually would handle them correctly today.

First things first, contrary to auto ref, an auto enum parameter bound to a compile-time constant (i.e. one that becomes enum) would stay a compile-time constant (and a run-time value would stay a run-time value), so forwarding auto enum is automatic (it requires nothing in terms of written code to happen). That is contrary to auto ref where no matter how the argument was passed (i.e. no matter whether the parameter becomes ref), the parameter itself is always an lvalue and binds to/prefers ref when passed to another function. This is why forwarding auto ref must be explicit.

Another aspect is that there is no enum ref parameters, so auto enum auto ref is non-orthogonal by design: It can become enum (pass as compile-time constant), ref (pass by reference) or none (pass by value).

Assuming the following code (which apart from additional auto enum is identical to the example of currently existing code):

void forwardTo(alias fun, T)(auto enum auto ref T arg)
{
    import core.lifetime: forward;
    fun(forward!arg);
}

How does forward handle the three cases? The cases ref and pass-by-value are clear, they are as they are today. As far as the function template implementation is concerned, the enum case is equivalent to:

void forwardTo(alias fun, T, T arg)()
{
    import core.lifetime: forward;
    fun(forward!arg);
}

That is valid code today!

void main()
{
    immutable myInt = 10;
    forwardTo!(
        (auto ref x) { pragma(msg, __traits(isRef, x)); },
        int,
        myInt
    );
}

It prints false. This means, forward passed the compile-time constant to the lambda unchanged and a compile-time constant is seen as an rvalue. Note that myInt could absolutely be bound as an lvalue.

The reason for this is that forward not only recognizes some storage classes (ref among them), it also checks if move would work and if it doesn’t, it does not use it. That is because template value parameters are rvalues and enum parameters would be rvalues as well, and move only works on lvalues.

May 07

On Tuesday, 7 May 2024 at 11:23:10 UTC, Quirin Schroll wrote:

>

You’re right in that forward better be mentioned in the DIP. Writing an answer to this post, I found out that not only could forward easily be adapted to recognize enum parameters and handle them properly by leaving them alone, forward actually would handle them correctly today.

Good to know; thanks for looking into this.

However, the main issue here is not forward--it's that in order to write correct generic code with this feature in place, D programmers will forever be obliged to defensively write auto enum auto ref on any parameter they intend to forward (and will have to go back and add auto enum to many of their existing auto ref parameters).

Most D programmers already forget to add auto ref to such parameters (see for example the vast swathes of Phobos which fail to compile when used with non-copyable types). Adding an additional thing for them to forget is, like I said, setting them up for failure.

(Also, auto enum auto ref is quite ugly and "attribute spam"-y, and some programmers will no doubt leave it out of their code intentionally in order to make the code look nicer--as they already do with existing attributes and parameter storage classes.)

May 07

On Tuesday, 7 May 2024 at 15:20:24 UTC, Paul Backus wrote:

>

On Tuesday, 7 May 2024 at 11:23:10 UTC, Quirin Schroll wrote:

>

You’re right in that forward better be mentioned in the DIP. Writing an answer to this post, I found out that not only could forward easily be adapted to recognize enum parameters and handle them properly by leaving them alone, forward actually would handle them correctly today.

Good to know; thanks for looking into this.

However, the main issue here is not forward--it's that in order to write correct generic code with this feature in place, D programmers will forever be obliged to defensively write auto enum auto ref on any parameter they intend to forward (and will have to go back and add auto enum to many of their existing auto ref parameters).

I don't think this is going to be the case. Consider that without explicitly forwarding, the difficulty is not in preserving refness, but non-refness. If care is not taken (e.g. via using forward), then a non-ref parameter suddenly turns into a ref parameter (implicitly). This is not the case for enum parameters -- enumness is preserved whether it's not an enum or is an enum.

Yes, generic code that wishes to forward enum-ness to another thing must use auto enum to accomplish this. However, comparing to the the current mechanism, things are pretty inferior. Switching from an enum parameter to a runtime parameter involves changing the place where parameters are put. You currently have to remember to create a nested template with an outer template that takes a tuple for this to work correctly (something which I doubt many generic functions do).

Put another way, if the cost of having the convenience of enum parameters is that we must also allow auto enum, I don't think this is a large enough drawback.

>

Most D programmers already forget to add auto ref to such parameters (see for example the vast swathes of Phobos which fail to compile when used with non-copyable types). Adding an additional thing for them to forget is, like I said, setting them up for failure.

(Also, auto enum auto ref is quite ugly and "attribute spam"-y, and some programmers will no doubt leave it out of their code intentionally in order to make the code look nicer--as they already do with existing attributes and parameter storage classes.)

Based on what Quirin explained, these attributes are not orthogonal. So auto ref enum or auto enum ref would be sufficient. It's at least a little better.

But... even auto enum ref Typename is quite noisy, I wish we had a combining storage class to do all the inference. auto ref is kind of ugly on its own, it would be nice to find a better way. Note that auto ref dates back to a time without @attributes and also at a time of extreme prejudice against adding new keywords.

-Steve

May 07

On Friday, 26 April 2024 at 09:39:52 UTC, Quirin Schroll wrote:

>

On Friday, 26 April 2024 at 02:47:18 UTC, Steven Schveighoffer wrote:

>

I have thoughts on reworking interpolation tuples if this got in...

Please, elaborate. I was somewhat absent from the forums while string interpolation took off. I read the docs, but it seems a lot of regulars like you have a lot more intuition about where interpolation tuples are useful. Not only could I use those ideas in the DIP, I’m honestly intrigued because I can’t imagine how they meaningfully interact.

I realize I forgot to expand on this.

In the current regime of IES, the compile-time strings are preserved via an attachment of the literal data to the type of the parameter. This necessitates always taking the IES via vararg template, and also results in some more advanced techniques required for introspecting the data.

Compared to a blueprint-style function (like writef), it's much more complex to explain and use. But the benefit is awesome -- perfect rewriting of code is possible (via string mixin) as if you called the new code without doing any runtime parsing.

But if we have an enum parameter mechanism, the compile-time data can become a simple array of literal data, and you can accept it via a runtime parameter (if you prefer runtime parsing, and less template bloat), or via an enum parameter (if you prefer to parse at compile time).

I outlined my thoughts on it here: https://forum.dlang.org/post/ojotybtiosjikmmcxemw@forum.dlang.org

(there I also acknowledged that I didn't know about your previous DIP, and supported it)

-Steve

May 07

On Tuesday, 7 May 2024 at 16:34:52 UTC, Steven Schveighoffer wrote:

>

I don't think this is going to be the case. Consider that without explicitly forwarding, the difficulty is not in preserving refness, but non-refness. If care is not taken (e.g. via using forward), then a non-ref parameter suddenly turns into a ref parameter (implicitly). This is not the case for enum parameters -- enumness is preserved whether it's not an enum or is an enum.

If a single function in a call stack forgets to pass a given parameter by ref, it will be copied. If your code relies on the parameter not being copied (e.g., if it has a disabled copy constructor), this failure to preserve ref results in a bug.

If a single function in a call stack forgets to pass a given parameter by enum, it will be evaluated at runtime. If your code relies on the parameter being evaluable at compile time (e.g., if it uses it as a template argument or in a static if condition), this failure to preserve enum results in a bug.

The two are exactly analogous.

>

Yes, generic code that wishes to forward enum-ness to another thing must use auto enum to accomplish this. However, comparing to the the current mechanism, things are pretty inferior. Switching from an enum parameter to a runtime parameter involves changing the place where parameters are put. You currently have to remember to create a nested template with an outer template that takes a tuple for this to work correctly (something which I doubt many generic functions do).

Put another way, if the cost of having the convenience of enum parameters is that we must also allow auto enum, I don't think this is a large enough drawback.

The argument here is essentially that it is worth it to make the library author's job more difficult in order to give the library users a better experience.

In general, I agree. But there's a point at which this argument falls apart. If you make the library author's job so difficult that they can no longer write correct code, it is library users who will ultimately suffer.

Personally, I think the ref storage class already crosses this line. The vast majority of generic D code does not handle non-copyable types correctly, and the design of ref is 100% to blame for this. Since enum uses the exact same design as ref, I think it's pretty reasonable to assume that it will cause the same kinds of problems.

So, when I evaluate this DIP, here's the tradeoff I'm looking at:

  • Pro: more convenient library APIs (e.g., format).
  • Con: more bugs in generic library code.

I don't think this is a good tradeoff.

May 07

On Tuesday, 7 May 2024 at 17:44:25 UTC, Paul Backus wrote:

>

If a single function in a call stack forgets to pass a given parameter by ref, it will be copied. If your code relies on the parameter not being copied (e.g., if it has a disabled copy constructor), this failure to preserve ref results in a bug.

If a single function in a call stack forgets to pass a given parameter by enum, it will be evaluated at runtime. If your code relies on the parameter being evaluable at compile time (e.g., if it uses it as a template argument or in a static if condition), this failure to preserve enum results in a bug.

Won't this always result in a compile error, not a runtime bug?

May 07

On Tuesday, 7 May 2024 at 22:32:04 UTC, Meta wrote:

>

On Tuesday, 7 May 2024 at 17:44:25 UTC, Paul Backus wrote:

>

If a single function in a call stack forgets to pass a given parameter by ref, it will be copied. If your code relies on the parameter not being copied (e.g., if it has a disabled copy constructor), this failure to preserve ref results in a bug.

If a single function in a call stack forgets to pass a given parameter by enum, it will be evaluated at runtime. If your code relies on the parameter being evaluable at compile time (e.g., if it uses it as a template argument or in a static if condition), this failure to preserve enum results in a bug.

Won't this always result in a compile error, not a runtime bug?

Yes. But in generic code, that compile error might not happen until a user instantiates it in a particular way that the library author hasn't thought of. (Again, see Phobos's handling of non-copyable types.)