Thread overview
Solving a constraint hiding an error in a function literal
Feb 03, 2023
Nick Treleaven
Feb 03, 2023
Nick Treleaven
Feb 09, 2023
WebFreak001
Feb 03, 2023
Salih Dincer
Feb 03, 2023
Paul Backus
Feb 04, 2023
Paul Backus
Feb 04, 2023
Salih Dincer
Feb 09, 2023
FeepingCreature
February 03, 2023

When a constraint for an alias parameter checks that calling it compiles, it instantiates any function literal template. That triggers any latent bugs in the user's function literal that couldn't be detected before IFTI instantiates it. Those bugs then cause the constraint to fail, because the literal does not compile. The constraint then removes the otherwise matching template from the list of candidates to use.

void f(alias a)() if (is(typeof(a()))) {}

void main()
{
    f!(x => blarg);
}

https://issues.dlang.org/show_bug.cgi?id=11907
https://issues.dlang.org/show_bug.cgi?id=14217

This is a common pattern in Phobos.
Initially I thought a constraint expression like __traits(compiles, a()) should be moved to a static if test with a helpful error message. std.algorithm.all was changed to use static assert instead:

https://issues.dlang.org/show_bug.cgi?id=13683
https://github.com/dlang/phobos/pull/6607/files

But that still swallows the actual compile error and just says the callable doesn't work without really saying why (why isn't it a unary predicate if it accepts range.front as an argument?). Alternatively, the constraint check could be removed and you will get an internal error actually stating the precise error in the callable's body. But it doesn't help with overloading (see below).

What if there was a new trait to solve this? Suppose __traits(callable, expr, args), that just does IFTI (if expr is a function template) and then checks that the parameter list of expr accepts args. It does not check that expr(args) actually compiles, it ignores any errors in the body of expr.

It doesn't check the return type, because that's not possible in general when the body does not compile and return type inference is needed. But overloading on the return type of a callable doesn't seem as important compared to overloading on the parameters of a callable.

The new trait can be used as a constraint or as a static if condition. As a constraint it can help to determine which overload matches. Imagine two overloads:

// body would call a(int)
void f(alias a)()
if (__traits(callable, a, int()))

// body would call a(int, int)
void f(alias a)()
if (__traits(callable, a, int(), int()))

The trait would allow one of the overloads to be selected based on the number of parameters that a has, even when the body of a contains an error. Then one of the overloads of f is instantiated and an internal template error will be raised that a has an error. Despite this being an internal error, it is far better than the Phobos status quo because you can see what's actually failing with a precise error message.

February 03, 2023

On 2/3/23 7:43 AM, Nick Treleaven wrote:

>

if (__traits(callable, a, int()))

GMTA?

https://forum.dlang.org/post/rj5hok$c6q$1@digitalmars.com

-Steve

February 03, 2023

On Friday, 3 February 2023 at 12:43:54 UTC, Nick Treleaven wrote:

>

The new trait can be used as a constraint or as a static if condition. As a constraint it can help to determine which overload matches. Imagine two overloads:

// body would call a(int)
void f(alias a)()
if (__traits(callable, a, int()))

// body would call a(int, int)
void f(alias a)()
if (__traits(callable, a, int(), int()))

I prefer to use static if inside the template to avoid similar situations. Otherwise, I have a difficult process to find where the error is.

SDB@79

February 03, 2023

On Friday, 3 February 2023 at 13:57:02 UTC, Steven Schveighoffer wrote:

>

On 2/3/23 7:43 AM, Nick Treleaven wrote:

>

if (__traits(callable, a, int()))

GMTA?

https://forum.dlang.org/post/rj5hok$c6q$1@digitalmars.com

Thanks, yes I probably remembered it from things read here in the past.

February 03, 2023

On Friday, 3 February 2023 at 12:43:54 UTC, Nick Treleaven wrote:

>

Initially I thought a constraint expression like __traits(compiles, a()) should be moved to a static if test with a helpful error message. std.algorithm.all was changed to use static assert instead:

https://issues.dlang.org/show_bug.cgi?id=13683
https://github.com/dlang/phobos/pull/6607/files

But that still swallows the actual compile error and just says the callable doesn't work without really saying why (why isn't it a unary predicate if it accepts range.front as an argument?). Alternatively, the constraint check could be removed and you will get an internal error actually stating the precise error in the callable's body. But it doesn't help with overloading (see below).

What if there was a new trait to solve this? Suppose __traits(callable, expr, args), that just does IFTI (if expr is a function template) and then checks that the parameter list of expr accepts args. It does not check that expr(args) actually compiles, it ignores any errors in the body of expr.

I think a better solution would be to have __traits(compiles) return the error message(s) when it fails, instead of just a boolean false. Then we could write code like:

enum compResult = __traits(compiles, expr);
static if (compResult) // implicitly converts to bool
{
    doStuffWith(expr);
}
else
{
    // print error message
    static assert(0, compResult.message);
}

Getting constraints to understand this would require compiler changes, but that shouldn't be too difficult. Might also be worth having static assert(__traits(compiles, expr)), without an explicit message argument, print the returned message.

February 03, 2023

On 2/3/23 1:55 PM, Paul Backus wrote:

>

On Friday, 3 February 2023 at 12:43:54 UTC, Nick Treleaven wrote:

>

Initially I thought a constraint expression like __traits(compiles, a()) should be moved to a static if test with a helpful error message. std.algorithm.all was changed to use static assert instead:

https://issues.dlang.org/show_bug.cgi?id=13683
https://github.com/dlang/phobos/pull/6607/files

But that still swallows the actual compile error and just says the callable doesn't work without really saying why (why isn't it a unary predicate if it accepts range.front as an argument?). Alternatively, the constraint check could be removed and you will get an internal error actually stating the precise error in the callable's body. But it doesn't help with overloading (see below).

What if there was a new trait to solve this? Suppose __traits(callable, expr, args), that just does IFTI (if expr is a function template) and then checks that the parameter list of expr accepts args. It does not check that expr(args) actually compiles, it ignores any errors in the body of expr.

I think a better solution would be to have __traits(compiles) return the error message(s) when it fails, instead of just a boolean false. Then we could write code like:

enum compResult = __traits(compiles, expr);
static if (compResult) // implicitly converts to bool
{
     doStuffWith(expr);
}
else
{
     // print error message
     static assert(0, compResult.message);
}

Getting constraints to understand this would require compiler changes, but that shouldn't be too difficult. Might also be worth having static assert(__traits(compiles, expr)), without an explicit message argument, print the returned message.

The problem with this is that your constraint then has to become:

foo(alias bar) if (__traits(compiles, exprUsingBar).hasOnlyImplementationIssues)

That hasOnlyImplementationIssues is going to be messy, difficult to implement, and basically redoing all that the compiler is already doing. Plus, if it's just a string, it might have to change with compiler versions. Essentially, changing error messages becomes a breaking change.

-Steve

February 04, 2023

On Friday, 3 February 2023 at 20:20:48 UTC, Steven Schveighoffer wrote:

>

The problem with this is that your constraint then has to become:

foo(alias bar) if (__traits(compiles, exprUsingBar).hasOnlyImplementationIssues)

That hasOnlyImplementationIssues is going to be messy, difficult to implement, and basically redoing all that the compiler is already doing. Plus, if it's just a string, it might have to change with compiler versions. Essentially, changing error messages becomes a breaking change.

Well, it has to become this if you specifically want the semantics of __traits(callable). If you just want to do a better job of surfacing error messages, which is what the original post focused on, this is unnecessary.

Either way, even if the semantics of __traits(callable) are desirable on their own merits, I think it is worth improving __traits(compiles), because (1) it is widely used in existing code, and (2) sometimes __traits(compiles) is actually what you want.

February 04, 2023

On Friday, 3 February 2023 at 20:20:48 UTC, Steven Schveighoffer wrote:

>

The problem with this is that your constraint then has to become:

foo(alias bar) if (__traits(compiles, exprUsingBar).hasOnlyImplementationIssues)

That hasOnlyImplementationIssues is going to be messy, difficult to implement, and basically redoing all that the compiler is already doing. Plus, if it's just a string, it might have to change with compiler versions. Essentially, changing error messages becomes a breaking change.

I'm trying to understand their, but neither I will understand their nor they will understand me! However, they can see that I am struggling with the following non-simple example for sink their differences:

// line 2:
auto foo(E)(E value = E.min)
{
   auto a = E.min; /*
   auto a = E.max; //*/
// line 7:
   assert(a == value);
   return 0;
}

auto bar(alias func)(int b = 0)
if (__traits( compiles, func(1) ))
{
  assert(func(b - 1));
  return 0;
}

void main()
{
  enum Eco { False, True }

  Eco eco;
  assert(eco == Eco.min);
//foo(1);    //core.exception.AssertError @source_file.d(7): Assertion failure
#line 1
  foo!Eco; // okay

//foo(++eco);//core.exception.AssertError @source_file.d(7): Assertion failure
#line 2
  //bar!foo; //core.exception.AssertError @source_file.d(7): Assertion failure

  alias fn = function int (int a) {
    import std.stdio;
    "funy a = ".writeln(a);
    return 0;
  };

  bar!fn(42); /* okay: "funy a = 41
               * core.exception.AssertError
               * @source_file.d(14): Assertion failure
               */

}

I love and appreciate you all. There is no my question, but I don't understand what is being mentioned in this discussion. Because I see that doing constraint with __traits is useless...

I can not see! 😀

SDB@79

February 09, 2023

On Friday, 3 February 2023 at 12:43:54 UTC, Nick Treleaven wrote:

>

When a constraint for an alias parameter checks that calling it compiles, it instantiates any function literal template. That triggers any latent bugs in the user's function literal that couldn't be detected before IFTI instantiates it. Those bugs then cause the constraint to fail, because the literal does not compile. The constraint then removes the otherwise matching template from the list of candidates to use.

void f(alias a)() if (is(typeof(a()))) {}

void main()
{
    f!(x => blarg);
}

https://issues.dlang.org/show_bug.cgi?id=11907
https://issues.dlang.org/show_bug.cgi?id=14217
...
What if there was a new trait to solve this? Suppose __traits(callable, expr, args), that just does IFTI (if expr is a function template) and then checks that the parameter list of expr accepts args. It does not check that expr(args) actually compiles, it ignores any errors in the body of expr.

Fully agreed. This is one of the biggest "simple" inadequacies in D's lambda-based metaprogramming. See also slide 9 from my 2020 DConf presentation http://dconf.org/2020/online/slides/mathis.odp "__traits(compiles) is Satan, the great Deceiver".

February 09, 2023

On Friday, 3 February 2023 at 13:57:02 UTC, Steven Schveighoffer wrote:

>

On 2/3/23 7:43 AM, Nick Treleaven wrote:

>

if (__traits(callable, a, int()))

GMTA?

https://forum.dlang.org/post/rj5hok$c6q$1@digitalmars.com

-Steve

I remember this thread now and I'm actually amazed we haven't been making any progress towards this yet, because this looks like such a simple and elegant solution we could use for new code and Phobos v2. (because it's a backwards incompatible change when suddenly other code paths would attempt to compile I think)

I'll try starting a DIP, maybe that's just everything we need to get going.