On Mon, Jan 14, 2019 at 04:31:06PM -0700, Jonathan M Davis via Digitalmars-d wrote:
> On Monday, January 14, 2019 2:57:28 PM MST Paul Backus via Digitalmars-d wrote:
> > On Monday, 14 January 2019 at 21:47:40 UTC, Steven Schveighoffer
> >
> > wrote:
> > > How many times have you written something like:
> > >
> > > void foo(T)(T t) if (__traits(compiles, t.bar())) // or
> > > is(typeof(t.bar()))
> > > {
> > >
> > >    t.bar();
> > >
> > > }
> > >
> > > And somehow, somewhere, this isn't called? Then you remove the constraint, and find that there's a syntax error in the bar template function (like a missing semicolon).

Yeah, I've run into this problem far too many times.  It's extremely annoying because the error is silent, and makes the compiler just move on to a different overload, leaving you scratching your head as to why it's not picking the "correct" overload.


> > Ideally, you would catch this in the unit tests for `bar`. You are writing unit tests, aren't you? ;)
[...]

The problem is not just trivial syntax errors like Steven's example. The problem is that sufficiently complex sig constraints will sometimes have unexpected corner cases that works in general, but fails for specific cases.

One nasty example that comes to mind is when the sig constraint only works with mutable types (or some other such qualified type) because you used an overly-restrictive test.  Initial unittests work because you're not expecting that changing the input to const would cause any trouble. Then some time later, you happen to need to pass a const object, and suddenly the compiler seemingly refuses to acknowledge that this overload of the template function exists, and insists on trying a different overload, or failing with a totally unhelpful error saying no overloads match.

If you're dealing with something with as many overloads as std.conv.toImpl, for example, good luck finding what went wrong. You first have to parse every single darned sig constraint on every single overload, just to find the one overload that *should* match, but doesn't.  Then you have to parse the sig constraint clause by clause just to determine which clause failed.  And *then*, if you're really unlucky and the clause is a complex one (that calls another template, say, like isXyzRange or something equally complex), *then* you have to find the definition of that helper template, then parse through every clause in *that* to find out what went wrong.  And if you're *truly* unlucky, the helper template is of the form:

	isBlahBlahBlah(T) = __traits(compiles, (T t) {
		... // arbitrarily complex code here
	});

then you have to copy-n-paste the stupid lambda and instantiate it with your exact version of T (and that's if the original template function's sig constraint hadn't munged it with things like Unqual or any of the other type-changing templates -- then good luck figuring out what T is for your particular function call), just to get the stupid compiler to tell you the stupid error that made it fail, because by default all errors are gagged inside template instantiations during IFTI, so there's nothing at all that even hints at what went wrong.

I've had to do this so many times when working with template-heavy code, that recently I've started to develop a distaste for sig constraints. Unless I'm working with complex overload sets where sig constraints are unavoidable, I'd rather just leave the function without any sig constraints -- then at least when something fails, you've a fighting chance of knowing what went wrong without going through the above dance *every* *single* *time*, because the compiler will now actually tell you what the lousy compile error was.

And if you're really cursed, you'd run into a case like Jonathan says, where there's a static if inside the template function that goes to the else clause because it doesn't like your template argument for whatever reason, so you just got switched to a different implementation with not a clue as to how it ended up there.  Given enough nested static if's, good luck figuring out just which one of them had the failing condition that sent you down the wrong rabbit hole -- the failure is completely silent; and remember, if the else-branch of the static if fails to compile, the compiler will just gag the error and move on to the next overload at the top level.  Good luck finding that one static if that failed.

(Oh, and did I mention, almost all of the above requires you to edit the *callee* code in order to find out where the template instantiation went? Good luck if it goes into some common templates used everywhere else, that if you change one line, something else unrelated in your program will stop compiling so you won't even get to the point of failure in the first place. Sometimes the only solution here is to copy-n-paste the entire lousy template function into a different module (possibly along with most/all of its dependencies) and call *that* instead, so that the compiler won't prematurely die in unrelated code.)

This is a very real problem, and an extremely annoying one.  In a hair-pullingly frustrating manner.  And of course, it makes you feel *really* good 2 hours later after all of this jazz to discover that the error came from a simple typo.


T

-- 
Life is too short to run proprietary software. -- Bdale Garbee

On 01/14/2019 03:58 PM, H. S. Teoh wrote:

> the error is silent, and makes the compiler just move
> on to a different overload, leaving you scratching your head as to why
> it's not picking the "correct" overload.

I've recently opened the following bug which I believe is due to the same underlying issue: A compilation error is silently skipped.

  https://issues.dlang.org/show_bug.cgi?id=19541

> (Oh, and did I mention, almost all of the above requires you to edit the
> *callee* code in order to find out where the template instantiation
> went?

Yep. Similar to my choice of words in that bug report: "the user [...] is now looking for unnecessary workarounds".

Ali

On 1/14/19 6:58 PM, H. S. Teoh wrote:
> On Mon, Jan 14, 2019 at 04:31:06PM -0700, Jonathan M Davis via Digitalmars-d wrote:
>> On Monday, January 14, 2019 2:57:28 PM MST Paul Backus via Digitalmars-d
>> wrote:
>>> On Monday, 14 January 2019 at 21:47:40 UTC, Steven Schveighoffer
>>>
>>> wrote:
>>>> How many times have you written something like:
>>>>
>>>> void foo(T)(T t) if (__traits(compiles, t.bar())) // or
>>>> is(typeof(t.bar()))
>>>> {
>>>>
>>>>     t.bar();
>>>>
>>>> }
>>>>
>>>> And somehow, somewhere, this isn't called? Then you remove the
>>>> constraint, and find that there's a syntax error in the bar
>>>> template function (like a missing semicolon).
> 
> Yeah, I've run into this problem far too many times.  It's extremely
> annoying because the error is silent, and makes the compiler just move
> on to a different overload, leaving you scratching your head as to why
> it's not picking the "correct" overload.
> 
> 
>>> Ideally, you would catch this in the unit tests for `bar`. You are
>>> writing unit tests, aren't you? ;)
> [...]
> 
> The problem is not just trivial syntax errors like Steven's example.
> The problem is that sufficiently complex sig constraints will sometimes
> have unexpected corner cases that works in general, but fails for
> specific cases.
> 
> One nasty example that comes to mind is when the sig constraint only
> works with mutable types (or some other such qualified type) because you
> used an overly-restrictive test.  Initial unittests work because you're
> not expecting that changing the input to const would cause any trouble.
> Then some time later, you happen to need to pass a const object, and
> suddenly the compiler seemingly refuses to acknowledge that this
> overload of the template function exists, and insists on trying a
> different overload, or failing with a totally unhelpful error saying no
> overloads match.

So to clarify a bit, I'm not expecting miracles here. If your code is valid, but just isn't callable in the way you expect it to be, I don't see how the compiler can figure out that you meant it another way.

But my overall point really is that we have a mechanism to say "Does this compile?", when we really mean "Can I use this thing this way?". I wondered if it wouldn't be worth having a way to assume that all code is syntactically valid, and just throw errors when it's not.

And now that I'm thinking about it some more, I'm completely wrong. It is little typos that are syntactically valid that fail to compile -- syntax errors are still flagged (unless you are using mixins).

So this doesn't help at all :(

But I still face the same problems you bring up.

> (Oh, and did I mention, almost all of the above requires you to edit the
> *callee* code in order to find out where the template instantiation
> went? Good luck if it goes into some common templates used everywhere
> else, that if you change one line, something else unrelated in your
> program will stop compiling so you won't even get to the point of
> failure in the first place. Sometimes the only solution here is to
> copy-n-paste the entire lousy template function into a different module
> (possibly along with most/all of its dependencies) and call *that*
> instead, so that the compiler won't prematurely die in unrelated code.)

Oh yeah, this is a really hard problem. This captures a lot of the frustration that I've had too.

I'm wondering instead of having a way to help with this, if we can't specify (for debugging purposes), "this overload should be used". In other words, get rid of the constraints and force the compiler to use a certain call. That would help IMMENSELY.

I'm actually thinking of a library mechanism to do this, which is horrible, and awful, but might actually work:

struct PickMe
{
   string file;
   size_t line;
}

enum isMyOverload(...) more cruft

enum ItPickedMe(T, string file == __FILE__, size_t line == __LINE__) = hasUDA!(PickMe) && anySatisfy!(isMyOverload, getUDAs!(T, PickMe))

void foo(T)(T t) if(ItPickedMe!T || __traits(compiles, t.bar()))
{
   t.bar();
}

I don't want to do this, but a compiler feature that allowed it would be cool. Having to make copies of templates to be able to debug things is so horrid.

-Steve

On Monday, 14 January 2019 at 21:47:40 UTC, Steven Schveighoffer wrote:
> How many times have you written something like:
>
> void foo(T)(T t) if (__traits(compiles, t.bar())) // or is(typeof(t.bar()))
> {
>    t.bar();
> }
>
> And somehow, somewhere, this isn't called? Then you remove the constraint, and find that there's a syntax error in the bar template function (like a missing semicolon).

struct A
{
    int f()()
    {
        return 0
    }
}

Like this? There's no way for it to ever compile.

On Monday, 14 January 2019 at 21:47:40 UTC, Steven Schveighoffer wrote:
> How many times have you written something like:

More times than I can count...

>
> void foo(T)(T t) if (__traits(compiles, t.bar())) // or is(typeof(t.bar()))
> {
>    t.bar();
> }
>
> [...]

I'm not sure how best to go about this.

On Tuesday, 15 January 2019 at 11:15:21 UTC, Atila Neves wrote:
> On Monday, 14 January 2019 at 21:47:40 UTC, Steven Schveighoffer wrote:
>> How many times have you written something like:
>
> More times than I can count...
>
>>
>> void foo(T)(T t) if (__traits(compiles, t.bar())) // or is(typeof(t.bar()))
>> {
>>    t.bar();
>> }
>>
>> [...]
>
> I'm not sure how best to go about this.

Sounds like a job for `static try` too keep things DRY:

void fun(T)(T t) if (__traits(compiles, t.foo()))
{
    t.foo();
}

void fun(T)(T t) if (__traits(compiles, t.bar()))
{
    t.bar();
}

// ----v

void fun(T)(T t)
{
    static try { t.bar(); }
    else static try { t.foo(); }
    else static assert(0, T.stringof ~ "does not support neither `foo`, nor `bar`");
}

On Tuesday, 15 January 2019 at 01:59:58 UTC, Steven Schveighoffer wrote:
>
> I don't want to do this, but a compiler feature that allowed it would be cool. Having to make copies of templates to be able to debug things is so horrid.
>
> -Steve

Maybe a sub-name system? Like:

    T* myFunc.withFoo(T)(T t) if(__traits(compiles, t.foo()));
    T* myFunc.withBar(T)(T t) if(__traits(compiles, t.bar()));

Then if you call `myFunc.withFoo(t)`, you get something like:

   Error: Cannot call withFoo overload of myFunc
   Error: because constraint if(__traits(compiles, t.foo())) is not satisfied

(it would be especially helpful with the "multiple constraints" DIP)

It would probably be pretty hard to implement, and make name resolution even more complicated, though.

On 1/15/19 3:26 AM, Kagamin wrote:
> On Monday, 14 January 2019 at 21:47:40 UTC, Steven Schveighoffer wrote:
>> How many times have you written something like:
>>
>> void foo(T)(T t) if (__traits(compiles, t.bar())) // or is(typeof(t.bar()))
>> {
>>    t.bar();
>> }
>>
>> And somehow, somewhere, this isn't called? Then you remove the constraint, and find that there's a syntax error in the bar template function (like a missing semicolon).
> 
> struct A
> {
>      int f()()
>      {
>          return 0
>      }
> }
> 
> Like this? There's no way for it to ever compile.

Indeed, I realized after posting [1] that it's really stupid typos that I make that are still valid syntax, but obviously unintentional. By that I mean, it will never compile.

Like:

void foo(T)(T t)
{
   int x = 1;
   t.fun(X);
}

What I was trying to say is that I wanted a way to separate "obvious" errors from errors that happen because of the template parameters. But I think there simply isn't a way to do it. The best we can do might be to try and force specific instantiations to see why they don't compile.

-Steve

[1] https://forum.dlang.org/post/q1jequ$r63$1@digitalmars.com

On 1/15/19 6:59 AM, Petar Kirov [ZombineDev] wrote:
> On Tuesday, 15 January 2019 at 11:15:21 UTC, Atila Neves wrote:
>> On Monday, 14 January 2019 at 21:47:40 UTC, Steven Schveighoffer wrote:
>>> How many times have you written something like:
>>
>> More times than I can count...
>>
>>>
>>> void foo(T)(T t) if (__traits(compiles, t.bar())) // or is(typeof(t.bar()))
>>> {
>>>    t.bar();
>>> }
>>>
>>> [...]
>>
>> I'm not sure how best to go about this.
> 
> Sounds like a job for `static try` too keep things DRY:
> 
> void fun(T)(T t) if (__traits(compiles, t.foo()))
> {
>      t.foo();
> }
> 
> void fun(T)(T t) if (__traits(compiles, t.bar()))
> {
>      t.bar();
> }
> 
> // ----v
> 
> void fun(T)(T t)
> {
>      static try { t.bar(); }
>      else static try { t.foo(); }
>      else static assert(0, T.stringof ~ "does not support neither `foo`, nor `bar`");
> }

It might be nice to have something like this. Note that the two code samples aren't equivalent -- something that defines both bar and foo would not work with the first iteration, but would work with the second.

The DRYness of the constraint is annoying too, but this misses the point that t.bar and t.foo may be intended to BOTH compile, but only one does because of a "stupid" error. And so the wrong path is taken, and you either get weird other errors, or you get no errors, but the code doesn't work right or is unintentionally lower performance.

Understanding WHY a template constraint fails is important, but when you have __traits(compiles), it's a very blunt instrument. There are so many reasons why it might not compile, and it's really hard to figure out the answer, especially if you don't control the templates. Overriding the constraints might be the easiest way to probe the answers out.

-Steve

On 1/14/19 5:50 PM, Nicholas Wilson wrote:
> On Monday, 14 January 2019 at 22:32:59 UTC, Steven Schveighoffer wrote:
>> On 1/14/19 5:25 PM, Iain Buclaw wrote:
>>> On Mon, 14 Jan 2019 at 22:50, Steven Schveighoffer via Digitalmars-d
>>> <digitalmars-d@puremagic.com> wrote:
>>>>
>>>>
>>>> Any thoughts?
>>>>
>>>
>>> Compile with -verrors=spec (or -Wspeculative if using gdc).
>>>
>>
>> But wouldn't that show all errors, not just syntax errors? That might be a bit difficult to wade through.
>>
> 
> Yep, the SNR on that switch is abominable.

Could there be a way to focus that switch on certain files/lines? That would be much more useful.

-Steve