Right now it's impossible to check "is there a valid (template) specialization for this function call" without also saying "and by the way, if there's a syntax error in the function, suppress the error and return false."

This makes many Phobos and library functions very annoying to use.

A simple (?) fix for this would be adding a new trait that checks if the expression resolves to a valid template specialization, without attempting to also check if the function body compiles, or else without suppressing internal syntax errors.

On Wed, Jan 15, 2020 at 03:13:08PM +0000, FeepingCreature via Digitalmars-d wrote:
> Right now it's impossible to check "is there a valid (template) specialization for this function call" without also saying "and by the way, if there's a syntax error in the function, suppress the error and return false."
> 
> This makes many Phobos and library functions very annoying to use.

Yeah, the error-gagging method of speculative template instantiation has been costing me lots of wasted time, too, esp. in Phobos range functions like .find, .filter, etc., because a single typo in the lambda argument causes the template to fail to instantiate, but the only information you get out of it is "cannot instantiate because it must satisfy isInputRange", but the reason it can't satisfy isInputRange is because a syntax error makes the type void (or _error_), but it doesn't tell you WHY the type is void or _error_.  Is it a syntax error? A mistyped symbol? A mismatching type? Who knows, you can't tease that information out of the compiler because errors are gagged. And no, turning on verbose output does not help because it floods you with truckloads of "errors" for *other* template instantiations that are completely unrelated to this one (and that actually *worked*, so that's just meaningless noise), and you're left to dig through the mountains of messages to find out which one might actually be relevant to your problem.

Furthermore, the error often crops up deep inside a long UFCS chain with multiple levels of nesting, so you can't exactly insert diagnostic messages in the middle to figure out where the problem is. The whole chain is one expression, after all, and all you'll get out of the compiler is that the entire expression failed to compile. But it could be as simple as a single typo deep inside one of the lambdas, or a missing import, and you've no idea where.

The only way I could diagnose problems of this sort was to copy-n-paste the entire chain, comment out most of the except the first N steps, and then iteratively uncomment them until it stops compiling. Then once it stops compiling and I identified the offending lambda, copy-n-paste the lambda body into an actual fake function with actual parameter types that the compiler is forced to compile on its own -- because otherwise all errors are gagged and it's impossible to know what the compiler didn't like about it -- just to get the actual error message out of the compiler.

And very often, it's a really trivial problem like a typo, a missing import, or forgetting to convert a type to the expected return type, etc.. The sort of error you'd expect the compiler to pinpoint with the exact line number and reason, and that you can fix in 30 seconds. But no thanks to gagged errors, it often takes more like 30 *minutes* just to narrow down the cause of the problem.

All in all, it makes for a very poor user experience. No wonder the naysayers have no good things to say about Phobos templates!


> A simple (?) fix for this would be adding a new trait that checks if the expression resolves to a valid template specialization, without attempting to also check if the function body compiles, or else without suppressing internal syntax errors.

+1000. This would've saved me HOURS of debugging trivial problems like missing imports and mistyped identifiers.


T

-- 
Chance favours the prepared mind. -- Louis Pasteur

On Wednesday, 15 January 2020 at 15:13:08 UTC, FeepingCreature wrote:
> Right now it's impossible to check "is there a valid (template) specialization for this function call" without also saying "and by the way, if there's a syntax error in the function, suppress the error and return false."


Another example:

The case which bothered me, and which I nagged FeepingCreature about leading to the post above, is that he has written us code to automatically JSON-decode to simple D structs, and we can override that automatic decoding where necessary, by writing a

T decode(T: SomeSpecialStruct)(const JSONValue value)

This works very well.... BUT: If I make an error such that my decode does not even compile, then I get no indication of this - a decoder is automatically generated instead, and at runtime I get strange behavior because the "wrong" decoder is being used.  Ideally I get decode errors, but it is easy to imagine cases where that would not even occur.

On 1/15/20 10:13 AM, FeepingCreature wrote:
> Right now it's impossible to check "is there a valid (template) specialization for this function call" without also saying "and by the way, if there's a syntax error in the function, suppress the error and return false."

I know what you mean, but they are not syntax errors (which ARE errors, even in templates). Which makes them really difficult to categorize.

> This makes many Phobos and library functions very annoying to use.
> 
> A simple (?) fix for this would be adding a new trait that checks if the expression resolves to a valid template specialization, without attempting to also check if the function body compiles, or else without suppressing internal syntax errors.

Generally, I use hasMember instead of __triats(compiles) and I'm better off for it. If something defines a member by a specific name, then it better be what I'm looking for, so try using it. If not, then you'll just have to name it something else. I'm not sure Phobos can take this road though.

-Steve

On Thu, Jan 16, 2020 at 11:00:03AM -0500, Steven Schveighoffer via Digitalmars-d wrote:
> On 1/15/20 10:13 AM, FeepingCreature wrote:
> > Right now it's impossible to check "is there a valid (template) specialization for this function call" without also saying "and by the way, if there's a syntax error in the function, suppress the error and return false."
> 
> I know what you mean, but they are not syntax errors (which ARE errors, even in templates). Which makes them really difficult to categorize.

Yeah, they are gagged errors that are nevertheless relevant to the problem at hand.  But it's very hard, from the compiler's standpoint, to know which gagged errors are relevant and which are not (most are not, in speculative template instantiations).

I'm tempted to propose that anything that passes sig constraints *should* force the compiler to treat any errors in the function body as hard (non-gagged) errors.  But then you also have sig constraints that fail due to gagged errors, and it's not clear at all which of those should be treated as hard errors.


[...]
> Generally, I use hasMember instead of __triats(compiles) and I'm better off for it. If something defines a member by a specific name, then it better be what I'm looking for, so try using it. If not, then you'll just have to name it something else. I'm not sure Phobos can take this road though.
[...]

Yeah, over time I've learned to avoid __traits(compiles) as much as
possible. It's just far too general, and unhelpful when problems occur.
(Why doesn't it compile? It could be any number of reasons, most of
which probably the person who wrote the __traits(compiles) didn't even
think of.)

But anyway, recent versions of dmd are now more helpful, by pointing out which sig constraints are failing, rather than just a blanket "cannot match template" error.  The next step IMO is to somehow display a brief summary of why that (or those) particular constraint(s) are failing. It's already quite helpful to know, e.g., that a template function failed to match because isInputRange!R failed. But isInputRange contains quite a number of further constraints; it would be nice to at least have an option to show the dirty details of where that particular constraint failed.

I've no idea how to implement such a thing, though.


T

-- 
Computerese Irregular Verb Conjugation: I have preferences.  You have biases.  He/She has prejudices. -- Gene Wirchenko

On Thursday, 16 January 2020 at 17:30:57 UTC, H. S. Teoh wrote:
> On Thu, Jan 16, 2020 at 11:00:03AM -0500, Steven Schveighoffer via Digitalmars-d wrote:
>> On 1/15/20 10:13 AM, FeepingCreature wrote:
>> > Right now it's impossible to check "is there a valid (template) specialization for this function call" without also saying "and by the way, if there's a syntax error in the function, suppress the error and return false."
>> 
>> I know what you mean, but they are not syntax errors (which ARE errors, even in templates). Which makes them really difficult to categorize.
>
> Yeah, they are gagged errors that are nevertheless relevant to the problem at hand.

I think he means they're semantic errors rather than parser (syntax) errors.

>
> I'm tempted to propose that anything that passes sig constraints *should* force the compiler to treat any errors in the function body as hard (non-gagged) errors.  But then you also have sig constraints that fail due to gagged errors, and it's not clear at all which of those should be treated as hard errors.
>

Would be nice, but would be a hard spec change. This is the sort of thing that D should have been doing from the beginning, but we may be too far in now for __traits(compiles) to be changed to do this. However, this is the behavior I would ask for for __traits(canInstantiateWith).

On 1/17/20 12:54 AM, FeepingCreature wrote:
> 
> Would be nice, but would be a hard spec change. This is the sort of thing that D should have been doing from the beginning, but we may be too far in now for __traits(compiles) to be changed to do this. However, this is the behavior I would ask for for __traits(canInstantiateWith).
> 

Just to point you at my previous thread on this: https://forum.dlang.org/post/q1jequ$r63$1@digitalmars.com

I think more tools for diagnosing why something doesn't happen would be good, but the request of "tell me if I was stupid" is I think impossible for the compiler to figure out.

-Steve

On Wednesday, 15 January 2020 at 15:13:08 UTC, FeepingCreature wrote:
> Right now it's impossible to check "is there a valid (template) specialization for this function call" without also saying "and by the way, if there's a syntax error in the function, suppress the error and return false."
>
> This makes many Phobos and library functions very annoying to use.
>
> A simple (?) fix for this would be adding a new trait that checks if the expression resolves to a valid template specialization, without attempting to also check if the function body compiles, or else without suppressing internal syntax errors.

Yes please (although I would say specialization and constraints, for all templates).

I worked around this in a commercial project a few years ago by creating an overload set of the passed symbol and a catch-all template, then instantiate and see if I got the catch-all (meaning no match), but obviously this is pretty nasty and doesn't work if one of the overloads of the original template is a catch-all as well.

On Friday, 17 January 2020 at 14:57:27 UTC, Steven Schveighoffer wrote:
> On 1/17/20 12:54 AM, FeepingCreature wrote:
>> 
>> Would be nice, but would be a hard spec change. This is the sort of thing that D should have been doing from the beginning, but we may be too far in now for __traits(compiles) to be changed to do this. However, this is the behavior I would ask for for __traits(canInstantiateWith).
>> 
>
> Just to point you at my previous thread on this: https://forum.dlang.org/post/q1jequ$r63$1@digitalmars.com
>
> I think more tools for diagnosing why something doesn't happen would be good, but the request of "tell me if I was stupid" is I think impossible for the compiler to figure out.
>
> -Steve

In other languages, such as C#, TypeScript and Rust, generic functions can use only properties of generic types iff such properties were required in the function generic type constraints. AFAIK Nim's Concepts [1] are also moving in that direction, even though their current way is more similar to D (I chatted with one of their core developers, a couple of months ago at a conference, but I may be misremembering some details).

In other words:

void use(T)(T obj)
where (hasMethod!(T, "callMe", int function(string, bool)))
{
    int x = obj.callMe("hello", true); // this works

    // the following would cause the template definition to
    // fail to type check (before even being instantiated):
    // obj.someThingElse = 3;
}

Of course, requiring complete constraints for D templates would be quite controversial, (although many languages are able to get away with that), but I think it could be a good if we could opt-into that on a per function basis (e.g. using `where`, instead of `if`).

If D has such a feature, I believe that libraries like phobos would use it extensively, while user (app) code would use current template duck typing model and only opt-into a stronger model when a developer wants to debug why his template code doesn't work as expected.

[1]: https://nim-lang.org/docs/manual_experimental.html#concepts

On 1/17/20 10:34 AM, Petar Kirov [ZombineDev] wrote:
> On Friday, 17 January 2020 at 14:57:27 UTC, Steven Schveighoffer wrote:
>> On 1/17/20 12:54 AM, FeepingCreature wrote:
>>>
>>> Would be nice, but would be a hard spec change. This is the sort of thing that D should have been doing from the beginning, but we may be too far in now for __traits(compiles) to be changed to do this. However, this is the behavior I would ask for for __traits(canInstantiateWith).
>>>
>>
>> Just to point you at my previous thread on this: https://forum.dlang.org/post/q1jequ$r63$1@digitalmars.com
>>
>> I think more tools for diagnosing why something doesn't happen would be good, but the request of "tell me if I was stupid" is I think impossible for the compiler to figure out.
>>
> 
> In other languages, such as C#, TypeScript and Rust, generic functions can use only properties of generic types iff such properties were required in the function generic type constraints. AFAIK Nim's Concepts [1] are also moving in that direction, even though their current way is more similar to D (I chatted with one of their core developers, a couple of months ago at a conference, but I may be misremembering some details).

Yes, concepts would be useful, but also would be limited for how D does duck typing. D's constraints are very much like concepts, but without having to declare everything up front for use in the function.

But it wouldn't work for UFCS either I don't think.

However, the real problem is things like typos that cause the function not to compile for reasons other than the intended failures.

For example:

T foo(T)(T x)
{
   return X + 5; // typo on parameter name
}

This will NEVER compile, because X will never exist. The intention was for it to compile with types that support addition to integers (i.e. it's EXPECTED to fail for strings), but instead it fails for all types. The result is that your code selects some other path when you don't want it to. Sometimes this just means it works but is slower, which is even harder to figure out.

But it's hard for the compiler to deduce intention from such typos. It would be great if the compiler could figure out this type of error, but I don't think it can.

The only "fix" really is to judiciously unittest with the calls you expect to work. And that generally doesn't happen (we are a lazy bunch!).

Perhaps, an "expected" clause or something to ensure it compiles for the expected types like: expected(T == int), which wouldn't unittest anything, just make sure it can compile for something that is expected.

-Steve