This is the discussion thread for the first round of Community Review of DIP 1036, "Formatted String Tuple Literals":

https://github.com/dlang/DIPs/blob/15537f9b36afa48f1c1cd57468e8c77f8f2499dd/DIPs/DIP1036.md

The review period will end at 11:59 PM ET on September 22, or when I make a post declaring it complete. Discussion in this thread may continue beyond that point.

Here in the discussion thread, you are free to discuss anything and everything related to the DIP. Express your support or opposition, debate alternatives, argue the merits, etc.

However, if you have any specific feedback on how to improve the proposal itself, then please post it in the feedback thread. The feedback thread will be the source for the review summary that I will write at the end of this review round. I will post a link to that thread immediately following this post. Just be sure to read and understand the Reviewer Guidelines before posting there:

https://github.com/dlang/DIPs/blob/master/docs/guidelines-reviewers.md

And my blog post on the difference between the Discussion and Feedback threads:

https://dlang.org/blog/2020/01/26/dip-reviews-discussion-vs-feedback/

Please stay on topic here. I will delete posts that are completely off-topic.

On Tuesday, 8 September 2020 at 10:59:31 UTC, Mike Parker wrote:

> However, if you have any specific feedback on how to improve the proposal itself, then please post it in the feedback thread. The feedback thread will be the source for the review summary that I will write at the end of this review round. I will post a link to that thread immediately following this post.

The feedback thread is here:

https://forum.dlang.org/post/vpgemqdqgorsuzqbxazw@forum.dlang.org

What's the purpose of toFormatString with a default spec? Why would you ever want to have a default??? That's just some hacky workaround from the previous DIP to circumvent some arbitrary rule. Why wouldn't you have some sort of type map instead that maps a format string to a type? All of that information is available.

The way 'hasAllSpec' seems to be intended to use just seems terrible. What happened to having an overload for printf() to handle the case of an interpolated string. Is that overload not happening? Implicitly converting to char* on a condition like 'hasAllSpec' is just asking for trouble. This special case isn't required, it will only introduce bugs; just add an overload to printf. It also requires the user to specify all the formatting themselves when all that information is already there for it to know what formatting specifier to use. That wouldn't fly in a intrrolated string library implementation and it definitely shouldn't fly as a language feature.

The 'toFormatStringImpl' and friends implementations should just be removed. Its simple enough that anyone can implement it, not.like there is a performance benefit anyways as it is just going to be using CTFE, like a library implementation. It is also too specific, something like an sql query would just put a number depending on the order of the parameters. So in that case it isnt really a specifier for the argument some much as which argument should go where. I can only imagine how confusing and error prone that is going to be, especially since you have to specify every single argument if you dont implement your own wrapper or overload to fix this.

On 9/8/20 8:10 AM, Avrina wrote:
> What's the purpose of toFormatString with a default spec? Why would you ever want to have a default??? That's just some hacky workaround from the previous DIP to circumvent some arbitrary rule. Why wouldn't you have some sort of type map instead that maps a format string to a type? All of that information is available.

Because the default spec should not be constrained by the language (e.g. always "%s" as DIP1027 specified). See for instance mysql prepared statements, where the spec is "?". This way the callee can decide what to put for default specs.

> 
> The way 'hasAllSpec' seems to be intended to use just seems terrible. What happened to having an overload for printf() to handle the case of an interpolated string. Is that overload not happening?

We could potentially handle it that way. But the idea of not having to handle it that way for this case is appealing. It means we don't have to write an overload for it unless you want to. There are potentially quite a few C libraries that use the format + parameters mechanism, and this allows a way to call those without having to write a wrapper for it.

> Implicitly converting to char* on a condition like 'hasAllSpec' is just asking for trouble. This special case isn't required, it will only introduce bugs; 

All vague statements, do you know of any specific trouble or bugs?

> just add an overload to printf. It also requires the user to specify all the formatting themselves when all that information is already there for it to know what formatting specifier to use. That wouldn't fly in a intrrolated string library implementation and it definitely shouldn't fly as a language feature.

This DIP doesn't provide for matching of parameters to their respective formats. It is one step above DIP1027, in that it allows a callee to specify the default format, and the format is available at compile-time. The main benefit is simply the fact that you CAN overload based on the fact that you received an interpolated string.

> 
> The 'toFormatStringImpl' and friends implementations should just be removed. Its simple enough that anyone can implement it, not.like there is a performance benefit anyways as it is just going to be using CTFE, like a library implementation. It is also too specific, something like an sql query would just put a number depending on the order of the parameters. So in that case it isnt really a specifier for the argument some much as which argument should go where. I can only imagine how confusing and error prone that is going to be, especially since you have to specify every single argument if you dont implement your own wrapper or overload to fix this.

These are implementation details, and not necessarily the final implementation. The real API is specified in the description.

The numbered SQL problem is definitely one that is troubling -- you have to put in a placeholder for the number, and then replace it with the correct value, which means 2 passes of compile-time string generation. However, this is something that could be improved upon if the type provides access to the raw spec sequence, or allows a different way to build the format string (I have some ideas).

Note that we are trying to prevent people from abusing the implementation to achieve "AST Macros" as Walter likes to put it. If we can provide a better way to generate the formatting, and still keep this property, it has a better chance of being accepted.

-Steve

On 9/8/20 6:59 AM, Mike Parker wrote:
> This is the discussion thread for the first round of Community Review of DIP 1036, "Formatted String Tuple Literals":
> 
> https://github.com/dlang/DIPs/blob/15537f9b36afa48f1c1cd57468e8c77f8f2499dd/DIPs/DIP1036.md 

I am really, really happy to see string interpolation coming to D, but I have a concern with the implementation as described in the DIP. Viz:

> An i"" string is not likely to be compatible with existing D string functions and variables. The compiler SHOULD provide helpful hints in error messages to help new users understand how to properly use the new feature. This message MAY link to a web page to educate the user on resolving the error.

When I am writing Javascript or Python, I use string interpolation quite a bit (even nicer now that Python3 has f-strings), and in my estimate the proportion of uses with print() (or other language equivalent) versus other uses is at most 50/50. i.e., string interpolation is OFTEN used in many contexts beyond simply printing a message to the user.

I read the sections "Usage in existing string-accepting functions" and understand `isInterpolationSpec`, and subsections "Wrong-use in unrelated function" and "On implicit conversions."

The authors argue:
> To avoid unintentional bugs in libraries that don't anticipate interpolated strings, this DIP does not recommend implicit conversion of the interpolation spec structure to string, ... The logic behind this decision is that, when making the additional effort to explicitly write format specifiers for all interpolated values, the user will have demonstrated significant understanding of how the feature works both in general and for a specific use case. In that scenario, we can allow implicit conversion so the interpolated string can be used by existing functions.

They then provide an example using printf.

Ignoring for a moment C functions, what is the anticipated issue "with existing functions [and] the safety of accidentally calling existing functions inappropriately" ?

Thanks again to the authors.  I really truly think this is important work and thank them, but if the barrier to entry for a SEEMINGLY "simple" feature is too high, this makes D much less approachable for new users, about which I believe we should be very concerned.

Kind regards

On 9/8/20 2:50 PM, James Blachly wrote:

As a tacky self-followup, I would add that I am not against the `interpolationSpec` specifically, but just arguing for more liberal approach to implicit conversion to D string, i.e., idup.

Normally I would suggest that implicit .idup should be accompanied by a compiler warning (instead of error, suggesting adding .idup), but I know how Walter feels about warnings.

On Tuesday, 8 September 2020 at 10:59:31 UTC, Mike Parker wrote:
> This is the discussion thread for the first round of Community Review of DIP 1036, "Formatted String Tuple Literals":
>
> https://github.com/dlang/DIPs/blob/15537f9b36afa48f1c1cd57468e8c77f8f2499dd/DIPs/DIP1036.md

When a DIP has to devote an equal amount of space to justifying the caveats and limitations of its proposal as it does to explaining the proposal in the first place (~1,500 words each), I think that is probably a sign that the proposal is not a very good one.

Maybe the authors can try getting it into C++23 instead. :^)

On 9/8/20 2:50 PM, James Blachly wrote:
> On 9/8/20 6:59 AM, Mike Parker wrote:
>> This is the discussion thread for the first round of Community Review of DIP 1036, "Formatted String Tuple Literals":
>>
>> https://github.com/dlang/DIPs/blob/15537f9b36afa48f1c1cd57468e8c77f8f2499dd/DIPs/DIP1036.md 
> 
> 
> I am really, really happy to see string interpolation coming to D, but I have a concern with the implementation as described in the DIP. Viz:
> 
>> An i"" string is not likely to be compatible with existing D string functions and variables. The compiler SHOULD provide helpful hints in error messages to help new users understand how to properly use the new feature. This message MAY link to a web page to educate the user on resolving the error.
> 
> When I am writing Javascript or Python, I use string interpolation quite a bit (even nicer now that Python3 has f-strings), and in my estimate the proportion of uses with print() (or other language equivalent) versus other uses is at most 50/50. i.e., string interpolation is OFTEN used in many contexts beyond simply printing a message to the user.
> 
> I read the sections "Usage in existing string-accepting functions" and understand `isInterpolationSpec`, and subsections "Wrong-use in unrelated function" and "On implicit conversions."

One thing to note here is that i"..." results not in a single thing, but a list of things. So it's not really possible to provide the mechanism we are providing AND have the whole thing implicitly convert to a single string. Much of the arguments about "wrong-use" are aimed at the previous DIP and why we feel this mechanism for the format string is better.

Another previous suggestion (by Adam actually) is to wrap the entire thing into a single type, and encapsulate the list of parameters inside that. If we did it that way, then you could potentially automatically convert the whole thing to a string implicitly using existing facilities.

But, you do lose some important capabilities. For instance, you would now be passing a copy of each item, instead of the item itself. This has issues that are not easily solved (for example forwarding refness, or binding to an alias).

To me it's a choice between not having to specify .idup when you want a string (which necessarily has to allocate, even if it's implicitly done) vs. compile time goodies that come from expanding a thing into a compile-time list. Not only that, but there are already functions which use this mechanism of format + args, and attaching this feature to those functions is going to be much smoother if the parameters are parameters and not stuffed into a struct.

This is why we went with the .idup route -- it's simple, easy to write, and already has a well-defined meaning. On top of that, it is a known signal that GC allocations are happening.

> 
> The authors argue:
>> To avoid unintentional bugs in libraries that don't anticipate interpolated strings, this DIP does not recommend implicit conversion of the interpolation spec structure to string, ... The logic behind this decision is that, when making the additional effort to explicitly write format specifiers for all interpolated values, the user will have demonstrated significant understanding of how the feature works both in general and for a specific use case. In that scenario, we can allow implicit conversion so the interpolated string can be used by existing functions.
> 
> They then provide an example using printf.
> 
> Ignoring for a moment C functions, what is the anticipated issue "with existing functions [and] the safety of accidentally calling existing functions inappropriately" ?

This is the part that's aimed at DIP1027, and why we felt you should be able to overload based on the format string type, and that it should not automatically work for things like the createWindow example. Specifically for people who are used to Javascript or Python (or others), and expect the function to work as a string generator. We hope that the implementation in the compiler is smart enough to suggest the correct usage when the argument type mismatch is detected.

> Thanks again to the authors.  I really truly think this is important work and thank them, but if the barrier to entry for a SEEMINGLY "simple" feature is too high, this makes D much less approachable for new users, about which I believe we should be very concerned.

It's a valid point. The full explanation of why we wrote the DIP this way is not essential to explaining the feature. In documentation, it might be good for ddoc to have a special notation for functions that accept formatted string tuple literals (i.e. collapse the interpolation string format and varargs that follow into one parameter). Then usage is simple, but one still has to understand the mechanisms to provide functions that accept such parameters. Though a simple tutorial and examples will go a long way.

BUT, one still has to be wary of how the first argument, if it is an interpolation literal, may consume multiple parameters for the function. For functions that are defined without varargs, it's going to be highly confusing. I would be very surprised if any interpolation string tuple accepting function is not a template varargs one.

-Steve

On Tuesday, 8 September 2020 at 13:32:10 UTC, Steven Schveighoffer wrote:
> On 9/8/20 8:10 AM, Avrina wrote:
>> What's the purpose of toFormatString with a default spec? Why would you ever want to have a default??? That's just some hacky workaround from the previous DIP to circumvent some arbitrary rule. Why wouldn't you have some sort of type map instead that maps a format string to a type? All of that information is available.
>
> Because the default spec should not be constrained by the language (e.g. always "%s" as DIP1027 specified). See for instance mysql prepared statements, where the spec is "?". This way the callee can decide what to put for default specs.

That doesn't answer the question. A type map to specifier would do the same thing, just better; actually doing the work for the user.

>> The way 'hasAllSpec' seems to be intended to use just seems terrible. What happened to having an overload for printf() to handle the case of an interpolated string. Is that overload not happening?
>
> We could potentially handle it that way. But the idea of not having to handle it that way for this case is appealing. It means we don't have to write an overload for it unless you want to. There are potentially quite a few C libraries that use the format + parameters mechanism, and this allows a way to call those without having to write a wrapper for it.

The way you have to pass the interpolated string for it to work makes it almost completely useless. Requiring all format specifiers to be defined by the user is counter intuitive compared to using an overload. There are many C libraries, they require you to write bindings in D anyways, it would not be that difficult to add an overload for interpolated strings.

>> Implicitly converting to char* on a condition like 'hasAllSpec' is just asking for trouble. This special case isn't required, it will only introduce bugs;
>
> All vague statements, do you know of any specific trouble or bugs?

The DIP lists one? The CreateWindow example, if the specifier is given, you can call a function with the wrong arguments that weren't intended. The only way to avoid this is if you specifically create an overload to avoid the implicit conversion. This should not be the default.

If there is any implicit conversion happening for an I telrpolated string, the only valid case would be one where it is converted to the finalized string. Which isn't possible the way this is being implemented.

>> just add an overload to printf. It also requires the user to specify all the formatting themselves when all that information is already there for it to know what formatting specifier to use. That wouldn't fly in a intrrolated string library implementation and it definitely shouldn't fly as a language feature.
>
> This DIP doesn't provide for matching of parameters to their respective formats. It is one step above DIP1027, in that it allows a callee to specify the default format, and the format is available at compile-time. The main benefit is simply the fact that you CAN overload based on the fact that you received an interpolated string.

Not sure what you are getting at here. You seemed to have taken a paragraph out of context on which it was written.

It shouldn't provide any formatting at all. See the below statement you cut off.

>> The 'toFormatStringImpl' and friends implementations should just be removed. Its simple enough that anyone can implement it, not.like there is a performance benefit anyways as it is just going to be using CTFE, like a library implementation. It is also too specific, something like an sql query would just put a number depending on the order of the parameters. So in that case it isnt really a specifier for the argument some much as which argument should go where. I can only imagine how confusing and error prone that is going to be, especially since you have to specify every single argument if you dont implement your own wrapper or overload to fix this.
>
> These are implementation details, and not necessarily the final implementation. The real API is specified in the description.

Yes, and the features they provide is what should be removed. There's no point to being able to call a C function with an interpolated string without an overload, see above.

> The numbered SQL problem is definitely one that is troubling -- you have to put in a placeholder for the number, and then replace it with the correct value, which means 2 passes of compile-time string generation. However, this is something that could be improved upon if the type provides access to the raw spec sequence, or allows a different way to build the format string (I have some ideas).

The sequence is accessible publically, at least with the given implementation as it is just a template. If that is not currently intended then the DIP implementation should be fixed, and that makes the situation with the 'toFormatString' default specifer much much worse. The user is forced to use it and can't implement their own type map to specifier to make interpolated strings function without having to have the user forcibly put specifiers in their  interpolated strings for types that don't match the default.

> Note that we are trying to prevent people from abusing the implementation to achieve "AST Macros" as Walter likes to put it. If we can provide a better way to generate the formatting, and still keep this property, it has a better chance of being accepted.
>
> -Steve

It'd be better if this wasn't accepted then, rather than adding a crippled feature that's counter intuitive for a user to use.

The discussion of the feature proposed was promising in the other DIP threads. What this ended up being isn't all that different than the other DIP.

On Wednesday, 9 September 2020 at 01:32:58 UTC, Avrina wrote:
> A type map to specifier would do the same thing, just better; actually doing the work for the user.

What would this look like?