On Tuesday, 8 September 2020 at 10:59:31 UTC, Mike Parker wrote:
> [...]

I think it's a little bit awkward and risky that things like

foo(Args...)(string a, Args args)
or I suppose also
foo(string a, int b)

will get more than one parameter with a call like

foo(i"hello $i");

which is very counter-intuitive to all existing concepts (this is a case that would pass multiple arguments from something which looks just like one single string)

For example the SQL example shows very nicely why doing this without the library author allowing it is a bad idea. You argue that an attribute for the entire interpolated type is a bad idea and I agree with that, however an attribute just for allowing the compiler to expand interpolated strings to multiple arguments like this would be a great way to avoid unintended potentially dangerous usage.

I really like the idea of the special _d_interpolated_string type containing all the parts as that will make usage very flexible. However I really dislike the intended API usage with `toFormatString` and `hasAllSpecs`. To me it seems very specialized just for the printf function and not really designed to be usable for functions performing simple concatenation like text(), which I think are a more common use. It looks like directly accessing "Parts" was not intended, but direct access would be the only way to reasonably and efficiently implement string interpolation as a user (without doing error-prone string replace operations which would break as soon as a user manually types in a % character)

Instead of forcing the % syntax onto users using toFormatString, can we specify a proper API that could be implemented without it breaking with a simple

i"You scored 100% with $points!".idup

?

On Wednesday, 9 September 2020 at 08:12:05 UTC, WebFreak001 wrote:
> I think it's a little bit awkward and risky that things like
>
> foo(Args...)(string a, Args args)
> or I suppose also
> foo(string a, int b)
>
> will get more than one parameter with a call like
>
> foo(i"hello $i");


That would be a type mismatch error since i"..."'s first argument is an "interpolated tuple spec", NOT a string. The compiler's error message would probably suggest you try ".idup" to convert it to a plain string. If you did that, it would just be a single argument.

The implicit conversion hasAllSpecs allows *only* happens if 1) the user provides a {} component for every interpolated item and 2) even so, it converts to `immutable(char)*` rather than `string`.

On 9/9/20 4:12 AM, WebFreak001 wrote:
> On Tuesday, 8 September 2020 at 10:59:31 UTC, Mike Parker wrote:
>> [...]
> 
> I think it's a little bit awkward and risky that things like
> 
> foo(Args...)(string a, Args args)
> or I suppose also
> foo(string a, int b)
> 
> will get more than one parameter with a call like
> 
> foo(i"hello $i");

That won't compile. The interpolation spec type will not implicitly convert to string.

> 
> which is very counter-intuitive to all existing concepts (this is a case that would pass multiple arguments from something which looks just like one single string)
> 
> For example the SQL example shows very nicely why doing this without the library author allowing it is a bad idea. You argue that an attribute for the entire interpolated type is a bad idea and I agree with that, however an attribute just for allowing the compiler to expand interpolated strings to multiple arguments like this would be a great way to avoid unintended potentially dangerous usage.

The intention is that you have to opt-in by specifying a template parameter that accepts the spec. An attribute is not necessary.

> 
> I really like the idea of the special _d_interpolated_string type containing all the parts as that will make usage very flexible. However I really dislike the intended API usage with `toFormatString` and `hasAllSpecs`. To me it seems very specialized just for the printf function and not really designed to be usable for functions performing simple concatenation like text(), which I think are a more common use. It looks like directly accessing "Parts" was not intended, but direct access would be the only way to reasonably and efficiently implement string interpolation as a user (without doing error-prone string replace operations which would break as soon as a user manually types in a % character)
> 
> Instead of forcing the % syntax onto users using toFormatString, can we specify a proper API that could be implemented without it breaking with a simple
> 
> i"You scored 100% with $points!".idup

You are right in all of this. I think we probably need to allow access to the parts, and this solves a lot of issues. We can leave everything else, including printf compatibility.

One of the goals of this mechanism is to start simple and expand if necessary. But it's going to be impossible to prevent people from introspecting the template parameters anyway. Just providing an alias to the parts is a simple thing, and allows much better code.

One thing that I cringe on is the SQL example, where each parameter needs a number. With access to the parts, this is a single pass. With only toFormatString to get the format string, you have to put in a searchable placeholder in the format string, and then replace that on a second pass.

I will talk with Adam and Mike and see how to proceed, as this requires a bit more work.

-Steve

On Wednesday, 9 September 2020 at 02:21:30 UTC, Adam D. Ruppe wrote:
> On Wednesday, 9 September 2020 at 01:32:58 UTC, Avrina wrote:
>> A type map to specifier would do the same thing, just better; actually doing the work for the user.
>
> What would this look like?

Don't have access to a computer ATM, not writing on a phone. I'll write it up when I can.

On 9/8/20 9:32 PM, Avrina wrote:
> On Tuesday, 8 September 2020 at 13:32:10 UTC, Steven Schveighoffer wrote:
>> On 9/8/20 8:10 AM, Avrina wrote:
>>> What's the purpose of toFormatString with a default spec? Why would you ever want to have a default??? That's just some hacky workaround from the previous DIP to circumvent some arbitrary rule. Why wouldn't you have some sort of type map instead that maps a format string to a type? All of that information is available.
>>
>> Because the default spec should not be constrained by the language (e.g. always "%s" as DIP1027 specified). See for instance mysql prepared statements, where the spec is "?". This way the callee can decide what to put for default specs.
> 
> That doesn't answer the question. A type map to specifier would do the same thing, just better; actually doing the work for the user.

Sorry, I didn't read the full question. So yes, a type map would work. But I think actually, if we give people access to the format tuple list (as I have brought up in the other subthread), a specific mechanism inside druntime is not necessary.

I still want to allow the implicit cast to const char* to allow one to use C functions without having to wrap them. And in that case, you need a mechanism to generate the format string anyway. And there's no reason not to expose it for other users (I'm thinking of mysql-native, which uses a single specifier for all parameters, '?'). We can handle the easiest case, and leave the more complex ones to the user.

> The way you have to pass the interpolated string for it to work makes it almost completely useless. Requiring all format specifiers to be defined by the user is counter intuitive compared to using an overload. There are many C libraries, they require you to write bindings in D anyways, it would not be that difficult to add an overload for interpolated strings.

The DIP does not prevent someone overloading printf to deal with it in your specified manner. BUT if you don't want to do it, there is no harm in allowing the call to proceed.

Now, printf could be overloaded, yes. We could make it work. It's not part of the DIP though. The point about writefln is salient because it will not work without an overload, and people will definitely expect writefln to work with interpolated strings.

> 
>>> Implicitly converting to char* on a condition like 'hasAllSpec' is just asking for trouble. This special case isn't required, it will only introduce bugs;
>>
>> All vague statements, do you know of any specific trouble or bugs?
> 
> The DIP lists one? The CreateWindow example, if the specifier is given, you can call a function with the wrong arguments that weren't intended. The only way to avoid this is if you specifically create an overload to avoid the implicit conversion. This should not be the default.

The key phrase in that is "if the specifier is given". Why would you add a specifier if you didn't understand what you are doing?

The point of that is for cases like i"my name is $name", which will NOT convert implicitly to a const char *. The thought process goes:

1. I've used interpolated strings in (Python | Javascript | ...).
2. Oooh! D has interpolated strings, let me just do that.
3. Compiler says "no, use .idup".

But if you are aware of the mechanisms of the interpolation spec, and you want to specify the formatting fields (e.g. if you are doing this in printf, you are already used to doing it), then it just works.

> If there is any implicit conversion happening for an I telrpolated string, the only valid case would be one where it is converted to the finalized string. Which isn't possible the way this is being implemented.

It is true this is not possible.

>>> The 'toFormatStringImpl' and friends implementations should just be removed. Its simple enough that anyone can implement it, not.like there is a performance benefit anyways as it is just going to be using CTFE, like a library implementation. It is also too specific, something like an sql query would just put a number depending on the order of the parameters. So in that case it isnt really a specifier for the argument some much as which argument should go where. I can only imagine how confusing and error prone that is going to be, especially since you have to specify every single argument if you dont implement your own wrapper or overload to fix this.
>>
>> These are implementation details, and not necessarily the final implementation. The real API is specified in the description.
> 
> Yes, and the features they provide is what should be removed. There's no point to being able to call a C function with an interpolated string without an overload, see above.

Programmers are lazy. If I can do:

import core.stdc.stdio : printf;

printf(i"hello ${%d}name");

That might be enough. It might bug me that I have to always specify formatting specs, and then I'm motivated to write an overload. But it's not necessary for the purposes of this language change.

>> The numbered SQL problem is definitely one that is troubling -- you have to put in a placeholder for the number, and then replace it with the correct value, which means 2 passes of compile-time string generation. However, this is something that could be improved upon if the type provides access to the raw spec sequence, or allows a different way to build the format string (I have some ideas).
> 
> The sequence is accessible publically, at least with the given implementation as it is just a template. If that is not currently intended then the DIP implementation should be fixed, and that makes the situation with the 'toFormatString' default specifer much much worse. The user is forced to use it and can't implement their own type map to specifier to make interpolated strings function without having to have the user forcibly put specifiers in their  interpolated strings for types that don't match the default.

The DIP implementation is an example, and while usable, was not intended as a defining part of the spec. In fact, I have conceived of a different mechanism that would break anything that depended on having that specific implementation, but would still satisfy the DIP requirements.

I have since changed my view that the template parameters should be visible. They are visible anyway by introspection, and as you and others have pointed out, the way you would have to use the intended interface to process formatted items with the format string means 2 passes over the format string at compile-time.

In other words, the compiler would (in this implementation) separate out the data for us in a nice way, but the only way to access it is to generate a string, and then have to split apart that string again to process it a different way. There's no need for that.

>> Note that we are trying to prevent people from abusing the implementation to achieve "AST Macros" as Walter likes to put it. If we can provide a better way to generate the formatting, and still keep this property, it has a better chance of being accepted.
>>
> 
> It'd be better if this wasn't accepted then, rather than adding a crippled feature that's counter intuitive for a user to use.

I agree. We will have to be convincing that allowing access to the format specifiers and string data separately does not equate to AST macros.

-Steve

On Tuesday, 8 September 2020 at 10:59:31 UTC, Mike Parker wrote:
> This is the discussion thread for the first round of Community Review of DIP 1036, "Formatted String Tuple Literals":

As per the rules, I'm responding here to Steven's post from the Feedback thread.

Steven Schveighoffer wrote:
> First, thanks for your opinion. We value all the opinions of people in the community, especially long-standing members with a lot of experience with D such as yourself.
>
> However, this entire post has no actionable or specific items, and really should have been added to the discussion thread instead. I'm responding to it here because in the past I have had cases where I posted feedback and got no response or changes, and it irked me.

Thanks for replying. The specific, actionable item in my post is: withdraw the DIP.

This kind of feedback ("related to the merits of the proposal rather than to the contents of the DIP") is explicitly permitted in Community Review by the Feedback thread rules, which is why I posted it there instead of in the Discussion thread.

The reason I gave that feedback, instead of more detailed feedback about the DIP's contents, is that I sincerely believe DIP 1036 cannot be salvaged. I have a lot of respect for you and Adam, and would hate to see you waste your time and effort on a proposal doomed to failure. I think you (or anyone else who wishes to take up the string-interpolation torch) would be much better off discarding DIP 1036 and starting a new proposal from scratch.

Regarding the specific points you replied to:

> The interface is actually simple to use. Without specific concerns, it's hard to address these comments further.

If the interface is simple to use, why does the DIP anticipate that new users will have so much trouble using it that they'll need "helpful hints" from the compiler, and possibly even a link to a web page (!), to get it right? I quote:

> An i"" string is not likely to be compatible with existing D string functions and variables. The compiler SHOULD provide helpful hints in error messages to help new users understand how to properly use the new feature. This message MAY link to a web page to educate the user on resolving the error.

To me, this does not pass the laugh test. [1] If you anticipate this being hard enough to get right that new users cannot do it without extensive hand-holding, maybe the problem is not with the users, but with the proposal itself.

> This measure of "complexity" is arbitrary, and has no bearing on the validity of the DIP. in fact, many already-accepted DIPS have relatively similar ratios of description and rationale.

Again, I quote:

> ### Justifications
>
> The **complexity** of the format spec may seem superfluous, however it serves four key roles:

Emphasis added. If you want to nitpick the wording here, take it up with your co-author.

> One thing to note is that this DIP has a prior very similar DIP, but just used a string for the formatting specification. The Justification section is ENTIRELY devoted to explaining why this DIP does not use that. In fact, we can remove the entire section, and it doesn't change anything about the DIPs features or benefits. But without a distinction between DIP1027 and this DIP, arguably it would be rejected on principle (why approve something that was just rejected).

I think members of this community sometimes give the language maintainers (currently Walter and Atila, previously Walter and Andrei) too little credit. They're not perfect, but generally speaking they are reasonable people with good judgment. Have there been any actual examples of a DIP being rejected on principle for being too similar to another previously-rejected DIP?

> This assessment is incorrect. There is an implicit conversion of the format specification to immutable char * in the circumstance that you have specified all format information. This makes it compatible with printf.

"X is true in the circumstance that Y" is not the same thing as "X is true." :)

Also, the fact that i"${%s}(foo)" and i"$(foo)" are equivalent when you pass them to `writefln` but not when you pass them to `printf` is another thing that doesn't pass the laugh test. You're not making it easy for me to tell my friends about D.

> Implicitly converting the format spec to string would result in undoubtedly incorrect function calls, and we address the concerns quite thoroughly. Even if it is a desired thing, it's not something we should provide. This is not the same as trying and failing. We intend misuse of string conversion to not compile.

I understand perfectly. My criticism of DIP 1036 is not that you have failed at what you set out to do, but that you have set out to do the wrong thing to begin with.

By the way, there's another part of that Scott Meyers talk that addresses the problem of accidentally passing arguments to the wrong parameters. The solution he shows there would also work for the example in the DIP, without requiring any of the elaborate machinery the DIP proposes.

> I would characterize the DIP as not only trying to address these issues, but succeeding. printf is supported for valid cases. writefln will be supported as expected. Conversion to string or immutable char * is possible with a library call which should be provided. There are no flaws I can see.

Adding dedicated overloads of writefln, idup, etc. to deal with interpolated strings might sound like a good solution, but it doesn't scale. The *best-case* scenario is that DIP 1036 creates a bunch of meaningless busywork for library maintainers; the more realistic scenario is that most libraries never get updated at all, and users have to either check every individual function they use for interpolated-string compatibility, or defensively spam .idup everywhere (at which point, you might as well just make interpolated strings use the GC to start with).

In other words, DIP 1036 is exactly the kind of thing Andrei is describing when he talks about Good Work [3]:

> Good Work begets more Good Work. Typically Good Work produces context, opportunity, and precedent for more of the same. The same reviewer who rubber stamped a piece of Good Work will have an idea how to produce more Good Work derived from it. The kind of environment where Good Work is revered encourages its creation, in a cycle that creates the illusion of progress. Because Good Work is complex, it produces "bug ripples" whereby increasingly complex Good Work fixes one bug but is liable to introduce others.

DIP 1027 had its issues, but at least it was simple to explain, simple to use, and didn't require the library and runtime to bend over backwards to support it. DIP 1036, by contrast, is hard to explain, fiddly to use, and requires extensive coordination between the compiler, runtime, and library to implement. In short, it's a step in the wrong direction.

[1] "Can I explain this to my friend, who doesn't know D, with a straight face?"
[2] https://www.youtube.com/watch?v=5tg1ONG18H8&t=46m14s
[3] https://forum.dlang.org/thread/q6plhj$1l9$1@digitalmars.com?page=15

On 9/9/20 5:41 PM, Paul Backus wrote:
> Adding dedicated overloads of writefln, idup, etc. to deal with interpolated strings might sound like a good solution, but it doesn't scale. The *best-case* scenario is that DIP 1036 creates a bunch of meaningless busywork for library maintainers; the more realistic scenario is that most libraries never get updated at all, and users have to either check every individual function they use for interpolated-string compatibility, or defensively spam .idup everywhere (at which point, you might as well just make interpolated strings use the GC to start with).

This (and the related points about complexity and surprise especially of new users made by Paul but also by myself and others) make me tend to believe that we should just add GC-only, implicit automatic conversion to Dstring, and be done with it.

Are there contemporary examples of other languages whose string interpolation is as complex as this proposals? I am really only most familiar with Javascript and Python, both of which use automatic memory management and the *string is a string is a string*.

James

On Wednesday, 9 September 2020 at 23:04:37 UTC, James Blachly wrote:
> Are there contemporary examples of other languages whose string interpolation is as complex as this proposals? I am really only most familiar with Javascript and Python, both of which use automatic memory management and the *string is a string is a string*.

Javascript is actually *very* similar to this dip, with the key difference that in D, we reuse existing normal function call syntax whereas Javascript invented a new magic function call syntax just for this feature.

See the "Tagged templates" section here: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Template_literals

Notice that they call it a "template literal" rather than an interpolated string since it is so much more than a string.

If you write

foo`${5}, ${23}`

in Javascript, it actually will rewrite that into a function call:

foo([ "", ", ", ", ", "" ], 5, 23);

I realize that's hard to read with all the quotes and commas... but it is essentially a dynamically-typed equivalent to what this DIP is talking about.

The first argument to foo is a string spec. In D, we represent it as a templated struct. Javascript instead represents that as an array of strings. But it is fundamentally the same idea: it gives the string pieces from around the placeholders and an entry representing the inserted item.

Then, the interpolated items are passed as separate arguments after the spec - just like we propose for D. The example on MDN for this uses JS' `...values` syntax to put it into an array, in D, we'd call that same thing `(Values...)(Values values)`, but again it is the same thing, just JS uses their dynamic typing whereas D uses a template.


Now, the obvious difference with Javascript is they have a default function. Above, I wrote foo`...`. Leave the foo part off and it uses the default function instead - which is identical to our proposed idup. JS can do this because they introduced a new function call syntax to the language. And they can do that because JS only has one kind of function call to begin with: obj.foo(x) (where obj is optional and will default to null).

D, by contrast, has several kinds of function calls: traditional foo(x), UFCS x.foo, and template foo!(x). Instead of inventing three new kinds of magic function call syntax, we just reuse what we already have. But, of course, that does have one small downside: there's no way to insert a default function like Javascript does without either more compiler magic or limiting it to just one case and forgetting about the others.

Instead we simply ask you to explicitly call the function you want in all cases. I doubt most users will even find this particular controversial after they try it in reality.

On Tuesday, 8 September 2020 at 10:59:31 UTC, Mike Parker wrote:
> This is the discussion thread for the first round of Community Review of DIP 1036, "Formatted String Tuple Literals":
>
> https://github.com/dlang/DIPs/blob/15537f9b36afa48f1c1cd57468e8c77f8f2499dd/DIPs/DIP1036.md
>
> The review period will end at 11:59 PM ET on September 22, or when I make a post declaring it complete. Discussion in this thread may continue beyond that point.
>
> Here in the discussion thread, you are free to discuss anything and everything related to the DIP. Express your support or opposition, debate alternatives, argue the merits, etc.
>
> However, if you have any specific feedback on how to improve the proposal itself, then please post it in the feedback thread. The feedback thread will be the source for the review summary that I will write at the end of this review round. I will post a link to that thread immediately following this post. Just be sure to read and understand the Reviewer Guidelines before posting there:
>
> https://github.com/dlang/DIPs/blob/master/docs/guidelines-reviewers.md
>
> And my blog post on the difference between the Discussion and Feedback threads:
>
> https://dlang.org/blog/2020/01/26/dip-reviews-discussion-vs-feedback/
>
> Please stay on topic here. I will delete posts that are completely off-topic.

Please support like this:

int i = 10;

string b = "test " ~ i;

writeln(b); // test 10

On Thursday, 10 September 2020 at 18:30:26 UTC, zoujiaqing wrote:
>
> Please support like this:
>
> int i = 10;
>
> string b = "test " ~ i;
>
> writeln(b); // test 10

Shouldn't a type like "i" in this example be automatically converted to string in cases like this?

Matheus.