On 12/12/19 12:07 AM, Jonathan Marler wrote:
> 
> Lowering interpolated strings to a format string and a tuple of values rather than just a tuple is less versatile than other solutions that have previously been proposed.  This DIP pigeon-holes interpolated strings for one use case.  Interpolated strings can be used for more than just creating strings.  They are just a pretty syntax to interleave string literals and expressions. This solution is assuming the user will only use this for string formatting, but that's not necessarily the case.

Indeed, one can potentially have more use cases than format string plus tuple. That is one thing I don't like about it. But any technique we use to forward tuples and string formatting to a function is going to require adjustment in some cases, and work as-is in others.

We did have problems with the straight tuple lowering (i.e. how do you tell which part was string literal and which parts were parameters). the new proposal handles that quite well in fact.

Not to say that this isn't solvable, but I do like that aspect of the new DIP -- the string literal parts are clearly defined. It definitely needs changing from its current mechanism (the requirement of % for placeholder is too limiting IMO).

One thing that might make things more useful -- if the compiler could define a new type istring, which is a derivative of string literal, and one could accept that as a parameter type for purposes of overloads if needed.

> My original proposal lowered it directly to a tuple of string literals and expressions (https://github.com/dlang/dmd/pull/7988) and Adam Ruppe's proposal (http://dpldocs.info/this-week-in-d/Blog.Posted_2019_05_13.html) lowered it to a custom type where the same tuple could be accessed via a field.  These solutions provide more versatility and are just as easy to print and format into strings with a single function call. Adam Ruppe's proposal was more complex than my own but improved versatility (i.e. added a method to convert it to a string without importing a module, allowed functions to overload on interpoated strings, provided access to the original raw un-interpolated string, etc).  The problem with this DIP is that it's both more complex than my original proposal and less versatile.

I think all of them are somewhat equally versatile. With a mechanism to specify how to generate the format string, such as H. S. Toeh suggested, I think Walter's proposal is just as useful as yours or Adam's. All of them provide somewhat similar transformations, just the ordering of the string parts and the non-string parts are different. Unfortunately, I don't know if there's a way to forward the "tuple-ness" of the result by wrapping it if you needed to convert something.

Even though Adam's proposal allows one to translate between what the compiler generates and what is needed, it copies the tuple into a type, which may not be what you want (e.g. Walter's example for apples, apples is an lvalue, but in Adam's it would always be an rvalue).

There are limitations in yours as well, I like the ability to specify the formatting of the individual items, which would be difficult if not clunky in your proposal.

> 
> The other problem with this DIP is how it handles consecutive interpolated strings.  Consider a case where you'd like to write a message with consecutive interpolated strings (see real life example here https://github.com/dlang/dmd/pull/7988/files#diff-685230fdddcb87fcbb4a344b264a2fb6L736): 
> 
> 
> // using this DIP
> writefln(i" ... %a ... ", i" ... %b ... ");
> // becomes
> writefln(" ... %s ... ", a, " ... %s ... ", b);
> // that doesn't work, so you'd have to do this
> writeln(i" ... $(a) ... ".format, i" ... $(b) ... ".format);

that does pose a slight problem, but one could potentially handle that with concatenation (i.e. concatenation of two interpolated strings results in the same thing as if you concatenated the two string literals and slapped i on the front).

> 
> I also don't see any mention of my proposal in the DIP in "Prior Work" or how this DIP compares to my proposal or Adam's.
> 

This should be rectified.

-Steve

On Thursday, 12 December 2019 at 17:01:01 UTC, Adam D. Ruppe wrote:
> On Thursday, 12 December 2019 at 15:41:54 UTC, aliak wrote:
>> Except you can actually assign it to a string?
>
> The javascript version is allowed to return whatever it wants. The default one returns string but you can do other things with it too like return objects or functions or whatever.
>
> https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Template_literals

That is a good observation. I forgot all about tagged template literals. We should totally steal some of that.

From the page:

> Template literals are enclosed by the back-tick (` `)  (grave accent) character instead of double or single quotes. Template literals can contain placeholders. These are indicated by the dollar sign and curly braces (${expression}). The expressions in the placeholders and the text between the back-ticks (` `) get passed to a function. The default function just concatenates the parts into a single string. If there is an expression preceding the template literal (tag here), this is called a "tagged template". In that case, the tag expression (usually a function) gets called with the template literal, which you can then manipulate before outputting.

try this in your browser's console:

val = 4;
function sql(strings, ...keys) { return {sql: strings.join("?"), keys} };
sql`select * from table where a = ${val} and b = ${val}`;

it will return: {"sql":"select * from table where a = ? and b = ?","keys":[4,4]}

> If you wanna talk about taking the best ideas from other languages, Javascript's template literals are actually really nice. They are so much more than a multi-line string or a way to concatenate things.

yes.

On Thursday, 12 December 2019 at 17:03:53 UTC, Atila Neves wrote:
> On Thursday, 12 December 2019 at 17:01:01 UTC, Adam D. Ruppe wrote:
>> On Thursday, 12 December 2019 at 15:41:54 UTC, aliak wrote:
>>> [...]
>>
>> The javascript version is allowed to return whatever it wants. The default one returns string but you can do other things with it too like return objects or functions or whatever.
>>
>> [...]
>
> It'd be great if you could expand on this with a list of examples of what D could do.

Tagged template literals are indeed quite nice in JS. Here's an example from the chalk npm package [1]:

```js
// Plain template literals (a.k.a. built-in string interpolation):
const str1 = `
CPU: ${chalk.red("90%")}
RAM: ${chalk.green("40%")}
DISK: ${chalk.yellow("70%")}
`;

// Same example but with tagged template literals - more DRY:
const str2 = chalk`
CPU: {red 90%}
RAM: {green 40%}
DISK: {yellow 70%}
`;

// Tagged template literals + interpolation of variables:
const cpu = 90;
const ram = 40;
const disk = 70;

const str3 = chalk`
CPU: {red ${cpu}%}
RAM: {green ${ram}%}
DISK: {yellow ${disk}%}
`;

Here's another example, this case from the es2015-i18n-tag library [2] that applies translations, locale and currency formatting:

const name = 'Steffen'
const amount = 1250.33
console.log(i18n`Hello ${ name }, you have ${ amount }:c in your bank account.`)
// Hallo Steffen, Sie haben US$ 1,250.33 auf Ihrem Bankkonto.
```

The D analog of a function that can be used as tagged template string literal would be:

```d
string interpolate(Args...)(string[] parts, Args args)
{
    import std.algorithm.iteration : joiner;
    import std.conv : text;
    import std.range : roundRobin;

    string[Args.length] stringifiedArgs;

    static foreach (idx, arg; args)
        stringifiedArgs[idx] = text(arg);

    return parts.roundRobin(stringifiedArgs[]).joiner.text;
}

static import term; // Assume some ANSI term coloring library

const string res = i"
CPU: ${term.red("90%")}
RAM: ${term.green("40%")}
DISK: ${term.yellow("70%")}
".interpolate;
```

[1]: https://github.com/chalk/chalk
[2]: https://github.com/skolmer/es2015-i18n-tag

On Thursday, 12 December 2019 at 23:31:21 UTC, Petar Kirov [ZombineDev] wrote:
> On Thursday, 12 December 2019 at 17:03:53 UTC, Atila Neves
> wrote:
>>
>> It'd be great if you could expand on this with a list of
>> examples of what D could do.
>
> Tagged template literals are indeed quite nice in JS. Here's an
> example from the chalk npm package [1]:
>
> ```js
> // Plain template literals (a.k.a. built-in string
> interpolation):
> const str1 = `
> CPU: ${chalk.red("90%")}
> RAM: ${chalk.green("40%")}
> DISK: ${chalk.yellow("70%")}
> `;

With no changes to DIP-1027:

  const str1 = format(i"
  CPU: %(chalk.red("90%"))
  RAM: %(chalk.green("40%"))
  DISK: %(chalk.yellow("70%"))
  ");

which lowers to

  const str1 = format("
  CPU: %s
  RAM: %s
  DISK: %s
  ", chalk.red("90%"), chalk.green("40%"), chalk.yellow("70%"));

> // Same example but with tagged template literals - more DRY:
> const str2 = chalk`
> CPU: {red 90%}
> RAM: {green 40%}
> DISK: {yellow 70%}
> `;

Likewise:

  with (chalk) {
      const str2 = format(i"
  CPU: %(90.percent.red)
  RAM: %(40.percent.green)
  DISK: %(70.percent.yellow)
  ");
  }

> // Tagged template literals + interpolation of variables:
> const cpu = 90;
> const ram = 40;
> const disk = 70;
>
> const str3 = chalk`
> CPU: {red ${cpu}%}
> RAM: {green ${ram}%}
> DISK: {yellow ${disk}%}
> `;

Likewise:

  with (chalk) {
      const str2 = format(i"
  CPU: %(cpu.red)
  RAM: %(ram.green)
  DISK: %(disk.yellow)
  ");
  }

> Here's another example, this case from the es2015-i18n-tag
> library [2] that applies translations, locale and currency
> formatting:
>
> const name = 'Steffen'
> const amount = 1250.33
> console.log(i18n`Hello ${ name }, you have ${ amount }:c in
> your bank account.`)
> // Hallo Steffen, Sie haben US$ 1,250.33 auf Ihrem Bankkonto.
> ```

Likewise:

  writeln(i18n(i"Hello %name, you have %{$}(amount) in your bank account."));

which lowers to:

  writeln(i18n("Hello %s, you have %$ in your bank account.", name, amount));

and normal function i18n can do the gettext() stuff, and handle
%$ itself, or indeed it could do nothing special with %$ but
leave it to custom format specifiers and `amount` being a
monetary object:

  https://wiki.dlang.org/Defining_custom_print_format_specifiers

So I must conclude:

1. This is all very cool, but

2. DIP-1027 can do all of it as currently specified.

3. Actually DIP-1027 is not as bad as I thought. In particular,
instead of documenting the feature in terms of its failures vs.
other languages, it can say "use format() if you mainly to
build a string" and "here's some cool stuff that this design
lets you do, that more traditional string interpolation could
not do."

So I respectfully withdraw my complaints and instead submit
support for some kind of SQL generalization with ?
placeholders.

On Thursday, 12 December 2019 at 17:03:53 UTC, Atila Neves wrote:
> It'd be great if you could expand on this with a list of examples of what D could do.

Of course, there's the examples of printf, writeln, sql query.. (One other addition I might make to my proposal is having the string version of what's in the thing too, but I digress). Those are the easy ones.

Note btw that you can even do a format!"%s"(a) type thing. format!i"#{a}" can be transformed.

But it can get pretty interesting to look beyond function calls. What about some kind of object literal?

int a;
string b;
JSONValue j = json!iq{
     a: #{a},
     b: #{b}
};

That specific syntax assumes interpolated token strings are a thing but it could just as well be

JSONValue j = json!i"
   a: #{a},
   b: #{b}
";

or %(a) or $a or whatever. I don't care.



The implementation would look something like:

JSONValue json(__d_interpolated_string s)() {
     JSONValue j;
     foreach(idx, member; s.tupleof) {
         static if(!(idx & 1)) {
           j.object[member.replace(",", "").strip] = JSONValue(member[idx + 1]());
          }
     }
     return j;
}

you know plus more sanity but you get the idea. All the data is in an object which we can pass to a template to do compile time checking and return type processing etc.

Of course, for json, you can do a built in AA and a constructor from it, just then the types have to be heterogeneous. Not a huge deal for json but it can get ugly nested. But we can do xml too, think react's jsx

int foo;
string bar = "<";

Element e = jsx!i`
   <foo value=#{foo}>#{bar}</foo>
`;

assert(e.tagName = "foo");
assert(e.attrs.value == foo);
assert(e.innerHTML == "&lt;");



This is almost interchangeable with the mixin(foo!"string") thing... in fact you could implement it that way, just instead of having foo!xx return a code string the compiler helps a bit more.

On Friday, 13 December 2019 at 00:05:54 UTC, mipri wrote:
> 3. Actually DIP-1027 is not as bad as I thought. In particular,
> instead of documenting the feature in terms of its failures vs.
> other languages, it can say "use format() if you mainly to
> build a string" and "here's some cool stuff that this design
> lets you do, that more traditional string interpolation could
> not do."

To be clear, some cool stuff is

1. Internationalization. Normally with string interpolation in
languages, the interpolation happens before anything can see
the string, so f.e. the Perl way to do internationalization
is just

  printf(gettext("I scanned %g %s."),
         $dir_scan_count,
         $dir_scan_count == 1 ?
           gettext("directory") : gettext("directories"),
  );
https://metacpan.org/pod/Locale::Maketext::TPJ13

despite "I scanned $count $directory" being the more normal
way to write that code in Perl.

2. SQL queries without SQL injection, if support for this is
added. It's precisely due to the popularity of string
interpolation that SQL injection even happens, and so again the
Perl way to do database queries is

  my $contained_sql = <<"";
  SELECT id FROM city_buildings
     WHERE  minLong >= ? AND maxLong <= ?
     AND    minLat  >= ? AND maxLat  <= ?

  my $contained = $dbh->selectcol_arrayref($contained_sql,undef,
                        $minLong, $maxLong, $minLat, $maxLat);
https://metacpan.org/pod/DBD::SQLite

Normally, string interpolation is a convenience that you can
use most of the time but have to put aside when things get
advanced. Here we have a proposed string interpolation that
fits the most common case while still being useful in more
advanced cases, which is instead a slightly less convenient in
certain cases.

The most common case:

  writeln("Hello, %s.".format(name)); // currently
  writefln(i"Hello, %name.");         // DIP-1027
  println("Hello, $name.");           // Kotlin

An advanced case:

  writeln(format(_("Hello, %s!"), name)); // currently (djtext)
  writeln(i"Hello, %name!".nu_gettext);   // DIP-1027
  // I can't find a Kotlin example.

An inconvenient case:

  // currently
  string[] greetings = [
      "Hello, %s".format(name),
      "Aloha, %s".format(name),
      "Hiya, %s".format(name),
      "Salutations, %s".format(name),
  ];

  // DIP-1027
  string[] greetings = [
      i"Hello, %name".format,
      i"Aloha, %name".format,
      i"Hiya, %name".format,
      i"Salutations, %name".format,
  ];

  // Kotlin
  val greetings = listOf(
      "Hello, $name",
      "Aloha, $name",
      "Hiya, $name",
      "Salutations, $name"
  )

Note the 'nu_gettext'. It can't be djtext's existing _() or
getdtext() functions because you can't accept the i""  string
without also accepting the parameters for the i"" string.

On Friday, 13 December 2019 at 00:06:15 UTC, Adam D. Ruppe wrote:
> But it can get pretty interesting to look beyond function calls. What about some kind of object literal?
>
> int a;
> string b;
> JSONValue j = json!iq{
>      a: #{a},
>      b: #{b}
> };

This doesn't work though:

  auto n = 4;
  [1,2,3].map!(format("a + %s", n)).each!writeln;
  // Error: variable n cannot be read at compile time

It doesn't even work to pass that format() to

  void ignore(string s)() { }

So I imagine i"" strings won't be useful in static arguments
except with static parameters.

On Friday, 13 December 2019 at 01:19:32 UTC, mipri wrote:
> This doesn't work though:
>
>   auto n = 4;
>   [1,2,3].map!(format("a + %s", n)).each!writeln;

Indeed, but it does work to pass seq!(() => n). That's why I've amended my proposal to do that instead. Then it inherits the CT-ness of n while still being usable in a runtime context.

It is similar to `lazy` parameters built into the language;

> So I imagine i"" strings won't be useful in static arguments
> except with static parameters.


auto interpolate(T...)() {
        import std.conv;
        string s;
        foreach(t; T)
                static if(is(typeof(t) == string))
                        s ~= t;
                else
                        s ~= to!string(t());
        return s;
}

void main() {
        int n;
        enum n2 = 56;
        // runtime variable works for runtime, but errors if `enum s;`
        string s = interpolate!("this works: ", () => n);
        // CT variable works in either cse
        enum e = interpolate!("this works too ", () => n2);
        import std.stdio;
        writeln(s);
        writeln(e);
}



In my original proposal I was just going to use a value, but the compiler-generated lambda is more flexible as seen here.

So

i"foo #{a}"

becomes

__DInterpolatedString!("foo ", () => a);

and you can do all kinds of magical things with that. (In my original proposal I rejected this hybrid library solution because of this very problem, but this little bit of magic allows it. The compiler simply rewrites the i"" string into a call to that template, then object.d can be responsible for doing the rest.)


If you like I could actually go ahead and write up a library implementation and we pretend the syntax lowering is in place to evaluate it.

On 13.12.19 02:30, Adam D. Ruppe wrote:
> On Friday, 13 December 2019 at 01:19:32 UTC, mipri wrote:
>> This doesn't work though:
>>
>>   auto n = 4;
>>   [1,2,3].map!(format("a + %s", n)).each!writeln;
> 
> Indeed, but it does work to pass seq!(() => n). That's why I've amended my proposal to do that instead. Then it inherits the CT-ness of n while still being usable in a runtime context.
> 
> It is similar to `lazy` parameters built into the language;

I think that's a bit hacky, e.g.:

int x=0;
auto s=i"$(++x)";
writeln(s); // 1
writeln(s); // 2

On Friday, 13 December 2019 at 02:10:04 UTC, Timon Gehr wrote:
> I think that's a bit hacky, e.g.:

yeah, can fix that with a lazily initialized cache.

but if it is built into the compiler we can fix details like this there too.