On Thursday, 6 December 2018 at 21:20:54 UTC, Mike Franklin wrote:
> 1) What is the fundamental missing language feature that is preventing us from implementing interpolated strings in the library?

To access symbols from the caller's scope.

> 2) What other use cases besides string interpolation would be enabled by adding such a feature?

Any thing from scala's implicit playbook.

On Thursday, 6 December 2018 at 16:19:12 UTC, Steven Schveighoffer wrote:
> [...]
> Most languages I know of create a string with toString-ing the interpolations.
>
> [...]
>
> The mixin/library solution is much less approachable. What's awesome about the language solution is that it just works without much extra understanding and verbosity, and need for using mixins. mixins are cool, but are best tucked away behind templates or generative functions. They shouldn't seen much in user code. Saying we should use a library solution for this is like saying we can replace foreach usage with a library foreach that lowers to some for-loop and explicit delegates (or maybe a mixin?). Yes, it could be done. No, it shouldn't be done. This is one of those types of syntax sugar that should not be ignored.

Well said.

I agree that lowering to a tuple opens up some cool possibilities, but what about the times when it is desired to just use string interpolation as just a plain string?
For example, if `i"a is ${a}."` is equivalent to 'tuple("a is ", a, ".")', then if you want to use the interpolated string as a string, you would have to do something like `text(i"a is ${a}.")`. If this is the case, how much better is this over a library implementation, which would look like this: `interp!"a is ${a}."`?

I am definitely not saying that a library solution is a better idea. I personally think that this only makes sense as a language feature. But I just think it would be best if `i"a is ${a}"` is more or less equivalent to `"a is "~a.to!string~"."`.

As you said, string interpolation is just (very nice) syntactic sugar over concatenation, so I think we should make this sugar as sweet as possible.

On 12/6/18 4:20 PM, Mike Franklin wrote:
> On Thursday, 6 December 2018 at 20:00:02 UTC, H. S. Teoh wrote:
>> Now, I'm also thinking about how to minimize the "surface area" of language change in order to make this proposal more likely to be accepted by W&A.
> 
> IMO the one thing that will likely get this across the finish line is to generalize it:
> 
> 1) What is the fundamental missing language feature that is preventing us from implementing interpolated strings in the library?
> 
> 2) What other use cases besides string interpolation would be enabled by adding such a feature?
> 
> If a compelling language feature that has more uses beyond string interpolation can be engineered and/or discovered, I think it will have a much better chance of being accepted.

It's really not what's enabled (clearly, things can still be used without string interpolation, and we can use mixins to simulate close to what we would want).

It's how one writes and reads code. The efficiency and elegance is not to be ignored.

I view `foreach` as a fundamental feature of D. Yet, it's completely unnecessary, given the existence of `for`. AND given the power of D's mixins, we can completely replace the functionality with a Foreach library type or function. Despite all that, I'd rather have foreach as a language construct, because it's so damned clean.

It's the boiling down of the syntax to the fundamental expression of the developer. I want to write the parameters to a function or to a template without having to reach for the comma-quote-separator-quote-comma sequence. Not only is this noisy and cumbersome, but it breaks up the data and the presentation in a way that is difficult to piece together.

writeln(a, " ", b, " ", c, " ", d);

is messy and distracting. It's like a photograph with lots of extraneous items taking away from what you should be looking at.

writefln("%s %s %s %s", a, b, c, d);

Better, I can envision how the result comes out. But get that string to be long and complicated, or add 5 more parameters, and now it's hard to figure out what variable goes where. It's not coupled to the location it belongs. Now the photograph has cutouts for your family members, and the actual photos of your family scrunched up together on the side, out of the picture. You just have to imagine what it would be like if they were in there.

writeln("$a $b $c $d");

Simple, elegant, obvious. With syntax highlighting, the symbols are identified easily. Its like a photograph with your family in focus against a nice background. I literally would have to explain the usage of this feature to nobody who is familiar with C. Or even PHP for that matter!

So it's not about what it enables, it's about enabling your expressiveness, and bringing focus to what is important when writing code.

Now, on top of that, if we do go with the lowering to tuples, we get even more power. We have so many more options than just a simple concatenated string. The DB query possibility makes my mouth water. And I've written a lot of DB queries with D. They are hard to write, and hard to read. Every time I add a stupid column, and then I have to add a new placeholder, and put the new value in the right place in the trailing arguments, it's a long process.

So everything this enables is doable, right now, in a library. It wouldn't be nearly as pretty, and the alternatives today without mixins are ugly, unfocused, and distracting. I won't say that we absolutely have to have this feature in the language, but I would say it would be a shining star of a feature when compared with other languages.

-Steve

On Thursday, 6 December 2018 at 21:32:22 UTC, H. S. Teoh wrote:
> Ah, so mixin templates *can* access symbols from the containing scope. That's nice... so actually, we can already implement string interpolation in the library.  The only complaint is the ugly syntax required `mixin blah!"abc ${def}"` instead of simply `blah!"abc ${def}"` or `blah("abc ${def}")`.

Library versions have existed for a while now. See for example `interp` from the scriptlike package on DUB [1], and Marler's own library version [2] linked in the original Github discussion.

[1] https://github.com/Abscissa/scriptlike/blob/v0.10.2/README.md#string-interpolation
[2] https://github.com/dlang/phobos/pull/6339

> I'm in favor of making the syntax less ugly, but I fear Andrei's objection is going to be that (1) this is already possible without a language change, and (2) the proposed language change would effectively only add syntactic sugar, which is unlikely to be enough justification for doing it.

Unless D someday adds AST macros, new syntax is always going to require language changes, so I don't think the fact that this is "only" a syntax change should disqualify it from consideration.

Whether or not the difference between `writeln(mixin(interp!"..."))` and `writeln(i"...")` is significant enough to be worth it is, of course, a value judgement, but given that string interpolation has a proven track-record in other languages, it's a 100% backwards-compatible change, and the implementation work has already been done, I think it stands a decent chance of being accepted. The most direct comparison I can find is DIP 1009, which added syntactic sugar for contracts, and was accepted "without objection."

On Thursday, 6 December 2018 at 21:20:54 UTC, Mike Franklin wrote:

> IMO the one thing that will likely get this across the finish line is to generalize it:
>
> 1) What is the fundamental missing language feature that is preventing us from implementing interpolated strings in the library?

AST macros :). That would give access to the caller's scope and it can replace the call expression with whatever the macro returns.

> 2) What other use cases besides string interpolation would be enabled by adding such a feature?

A whole lot of different use cases.

https://wiki.dlang.org/DIP50

--
/Jacob Carlborg

On 12/6/18 5:03 PM, o wrote:
> I agree that lowering to a tuple opens up some cool possibilities, but what about the times when it is desired to just use string interpolation as just a plain string?
> For example, if `i"a is ${a}."` is equivalent to 'tuple("a is ", a, ".")', then if you want to use the interpolated string as a string, you would have to do something like `text(i"a is ${a}.")`. If this is the case, how much better is this over a library implementation, which would look like this: `interp!"a is ${a}."`?

`text(...)` is the way to go. Or rather `i"a is $a".text`;

And no, it wouldn't be interp!"a is $a.", as you HAVE to put mixin in there (this is how scriptlike works).

Consider if you are actually writing a string interpolation to do a mixin, it would be:

mixin(mixin(interp!"..."));

> 
> I am definitely not saying that a library solution is a better idea. I personally think that this only makes sense as a language feature. But I just think it would be best if `i"a is ${a}"` is more or less equivalent to `"a is "~a.to!string~"."`.

I get what you are saying. But this solution is inferior for a few reasons:

1. It allocates. Maybe many times (lots of to!string calls allocate).
2. It loses information.
3. It depends on a library feature (i.e. unusable in betterC).

> As you said, string interpolation is just (very nice) syntactic sugar over concatenation, so I think we should make this sugar as sweet as possible.

It's sugar, but I'd prefer it to be D-flavored sugar. Expose what the developer wrote at compile-time, and let the language gurus work their magic. Concatenation is accessible, easily, but so are so many other things that make this very useful.

D is capable of doing so much, because of the ability to introspect, and discover at compile-time. I'm still freaking amazed that vibe.d can take a line like:

render!("myview.dt", foo, bar);

and process a DSL which has access to foo and bar, by those names, and outputs a compiled D function. I want to allow enabling such things with string interpolation too.

-Steve

On Thursday, 6 December 2018 at 21:47:02 UTC, Sebastiaan Koppe wrote:
> On Thursday, 6 December 2018 at 21:20:54 UTC, Mike Franklin wrote:
>> 1) What is the fundamental missing language feature that is preventing us from implementing interpolated strings in the library?
>
> To access symbols from the caller's scope.
>
>> 2) What other use cases besides string interpolation would be enabled by adding such a feature?
>
> Any thing from scala's implicit playbook.

There's a reason why there's no way to access the callers scope without a mixin...because it breaks encapsulation.  Imagine this:

int x = 10;
foo();


What is the value of x after this?  If a function (or whatever foo is) could access the caller's scope, then it could be anything.  This adds a very large mental burden on the reader because now every function call could modify any local state at any point.  That's why you need this:

int x = 10;
mixin foo();

Now you've got a flag to indicate that you need to understand foo before you can make any assumptions about your local state.  There's a balance between power and reasonable encapsulation.  If you make functions too powerful, then understanding code becomes a nightmare.

Interpolated strings are a special case because even though they can access the caller's scope, they do this explicitly:

int x = 10;
foo(i"$(x=20)");

You can clearly see that x is being set to 20, you don't need to know the implementation specifics of interpolated strings to see that.

A feature that allows functions and/or templates to access the caller's state implicitly would allow us to implement interpolated strings, however, this would come at the cost of adding a very large mental burden to understand any and all D code going forward.

On that note, you could make an argument that you should be able to access the caller's state in a "read-only" capacity, but that can get complicated.  Say you have this:

int x = 10;
int* y = &x;
foo();

What's the value of x after this?  What if we define foo as:

void foo(CallerScope s = __CALLER_SCOPE__)
{
    *(cast(int*)s.y) = 20;
}

We did have to cast `y` from const(int*) to int*, but we were able to change x nonetheless. You end up with the same problem that you have to understand every function you call at a much deeper level to know when it can modify your local state.

On Thursday, 6 December 2018 at 22:04:13 UTC, Steven Schveighoffer wrote:
> [...]
> Now, on top of that, if we do go with the lowering to tuples, we get even more power. We have so many more options than just a simple concatenated string. The DB query possibility makes my mouth water. And I've written a lot of DB queries with D. They are hard to write, and hard to read. Every time I add a stupid column, and then I have to add a new placeholder, and put the new value in the right place in the trailing arguments, it's a long process.
>
> -Steve

While I agree that lowering to tuples opens awesome possibilities, it is rather different than string interpolation. "string interpolation" should lower to a string, so an interpolated string can be used identically to any other string. It just makes a lot more sense if an "interpolated string" actually is a string, not a tuple.

String interpolation (at least using the 'i' prefix) should produce a string. If 'tuple interpolation' really is beneficial, maybe it could be added as a different string prefix
i.e `assert(t"my name is $name" == tuple("my name is ", name))`
and `assert(i"my name is $name" == "my name is Jeff")`

Just think how much nicer this is (lowering to a string):
`foo(i"My name is $name");`

Compared to this (lowering to a tuple, requiring a conversion to a string):
```
import std.conv: text;
foo(text(i"My name is $name"));
```

On Thursday, 6 December 2018 at 22:34:14 UTC, o wrote:
> Just think how much nicer this is (lowering to a string):
> `foo(i"My name is $name");`
>
> Compared to this (lowering to a tuple, requiring a conversion to a string):
> ```
> import std.conv: text;
> foo(text(i"My name is $name"));
> ```

With UFCS it looks a lot nicer already:
```
foo(i"My name is $name".text);
```
How often do you actually need to pass an interpolated string to a function in string form? That's not a rethorical question, it is important to consider. If 90% of string interpolation happens in writeln / logging then it's not worth forcing the use of phobos + GC + eager toString and concatenation just to make the remaining 10% of cases a bit easier.

On Thursday, 6 December 2018 at 22:34:14 UTC, o wrote:
> Just think how much nicer this is (lowering to a string):
> `foo(i"My name is $name");`
>
> Compared to this (lowering to a tuple, requiring a conversion to a string):
> ```
> import std.conv: text;
> foo(text(i"My name is $name"));
> ```

You can write `foo` in such a way that the user never has to manually insert a call to `text`:

void foo(string s)
{
    // do whatever
}

void foo(Args...)(Args args)
{
    import std.conv: text;
    foo(text(args));
}

In fact, you can even encapsulate this pattern in a mixin:

template interpArgs(alias fun)
{
    import std.format: format;

    enum interpArgs = q{
        auto %1$s(Args...)(Args args)
        {
            import std.conv: text;
            return %1$s(text(args));
        }
    }.format(__traits(identifier, fun));
}

mixin(interpArgs!foo);