On Tuesday, 9 January 2024 at 23:21:34 UTC, Walter Bright wrote:

To sum up, it works with nested istrings poorly; it may even be sensible to forbid them entirely for DIP1027. Glad we’ve reached a consensus on this point. This case doesn’t seem crucial at the moment though; now we can focus on more relevant questions.

DIP1036 provides full CTFE capabilities at your disposal. You can validate anything about a format string; any compile-time-executable hypothetical validateSql(query) will fit. I guess none of the examples presented so far featured such validation because it usually tends to be long and not illustrative.

However, another Adam’s example does perform non-trivial compile-time validation. Here is how it is implemented.

They check a lot more. I agree it is hard to spot the error messages in the linked post so I’ll copy them here:

relation "snapshotsssss" does not exist. SQL: select size_mm, size_px from snapshotsssss where snapshot_id = $1

role "dummyuser" can't select on table "snapshots". SQL: select size_mm, size_px from snapshots where snapshot_id = $1

As you can see, they check sophisticated business logic expressed in terms of relational databases. And all of that happens at compile time. Isn’t that a miracle?

Err… every formatted writer has to be a template anyway, doesn’t it? It needs to accept argument lists that may contain values of arbitrary types.

Yeah. I admit this is a problem. As a rule of thumb, the most obvious code should yield the best results. With DIP1036, this is not the case at the moment: when you pass an interpolation sequence to a function not specifically designed for it, it wastes more stack space than necessary and passes useless junk in registers.

Others have mentioned that DIP1027 performs much worse in terms of speed (due to runtime parsing). While that is undoubtable, I think DIP1036 should be tweaked to behave as good as possible.

There was an idea in this thread to improve the ABI so that it ignores empty structs, but I’m rather sceptical about it.

Instead, let us note there are basically two patterns of usage for istrings:

1. Passing to a function that processes an istring and does something non-trivial. execi is a good example.
2. Passing to a function that simply stringifies every fragment, one after another. writeln is a good example.

Something counterintuitive, case 1 is easier to address: the function already traverses the received sequence and transforms it. So it is only necessary to write it in such way that it is inline-friendly.

By the way, what functions do we have in Phobos that fall into the case-2 category? write/writeln, std.conv.text, std.logger.core.log, and… is that all? Must be something else!..

Turns out there are only a handful of relevant functions in the entire stdlib. It shouldn’t be hard to put a filter in each of them. It also hints they are probably not that common in the wild.

However, when one encounters a third-party write-like function that is unaware of InterpolationHeader/etc., they should have a means to fix it from outside, i.e., without touching its source and ideally without writing a wrapper by hand. Unfortunately, I could not come up with a satisfactory solution for this. Will keep thinking. Perhaps someone else manages to find it faster.

An idea in a different direction. Currently, InterpolationHeader/etc. structs interoperate with write-like functions seamlessly (at the expense of passing zero-sized arguments) due to the fact they all have an appropriate toString method. If we remove those methods (and do nothing else), then write(i"a$(x)b") would produce something like:

InterpolationHeader()InterpolatedLiteral!"a"()InterpolatedExpression!"x"()42InterpolatedLiteral!"b"()InterpolationFooter()

The program, rather than introducing a silent inefficiency, immediately tells the user they need to account for these types.

And one more idea. Current implementation of DIP1036 can emit empty chunks—i.e., InterpolatedLiteral!""—see for example i"$(x)". If I was making a guess why it does so, I would say it strives to produce consistent, regular sequences. On the one hand, it might ease the job of interpolation-sequence handlers: they can count on the fact that expressions and literals always alternate inside a sequence. On the other, they have to check if a literal is empty and drop it if it is so it actually makes their job harder.

I do not know whether not producing empty literals in the first place would be a positive or negative change. But it is something worth to consider.

Slightly off-topic: when I was thinking about this, I was astonished by the fact istrings can work with readf/formattedRead/scanf. Just wanted to share this observation.

readf(i" $(&x) $(&y)");

This is true. I suppose the docs should mention InterpolationHeader and friends when talking about istrings, explain what an istring is lowered to, and show examples. Then a programmer who’ve read the docs will have a mental association between “istring” and “InterpolationHeader/Footer/etc.” Those who don’t read the docs—well, they won’t have. Only googling will save them.

To be honest, I’m not concerned about this point too much.

I’d argue it is wonderful that execi cannot be called with a pre-constructed string. The API should provide another function instead—say, execDynamicStatement(Sqlite, string, Args...). execi should be used for statically known SQL with interpolated arguments, and execDynamicStatement—for arbitrary SQL constructed at runtime. A verbose name is intentional to discourage its usage in favour of execi.

That makes sense. Though you’ll never guess what beast can be spawned by uncareful refactoring. Extra protection won’t harm, especially if it’s zero-cost.

P.S. Zero-initialization of variables is one of D’s cool features, indeed.

On Monday, 8 January 2024 at 03:05:17 UTC, Walter Bright wrote:

int x=readln.strip.split.to!int;
db.execi(xxx!i"INSERT INTO sample VALUES ($(id), $(2*x))");

template tuple(A ...) { alias tuple = A; }

Yes! It would be brilliant if alias could refer to any Expression, not just symbols. If that was the case, we could just pass InterpolationHeader/Footer/etc. to template parameters (as opposed to runtime parameters, where they go now).

// Desired syntax:
db.execi!i"INSERT INTO sample VALUES ($(id), $(2*x))";
// Desugars to:
db.execi!(
    InterpolationHeader(),
    InterpolatedLiteral!"INSERT INTO sample VALUES ("(),
    InterpolatedExpression!"id"(),
    id,
    InterpolatedLiteral!", "(),
    InterpolatedExpression!"2*x"(),
    2*x, // Currently illegal (`2*x` is not aliasable).
    InterpolatedLiteral!")"(),
    InterpolationFooter(),
);
// `execi!(...)` would expand to:
db.execImpl("INSERT INTO sample VALUES (?1, ?2)", id, 2*x);

With this approach, they are processed entirely via compile-time sequence manipulations. Zero-sized structs are never passed as arguments. Inlining is not necessary to get rid of them.

An example with writeln (or just about any function alike):

writeln(interpolate!i"prefix $(baz + 4) suffix");
// Desugars to:
writeln(interpolate!(
    InterpolationHeader(),
    InterpolatedLiteral!"prefix "(),
    InterpolatedExpression!"baz + 4"(),
    baz + 4,
    InterpolatedLiteral!" suffix"(),
    InterpolationFooter(),
));
// `interpolate!(...)` would expand to:
writeln("prefix ", baz + 4, " suffix");

On Wednesday, 10 January 2024 at 15:07:42 UTC, Nickolay Bukreyev wrote:

writeln(interpolate!i"prefix $(baz + 4) suffix");
// Desugars to:
writeln(interpolate!(
    InterpolationHeader(),
    InterpolatedLiteral!"prefix "(),
    InterpolatedExpression!"baz + 4"(),
    baz + 4,
    InterpolatedLiteral!" suffix"(),
    InterpolationFooter(),
));
// `interpolate!(...)` would expand to:
writeln("prefix ", baz + 4, " suffix");

Well, InterpolatedLiteral and InterpolatedExpression don’t have to be templates anymore:

writeln(interpolate!i"prefix $(baz + 4) suffix");
// Desugars to:
writeln(interpolate!(
    InterpolationHeader(),
    InterpolatedLiteral("prefix "),
    InterpolatedExpression("baz + 4"),
    baz + 4,
    InterpolatedLiteral(" suffix"),
    InterpolationFooter(),
));
// `interpolate!(...)` would expand to:
writeln("prefix ", baz + 4, " suffix");

On Wednesday, 10 January 2024 at 19:53:48 UTC, Walter Bright wrote:

It looks very similar to what I presented in my later posts (this and one following). It’s inspiring: we are probably getting closer to common understanding of things.

Exactly. Let me try to explain why DIP1036 is doing what it is doing. For illustrative purposes, I’ll be drastically simplifying code; please excuse me for that.

Let there be foo, a function that would like to receive an istring. Inside it, we would like to transform its argument list at compile time into a new argument list. So what we essentially want is to pass an istring to a template parameter so that it is available to foo at compile time:

int x;
foo!(cast(Format)"prefix ", 2 * x); // foo!(alias Format, alias int)()

Unfortunately, this does not work because 2 * x cannot be passed to an alias parameter. This is the root of the problem. The only way to do that is to pass them to runtime parameters:

int x;
foo(cast(Format)"prefix ", 2 * x); // foo!(Format, int)(Format, int)

However, now foo cannot access the format string at compile time—its type is simply Format, and its value becomes known only at runtime. So we encode the value into the type:

int x;
foo(Format!"prefix "(), 2 * x); // foo!(Format!"prefix ", int)(Format!"prefix ", int)

This is more or less what DIP1036 is doing at the moment. Hope it became clear now.

I’d say DIP1036, as we see it now, relies on a clever workaround of a limitation imposed by the language. If that limitation is gone, the DIP will become simpler.