January 14

On Sunday, 14 January 2024 at 08:23:47 UTC, Richard (Rikki) Andrew Cattermole wrote:

>

For instance suffix strings, positional arguments are not handled here and if something wasn't an interpolated string.

Yes, suffix strings. There’s simply nowhere to put them in. We cannot even attach them to the last argument as @ISuffix… because there can be no arguments at all! (Not to mention this would break symmetry—which I’m not fond of.) Until there is a satisfactory solution, I cannot compare inline attributes with other interpolation approaches.

January 15
On 15/01/2024 6:12 AM, Nickolay Bukreyev wrote:
> On Sunday, 14 January 2024 at 08:23:47 UTC, Richard (Rikki) Andrew Cattermole wrote:
>> For instance suffix strings, positional arguments are not handled here and if something wasn't an interpolated string.
> 
> Yes, suffix strings. There’s simply nowhere to put them in. We cannot even attach them to the last argument as `@ISuffix`… because there can be no arguments at all! (Not to mention this would break symmetry—which I’m not fond of.) Until there is a satisfactory solution, I cannot compare inline attributes with other interpolation approaches.

There is a solution.

Use a positional argument instead, set to -1.

Quite easy really :)
January 14
I'll summarize up front, and so one can skip the detailed reply below. It's obvious there's no meeting of the minds here. I'll just explain where I'm coming from.

Features doing simple things should be simple. That's the case with 1027. Doing complicated things should be expected to be more complicated to use. With 1036, it starts out at the other end. It's complicated, with complexity that one has to write additional code to make it simple again (the "filter out the unnecessary templates" thing we talked about).

It's fine that transmorgrifying the string to adapt it to SQL has complexities to it. That's expected. If that's all string interpolation would be used for, that's ok.

But people want to use string interpolation for mixins, etc., all that have nothing to do with SQL. The transmorgrification should not be necessary for those cases, it should just work. But it doesn't, if the filter isn't applied there's a significant runtime and code space cost.

Simple things should be simple, complicated things should expect complexity. But 1036 has simple things being complicated. Doing SQL is about the same level of complexity for 1027 as 1036. For doing simple things, 1036 remains complicated, and 1027 gets simple.

That's the extent of my unease with it.

Does 1027 do *everything* 1036 does? No. There's the Format can be implicitly converted to a string thing. So there's a tradeoff. Do we trade off a minor thing for a fair amount of complexity that yes, the user will see? Every feature in D is a compromise of one sort or another. I fall on one side of that, you the other.

Another illustration of this idea. For algorithmic code, I invented opApply(). But after experience with it, I slowly grew to dislike it because I could never remember how it worked, and would have to reread the documentation on it. The implementation of it is also ugly, as it has to rewrite the code that surrounds it. Heaven help anyone who has to read the code gen output of that. I much prefer the later approach using lambdas. They're simple, easy to remember and use. There isn't much implementation for them; they grew naturally out of other features in the language.

The one thing that lambdas don't do that opApply does is recursion. Some klunky code has to be written to make a lambda do recursion. But that's ok, as there are relatively few needs for recursion, so the simple cases with lambdas are simple, and the complicated cases are more work. This is as it should be.

Nothing comes for free, everything is a compromise.

----------------------------

On 1/13/2024 9:12 AM, Steven Schveighoffer wrote:
> This doesn't help, as an enum implicitly converts to its base type.

It's a point for 1036.


> In none of the proposals I have written or supported, has it been meant for `writef`. I don't understand the desire to hook `writef` and `format`.

A format string is needed to support formats like "%03d". I understand that is not needed for SQL, but support for formats is not so straightforward with 1036.


> Me too. But shouldn't we prefer compiler errors? Shouldn't we use the type system for what it is intended?

We should prefer compiler errors, I agree. But everything has a cost to it.


> I've literally left bugs like this in code for years without noticing until the actual thing (an exception) was printed, and then it was hours to figure out what was happening.

When I've had an exception printed, I'll grep the code base for the message to see where it came from. The message should also say what was wrong. If it's a generic exception, I'll use the debugger to find where it came from. I doubt a format string error would be hard to track down, as it's only one level below the cause of the error.


> This would be a step up, but still doesn't fix the format specifier problem.

I'm not seeing a format specifier problem. %s, ?1, what's the difference? What if the string literals generated by 1036 have a stray ?1 in them? If there's a check for that, I missed it.


> Without a trailer, this isn't solvable technically.

It can by counting arguments and the %s.


> I just meant that it isn't a requirement to disallow them somehow.

I did mention a way it can work.


> The SQL example *DOES NOT* generate a format string, I've told you this multiple times. It generates a string with placeholders. There is no formatting. In fact, the C function doesn't even accept the parameters, those happen later after you generate the prepared statement.

CTFE work is required to generate both "hello ?1 ?2" and "hello %s %s". I understand your point that the latter requires an unacceptable level of CTFE processing, though I don't agree with the "unacceptable" part. I don't see how 1036 is going to deal with stray ? in the string literals without CTFE.


> Things other than SQL *do not require building a string*.

Building a format string enables formats other than %s. For example, %03d. Adam wrote a library function for that, https://github.com/adamdruppe/interpolation-examples/blob/master/lib/format.d which does a lot of CTFE. Noted in it is "the format string is built at compile time".
	>> I suspect you routinely use CTFE for far, far more complex tasks. This is a
>> rounding error.
> Wait, so generating an extra template is a bridge too far, but parsing a DSL at compile time is a rounding error?

It's ((number of args) - 1)*2+3 templates. Looking at the object file, they are templates with pretty long names, along with their struct instantiations, which is not free in compiler time or memory consumption.


> Not to mention that concatenation is easy. I can do it in one line (if I don't really care about performance). The same cannot be said for parsing.

As you mentioned before, the code only has to be written once!

January 15
On 1/15/24 02:27, Walter Bright wrote:
> I'll summarize up front, and so one can skip the detailed reply below. It's obvious there's no meeting of the minds here. I'll just explain where I'm coming from.
> ...

Well, it is sad that you feel this way.

> Features doing simple things should be simple. That's the case with 1027.

It's not the case. Format strings are not simpler, you even got it wrong. In fact, you got it wrong in the DIP1027 specification, in the DIP1027 implementation and now in the format string parser within the `execi` example.

> Doing complicated things should be expected to be more complicated to use. With 1036, it starts out at the other end. It's complicated, with complexity that one has to write additional code to make it simple again (the "filter out the unnecessary templates" thing we talked about).

That's DIP1036e, not DIP1036. DIP1036 does not have templates separating the arguments, it only has a header at the start.

> 
> It's fine that transmorgrifying the string to adapt it to SQL has complexities to it. That's expected. If that's all string interpolation would be used for, that's ok.
> ...

DIP1027 is not simpler.

> But people want to use string interpolation for mixins, etc., all that have nothing to do with SQL. The transmorgrification should not be necessary for those cases, it should just work.

It does not with DIP1027, you still need to call some function, same as with DIP1036e, and that function itself will be more complex and harder to get right. (Though that really hardly matters as the functions needed for simple use cases are in Phobos anyway.)

> But it doesn't, if the filter isn't applied there's a significant runtime and code space cost.
> ...

DIP1036e can be slightly inefficient by default.
DIP1027 is unsafe by default. Companies went out of business due to a SQL injection attack. People's private information got leaked.

I really do not understand why instead of using DIP1036e as a sane starting point and addressing the issues with efficiency, you use DIP1027 as a starting point make it only slightly safer, relying on the library author to get things right in terms of safety, while nudging them into the direction of doing it unsafely. D has a long track record of encouraging safer patterns, and this should not be an exception.

> Simple things should be simple, complicated things should expect complexity. But 1036 has simple things being complicated. Doing SQL is about the same level of complexity for 1027 as 1036. For doing simple things, 1036 remains complicated, and 1027 gets simple.
> ...

That is not true.

> That's the extent of my unease with it.
> ...

Well, then I think you should be able to see it my way, by recognizing that format strings are not actually simpler.

> Does 1027 do *everything* 1036 does? No. There's the Format can be implicitly converted to a string thing.

I am amazed this is still the only drawback that comes to your mind.

> So there's a tradeoff. Do we trade off a minor thing for a fair amount of complexity that yes, the user will see? Every feature in D is a compromise of one sort or another. I fall on one side of that, you the other.
> ...

I think not having istrings at all is significantly better than having DIP1027.

January 15
On Monday, 15 January 2024 at 01:27:00 UTC, Walter Bright wrote:
> Simple things should be simple, complicated things should expect complexity.

Indeed.

> But 1036 has simple things being complicated.

Maybe, but it does is uniformly, consistently and extensibly.

> Doing SQL is about the same level of complexity for 1027 as 1036. For doing simple things, 1036 remains complicated, and 1027 gets simple.

1027 does it deceptively simply, wrong and unsafe.
Level of complexity is completely irrelevant here.
January 15
On 1/15/24 04:41, Timon Gehr wrote:
> 
>> Doing complicated things should be expected to be more complicated to use. With 1036, it starts out at the other end. It's complicated, with complexity that one has to write additional code to make it simple again (the "filter out the unnecessary templates" thing we talked about).
> 
> That's DIP1036e, not DIP1036. DIP1036 does not have templates separating the arguments, it only has a header at the start.

Seems I got that mixed up myself, with YAIDIP.

DIP1036 in fact does have the templates interspersed:
https://github.com/dlang/DIPs/blob/master/DIPs/other/DIP1036.md

But YAIDIP only has a header:
https://github.com/John-Colvin/YAIDIP
January 15

On Monday, 15 January 2024 at 01:27:00 UTC, Walter Bright wrote:

>

I'll summarize up front, and so one can skip the detailed reply below. It's obvious there's no meeting of the minds here. I'll just explain where I'm coming from.

Don't argue anymore, there's no result. Use two switches and experiment first.

January 15

On Monday, 15 January 2024 at 01:27:00 UTC, Walter Bright wrote:

>

I'll summarize up front, and so one can skip the detailed reply below. It's obvious there's no meeting of the minds here. I'll just explain where I'm coming from.

Thank you for your thorough explanation. Such things really help us to understand one another better.

>

Simple things should be simple, complicated things should expect complexity.

Agreed.

Could you take a look at this post please? I think it presented a valuable counter-argument to some of the points you addressed.

January 15
On 15/01/2024 2:27 PM, Walter Bright wrote:
> I'll summarize up front, and so one can skip the detailed reply below. It's obvious there's no meeting of the minds here. I'll just explain where I'm coming from.
> 
> Features doing simple things should be simple. That's the case with 1027. Doing complicated things should be expected to be more complicated to use. With 1036, it starts out at the other end. It's complicated, with complexity that one has to write additional code to make it simple again (the "filter out the unnecessary templates" thing we talked about).
> 
> It's fine that transmorgrifying the string to adapt it to SQL has complexities to it. That's expected. If that's all string interpolation would be used for, that's ok.
> 
> But people want to use string interpolation for mixins, etc., all that have nothing to do with SQL. The transmorgrification should not be necessary for those cases, it should just work. But it doesn't, if the filter isn't applied there's a significant runtime and code space cost.
> 
> Simple things should be simple, complicated things should expect complexity. But 1036 has simple things being complicated. Doing SQL is about the same level of complexity for 1027 as 1036. For doing simple things, 1036 remains complicated, and 1027 gets simple.

As others have said, 1027 is not simple. In fact it is the sort of feature when used will be considered a program security risk that companies will forbid the use of. A little bit of bad syntax sugar is not worth losing money over and it will happen, the question is when not if.

It requires an extensive infrastructure to process, and even then you are unlikely to get it right. As a feature it is a net negative. Almost every person who has had an opinion on such a feature has expressed dislike and out right thinking it is problematic. It cannot go in.

> Another illustration of this idea. For algorithmic code, I invented opApply(). But after experience with it, I slowly grew to dislike it because I could never remember how it worked, and would have to reread the documentation on it. The implementation of it is also ugly, as it has to rewrite the code that surrounds it. Heaven help anyone who has to read the code gen output of that. I much prefer the later approach using lambdas. They're simple, easy to remember and use. There isn't much implementation for them; they grew naturally out of other features in the language.

I on the other hand quite like it.

> On 1/13/2024 9:12 AM, Steven Schveighoffer wrote:
>> This doesn't help, as an enum implicitly converts to its base type.
> 
> It's a point for 1036.
> 
> 
>> In none of the proposals I have written or supported, has it been meant for `writef`. I don't understand the desire to hook `writef` and `format`.
> 
> A format string is needed to support formats like "%03d". I understand that is not needed for SQL, but support for formats is not so straightforward with 1036.

It is also not straight forward for 1027 either.

People are moving away from percentage escape formats. They are variable length and therefore cannot be parsed generically.

Whereas f-string's which is what everyone is moving towards are incredibly easy to parse for their format. I have mine working with both formatting and date/time (which has its format based upon PHP's percentage escape yuck).

Overall I'm happy with it.

>> Me too. But shouldn't we prefer compiler errors? Shouldn't we use the type system for what it is intended?
> 
> We should prefer compiler errors, I agree. But everything has a cost to it.

Yeah turn around time.

Sure compilation might be slower by a few millisecond, but hey lets use more of the developer time which is far more constly and could cost hours instead!


Realistically though, have you reviewed my proposal for call site UDA's?

In a very short space of time, while I was tired I was able to mock up getopt without any additional templates.

Later on whilst tired and not thinking straight I was asked to demonstrate how SQL with string interpolation could work. I wrote the few lines to do that succinctly in a matter of a couple of minutes.

I don't understand the problem here. We have a solution that appears to tick the design requirements you have expressed and you have not so far reviewed it. Please do.
January 14
On 1/13/2024 1:24 AM, Timon Gehr wrote:
> It's not rewritten like that with DIP1027.

I know. Based on our discussions, several improvements need to be made to DIP1027. I'm basing my argument on what DIP1027 would be with those improvements. For example, changing the type of the format string from `string` to `Format`, doing proper escaping of %, handling tuple arguments, handling nested interpolations, etc. The draft implementation of it also lacks things like proper parsing of `(expression)`, which is straightforward to fix and not any fundamental flaw in the design.

I also am not criticizing things in DIP1036e that should be improved, as that is irrelevant to the fundamental issues discussed here. The current implementation/spec seems to be missing things like escaping stray ? that could appear in the string literals.

I.e. we should both be discussing what the two proposals *could* be, rather than what they *are* with their feet of clay.