On Friday, 30 August 2024 at 11:18:10 UTC, kdevel wrote:
> Ideally compilation of such unadorned writes would fail.

for what its worth, after starting to use this more, I'm actually inclined to agree with you.

The idea behind 1036e is that it allows the language to provide the pieces that library authors can use to make things that do the right thing. Two existing library functions work with it out of the box in a reasonably good way: std.conv.text and std.stdio.writeln.

I've seen people in the wild using std.conv.text to do things like make sql queries - exactly what you're not supposed to do. We want some way to do a toString, but it should be a bit of a pain: doing it the right thing should be the easy way. Some of this is just because library support is still a bit immature - I still haven't even put it in place in all the places arsd wants it - but part of it means committing to the change too.

So like we currently have `db.query(string, args...)` and... maybe that should be deprecated, so people stop trying to do `db.query(text(istring))`. Maybe text(istring) itself should be deprecated. It is some breakage though that requires users update their compiler so that makes it more complicated. Also, you sometimes DO want a runtime string, so it probably more of a rename than a remove.

I didn't think about using writeln directly in a cgi context. There's MUCH better ways to make cgi programs! And much better ways to construct html too. And those libraries tend to make you be more careful with these cases. But indeed, writeln bypasses all that. I do think this is a harder case though... outright banning writeln in these cases isn't super realistic so the question is where you balance people's capability of doing it wrong with the flexibility to build anything on top at all.

A lot of it comes back to making the better way the easy way: use my cgi library, and `cgi.write(i"istring")` is a compile error. But it still doesn't prevent you from bypassing the library and doing it wrong. Still, my feeling is if you make the right way just as easy as the wrong way, at least it helps point people the right way.

On Fri, Aug 30, 2024 at 02:36:56PM +0000, Steven Schveighoffer via Digitalmars-d wrote:
> On Friday, 30 August 2024 at 12:07:47 UTC, kdevel wrote:
[...]
> > With post-1036e D the user has now three equally potent ways to shoot theirself in the foot:
> > 
> > 1.
> > ```
> >      data = "alert (-1)";
> >      writeln ("<script>" ~ data ~ "</script>");
> > ```
> > 
> > 2.
> > ```
> >      data = "alert (-1)";
> >      writeln (format!"<script>%s</script>" (data));
> > ```
> > 
> > 3.
> > ```
> >      data = "alert (-1)";
> >      writeln (i"<script>$(data)</script>");
> > ```
> 
> 4.
> ```d
> writefln("<script>%s</script>", data);
> ```
[...]

Actually, the footgun here is just one: writeln + CGI.  The rest are merely specific instances of the same thing: writing unvetted data to your output stream.  No amount of cleverness is going to prevent this; you, as the programmer, are responsible for making sure the program logic properly vets all data before sending them to the output stream. If some coder wannabe can't figure out how to properly segregate their input data from their output data, no amount of programming language features will be able to help them.  The program logic has to be structured in such a way that *all* input data is properly escaped, or *all* output data is properly encoded.  The latter is much harder; recoding input data is recommended.  If the fundamental program logic is flawed, no magic feature will be able to fix the resulting problems.


T

-- 
Study gravitation, it's a field with a lot of potential.

On Friday, 30 August 2024 at 14:36:56 UTC, Steven Schveighoffer wrote:
> Yes, I purposely said log, because that's the intention of `writeln`.

Gonna disagree with this: writeln's intention is to write a line to the stdout stream. It has no more semantic knowledge; it is fundamentally a low level function with a wide variety of high level uses.

You can use it for logging to the console, but it might also go to a file, but it might also go to a user for interactive use ("Enter your name: "), and it might also be structured output intended to be consumed by another program: ./yourapp | whatever.

CGI, on the low level, works as one of those unix pipelines. You ought to be more careful with that, but writeln has no idea how it is actually going to be used.

> For instance, what if you call a function that logs something to the console? That gets included in your html.

They're supposed to use the stderr stream for that kind of thing, which is traditionally redirected to the server's log file.

But yeah, writeln is a low level function and you have to treat it as such - limiting what it is allowed to do can help one use case but hurt another, so that's hard to say... but at the same time, users really shouldn't be encouraged to use it. There should be high level intent libs that do those jobs all better and easier.

On Friday, 30 August 2024 at 15:09:39 UTC, H. S. Teoh wrote:
> The program logic has to be structured in such a way that *all* input data is properly escaped,

What does that mean? Can, after such input escaping, Bobby Tables first name still be stored in the database?

> or *all* output data is properly encoded.  The latter is much harder; recoding input data is recommended.

By whom? And "recode" (re-encode) into which code?

The web is full of content recommending the opposite:

- https://benhoyt.com/writings/dont-sanitize-do-escape/

"Every so often developers talk about “sanitizing user input” to prevent cross-site scripting attacks. This is well-intentioned, but leads to a false sense of security, and sometimes mangles perfectly good input."

- https://cheatsheetseries.owasp.org/cheatsheets/Cross_Site_Scripting_Prevention_Cheat_Sheet.html



- https://security.stackexchange.com/questions/32394/when-to-escape-user-input (2013)

"In general, you want to keep strings as strings, and delegate any encoding or escaping to specialized functions which do that well. For instance, for SQL, you use prepared statements. With HTML from a PHP context, you would use htmlspecialchars()."

- https://lukeplant.me.uk/blog/posts/why-escape-on-input-is-a-bad-idea/

"The right way to handle issues with untrusted data is:

    Filter on input, escape on output

This means that you validate or limit data that comes in (filter), but only transform (escape or encode) it at the point you are sending it as output to another system that requires the encoding. It has been standard best practice since just about forever [citation required].

[...]

First of all, escape-on-input is just wrong – you've taken some input and applied some transformation that is totally irrelevant to that data. If, taking our example, you have some data collected by HTTP POST or GET parameters, applying HTML escaping to it is a layering violation – it mixes an output formatting concern into input handling. Layering violations make your code much harder to understand and maintain, because you have to take into account other layers instead of letting each component and layer do its own job.

Doing things ‘right’ is very important, even if doing them ‘wrong’ seems to work and you are tempted to be dismissive of ‘theoretical’ concerns about purity etc. When you have to maintain code, you will be very glad if things are in the right place, and not full of hacks and surprises."