Hi everyone, as I had already announced in the discord channel, I was wondering if any of you would like to try and do some tests on my http server (serverino). I don't mean a stress test/ddos, of course. I'm interested in request parsing errors or any bug that can crash the server (5xx error).

Source: https://github.com/trikko/serverino/
Online using nginx as proxy: http://test.andreafontana.it (also https)
Online into the wild listening on port 57123.

Andrea

On Saturday, 13 May 2023 at 09:03:22 UTC, Andrea Fontana wrote:

Not bad. What I found in 10 minutes:

• LF line endings are not accepted
• Host header is mandatory, but not for nginx
• Raw UTF-8 gets mangled in URL and POST parameters, you might be decoding those twice
• multipart/form-data encoding errors are silently discarded
• The server seems to handle application/x-www-form-urlencoded very differently from multipart/form-data? Even though they're both alternative options for HTML <form> parameters, and one is somewhat of a superset of the other

Hope this helps.

On Saturday, 13 May 2023 at 11:21:53 UTC, Vladimir Panteleev wrote:

I've seen your tests! Thank you Vladimir!

Do you mean as line separator in headers? I know some (old?) clients use it but I think HTTP protocol requires CRLF

Interesting, could you please give me an example?

It is (and a warning is shown on server error log). Probably you're right and I should send back a 400 Bad Request. Or something else?

Yes, somewhat. But I can't really build a superset, that's why they are managed in two different ways.

Sure! Thanks!

On Saturday, 13 May 2023 at 11:32:39 UTC, Andrea Fontana wrote:

From RFC:

«Although the line terminator for the start-line and fields is the sequence CRLF, a recipient MAY recognize a single LF as a line terminator and ignore any preceding CR»

Of course MAY means it is optional (rfc2119). I don't think I'm going to implement a special case for this, it is rarely used by old clients in 2023 :)

Good point, anyway.

Andrea

On Saturday, 13 May 2023 at 09:03:22 UTC, Andrea Fontana wrote:

Have you already fuzzed your server code?

https://johanengelen.github.io/ldc/2018/01/14/Fuzzing-with-LDC.html

cheers,
Johan

On Saturday, 13 May 2023 at 11:32:39 UTC, Andrea Fontana wrote:

Ah, OK. I thought the specification allowed either.

I get a 400 with 1.0 too.

printf 'GET /?ппп=ĂÎȘȚ HTTP/1.0\r\nHost: test.andreafontana.it\r\n\r\n' | nc -v test.andreafontana.it 57123

It returns mojibake. However, only for URL and form parameters.

Normally these get percent-encoded by user-agents though.

On Saturday, 13 May 2023 at 22:25:28 UTC, Vladimir Panteleev wrote:

printf 'GET /?ппп=ĂÎȘȚ HTTP/1.0\r\nHost: test.andreafontana.it\r\n\r\n' | nc -v test.andreafontana.it 57123

Hmm I don't think you can use utf-8 encoding in your request. I think everything must be encoded as old US-ASCII.

How can I understand in advance what encoding you're using, otherwise? You could use utf-8 or big5 but I couldn't tell, or am I missing something?

Andrea

On Saturday, 13 May 2023 at 22:25:28 UTC, Vladimir Panteleev wrote:

printf 'GET /?ппп=ĂÎȘȚ HTTP/1.0\r\nHost: test.andreafontana.it\r\n\r\n' | nc -v test.andreafontana.it 57123

Hmm I don't think you can use utf-8 encoding in your request. I think everything must be encoded as old US-ASCII.

How can I understand in advance what encoding you're using, otherwise? You could use utf-8 or big5 but I couldn't tell, or am I missing something?

Andrea

On Sunday, 14 May 2023 at 10:56:29 UTC, Andrea Fontana wrote:

Well, bytes are bytes until you decide to look at them in a certain way. Yea, the input may be invalid as per the spec; however, if mojibake indicates that you're decoding them twice, you're probably doing something that's at least unnecessarily inefficient.

Maybe you're passing the bytes as char arrays to std.algorithm, which produces dchars, which are then being cast into char before decoding again? I think that would produce this sort of mojibake.

On Sunday, 14 May 2023 at 10:56:29 UTC, Andrea Fontana wrote:

Oh also, I noticed that bad UTF-8 in URLs is rejected. Unless you're decoding UTF for the purpose of validating that further logic doesn't have to deal with bad UTF-8, that also indicates a potential inefficiency. Web servers don't need to do any UTF-8 decoding, but it's very easy to do it accidentally in D.