[SAOC 2024] - Leverage dmd as a library in D-Scanner in order to extend its functionality - Weekly Update 7

Nov 03

Vladiwostok

Nov 04

Richard (Rikki) Andrew Cattermole

Nov 10

Vladiwostok

Nov 11

Richard (Rikki) Andrew Cattermole

November 03

[SAOC 2024] - Leverage dmd as a library in D-Scanner in order to extend its functionality - Weekly Update 7

Posted by Vladiwostok

Permalink

Vladiwostok

Permalink

Greetings,

I've spent this week doing debugging on my Windows machine.

Delimited strings do add a single \r\n sequence for line endings on Windows machines. However, when code is written in a file (used for testing in D-Scanner) with std.file.write, an additional '\r' char is prepended to the sequence, resulting in \r\r\n.

There's no bugs in the compiler; when the lexer encounters a '\r' char, it peeks to the next one to see if it's a standalone carriage return, or if its a CRLF sequence. Both cases lead to incrementing the line counts, causing the tokens to have different locations compared to what would be expected in the autofix test code.

I've made a fix in D-Scanner that sanitizes the code read from a file, before passing it to the lexer: https://github.com/Dlang-UPB/D-scanner/pull/151/commits/6c256a9f7835dd510868d4556b4326263528987c

On Monday, 4 November 2024 at 08:53:21 UTC, Richard (Rikki) Andrew Cattermole wrote: > Something about this isn't sounding right. > > https://github.com/dlang/phobos/blob/master/std/file.d#L861 > > There is no handling of new lines, in fact it doesn't even know that it is textual. `rawWrite` fixed the issue, no additional carriage returns are added in the test file: https://github.com/Dlang-UPB/D-scanner/commit/8db5a658134994cd1f388ca4b5613a5bb24870a2 Any idea why?

On 11/11/2024 12:04 PM, Vladiwostok wrote: > On Monday, 4 November 2024 at 08:53:21 UTC, Richard (Rikki) Andrew Cattermole wrote: >> Something about this isn't sounding right. >> >> https://github.com/dlang/phobos/blob/master/std/file.d#L861 >> >> There is no handling of new lines, in fact it doesn't even know that it is textual. > > `rawWrite` fixed the issue, no additional carriage returns are added in the test file: > https://github.com/Dlang-UPB/D-scanner/ commit/8db5a658134994cd1f388ca4b5613a5bb24870a2 > > Any idea why? Not a behavior of WinAPI, its the libc doing it: https://github.com/dlang/phobos/blob/master/std/stdio.d#L2976C57-L2976C64 https://learn.microsoft.com/en-us/cpp/c-runtime-library/reference/setmode?view=msvc-170#remarks Took me a bit to find.

Forums