Thread overview
[Issue 18844] std.utf.decode skips valid character on invalid multibyte sequence
May 09, 2018
FeepingCreature
Dec 17, 2022
Iain Buclaw
Apr 26, 2023
Ate Eskola
May 09, 2018
https://issues.dlang.org/show_bug.cgi?id=18844

--- Comment #1 from FeepingCreature <default_357-line@yahoo.de> ---
Repro:

    string s = cast(string) [cast(ubyte) 'ä', 't'];
    size_t i = 0;
    auto ch = decode!(UseReplacementDchar.yes, string)(s, i);
    writefln("ch = %s, i = %s, should be 1", ch, i);

ch = �, i = 2, should be 1.

--
December 17, 2022
https://issues.dlang.org/show_bug.cgi?id=18844

Iain Buclaw <ibuclaw@gdcproject.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Priority|P1                          |P4

--
April 26, 2023
https://issues.dlang.org/show_bug.cgi?id=18844

Ate Eskola <Ajieskola@gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |Ajieskola@gmail.com

--- Comment #2 from Ate Eskola <Ajieskola@gmail.com> ---
Additional observation: the documentation is misleading with both the present and proposed behaviour. It says "If the code point is not well-formed, then a UTFException is thrown and index remains unchanged.".

Well, we don't throw here since we're using replacement characters, so maybe it's a hint that the part about index remaining unchanged does not apply either. On the other hand, the documentation doesn't say what happens to the index instead. At least for me, it gave the wrong impression that index wouldn't be advanced.

--