Thread overview
[Issue 14861] Error in stdio.d in LockingTextReader.readFront()
Aug 02, 2015
MGW
Sep 01, 2015
Vladimir Panteleev
Sep 01, 2015
ag0aep6g@gmail.com
Oct 05, 2015
ag0aep6g@gmail.com
Oct 06, 2015
Walter Bright
August 02, 2015
https://issues.dlang.org/show_bug.cgi?id=14861

--- Comment #1 from MGW <mgw@yandex.ru> ---
Size ring buffer = 16384 bytes in struct _iobuf*. If Utf-8 sequence is broken off on buffer boundary, it is impossible to return earlier read characters, as in the buffer absolutely other data.

--
September 01, 2015
https://issues.dlang.org/show_bug.cgi?id=14861

Vladimir Panteleev <thecybershadow@gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |thecybershadow@gmail.com
           Hardware|x86                         |All
           Severity|critical                    |regression

--- Comment #2 from Vladimir Panteleev <thecybershadow@gmail.com> ---
This seems to be Windows-only - I can't reproduce it on Linux.

This seems to be a regression, introduced in: https://github.com/D-Programming-Language/phobos/pull/2663

--
September 01, 2015
https://issues.dlang.org/show_bug.cgi?id=14861

ag0aep6g@gmail.com changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|                            |pull
                 CC|                            |ag0aep6g@gmail.com
           Assignee|nobody@puremagic.com        |ag0aep6g@gmail.com

--- Comment #3 from ag0aep6g@gmail.com ---
Reduced test case:

----
import std.stdio;
import std.array: replicate;

void main()
{
    File fw = File("panic.csv", "w");
    fw.rawWrite("a".replicate(16383) ~ "\xD1\x91\xD1\x82");
        /* \xD1\x91 = U+0451 CYRILLIC SMALL LETTER IO */
        /* \xD1\x82 = U+0442 CYRILLIC SMALL LETTER TE */
    fw.close();

    File fr = File("panic.csv", "r");
    fr.rawRead(new char[16383]);

    auto ltr = LockingTextReader(fr);
    assert(ltr.front == '\u0451'); /* passes */
    ltr.popFront(); /* "Invalid UTF-8 sequence" */
    assert(ltr.front == '\u0442');
    ltr.popFront();
    assert(ltr.empty);
}
----

LockingTextReader essentially does this:

----
    auto fps = fr.getFP();
    auto fp = cast(_iobuf*) fps;

    assert(FGETC(fp) == '\xD1'); /* passes */
    assert(FGETC(fp) == '\x91'); /* passes */

    assert(ungetc('\x91', fps) == '\x91'); /* passes */
    assert(ungetc('\xD1', fps) == '\xD1'); /* passes */

    assert(FGETC(fp) == '\xD1'); /* passes */
    assert(FGETC(fp) == '\x91'); /* fails */
----

The problem is that ungetc is called multiple times. Apparently, the Windows 32 C runtime doesn't like that under these specific circumstances.

Checking the documentation for ungetc, calling it more than once is actually not guaranteed to work.

Here's a PR that replaces the ungetc calls with ftell/fseek: https://github.com/D-Programming-Language/phobos/pull/3622

--
October 05, 2015
https://issues.dlang.org/show_bug.cgi?id=14861

--- Comment #4 from ag0aep6g@gmail.com ---
(In reply to ag0aep6g from comment #3)
> Here's a PR that replaces the ungetc calls with ftell/fseek: https://github.com/D-Programming-Language/phobos/pull/3622

That one's been closed. New PR to make LockingTextReader a char range: https://github.com/D-Programming-Language/phobos/pull/3696

--
October 06, 2015
https://issues.dlang.org/show_bug.cgi?id=14861

Walter Bright <bugzilla@digitalmars.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |bugzilla@digitalmars.com

--- Comment #5 from Walter Bright <bugzilla@digitalmars.com> ---
Looks like competing pull requests!

https://github.com/D-Programming-Language/phobos/pull/3698

--
November 13, 2015
https://issues.dlang.org/show_bug.cgi?id=14861

--- Comment #6 from github-bugzilla@puremagic.com ---
Commits pushed to master at https://github.com/D-Programming-Language/phobos

https://github.com/D-Programming-Language/phobos/commit/196f251ff443a91b82dcead5af6ad2ad54f518fd fix Issue 14861 - Error in stdio.d in LockingTextReader.readFront()

https://github.com/D-Programming-Language/phobos/commit/99741241a1f57cffe61dad74c1f97a0822be872e Merge pull request #3698 from WalterBright/fix14861

[REG] fix Issue 14861 - Error in stdio.d in LockingTextReader.readFront()

--
January 03, 2016
https://issues.dlang.org/show_bug.cgi?id=14861

--- Comment #7 from github-bugzilla@puremagic.com ---
Commits pushed to stable at https://github.com/D-Programming-Language/phobos

https://github.com/D-Programming-Language/phobos/commit/196f251ff443a91b82dcead5af6ad2ad54f518fd fix Issue 14861 - Error in stdio.d in LockingTextReader.readFront()

https://github.com/D-Programming-Language/phobos/commit/99741241a1f57cffe61dad74c1f97a0822be872e Merge pull request #3698 from WalterBright/fix14861

--