On Tuesday, 2 June 2015 at 20:56:41 UTC, Steven Schveighoffer wrote:
> On 6/2/15 4:43 PM, Steven Schveighoffer wrote:
>> On 6/2/15 3:35 PM, anonymous wrote:
>>
>>> It's an embarrassing issue that comes up again and again. I really hope
>>> a fix makes it through soon.
>>>
>>> https://issues.dlang.org/show_bug.cgi?id=14005
>>
>> I can reproduce this one. I'll figure this out.
>
> Hm... I think the issue might possibly be solved. What version of the compiler are you using?
>
> -Steve

I'm on dmd v2.067 (according to dmd --version), using Arch Linux kernel version 4.0.4-2. I installed dmd from the Arch community repo just a couple days ago.

On Tuesday, 2 June 2015 at 21:10:20 UTC, Stiff wrote:
> On Tuesday, 2 June 2015 at 20:56:41 UTC, Steven Schveighoffer wrote:
>> On 6/2/15 4:43 PM, Steven Schveighoffer wrote:
>>> On 6/2/15 3:35 PM, anonymous wrote:
>>>
>>>> It's an embarrassing issue that comes up again and again. I really hope
>>>> a fix makes it through soon.
>>>>
>>>> https://issues.dlang.org/show_bug.cgi?id=14005
>>>
>>> I can reproduce this one. I'll figure this out.
>>
>> Hm... I think the issue might possibly be solved. What version of the compiler are you using?
>>
>> -Steve
>
> I'm on dmd v2.067 (according to dmd --version), using Arch Linux kernel version 4.0.4-2. I installed dmd from the Arch community repo just a couple days ago.

Sorry, x86_64.

On Tuesday, 2 June 2015 at 21:15:19 UTC, Stiff wrote:
> On Tuesday, 2 June 2015 at 21:10:20 UTC, Stiff wrote:
>> On Tuesday, 2 June 2015 at 20:56:41 UTC, Steven Schveighoffer wrote:
>>> On 6/2/15 4:43 PM, Steven Schveighoffer wrote:
>>>> On 6/2/15 3:35 PM, anonymous wrote:
>>>>
>>>>> It's an embarrassing issue that comes up again and again. I really hope
>>>>> a fix makes it through soon.
>>>>>
>>>>> https://issues.dlang.org/show_bug.cgi?id=14005
>>>>
>>>> I can reproduce this one. I'll figure this out.
>>>
>>> Hm... I think the issue might possibly be solved. What version of the compiler are you using?
>>>
>>> -Steve
>>
>> I'm on dmd v2.067 (according to dmd --version), using Arch Linux kernel version 4.0.4-2. I installed dmd from the Arch community repo just a couple days ago.
>
> Sorry, x86_64.

...and for what it's worth, it looks like my second line has 2046 characters, including the newline, which is sorta consistent with Iain's test case.

On 6/2/15 5:22 PM, Stiff wrote:
> On Tuesday, 2 June 2015 at 21:15:19 UTC, Stiff wrote:
>> On Tuesday, 2 June 2015 at 21:10:20 UTC, Stiff wrote:
>>> I'm on dmd v2.067 (according to dmd --version), using Arch Linux
>>> kernel version 4.0.4-2. I installed dmd from the Arch community repo
>>> just a couple days ago.
>>
>> Sorry, x86_64.
>
> ....and for what it's worth, it looks like my second line has 2046
> characters, including the newline, which is sorta consistent with Iain's
> test case.

OK. I can reproduce now with all 3 test cases. I tweaked the line sizes a little bit. Noting that it fails on 2.067.0 and 2.067.1 on Linux x64, but passes just fine on OSX same versions.

FWIW, you don't need anything except foreach(line; somefile.byLine()) {} as your main code.

I think the correct fix is to fix readln as outlined in Ranier's pull request: https://github.com/D-Programming-Language/phobos/pull/2794

But that seems to be stalled. I will at least try to fix the assumeSafeAppend calls so they are not crashing.

-Steve

On 06/02/2015 01:56 PM, Steven Schveighoffer wrote:
> On 6/2/15 4:43 PM, Steven Schveighoffer wrote:
>> On 6/2/15 3:35 PM, anonymous wrote:
>>
>>> It's an embarrassing issue that comes up again and again. I really hope
>>> a fix makes it through soon.
>>>
>>> https://issues.dlang.org/show_bug.cgi?id=14005
>>
>> I can reproduce this one. I'll figure this out.
>
> Hm... I think the issue might possibly be solved. What version of the
> compiler are you using?
>
> -Steve
>

I had the impression that it was solved at some point but the problem came back. I had to remove byLine from my book project altogether so that the sample code can be tested: :)


https://bitbucket.org/acehreli/ddili/commits/45c183d078e144d68b96541c858e8fd43b4734e9#Lsrc/codetester.dF257

Ali

On Tuesday, 2 June 2015 at 20:56:41 UTC, Steven Schveighoffer wrote:
> Hm... I think the issue might possibly be solved. What version of the compiler are you using?

I think 14005 (and 14578) are fixed in git head, because there's no assumeSafeAppend in byLine anymore.

But the underlying issue 13856 is still there: readln still stomps happily over the bounds of its buffer.

On Tuesday, 2 June 2015 at 21:35:54 UTC, Steven Schveighoffer wrote:
>
> OK. I can reproduce now with all 3 test cases. I tweaked the line sizes a little bit. Noting that it fails on 2.067.0 and 2.067.1 on Linux x64, but passes just fine on OSX same versions.
>
> FWIW, you don't need anything except foreach(line; somefile.byLine()) {} as your main code.
>
> I think the correct fix is to fix readln as outlined in Ranier's pull request: https://github.com/D-Programming-Language/phobos/pull/2794
>
> But that seems to be stalled. I will at least try to fix the assumeSafeAppend calls so they are not crashing.
>
> -Steve

So...in the meantime, I'll just pad my input I guess?

On Tuesday, 2 June 2015 at 21:43:45 UTC, Stiff wrote:
> So...in the meantime, I'll just pad my input I guess?

It's a mess and I'm not sure what works and what doesn't, but here are some options:

byLineCopy: Could be fine as it doesn't reuse any buffers.

readln without passing a buffer: as above.

readln with passing a buffer: Problematic, readln won't respect the boundaries of the buffer (issue 13856). But it may work ok, if you give its very own GC allocation.

byLine from git head: If you can build phobos from source, the InvalidMemoryOperationError doesn't seem to happen there anymore. readln is still problematic, though.

On 6/2/15 5:42 PM, anonymous wrote:
> On Tuesday, 2 June 2015 at 20:56:41 UTC, Steven Schveighoffer wrote:
>> Hm... I think the issue might possibly be solved. What version of the
>> compiler are you using?
>
> I think 14005 (and 14578) are fixed in git head, because there's no
> assumeSafeAppend in byLine anymore.

Yep, I can confirm that.

Stiff, sorry to say that you are stuck unless you want to use the bleeding edge of D, which means building from source. Wish I had a better answer.

And in reality, even if it needed a fix and I provided it, this was going to be your option.

-Steve

On Tuesday, 2 June 2015 at 22:36:36 UTC, Steven Schveighoffer wrote:
> On 6/2/15 5:42 PM, anonymous wrote:
>> On Tuesday, 2 June 2015 at 20:56:41 UTC, Steven Schveighoffer wrote:
>>> Hm... I think the issue might possibly be solved. What version of the
>>> compiler are you using?
>>
>> I think 14005 (and 14578) are fixed in git head, because there's no
>> assumeSafeAppend in byLine anymore.
>
> Yep, I can confirm that.
>
> Stiff, sorry to say that you are stuck unless you want to use the bleeding edge of D, which means building from source. Wish I had a better answer.
>
> And in reality, even if it needed a fix and I provided it, this was going to be your option.
>
> -Steve

Thanks for the workarounds, Steve and anonymous. I ended up getting around it by just using awk to pad any of my input lines that were around 2048 chars with a few spaces, since I have access to that file. Not particularly convenient, but it works.

Seeing as this is only my second day of using D (thus why I posted this to .learn), I'll try to get a grip on the language itself before I start playing with the bleeding edge. :)