June 02, 2015 Re: InvalidMemoryOperationError from File.byLine()? | ||||
---|---|---|---|---|
| ||||
Posted in reply to Steven Schveighoffer | On Tuesday, 2 June 2015 at 20:56:41 UTC, Steven Schveighoffer wrote:
> On 6/2/15 4:43 PM, Steven Schveighoffer wrote:
>> On 6/2/15 3:35 PM, anonymous wrote:
>>
>>> It's an embarrassing issue that comes up again and again. I really hope
>>> a fix makes it through soon.
>>>
>>> https://issues.dlang.org/show_bug.cgi?id=14005
>>
>> I can reproduce this one. I'll figure this out.
>
> Hm... I think the issue might possibly be solved. What version of the compiler are you using?
>
> -Steve
I'm on dmd v2.067 (according to dmd --version), using Arch Linux kernel version 4.0.4-2. I installed dmd from the Arch community repo just a couple days ago.
|
June 02, 2015 Re: InvalidMemoryOperationError from File.byLine()? | ||||
---|---|---|---|---|
| ||||
Posted in reply to Stiff | On Tuesday, 2 June 2015 at 21:10:20 UTC, Stiff wrote:
> On Tuesday, 2 June 2015 at 20:56:41 UTC, Steven Schveighoffer wrote:
>> On 6/2/15 4:43 PM, Steven Schveighoffer wrote:
>>> On 6/2/15 3:35 PM, anonymous wrote:
>>>
>>>> It's an embarrassing issue that comes up again and again. I really hope
>>>> a fix makes it through soon.
>>>>
>>>> https://issues.dlang.org/show_bug.cgi?id=14005
>>>
>>> I can reproduce this one. I'll figure this out.
>>
>> Hm... I think the issue might possibly be solved. What version of the compiler are you using?
>>
>> -Steve
>
> I'm on dmd v2.067 (according to dmd --version), using Arch Linux kernel version 4.0.4-2. I installed dmd from the Arch community repo just a couple days ago.
Sorry, x86_64.
|
June 02, 2015 Re: InvalidMemoryOperationError from File.byLine()? | ||||
---|---|---|---|---|
| ||||
Posted in reply to Stiff | On Tuesday, 2 June 2015 at 21:15:19 UTC, Stiff wrote:
> On Tuesday, 2 June 2015 at 21:10:20 UTC, Stiff wrote:
>> On Tuesday, 2 June 2015 at 20:56:41 UTC, Steven Schveighoffer wrote:
>>> On 6/2/15 4:43 PM, Steven Schveighoffer wrote:
>>>> On 6/2/15 3:35 PM, anonymous wrote:
>>>>
>>>>> It's an embarrassing issue that comes up again and again. I really hope
>>>>> a fix makes it through soon.
>>>>>
>>>>> https://issues.dlang.org/show_bug.cgi?id=14005
>>>>
>>>> I can reproduce this one. I'll figure this out.
>>>
>>> Hm... I think the issue might possibly be solved. What version of the compiler are you using?
>>>
>>> -Steve
>>
>> I'm on dmd v2.067 (according to dmd --version), using Arch Linux kernel version 4.0.4-2. I installed dmd from the Arch community repo just a couple days ago.
>
> Sorry, x86_64.
...and for what it's worth, it looks like my second line has 2046 characters, including the newline, which is sorta consistent with Iain's test case.
|
June 02, 2015 Re: InvalidMemoryOperationError from File.byLine()? | ||||
---|---|---|---|---|
| ||||
Posted in reply to Stiff | On 6/2/15 5:22 PM, Stiff wrote: > On Tuesday, 2 June 2015 at 21:15:19 UTC, Stiff wrote: >> On Tuesday, 2 June 2015 at 21:10:20 UTC, Stiff wrote: >>> I'm on dmd v2.067 (according to dmd --version), using Arch Linux >>> kernel version 4.0.4-2. I installed dmd from the Arch community repo >>> just a couple days ago. >> >> Sorry, x86_64. > > ....and for what it's worth, it looks like my second line has 2046 > characters, including the newline, which is sorta consistent with Iain's > test case. OK. I can reproduce now with all 3 test cases. I tweaked the line sizes a little bit. Noting that it fails on 2.067.0 and 2.067.1 on Linux x64, but passes just fine on OSX same versions. FWIW, you don't need anything except foreach(line; somefile.byLine()) {} as your main code. I think the correct fix is to fix readln as outlined in Ranier's pull request: https://github.com/D-Programming-Language/phobos/pull/2794 But that seems to be stalled. I will at least try to fix the assumeSafeAppend calls so they are not crashing. -Steve |
June 02, 2015 Re: InvalidMemoryOperationError from File.byLine()? | ||||
---|---|---|---|---|
| ||||
Posted in reply to Steven Schveighoffer | On 06/02/2015 01:56 PM, Steven Schveighoffer wrote: > On 6/2/15 4:43 PM, Steven Schveighoffer wrote: >> On 6/2/15 3:35 PM, anonymous wrote: >> >>> It's an embarrassing issue that comes up again and again. I really hope >>> a fix makes it through soon. >>> >>> https://issues.dlang.org/show_bug.cgi?id=14005 >> >> I can reproduce this one. I'll figure this out. > > Hm... I think the issue might possibly be solved. What version of the > compiler are you using? > > -Steve > I had the impression that it was solved at some point but the problem came back. I had to remove byLine from my book project altogether so that the sample code can be tested: :) https://bitbucket.org/acehreli/ddili/commits/45c183d078e144d68b96541c858e8fd43b4734e9#Lsrc/codetester.dF257 Ali |
June 02, 2015 Re: InvalidMemoryOperationError from File.byLine()? | ||||
---|---|---|---|---|
| ||||
Posted in reply to Steven Schveighoffer | On Tuesday, 2 June 2015 at 20:56:41 UTC, Steven Schveighoffer wrote:
> Hm... I think the issue might possibly be solved. What version of the compiler are you using?
I think 14005 (and 14578) are fixed in git head, because there's no assumeSafeAppend in byLine anymore.
But the underlying issue 13856 is still there: readln still stomps happily over the bounds of its buffer.
|
June 02, 2015 Re: InvalidMemoryOperationError from File.byLine()? | ||||
---|---|---|---|---|
| ||||
Posted in reply to Steven Schveighoffer | On Tuesday, 2 June 2015 at 21:35:54 UTC, Steven Schveighoffer wrote:
>
> OK. I can reproduce now with all 3 test cases. I tweaked the line sizes a little bit. Noting that it fails on 2.067.0 and 2.067.1 on Linux x64, but passes just fine on OSX same versions.
>
> FWIW, you don't need anything except foreach(line; somefile.byLine()) {} as your main code.
>
> I think the correct fix is to fix readln as outlined in Ranier's pull request: https://github.com/D-Programming-Language/phobos/pull/2794
>
> But that seems to be stalled. I will at least try to fix the assumeSafeAppend calls so they are not crashing.
>
> -Steve
So...in the meantime, I'll just pad my input I guess?
|
June 02, 2015 Re: InvalidMemoryOperationError from File.byLine()? | ||||
---|---|---|---|---|
| ||||
Posted in reply to Stiff | On Tuesday, 2 June 2015 at 21:43:45 UTC, Stiff wrote:
> So...in the meantime, I'll just pad my input I guess?
It's a mess and I'm not sure what works and what doesn't, but here are some options:
byLineCopy: Could be fine as it doesn't reuse any buffers.
readln without passing a buffer: as above.
readln with passing a buffer: Problematic, readln won't respect the boundaries of the buffer (issue 13856). But it may work ok, if you give its very own GC allocation.
byLine from git head: If you can build phobos from source, the InvalidMemoryOperationError doesn't seem to happen there anymore. readln is still problematic, though.
|
June 02, 2015 Re: InvalidMemoryOperationError from File.byLine()? | ||||
---|---|---|---|---|
| ||||
Posted in reply to anonymous | On 6/2/15 5:42 PM, anonymous wrote:
> On Tuesday, 2 June 2015 at 20:56:41 UTC, Steven Schveighoffer wrote:
>> Hm... I think the issue might possibly be solved. What version of the
>> compiler are you using?
>
> I think 14005 (and 14578) are fixed in git head, because there's no
> assumeSafeAppend in byLine anymore.
Yep, I can confirm that.
Stiff, sorry to say that you are stuck unless you want to use the bleeding edge of D, which means building from source. Wish I had a better answer.
And in reality, even if it needed a fix and I provided it, this was going to be your option.
-Steve
|
June 02, 2015 Re: InvalidMemoryOperationError from File.byLine()? | ||||
---|---|---|---|---|
| ||||
Posted in reply to Steven Schveighoffer | On Tuesday, 2 June 2015 at 22:36:36 UTC, Steven Schveighoffer wrote:
> On 6/2/15 5:42 PM, anonymous wrote:
>> On Tuesday, 2 June 2015 at 20:56:41 UTC, Steven Schveighoffer wrote:
>>> Hm... I think the issue might possibly be solved. What version of the
>>> compiler are you using?
>>
>> I think 14005 (and 14578) are fixed in git head, because there's no
>> assumeSafeAppend in byLine anymore.
>
> Yep, I can confirm that.
>
> Stiff, sorry to say that you are stuck unless you want to use the bleeding edge of D, which means building from source. Wish I had a better answer.
>
> And in reality, even if it needed a fix and I provided it, this was going to be your option.
>
> -Steve
Thanks for the workarounds, Steve and anonymous. I ended up getting around it by just using awk to pad any of my input lines that were around 2048 chars with a few spaces, since I have access to that file. Not particularly convenient, but it works.
Seeing as this is only my second day of using D (thus why I posted this to .learn), I'll try to get a grip on the language itself before I start playing with the bleeding edge. :)
|
Copyright © 1999-2021 by the D Language Foundation