Jump to page: 1 2
Thread overview
Bug in csv or byLine ?
Jan 08, 2016
Guillaume Chatelet
Jan 08, 2016
Tobi G.
Jan 08, 2016
Guillaume Chatelet
Jan 08, 2016
Tobi G.
Jan 08, 2016
Guillaume Chatelet
Jan 10, 2016
Keywan Ghadami
Jan 10, 2016
Tobi G.
Jan 10, 2016
Tobi G.
Jan 10, 2016
Jesse Phillips
Jan 10, 2016
Tobi G.
Jan 11, 2016
Guillaume Chatelet
January 08, 2016
$ cat debug.csv
timestamp,curr_property
2015-12-01 06:07:55,7035
----------------------------------------------------------------
$ cat process.d
import std.stdio;
import std.csv;
import std.algorithm;
import std.file;

void main(string[] args) {
  version (Fail) {
    File(args[1], "r").byLine.joiner("\n").csvReader.each!writeln;
  } else {
    readText(args[1]).csvReader.each!writeln;
  }
}
----------------------------------------------------------------
$ dmd -run ./process.d debug.csv
["timestamp", "curr_property"]
["2015-12-01 06:07:55", "7035"]
----------------------------------------------------------------
$ dmd -version=Fail -run ./process.d debug.csv
["timestamp", "curr_property"]
["2015-12-01 06:07:55", "7035"]
core.exception.AssertError@std/algorithm/iteration.d(2027): Assertion failure
----------------
??:? _d_assert [0x4633d3]
??:? void std.algorithm.iteration.__assert(int) [0x46d770]
??:? pure @property @safe dchar std.algorithm.iteration.joiner!(std.stdio.File.ByLine!(char, char).ByLine, immutable(char)[]).joiner(std.stdio.File.ByLine!(char, char).ByLine, immutable(char)[]).Result.front() [0x44eaf0]
??:? void std.csv.CsvReader!(immutable(char)[], 1, std.algorithm.iteration.joiner!(std.stdio.File.ByLine!(char, char).ByLine, immutable(char)[]).joiner(std.stdio.File.ByLine!(char, char).ByLine, immutable(char)[]).Result, dchar, immutable(char)[][]).CsvReader.popFront() [0x44f7fc]
??:? void std.algorithm.iteration.__T4eachS183std5stdio7writelnZ.each!(std.csv.CsvReader!(immutable(char)[], 1, std.algorithm.iteration.joiner!(std.stdio.File.ByLine!(char, char).ByLine, immutable(char)[]).joiner(std.stdio.File.ByLine!(char, char).ByLine, immutable(char)[]).Result, dchar, immutable(char)[][]).CsvReader).each(std.csv.CsvReader!(immutable(char)[], 1, std.algorithm.iteration.joiner!(std.stdio.File.ByLine!(char, char).ByLine, immutable(char)[]).joiner(std.stdio.File.ByLine!(char, char).ByLine, immutable(char)[]).Result, dchar, immutable(char)[][]).CsvReader) [0x4608f7]
??:? _Dmain [0x44bc93]


Any idea ?
January 08, 2016
On Friday, 8 January 2016 at 09:59:26 UTC, Guillaume Chatelet wrote:
>
> Any idea ?


No, sorry. Under Windows DMD v2.069.2 it works perfectly in both cases.

Which compiler do you use?

You could run DMD with the -g option. This will print often more useful output, if it fails.


togrue
January 08, 2016
On Friday, 8 January 2016 at 12:07:05 UTC, Tobi G. wrote:
> No, sorry. Under Windows DMD v2.069.2 it works perfectly in both cases.
>
> Which compiler do you use?

- DMD64 D Compiler v2.069.2 on Linux.
- LDC 0.16.1 (DMD v2.067.1, LLVM 3.7.0)

So if it works on windows I guess it's a problem with the File implementation.

> You could run DMD with the -g option. This will print often more useful output, if it fails.

-g didn't bring much.

core.exception.AssertError@std/algorithm/iteration.d(2027): Assertion failure
----------------
??:? _d_assert [0x4a9c33]
??:? void std.algorithm.iteration.__assert(int) [0x4b8048]
/usr/include/dmd/phobos/std/algorithm/iteration.d:2027 pure @property @safe dchar std.algorithm.iteration.joiner!(std.stdio.File.ByLine!(char, char).ByLine, immutable(char)[]).joiner(std.stdio.File.ByLine!(char, char).ByLine, immutable(char)[]).Result.front() [0x495330]
/usr/include/dmd/phobos/std/csv.d:1018 void std.csv.CsvReader!(immutable(char)[], 1, std.algorithm.iteration.joiner!(std.stdio.File.ByLine!(char, char).ByLine, immutable(char)[]).joiner(std.stdio.File.ByLine!(char, char).ByLine, immutable(char)[]).Result, dchar, immutable(char)[][]).CsvReader.popFront() [0x49608c]
/usr/include/dmd/phobos/std/algorithm/iteration.d:881 void std.algorithm.iteration.__T4eachS183std5stdio7writelnZ.each!(std.csv.CsvReader!(immutable(char)[], 1, std.algorithm.iteration.joiner!(std.stdio.File.ByLine!(char, char).ByLine, immutable(char)[]).joiner(std.stdio.File.ByLine!(char, char).ByLine, immutable(char)[]).Result, dchar, immutable(char)[][]).CsvReader).each(std.csv.CsvReader!(immutable(char)[], 1, std.algorithm.iteration.joiner!(std.stdio.File.ByLine!(char, char).ByLine, immutable(char)[]).joiner(std.stdio.File.ByLine!(char, char).ByLine, immutable(char)[]).Result, dchar, immutable(char)[][]).CsvReader) [0x4a5063]
./process.d:8 _Dmain [0x49226c]

January 08, 2016
On Friday, 8 January 2016 at 12:13:59 UTC, Guillaume Chatelet wrote:
> On Friday, 8 January 2016 at 12:07:05 UTC, Tobi G. wrote:
>> No, sorry. Under Windows DMD v2.069.2 it works perfectly in both cases.
>>
>> Which compiler do you use?
>
> - DMD64 D Compiler v2.069.2 on Linux.
> - LDC 0.16.1 (DMD v2.067.1, LLVM 3.7.0)

I ran it now under Linux/Ubuntu DMD64 D Compiler v2.069.2

But both still worked..

Are there some characters in your input data which are invalid and not displayed in the forum?
(multiple empty lines after the actual csv data for example)

togrue
January 08, 2016
On Friday, 8 January 2016 at 13:22:40 UTC, Tobi G. wrote:
> On Friday, 8 January 2016 at 12:13:59 UTC, Guillaume Chatelet wrote:
>> On Friday, 8 January 2016 at 12:07:05 UTC, Tobi G. wrote:
>>> No, sorry. Under Windows DMD v2.069.2 it works perfectly in both cases.
>>>
>>> Which compiler do you use?
>>
>> - DMD64 D Compiler v2.069.2 on Linux.
>> - LDC 0.16.1 (DMD v2.067.1, LLVM 3.7.0)
>
> I ran it now under Linux/Ubuntu DMD64 D Compiler v2.069.2
>
> But both still worked..
>
> Are there some characters in your input data which are invalid and not displayed in the forum?
> (multiple empty lines after the actual csv data for example)
>
> togrue

Indeed there's an empty line at the end of the csv.

Interestingly enough if I try with DMD64 D Compiler v2.069, the Fail version runs fine but the normal version returns:
std.csv.CSVException@/usr/include/dlang/dmd/std/csv.d(1246): Row 3's length 1 does not match previous length of 2.
January 10, 2016
On Friday, 8 January 2016 at 13:53:06 UTC, Guillaume Chatelet wrote:
> On Friday, 8 January 2016 at 13:22:40 UTC, Tobi G. wrote:
>> On Friday, 8 January 2016 at 12:13:59 UTC, Guillaume Chatelet wrote:
>>> On Friday, 8 January 2016 at 12:07:05 UTC, Tobi G. wrote:
>>>> No, sorry. Under Windows DMD v2.069.2 it works perfectly in both cases.
>>>>
>>>> Which compiler do you use?
>>>
>>> - DMD64 D Compiler v2.069.2 on Linux.
>>> - LDC 0.16.1 (DMD v2.067.1, LLVM 3.7.0)
>>
>> I ran it now under Linux/Ubuntu DMD64 D Compiler v2.069.2
>>
>> But both still worked..
>>
>> Are there some characters in your input data which are invalid and not displayed in the forum?
>> (multiple empty lines after the actual csv data for example)
>>
>> togrue
>
> Indeed there's an empty line at the end of the csv.
>
> Interestingly enough if I try with DMD64 D Compiler v2.069, the Fail version runs fine but the normal version returns:
> std.csv.CSVException@/usr/include/dlang/dmd/std/csv.d(1246): Row 3's length 1 does not match previous length of 2.
I still do not understand half of the syntax(still learning) but my guess is that it is a bug in the csv reader:
In https://github.com/D-Programming-Language/phobos/blob/67c95e6de21d5d627e3c57128b4d6e332c82f785/std/csv.d line 1020 calls popfront on the input without checking that it input is empty.
But this only happen in special case if the last line is \r\n if i understand it correctly.
I might be totally wrong because it does not match your stackstrace and i did not yet understand everything about this.
January 10, 2016
On Sunday, 10 January 2016 at 09:41:16 UTC, Keywan Ghadami wrote:
> On Friday, 8 January 2016 at 13:53:06 UTC, Guillaume Chatelet wrote:
> I still do not understand half of the syntax(still learning) but my guess is that it is a bug in the csv reader:
> In https://github.com/D-Programming-Language/phobos/blob/67c95e6de21d5d627e3c57128b4d6e332c82f785/std/csv.d line 1020 calls popfront on the input without checking that it input is empty.
> But this only happen in special case if the last line is \r\n if i understand it correctly.
> I might be totally wrong because it does not match your stackstrace and i did not yet understand everything about this.

Yes. It looks like there is no case to handle '\r' - line endings correctly.
https://github.com/D-Programming-Language/phobos/blob/67c95e6de21d5d627e3c57128b4d6e332c82f785/std/csv.d?L=1020#L1020
If '\r' is the last character it will fail, (because it tryes to read the next character).

enum data =
"name, surname, age
Joe, Joker, 99\r";  // WILL FAIL


import std.csv, std.stdio;

void main(string[] args)
{
    auto reader = csvReader(data);
	
    foreach(entry; reader)
    {
        writeln(entry);
    }
}


togrue

January 10, 2016
The bug has been fixed...
January 10, 2016
On Sunday, 10 January 2016 at 18:09:23 UTC, Tobi G. wrote:
> The bug has been fixed...

Do you have a link for the fix? Is there a BugZilla entry?
January 10, 2016
On Sunday, 10 January 2016 at 19:07:52 UTC, Jesse Phillips wrote:
> On Sunday, 10 January 2016 at 18:09:23 UTC, Tobi G. wrote:
>> The bug has been fixed...
>
> Do you have a link for the fix? Is there a BugZilla entry?

Yes sure..

https://issues.dlang.org/show_bug.cgi?id=15545

and the fix at github

https://github.com/D-Programming-Language/phobos/pull/3917


togrue
« First   ‹ Prev
1 2