Thread overview | ||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
March 08, 2015 [Issue 14256] Poor IO performance on 64-bit dmd 2.066 (OS X) | ||||
---|---|---|---|---|
| ||||
https://issues.dlang.org/show_bug.cgi?id=14256 --- Comment #1 from Paul M <pmmagic@gmail.com> --- For testing purposes, here is a link to a tarball of file that I was using for testing purposes: ftp://flybase.net/genomes/Drosophila_melanogaster/dmel_r6.03_FB2014_06/gff/dmel-all-no-analysis-r6.03.gff.gz When decompressed this gives a 471Mb file with 3615492 lines. -- |
March 08, 2015 [Issue 14256] Poor IO performance on 64-bit dmd 2.066 (OS X) | ||||
---|---|---|---|---|
| ||||
https://issues.dlang.org/show_bug.cgi?id=14256 Ivan Kazmenko <gassa@mail.ru> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |gassa@mail.ru --- Comment #2 from Ivan Kazmenko <gassa@mail.ru> --- I tested the same on Win64 with dmd -m32 and dmd -m64 (options "-O -release -inline -noboundscheck"). I have three test files: 1000000 lines of length 100 each (gen1.d), 10000 lines of length 10000 each (gen2.d), 100 lines of length 1000000 each (gen3.d). Here are my test results: Entry test1 test2 test3 Number of lines 1000000 10000 100 Length of each line 100 10000 1000000 ------------------------------------------------------------------------ Python 2.7.5 x32: 0.60 0.41 0.36 Python 2.7.7 x64: 0.59 0.42 0.36 DMD 2.067.0-b3 byLine -m32: 0.53 1.27 1.77 DMD 2.067.0-b3 byLine -m64: 2.32 1.98 2.06 DMD 2.067.0-b3 readln -m32: 0.52 1.29 1.89 DMD 2.067.0-b3 readln -m64: 2.33 2.02 2.05 DMD 2.067.0-b3 read+splitLines -m32: 0.40 0.39 0.37 DMD 2.067.0-b3 read+splitLines -m64: 0.37 0.28 0.28 Here, I see at least two separate concerns: 1. The functions byLine and readln suffer from long lines with -m32. 2. The functions byLine and readln are a few times slower than possible with -m64. The behavior of read + splitLines is satisfactory (outperforms Python in both 32-bit and 64-bit mode) in my setup. -- |
March 08, 2015 [Issue 14256] Poor IO performance on 64-bit dmd 2.066 (OS X) | ||||
---|---|---|---|---|
| ||||
https://issues.dlang.org/show_bug.cgi?id=14256 --- Comment #3 from Ivan Kazmenko <gassa@mail.ru> --- Created attachment 1482 --> https://issues.dlang.org/attachment.cgi?id=1482&action=edit test1 generator (1000000 lines of length 100) -- |
March 08, 2015 [Issue 14256] Poor IO performance on 64-bit dmd 2.066 (OS X) | ||||
---|---|---|---|---|
| ||||
https://issues.dlang.org/show_bug.cgi?id=14256 --- Comment #4 from Ivan Kazmenko <gassa@mail.ru> --- Created attachment 1483 --> https://issues.dlang.org/attachment.cgi?id=1483&action=edit test2 generator (10000 lines of length 10000) -- |
March 08, 2015 [Issue 14256] Poor IO performance on 64-bit dmd 2.066 (OS X) | ||||
---|---|---|---|---|
| ||||
https://issues.dlang.org/show_bug.cgi?id=14256 --- Comment #5 from Ivan Kazmenko <gassa@mail.ru> --- Created attachment 1484 --> https://issues.dlang.org/attachment.cgi?id=1484&action=edit test3 generator (100 lines of length 1000000) -- |
March 08, 2015 [Issue 14256] Poor IO performance on 64-bit dmd 2.066 (OS X) | ||||
---|---|---|---|---|
| ||||
https://issues.dlang.org/show_bug.cgi?id=14256 --- Comment #6 from Ivan Kazmenko <gassa@mail.ru> --- Created attachment 1485 --> https://issues.dlang.org/attachment.cgi?id=1485&action=edit Python2 program -- |
March 08, 2015 [Issue 14256] Poor IO performance on 64-bit dmd 2.066 (OS X) | ||||
---|---|---|---|---|
| ||||
https://issues.dlang.org/show_bug.cgi?id=14256 --- Comment #7 from Ivan Kazmenko <gassa@mail.ru> --- Created attachment 1486 --> https://issues.dlang.org/attachment.cgi?id=1486&action=edit Original D program by Paul M -- |
March 08, 2015 [Issue 14256] Poor IO performance on 64-bit dmd 2.066 (OS X) | ||||
---|---|---|---|---|
| ||||
https://issues.dlang.org/show_bug.cgi?id=14256 --- Comment #8 from Ivan Kazmenko <gassa@mail.ru> --- Created attachment 1487 --> https://issues.dlang.org/attachment.cgi?id=1487&action=edit readln version suggested by Daniel Kozak (http://stackoverflow.com/a/28926137/1488799) -- |
March 08, 2015 [Issue 14256] Poor IO performance on 64-bit dmd 2.066 (OS X) | ||||
---|---|---|---|---|
| ||||
https://issues.dlang.org/show_bug.cgi?id=14256 Ivan Kazmenko <gassa@mail.ru> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #1487|test_readln.d |test_read_splitLines.d filename| | Attachment #1487|readln version suggested by |read + splitLines version description|Daniel Kozak |by Paul M |(http://stackoverflow.com/a | |/28926137/1488799) | -- |
March 08, 2015 [Issue 14256] Poor IO performance on 64-bit dmd 2.066 (OS X) | ||||
---|---|---|---|---|
| ||||
https://issues.dlang.org/show_bug.cgi?id=14256 --- Comment #9 from Ivan Kazmenko <gassa@mail.ru> --- Created attachment 1488 --> https://issues.dlang.org/attachment.cgi?id=1488&action=edit readln version suggested by Daniel Kozak (http://stackoverflow.com/a/28926137/1488799) -- |
Copyright © 1999-2021 by the D Language Foundation