Jump to page: 1 2
Thread overview
[Issue 14256] Poor IO performance on 64-bit dmd 2.066 (OS X)
Mar 08, 2015
Paul M
Mar 08, 2015
Ivan Kazmenko
Mar 08, 2015
Ivan Kazmenko
Mar 08, 2015
Ivan Kazmenko
Mar 08, 2015
Ivan Kazmenko
Mar 08, 2015
Ivan Kazmenko
Mar 08, 2015
Ivan Kazmenko
Mar 08, 2015
Ivan Kazmenko
Mar 08, 2015
Ivan Kazmenko
Mar 08, 2015
Ivan Kazmenko
Jun 25, 2017
Vladimir Panteleev
Jun 25, 2017
Ivan Kazmenko
Jun 25, 2017
Vladimir Panteleev
Jun 25, 2017
Vladimir Panteleev
Jun 28, 2017
Jon Degenhardt
Jun 28, 2017
Vladimir Panteleev
Dec 17, 2022
Iain Buclaw
March 08, 2015
https://issues.dlang.org/show_bug.cgi?id=14256

--- Comment #1 from Paul M <pmmagic@gmail.com> ---
For testing purposes, here is a link to a tarball of  file that I was using for testing purposes:

ftp://flybase.net/genomes/Drosophila_melanogaster/dmel_r6.03_FB2014_06/gff/dmel-all-no-analysis-r6.03.gff.gz

When decompressed this gives a 471Mb file with 3615492 lines.

--
March 08, 2015
https://issues.dlang.org/show_bug.cgi?id=14256

Ivan Kazmenko <gassa@mail.ru> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |gassa@mail.ru

--- Comment #2 from Ivan Kazmenko <gassa@mail.ru> ---
I tested the same on Win64 with dmd -m32 and dmd -m64 (options "-O -release
-inline -noboundscheck").

I have three test files:
1000000 lines of length 100 each (gen1.d),
10000 lines of length 10000 each (gen2.d),
100 lines of length 1000000 each (gen3.d).

Here are my test results:

Entry                                       test1     test2     test3 Number of lines                           1000000     10000       100 Length of each line                           100     10000   1000000
------------------------------------------------------------------------
Python 2.7.5 x32:                            0.60      0.41      0.36 Python 2.7.7 x64:                            0.59      0.42      0.36 DMD 2.067.0-b3 byLine -m32:                  0.53      1.27      1.77 DMD 2.067.0-b3 byLine -m64:                  2.32      1.98      2.06 DMD 2.067.0-b3 readln -m32:                  0.52      1.29      1.89 DMD 2.067.0-b3 readln -m64:                  2.33      2.02      2.05 DMD 2.067.0-b3 read+splitLines -m32:         0.40      0.39      0.37 DMD 2.067.0-b3 read+splitLines -m64:         0.37      0.28      0.28

Here, I see at least two separate concerns:

1. The functions byLine and readln suffer from long lines with -m32.
2. The functions byLine and readln are a few times slower than possible with
-m64.

The behavior of read + splitLines is satisfactory
(outperforms Python in both 32-bit and 64-bit mode) in my setup.

--
March 08, 2015
https://issues.dlang.org/show_bug.cgi?id=14256

--- Comment #3 from Ivan Kazmenko <gassa@mail.ru> ---
Created attachment 1482
  --> https://issues.dlang.org/attachment.cgi?id=1482&action=edit
test1 generator (1000000 lines of length 100)

--
March 08, 2015
https://issues.dlang.org/show_bug.cgi?id=14256

--- Comment #4 from Ivan Kazmenko <gassa@mail.ru> ---
Created attachment 1483
  --> https://issues.dlang.org/attachment.cgi?id=1483&action=edit
test2 generator (10000 lines of length 10000)

--
March 08, 2015
https://issues.dlang.org/show_bug.cgi?id=14256

--- Comment #5 from Ivan Kazmenko <gassa@mail.ru> ---
Created attachment 1484
  --> https://issues.dlang.org/attachment.cgi?id=1484&action=edit
test3 generator (100 lines of length 1000000)

--
March 08, 2015
https://issues.dlang.org/show_bug.cgi?id=14256

--- Comment #6 from Ivan Kazmenko <gassa@mail.ru> ---
Created attachment 1485
  --> https://issues.dlang.org/attachment.cgi?id=1485&action=edit
Python2 program

--
March 08, 2015
https://issues.dlang.org/show_bug.cgi?id=14256

--- Comment #7 from Ivan Kazmenko <gassa@mail.ru> ---
Created attachment 1486
  --> https://issues.dlang.org/attachment.cgi?id=1486&action=edit
Original D program by Paul M

--
March 08, 2015
https://issues.dlang.org/show_bug.cgi?id=14256

--- Comment #8 from Ivan Kazmenko <gassa@mail.ru> ---
Created attachment 1487
  --> https://issues.dlang.org/attachment.cgi?id=1487&action=edit
readln version suggested by Daniel Kozak
(http://stackoverflow.com/a/28926137/1488799)

--
March 08, 2015
https://issues.dlang.org/show_bug.cgi?id=14256

Ivan Kazmenko <gassa@mail.ru> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Attachment #1487|test_readln.d               |test_read_splitLines.d
           filename|                            |
   Attachment #1487|readln version suggested by |read + splitLines version
        description|Daniel Kozak                |by Paul M
                   |(http://stackoverflow.com/a |
                   |/28926137/1488799)          |

--
March 08, 2015
https://issues.dlang.org/show_bug.cgi?id=14256

--- Comment #9 from Ivan Kazmenko <gassa@mail.ru> ---
Created attachment 1488
  --> https://issues.dlang.org/attachment.cgi?id=1488&action=edit
readln version suggested by Daniel Kozak
(http://stackoverflow.com/a/28926137/1488799)

--
« First   ‹ Prev
1 2