October 03, 2015 buffered output to files | ||||
---|---|---|---|---|
| ||||
In this great article [1] there is a brief section on buffered output to files. Also in this thread [2] I was advised to use explicitly buffered output for maximum performance. This left me perplexed: surely any high-level routines already use buffered IO, no? [1] http://nomad.so/2015/09/working-with-files-in-the-d-programming-language/ [2] http://forum.dlang.org/post/ydgmzhlspvvvrbeemrqf@forum.dlang.org In order to get a better understanding of the issue, I decided to do a bit of simple testing with kind with the kind of file writing I have to do very often: millions of shortish formatted data records. My simple test program is here: http://dpaste.dzfl.pl/329023e651c4. Time to write 10 million formatted records (average of 10 runs) using format ("%d %d %0.8f", i, i*13, i/1e7) with different methods: GDC DMD f0: 12.0s 15.2s std.stdio.writef f1: 10.0s 13.6s 1Mb buffer std.array.appender; std.stdio.rawWrite f2: 10.3s 13.6s 300Mb buffer std.outbuffer; std.file.write f3: 4.9s 5.0s std.stdc.fprintf f4: 4.8s 4.8s 30b buffer std.stdc.sprintf (no file output) * gdc (Debian 5.2.1-17) 5.2.1 20150911 :: -O3 -frelease -march=native * dmd DMD64 D Compiler v2.068.1 :: -O -release -inline Some observations: * the two double-buffered output approaches (f1 and f2) are roughly the same * the performance gain from the extra buffering was not huge (about 20%) * good old fprintf seems to be my best bet :-( These results may be very specific to my conditions, but I throw them out there just in case others may be interested. |
October 04, 2015 Re: buffered output to files | ||||
---|---|---|---|---|
| ||||
Posted in reply to Gerald Jansen | On Saturday, 3 October 2015 at 22:21:08 UTC, Gerald Jansen wrote: > My simple test program is here: http://dpaste.dzfl.pl/329023e651c4. An alternative link to the program (that doesn't try to run it) http://codepad.org/FbHJJqYM. |
Copyright © 1999-2021 by the D Language Foundation