Thread overview
File I/O performance pitfalls
Oct 26, 2019
9898287
Oct 26, 2019
rikki cattermole
Oct 26, 2019
9898287
October 26, 2019
Hi I want to find out what's causing my file writes to be so slow. I'm setting up buffer and locking the file and writing them to the file.

$ cat d.d && ldc2 -O5  d.d && time ./d >> /dev/null
void main() {
    import std.stdio;
    stdout.setvbuf(4096);
    stdout.lock();

    foreach(i; 0 .. 900_000_000)
        writef("Hello, {}\n", i);
}
real    0m58.790s
user    0m58.545s
sys     0m0.232s

For comparison, I come from the Rust land and here's somewhat equivalent code which is consistently faster:

$ cat rust.rs && rustc -C opt-level=3 rust.rs && time ./rust >> /dev/null
use std::io::Write;

fn main() {
    let stdout = std::io::stdout();
    let lock = stdout.lock();
    let mut buf = std::io::BufWriter::new(lock);
    for i in 0 .. 900_000_000 {
        buf.write_fmt(format_args!(
            "Hello, {}\n", i
        )).unwrap();
    }
}

real    0m46.502s
user    0m46.263s
sys     0m0.228s

What am I missing?
October 26, 2019
On 26/10/2019 2:27 PM, 9898287 wrote:
> Hi I want to find out what's causing my file writes to be so slow. I'm setting up buffer and locking the file and writing them to the file.
> 
> $ cat d.d && ldc2 -O5  d.d && time ./d >> /dev/null

You probably want -O3 not -O5.

> void main() {
>      import std.stdio;
>      stdout.setvbuf(4096);
>      stdout.lock();
> 
>      foreach(i; 0 .. 900_000_000)
>          writef("Hello, {}\n", i);

I assume what you intended is:
writeln("Hello, ", i);

Also for format functions in D you should prefer the templated variation as it provides compile time verification of arguments e.g.

writef!"Hello, {}\n%d"(i);

Please note that {} is not a format specifier.
printf style functions (which is what Phobos uses as the basis) for format specifiers begin with a percentage sign.
October 26, 2019
On Saturday, 26 October 2019 at 02:42:04 UTC, rikki cattermole wrote:
> On 26/10/2019 2:27 PM, 9898287 wrote:
>> [...]
>
> You probably want -O3 not -O5.
>
>>  [...]
>
> I assume what you intended is:
> writeln("Hello, ", i);
>
> Also for format functions in D you should prefer the templated variation as it provides compile time verification of arguments e.g.
>
> writef!"Hello, {}\n%d"(i);
>
> Please note that {} is not a format specifier.
> printf style functions (which is what Phobos uses as the basis) for format specifiers begin with a percentage sign.

Thank you for pointing that out.