This D2 program runs in about 5.13 seconds on my PC:


import std.stdio;
void main() {
    auto f = File("bytes_test.dat", "wb");
    ubyte[3] RGB;
    foreach (_; 0 .. 1_000_000)
        f.rawWrite(RGB);
}



While this C program runs in about 0.14 seconds:


#include <stdio.h>
int main() {
    FILE *f = fopen("bytes_test.dat", "wb");
    unsigned char RGB[3] = {0};
    int i;
    for (i = 0; i < 1000000; i++)
        fwrite(RGB, 1, 3, f);
    return 0;
}


Is my D2 program wrong, or is File.rawWrite in need of some improvements?

(Writing 3 bytes at a time is not efficient, but the C code shows that the runtime is acceptable for me for small files).

Bye,
bearophile

29.11.2011 15:57, bearophile пишет:
> This D2 program runs in about 5.13 seconds on my PC:
>
>
> import std.stdio;
> void main() {
>      auto f = File("bytes_test.dat", "wb");
>      ubyte[3] RGB;
>      foreach (_; 0 .. 1_000_000)
>          f.rawWrite(RGB);
> }
>
>
>
> While this C program runs in about 0.14 seconds:
>
>
> #include<stdio.h>
> int main() {
>      FILE *f = fopen("bytes_test.dat", "wb");
>      unsigned char RGB[3] = {0};
>      int i;
>      for (i = 0; i<  1000000; i++)
>          fwrite(RGB, 1, 3, f);
>      return 0;
> }
>
>
> Is my D2 program wrong, or is File.rawWrite in need of some improvements?
>
> (Writing 3 bytes at a time is not efficient, but the C code shows that the runtime is acceptable for me for small files).
>
> Bye,
> bearophile

Your OS is Windows, right? On Windows, rawWrite and rawRead always flushes stream, sets binary mode, reads/writes, flushes stream again, sets previous mode. This is definitely unnecessary slow - at least it should change mode only if needed (the file is opened in a text mode). The better solution is to change all other functions so the file mode will be changed lazily (for a text file, raw* functions will change file mode and leave file in this mode until a call of a function, that really needs a file to be in a text mode).

By the way, why this changing mode behaviour is Windows only? Yes, it is clearly documented that this is the case, but it isn't documented _why_ this is the case.

Quick link to the current rawRead implementation (for happy owners of Google's Chrome, the fastest beautiful web browser that can BSOD your Windows even without administrator rights) (the link isn't for Firefox: it will slow down the browser and be displayed incorrectly):
https://github.com/D-Programming-Language/phobos/blob/master/std/stdio.d#L458

On 29.11.2011 16:00, Denis Shelomovskij wrote:
>
> Your OS is Windows, right? On Windows, rawWrite and rawRead always
> flushes stream, sets binary mode, reads/writes, flushes stream again,
> sets previous mode. This is definitely unnecessary slow - at least it
> should change mode only if needed (the file is opened in a text mode).
> The better solution is to change all other functions so the file mode
> will be changed lazily (for a text file, raw* functions will change file
> mode and leave file in this mode until a call of a function, that really
> needs a file to be in a text mode).
>
> By the way, why this changing mode behaviour is Windows only? Yes, it is
> clearly documented that this is the case, but it isn't documented _why_
> this is the case.

Text mode means that \n in strings is translated to \r\n when writing, and the other way when reading. That way you can use only LF (\n) to get linefeeds on all platforms, even if Windows uses CRLF.

On 11/29/2011 08:00 AM, Denis Shelomovskij wrote:
> 29.11.2011 15:57, bearophile пишет:
>> This D2 program runs in about 5.13 seconds on my PC:
>>
>>
>> import std.stdio;
>> void main() {
>> auto f = File("bytes_test.dat", "wb");
>> ubyte[3] RGB;
>> foreach (_; 0 .. 1_000_000)
>> f.rawWrite(RGB);
>> }
>>
>>
>>
>> While this C program runs in about 0.14 seconds:
>>
>>
>> #include<stdio.h>
>> int main() {
>> FILE *f = fopen("bytes_test.dat", "wb");
>> unsigned char RGB[3] = {0};
>> int i;
>> for (i = 0; i< 1000000; i++)
>> fwrite(RGB, 1, 3, f);
>> return 0;
>> }
>>
>>
>> Is my D2 program wrong, or is File.rawWrite in need of some improvements?
>>
>> (Writing 3 bytes at a time is not efficient, but the C code shows that
>> the runtime is acceptable for me for small files).
>>
>> Bye,
>> bearophile
>
> Your OS is Windows, right? On Windows, rawWrite and rawRead always
> flushes stream, sets binary mode, reads/writes, flushes stream again,
> sets previous mode. This is definitely unnecessary slow - at least it
> should change mode only if needed (the file is opened in a text mode).
> The better solution is to change all other functions so the file mode
> will be changed lazily (for a text file, raw* functions will change file
> mode and leave file in this mode until a call of a function, that really
> needs a file to be in a text mode).
>
> By the way, why this changing mode behaviour is Windows only? Yes, it is
> clearly documented that this is the case, but it isn't documented _why_
> this is the case.
>
> Quick link to the current rawRead implementation (for happy owners of
> Google's Chrome, the fastest beautiful web browser that can BSOD your
> Windows even without administrator rights) (the link isn't for Firefox:
> it will slow down the browser and be displayed incorrectly):
> https://github.com/D-Programming-Language/phobos/blob/master/std/stdio.d#L458
>
My firefox handled the link just fine. Ofcourse my dev machine is an 8 core i7 with 16 GB of ram.....

-Kai Meyer

On 11/29/2011 01:20 PM, torhu wrote:
> On 29.11.2011 16:00, Denis Shelomovskij wrote:
>>
>> Your OS is Windows, right? On Windows, rawWrite and rawRead always
>> flushes stream, sets binary mode, reads/writes, flushes stream again,
>> sets previous mode. This is definitely unnecessary slow - at least it
>> should change mode only if needed (the file is opened in a text mode).
>> The better solution is to change all other functions so the file mode
>> will be changed lazily (for a text file, raw* functions will change file
>> mode and leave file in this mode until a call of a function, that really
>> needs a file to be in a text mode).
>>
>> By the way, why this changing mode behaviour is Windows only? Yes, it is
>> clearly documented that this is the case, but it isn't documented _why_
>> this is the case.
>
> Text mode means that \n in strings is translated to \r\n when writing,
> and the other way when reading. That way you can use only LF (\n) to get
> linefeeds on all platforms, even if Windows uses CRLF.

The "has no effect" part in the following paragraph has been news to me. From 'man fopen' on my Lubuntu 11 based system:

<quote>
The  mode string can also include the letter 'b' either as a last char‐
acter or as a character between the characters in any of the  two-char‐
acter strings described above.  This is strictly for compatibility with
C89 and has no effect; the 'b' is ignored on all POSIX conforming  sys‐
tems,  including Linux.  (Other systems may treat text files and binary
files differently, and adding the 'b' may be a good idea if you do  I/O
to a binary file and expect that your program may be ported to non-Unix
environments.)
</quote>

Ali