I need to create a very simple program module that prints some runtime generated strings to the standard output. Ok, that's not a problem. The real problem is that I need to output these as fast as possible. The output consists of big string blocks (>100.000 chars) that mostly contain only two different characters.

I've ported the program from C. The original program doesn't use any sort of buffering, just some printf:s like for(i=1;i<n;i++) if (foobar()) printf("*.-");

Here are some simple benchmarks using the GNU time utility:


dmd test.c -o test -O3 -s
time ./test_c_printf > /dev/null
real	0m0.117s

dmd test.d -O -release -inline -version=printf
time ./test > /dev/null
real	0m0.140s

dmd test.d -O -release -inline -version=writef
time ./test > /dev/null
real	0m0.225s

How come writef() is so much slower than printf()?
Is there a fast way to do buffered output. I tried to use dynamic & static arrays and printf the full arrays, but that wasn't actually any faster.

You may wish to try Mango.io (Stdout)?  The IO model is designed to take advantage of buffering, and I once ran into a problem with the Win32 console where it barfed on more than 32K at one time :-)

To illustrate, there's a specific subclass to handle that condition at the end of this module: http://svn.dsource.org/projects/mango/trunk/mango/io/DeviceConduit.d

Mango.io has formatting options similar to printf(), the full gamut of discrete IO methods, and a whole lot more besides. The mango.io equivalent to writef() should be faster in that it emits arrays of utf8, rather than one dchar at a time; and Stdout employs buffering by default. It's worth taking a look here http://mango.dsource.org/ and here http://www.dsource.org/forums/viewtopic.php?t=148

Those who prefer OOP may well prefer Mango.io over the phobos IO package(s) ~ they're designed for different audiences.

- Kris


"Jari-Matti Mäkelä" <jmjmak@invalid_utu.fi> wrote in message news:djjlb6$1l22$1@digitaldaemon.com...
>I need to create a very simple program module that prints some runtime generated strings to the standard output. Ok, that's not a problem. The real problem is that I need to output these as fast as possible. The output consists of big string blocks (>100.000 chars) that mostly contain only two different characters.
>
> I've ported the program from C. The original program doesn't use any sort of buffering, just some printf:s like for(i=1;i<n;i++) if (foobar()) printf("*.-");
>
> Here are some simple benchmarks using the GNU time utility:
>
>
> dmd test.c -o test -O3 -s
> time ./test_c_printf > /dev/null
> real 0m0.117s
>
> dmd test.d -O -release -inline -version=printf
> time ./test > /dev/null
> real 0m0.140s
>
> dmd test.d -O -release -inline -version=writef
> time ./test > /dev/null
> real 0m0.225s
>
> How come writef() is so much slower than printf()?
> Is there a fast way to do buffered output. I tried to use dynamic & static
> arrays and printf the full arrays, but that wasn't actually any faster.

In article <djjlb6$1l22$1@digitaldaemon.com>, =?ISO-8859-1?Q?Jari-Matti_M=E4kel=E4?= says...
>
>I need to create a very simple program module that prints some runtime generated strings to the standard output. Ok, that's not a problem. The real problem is that I need to output these as fast as possible. The output consists of big string blocks (>100.000 chars) that mostly contain only two different characters.
..
>How come writef() is so much slower than printf()?

writef does some fancy things that printf doesn't, though I had expected performance to be comparable to printf for ASCII character strings.  The implementation has changed since I last looked at it, but you might want to look at the source for writef in std.format.d (doFormat) to see if you can find any hangups.

>Is there a fast way to do buffered output. I tried to use dynamic & static arrays and printf the full arrays, but that wasn't actually any faster.

If you just want to print ASCII strings as fast as possible, printf and writef are probably not ideal.  You may be better off with puts or fwrite, and possibly even messing with the buffer settings using setbuf/setvbuf.

Sean

printf and writef are already buffered - I believe stdout is line-buffered. Posting the actual code might help people help you. I also recommend using the -profile compiler option and/or std.perf. If you like you can also wrap a std.stream.Stream with a std.stream.BufferedStream but I suspect that you can do what you want with regular C buffering. See for example setvbuf.

"Jari-Matti Mäkelä" <jmjmak@invalid_utu.fi> wrote in message news:djjlb6$1l22$1@digitaldaemon.com...
>I need to create a very simple program module that prints some runtime generated strings to the standard output. Ok, that's not a problem. The real problem is that I need to output these as fast as possible. The output consists of big string blocks (>100.000 chars) that mostly contain only two different characters.
>
> I've ported the program from C. The original program doesn't use any sort of buffering, just some printf:s like for(i=1;i<n;i++) if (foobar()) printf("*.-");
>
> Here are some simple benchmarks using the GNU time utility:
>
>
> dmd test.c -o test -O3 -s
> time ./test_c_printf > /dev/null
> real 0m0.117s
>
> dmd test.d -O -release -inline -version=printf
> time ./test > /dev/null
> real 0m0.140s
>
> dmd test.d -O -release -inline -version=writef
> time ./test > /dev/null
> real 0m0.225s
>
> How come writef() is so much slower than printf()?
> Is there a fast way to do buffered output. I tried to use dynamic & static
> arrays and printf the full arrays, but that wasn't actually any faster.

Ben Hinkle wrote:
> printf and writef are already buffered - I believe stdout is line-buffered. Posting the actual code might help people help you. I also recommend using the -profile compiler option and/or std.perf. If you like you can also wrap a std.stream.Stream with a std.stream.BufferedStream but I suspect that you can do what you want with regular C buffering. See for example setvbuf.
> 

Yes, you're right. writef & printf are already buffered. One can see this with multithreaded CLI applications.

Unfortunately the original code isn't mine. I need to ask for permission if I want to post some code here. Ok, I'll give a shot at trying the std.stream-functions. Thanks.

The strangest thing here is that according to the time program the c-version does much greater amount of work in kernel mode. How is it possible at all that the D program seems to do the same I/O in user mode?

Sean Kelly wrote:
>>How come writef() is so much slower than printf()?
> 
> 
> writef does some fancy things that printf doesn't, though I had expected
> performance to be comparable to printf for ASCII character strings.  The
> implementation has changed since I last looked at it, but you might want to look
> at the source for writef in std.format.d (doFormat) to see if you can find any
> hangups.
> 
writef is very useful and flexible, but also pretty slow. The std.format.d is too complex for me to optimize. I need to take a second look at it sometime, but for now another approach may be more useful.