Brad Roberts wrote:
> bearophile wrote:
>  >> What the heck is going on? When does memory mapping actually help?<
>> You are scanning the file linearly, and the memory window you use is
>> probably very small. In such situation a memory mapping is probably
>> not the best thing. A memory mapping is useful when you for example
>> operate with random access on a wider sliding window on the file.
> 
> You can drop the 'sliding' part.  mmap tends to help when doing random
> access (or sequential but non-contiguous maybe) over a file.  Pure
> streaming is handled pretty well by both patterns.  One nicity with mmap
> is that you can hint to the os how you'll be using it via madvise.  You
> can't do that with [f]read.

Hey Brad,

Nice advice on madvise, didn't know about it. Just in case it might be useful to someone, trying madvise with any of the four possible policies did not yield any noticeable change in timing for my particular test.


Andrei

Andrei Alexandrescu wrote:
> Nice advice on madvise, didn't know about it. Just in case it might be useful to someone, trying madvise with any of the four possible policies did not yield any noticeable change in timing for my particular test.

If you can build 4 windows executables, I can time them on my machine, and we can see if windows behaves differently.

Andrei Alexandrescu wrote:
> This all would make perfect sense if the performance was about the same in the two cases. But in fact memory mapping introduced a large *pessimization*. Why? I am supposedly copying less data and doing less 

Pessimization? What a great word! I've never heard that before!

--benji

Benji Smith wrote:
> Andrei Alexandrescu wrote:
>> This all would make perfect sense if the performance was about the same in the two cases. But in fact memory mapping introduced a large *pessimization*. Why? I am supposedly copying less data and doing less 
> 
> Pessimization? What a great word! I've never heard that before!
> 
> --benji

I've heard it first from Scott Meyers.

Andrei

Walter Bright Wrote:

> Andrei Alexandrescu wrote:
> > Nice advice on madvise, didn't know about it. Just in case it might be useful to someone, trying madvise with any of the four possible policies did not yield any noticeable change in timing for my particular test.
> 
> If you can build 4 windows executables, I can time them on my machine, and we can see if windows behaves differently.

By default windows does random access optimisation simply sucking file into cache which is faster (on XP) than sequential access optimisation. It will behave quite good if all 400MB fit in your file cache.

Wed, 18 Feb 2009 20:56:16 -0800, Andrei Alexandrescu wrote:

> Benji Smith wrote:
>> Andrei Alexandrescu wrote:
>>> This all would make perfect sense if the performance was about the same in the two cases. But in fact memory mapping introduced a large *pessimization*. Why? I am supposedly copying less data and doing less
>> 
>> Pessimization? What a great word! I've never heard that before!
>> 
>> --benji
> 
> I've heard it first from Scott Meyers.

I've heard this term in connection with premature optimization discussions.  Like, premature optimization is investing time in improving something that doesn't really need to be improved.  On the other hand, pessimization is doing something which is easy to avoid and is almost guaranteed to slow you down.  Like using post-increment in C++.