Hi,

I have an input range.
I use the map! on it to transform using a function.
Finally I use filter! on the transformed data.

When I do a foreach on this, i noticed, that the transform function is called twice for each element.
If I remove the filter! it does only one transform function call per element.

Is this because the function is NOT PURE, so Phobos thinks that the function can give different results for the same input?

If I modify the function to be pure, will the transform function be called only once?

On Friday, 29 November 2019 at 15:30:22 UTC, realhet wrote:
> Hi,
>
> I have an input range.
> I use the map! on it to transform using a function.
> Finally I use filter! on the transformed data.
>
> When I do a foreach on this, i noticed, that the transform function is called twice for each element.
> If I remove the filter! it does only one transform function call per element.
>
> Is this because the function is NOT PURE, so Phobos thinks that the function can give different results for the same input?
>
> If I modify the function to be pure, will the transform function be called only once?

It's actually a much simpler reason: filter calls .front twice for each element in its input (once to check if the value satisfies the predicate, and then again to return the value if it does), and the range returned by map doesn't save the value of .front between calls, so it has to re-compute it each time by calling the transform function.

You can work around this using std.algorithm.iteration.cache, a range wrapper that calls .front on the underlying range exactly once, and then saves the value and reuses it for subsequent calls:

myRange
    .map!transform
    .cache
    .filter!pred
    .each!doSomething;

On Friday, 29 November 2019 at 15:30:22 UTC, realhet wrote:
> ...

Unfortunately function purity is not the answer.

I put a very long calculation into the transform function which is called from "map!".
And the "filter!" is making the "map!" call my function 2 times:
First for the "filter!" to decide it allows the element or not.
And second when the map! is passing it to the foreach() where I write the elements out.

Is there a solution for this performance issue?

Using functional programming, my program looks beautiful. But I need it to be optimal too.

//To be easier to understand, I want to make the following:
foreach(a; input){
  auto temp = slowFunction(a);
  if(check(temp))
    writeln(temp);
}

//But the elegant functional form ->
input.map!slowFunction.filter!(a => check(a)).each!writeln;

//Produces the following unoptimal solution:
foreach(a; input){
  if(check(slowFunction(a)))
    writeln(slowFunction(a));
}

Is there a functional way to fix this?

Thank you in advance!

On Friday, 29 November 2019 at 15:49:24 UTC, Paul Backus wrote:
> It's actually a much simpler reason: filter calls .front twice for each element in its input (once to check if the value satisfies the predicate, and then again to return the value if it does), and the range returned by map doesn't save the value of .front between calls, so it has to re-compute it each time by calling the transform function.

This makes it clear.

In my case a cache which can access all the previous elements would be a too big thing, so I will do the filtering in a later stage manually. But most importantly, now I know what's going on, Thank You!

On Friday, 29 November 2019 at 15:54:13 UTC, realhet wrote:
> On Friday, 29 November 2019 at 15:49:24 UTC, Paul Backus wrote:
>> It's actually a much simpler reason: filter calls .front twice for each element in its input (once to check if the value satisfies the predicate, and then again to return the value if it does), and the range returned by map doesn't save the value of .front between calls, so it has to re-compute it each time by calling the transform function.
>
> This makes it clear.
>
> In my case a cache which can access all the previous elements would be a too big thing, so I will do the filtering in a later stage manually. But most importantly, now I know what's going on, Thank You!

The "cache" wrapper only caches one element at a time. The cached value is discarded after each call to .popFront.