I saw a thread a few days ago about somebody wanting a few UFCS-based convenience functions, so I thought that I'd take the opportunity to make a small contribution to Phobos. Currently I have four small functions: each, exhaust, perform, and tap, and would like some feedback.

each is designed to perform operations with side-effects on each range element. To actually change the elements of the range, each element must be accepted by reference.

Range each(alias fun, Range)(Range r)
if (isInputRange!(Unqual!Range))
{
    alias unaryFun!fun _fun;
    foreach (ref e; r)
    {
        fun(e);
    }

    return r;
}

//Prints [-1, 0, 1]
[1, 2, 3].each!((ref i) => i -= 2).writeln;


exhaust iterates a range until it is exhausted. It also has the nice feature that if range.front is callable, exhaust will call it upon each iteration.

Range exhaust(Range)(Range r)
if (isInputRange!(Unqual!Range))
{

    while (!r.empty)
    {
        r.front();
        r.popFront();
    }

    return r;
}

//Writes "www.dlang.org". x is an empty MapResult range.
auto x = "www.dlang.org"
         .map!((c) { c.write; return false; })
         .exhaust;

//Prints []
[1, 2, 3].exhaust.writeln;


perform is pretty badly named, but I couldn't come up with a better one. It can be inserted in a UFCS chain and perform some operation with side-effects. It doesn't alter its argument, just returns it for the next function in the chain.

T perform(alias dg, T)(ref T val)
{
    dg();

    return val;
}

//Prints "Mapped: 2 4"
[1, 2, 3, 4, 5]
.filter!(n => n < 3)
.map!(n => n * n)
.perform!({write("Mapped: ");})
.each!(n => write(n, " "));


Lastly is tap, which takes a value and performs some mutating operation on it. It then returns the value.

T tap(alias dg, T)(auto ref T val)
{
    dg(val);

    return val;
}

class Foo
{
    int x;
    int y;
}

auto f = (new Foo).tap!((f)
{
    f.x = 2;
    f.y = 3;
});

//Prints 2 3
writeln(f.x, " ", f.y);

struct Foo2
{
    int x;
    int y;
}

//Need to use ref for value types
auto f2 = Foo2().tap!((ref f)
{
    f.x = 3;
    f.y = 2;
});

//Prints 3 2
writeln(f2.x, " ", f2.y);


Do you think these small functions have a place in Phobos? I think each and exhaust would be best put into std.range, but I'm not quite sure where perform and tap should go. Also, there's that horrible name for perform, for which I would like to come up with a better name.

On Sunday, June 02, 2013 04:57:53 Meta wrote:
> I saw a thread a few days ago about somebody wanting a few UFCS-based convenience functions, so I thought that I'd take the opportunity to make a small contribution to Phobos. Currently I have four small functions: each, exhaust, perform, and tap, and would like some feedback.
> 
> each is designed to perform operations with side-effects on each range element. To actually change the elements of the range, each element must be accepted by reference.
> 
> Range each(alias fun, Range)(Range r)
> if (isInputRange!(Unqual!Range))
> {
>      alias unaryFun!fun _fun;
>      foreach (ref e; r)
>      {
>          fun(e);
>      }
> 
>      return r;
> }
> 
> //Prints [-1, 0, 1]
> [1, 2, 3].each!((ref i) => i -= 2).writeln;

For reference type ranges and input ranges which are not forward ranges, this will consume the range and return nothing. It would have to accept only forward ranges and save the result before iterating over it. Also, range-based functions should not be strict (i.e. not lazy) without good reason. And I don't see much reason to make this strict. Also, it's almost the same thing as map. Why not just use map? The predicate can simply return the same value after it's operated on it.

If we did add this, I'd argue that transform is a better name, but I'm still inclined to think that it's not worth adding.

> exhaust iterates a range until it is exhausted. It also has the nice feature that if range.front is callable, exhaust will call it upon each iteration.
> 
> Range exhaust(Range)(Range r)
> if (isInputRange!(Unqual!Range))
> {
> 
>      while (!r.empty)
>      {
>          r.front();
>          r.popFront();
>      }
> 
>      return r;
> }
> 
> //Writes "www.dlang.org". x is an empty MapResult range.
> auto x = "www.dlang.org"
>           .map!((c) { c.write; return false; })
>           .exhaust;
> 
> //Prints []
> [1, 2, 3].exhaust.writeln;

The callable bit won't work. It'll just call front. You'd have to do something like

static if(isCallable!(ElementType!R))
    r.front()();

Also, if front were pure, then calling it and doing nothing with its return value would result in a compilation error. The same goes if the element type is a pure callable. And even if this did work exactly as you intended. I think that assuming that someone exhausting the range would would what front returns to be called is a bad idea. Maybe they do, maybe they don't, I'd expect that in most cases, they wouldn't. If that's what they want, they can call map before calling exhaust.

> perform is pretty badly named, but I couldn't come up with a better one. It can be inserted in a UFCS chain and perform some operation with side-effects. It doesn't alter its argument, just returns it for the next function in the chain.
> 
> T perform(alias dg, T)(ref T val)
> {
>      dg();
> 
>      return val;
> }
> 
> //Prints "Mapped: 2 4"
> [1, 2, 3, 4, 5]
> .filter!(n => n < 3)
> .map!(n => n * n)
> .perform!({write("Mapped: ");})
> .each!(n => write(n, " "));

So, you want to have a function which you pass something (including a range) and then returns that same value after calling some other function? Does this really buy you much over just splitting up the expression - you're already giving a multline example anyway.

auto foo = [1, 2, 3, 4, 4].filt!(n => n < 3)().map!(n => n * n)();
write("Mapped: ");
foo.each!(n => write(n, "")();

And I think that this is a perfect example of something that should just be done with foreach anyway. Not to mention, if you're calling very many functions, you're going to need to use multiple lines, in which case chaining the functions like that doesn't buy you much. All you end up doing is taking what would normally be a sequence of statements and turned it into one multiline statement. I don't think that this buys us much, especially when it's just calling one function which does nothing on any object in the chain.

> Lastly is tap, which takes a value and performs some mutating operation on it. It then returns the value.
> 
> T tap(alias dg, T)(auto ref T val)
> {
>      dg(val);
> 
>      return val;
> }
> 
> class Foo
> {
>      int x;
>      int y;
> }
> 
> auto f = (new Foo).tap!((f)
> {
>      f.x = 2;
>      f.y = 3;
> });
> 
> //Prints 2 3
> writeln(f.x, " ", f.y);
> 
> struct Foo2
> {
>      int x;
>      int y;
> }
> 
> //Need to use ref for value types
> auto f2 = Foo2().tap!((ref f)
> {
>      f.x = 3;
>      f.y = 2;
> });
> 
> //Prints 3 2
> writeln(f2.x, " ", f2.y);

Why do you need tap? So that you can use an anonymous function? If it had a name, you'd just use it with UFCS. I'd argue that this use case is minimal enough that you might as well just give it a name and then use UFCS if you really want to use UFCS, and if you want an anonymous function, what's the real gain of chaining it with UFCS anyway? It makes the expression much harder to read if you try and chain calls on the anonymous function.

UFCS' main purpose is making it so that a function can be called on multiple types in the same manner (particularly where it could be a member function in some cases and a free function in others), and it just so happens to make function chaining cleaner in some cases. But there's no reason to try and turn all function calls in UFCS calls, and I think that perform and tap are taking it too far.

- Jonathan M Davis

On Sunday, 2 June 2013 at 02:57:56 UTC, Meta wrote:
> I saw a thread a few days ago about somebody wanting a few UFCS-based convenience functions, so I thought that I'd take the opportunity to make a small contribution to Phobos. Currently I have four small functions: each, exhaust, perform, and tap, and would like some feedback.
>
> each is designed to perform operations with side-effects on each range element. To actually change the elements of the range, each element must be accepted by reference.
>
> Range each(alias fun, Range)(Range r)
> if (isInputRange!(Unqual!Range))
> {
>     alias unaryFun!fun _fun;
>     foreach (ref e; r)
>     {
>         fun(e);
>     }
>
>     return r;
> }
>
> //Prints [-1, 0, 1]
> [1, 2, 3].each!((ref i) => i -= 2).writeln;
>
>
> exhaust iterates a range until it is exhausted. It also has the nice feature that if range.front is callable, exhaust will call it upon each iteration.
>
> Range exhaust(Range)(Range r)
> if (isInputRange!(Unqual!Range))
> {
>
>     while (!r.empty)
>     {
>         r.front();
>         r.popFront();
>     }
>
>     return r;
> }
>
> //Writes "www.dlang.org". x is an empty MapResult range.
> auto x = "www.dlang.org"
>          .map!((c) { c.write; return false; })
>          .exhaust;
>
> //Prints []
> [1, 2, 3].exhaust.writeln;
>
>
> perform is pretty badly named, but I couldn't come up with a better one. It can be inserted in a UFCS chain and perform some operation with side-effects. It doesn't alter its argument, just returns it for the next function in the chain.
>
> T perform(alias dg, T)(ref T val)
> {
>     dg();
>
>     return val;
> }
>
> //Prints "Mapped: 2 4"
> [1, 2, 3, 4, 5]
> .filter!(n => n < 3)
> .map!(n => n * n)
> .perform!({write("Mapped: ");})
> .each!(n => write(n, " "));
>
>
> Lastly is tap, which takes a value and performs some mutating operation on it. It then returns the value.
>
> T tap(alias dg, T)(auto ref T val)
> {
>     dg(val);
>
>     return val;
> }
>
> class Foo
> {
>     int x;
>     int y;
> }
>
> auto f = (new Foo).tap!((f)
> {
>     f.x = 2;
>     f.y = 3;
> });
>
> //Prints 2 3
> writeln(f.x, " ", f.y);
>
> struct Foo2
> {
>     int x;
>     int y;
> }
>
> //Need to use ref for value types
> auto f2 = Foo2().tap!((ref f)
> {
>     f.x = 3;
>     f.y = 2;
> });
>
> //Prints 3 2
> writeln(f2.x, " ", f2.y);
>
>
> Do you think these small functions have a place in Phobos? I think each and exhaust would be best put into std.range, but I'm not quite sure where perform and tap should go. Also, there's that horrible name for perform, for which I would like to come up with a better name.

You may find this forum discussion from several months ago interesting.

http://forum.dlang.org/post/kglo9d$rjf$1@digitalmars.com

Confusingly, your each() seems to be fairly similar to what Andrei wanted tap() used for.  Andrei didn't care for the tap() you propose but loved the idea of a tap() function that works like unix tee.

I like exhaust() as I just had to write something similar.  I like perform() just because I love UFCS range chains and anything to avoid those extra statements is alright in my book.  This is probably not a majority opinion though.  I can't think of a better name either though.

On Sunday, 2 June 2013 at 04:10:15 UTC, Jonathan M Davis wrote:
> On Sunday, June 02, 2013 04:57:53 Meta wrote:
> The callable bit won't work. It'll just call front. You'd have to do something
> like
>
> static if(isCallable!(ElementType!R))
>     r.front()();
>
> Also, if front were pure, then calling it and doing nothing with its return
> value would result in a compilation error. The same goes if the element type
> is a pure callable.

Calling front is kind of the point of exhaust(), otherwise you'd use takeNone().  You wouldn't use this if front were pure because the only reason you'd want exhaust is if you were (ab)using side effects (like I was the other day on D.learn).  Having it error out if you were using it on a range with pure front() is actually a good thing because you've made some error in your reasoning if you think you want exhaust() to run in that situation. processSideEffects() is probably too long of name.

>  And even if this did work exactly as you intended. I think
> that assuming that someone exhausting the range would would what front returns
> to be called is a bad idea. Maybe they do, maybe they don't, I'd expect that
> in most cases, they wouldn't. If that's what they want, they can call map
> before calling exhaust.
>

Sticking a map before exhaust without it calling front() would accomplish nothing. I know this because my own little toy eat() just called popFront() originally on a Map range and nothing happened.  You'd be skipping map's function if you don't call front.

> For reference type ranges and input ranges which are not forward ranges, this
> will consume the range and return nothing.

I originally wrote it to accept forward ranges and use save, but I wanted to make it as inclusive as possible. I guess I overlooked the case of ref ranges. As for ranges that aren't forward ranges, consider a simple input range.

struct InputRange
{
    int[] arr = [1, 2, 3, 4, 5];

    int front() { return arr.front; }

    bool empty() { return arr.empty; }

    void popFront() { return arr.popFront; }
}

writeln(isForwardRange!InputRange); //False

Range()
.each!(n => write(n, " "))
.map!(n => n * n)
.writeln;

This outputs 1 2 3 4 5 [1, 4, 9, 16, 25], so each is not returning an empty range. I believe this is because r in this case is a value type range, and the foreach loop makes a copy of it. This does still leave the problem of reference type ranges.

> Also, range-based
> functions should not be strict (i.e. not lazy) without good reason. And I
> don't see much reason to make this strict.

It's not lazy because it's intended to perform some mutating or otherwise side-effectful operation. Map doesn't play well with side effects, partially because of its laziness. A very contrived example:

auto arr = [1, 2, 3, 4].map!(n => n.writeln); //Now what?

It's not clear now what to do with the result. You could try a for loop:

foreach (n; arr) n(); //Error: n cannot be of type void

But that doesn't work. A solution would be to modify the function you pass to map:

auto arr = [1, 2, 3, 4].map!((n) { n.writeln; return n; });

foreach (n; arr) {} //Prints 1 2 3 4

But that's both ugly and verbose. each also has the advantage of being able to return the original range (possibly modified), whereas map must return a MapResult due to its laziness, and you need that extra array call to bludgeon it into the correct form. each is also more efficient in that it doesn't need to return a copy of the data passed to it. It simply mutates it in-place.

> Also, it's almost the same thing as map. Why not just use map? The predicate can simply return the same value
> after it's operated on it.

See above. There are some cases where map is clunky to work with due to it being non-strict.

> If we did add this, I'd argue that transform is a better name, but I'm still
> inclined to think that it's not worth adding.

I chose the name each because it's a common idiom in a couple of other languages (Javascript, Ruby and Rust off the top of my head), and because I think it underlines the fact that each is meant to perform side-effectful operations.

>> exhaust iterates a range until it is exhausted. It also has the
>> nice feature that if range.front is callable, exhaust will call
>> it upon each iteration.
>> 
>> Range exhaust(Range)(Range r)
>> if (isInputRange!(Unqual!Range))
>> {
>> 
>>      while (!r.empty)
>>      {
>>          r.front();
>>          r.popFront();
>>      }
>> 
>>      return r;
>> }
>> 
>> //Writes "www.dlang.org". x is an empty MapResult range.
>> auto x = "www.dlang.org"
>>           .map!((c) { c.write; return false; })
>>           .exhaust;
>> 
>> //Prints []
>> [1, 2, 3].exhaust.writeln;
>
> The callable bit won't work. It'll just call front. You'd have to do something
> like
>
> static if(isCallable!(ElementType!R))
>     r.front()();

I was having some trouble with writing exhaust and forgot all about ElementType. I'll change that.

> Also, if front were pure, then calling it and doing nothing with its return
> value would result in a compilation error. The same goes if the element type
> is a pure callable.

Is this true for all pure functions? That seems like kind of strange behaviour to me, and doesn't really make sense given the definition of functional purity.

> And even if this did work exactly as you intended. I think
> that assuming that someone exhausting the range would would what front returns
> to be called is a bad idea. Maybe they do, maybe they don't, I'd expect that
> in most cases, they wouldn't. If that's what they want, they can call map
> before calling exhaust.

I think the original reason that somebody wanted exhaust was because map is lazy and they wanted a function which could take the result of map and consume it while calling front each time. Otherwise, there wouldn't be much reason to have this, as there is takeNone and popFrontN.

> So, you want to have a function which you pass something (including a range)
> and then returns that same value after calling some other function? Does this
> really buy you much over just splitting up the expression - you're already
> giving a multline example anyway.

It gives you the advantage of not having to split your UFCS chain up, which I personally find valuable, and I think other people would as well. I think it's quite similar to the various side-effectful monads in Haskell, which don't do anything with their argument other than return it, but perform some operation with side-effects in the process. I'll try to think up a better example for this, because I think it can be quite useful in certain circumstances.

> And I think that this is a perfect example of something that should just be
> done with foreach anyway. Not to mention, if you're calling very many
> functions, you're going to need to use multiple lines, in which case chaining
> the functions like that doesn't buy you much. All you end up doing is taking
> what would normally be a sequence of statements and turned it into one
> multiline statement. I don't think that this buys us much, especially when
> it's just calling one function which does nothing on any object in the chain.

See above. I think there is quite a high value in not having to define extra variables and split up your UFCS chain halfway through, which which is somewhat obfuscatory, I think.

> Why do you need tap? So that you can use an anonymous function? If it had a
> name, you'd just use it with UFCS. I'd argue that this use case is minimal
> enough that you might as well just give it a name and then use UFCS if you
> really want to use UFCS, and if you want an anonymous function, what's the
> real gain of chaining it with UFCS anyway? It makes the expression much harder
> to read if you try and chain calls on the anonymous function.

One way in which tap can be useful is that you can perform some operations on data in the middle of a UFCS chain and then go about your business. It's probably the least useful of the four.

> UFCS' main purpose is making it so that a function can be called on multiple
> types in the same manner (particularly where it could be a member function in
> some cases and a free function in others), and it just so happens to make
> function chaining cleaner in some cases. But there's no reason to try and turn
> all function calls in UFCS calls, and I think that perform and tap are taking
> it too far.

Personally, I prefer function-chaining style more, as I think it's more aesthetic and more amenable to Walter's notion of component programming. For someone who doesn't use UFCS that much, these functions will seem almost useless, as their entire functionality can be duplicated by using some other construct. I wrote them to allow more versatile UFCS chains, so you don't have to break them up.

I think that people who heavily use UFCS, on the other hand, will find these quite useful in different situations.

Meta:

> perform is pretty badly named, but I couldn't come up with a better one. It can be inserted in a UFCS chain and perform some operation with side-effects. It doesn't alter its argument, just returns it for the next function in the chain.
>
> T perform(alias dg, T)(ref T val)
> {
>     dg();
>
>     return val;
> }
>
> //Prints "Mapped: 2 4"
> [1, 2, 3, 4, 5]
> .filter!(n => n < 3)
> .map!(n => n * n)
> .perform!({write("Mapped: ");})
> .each!(n => write(n, " "));

I'd like something like this in Phobos, but I'd like it to have a better name. But in most (all?) cases what I want to put inside such perform is a printing function, so I have opened this:

http://d.puremagic.com/issues/show_bug.cgi?id=9882



> exhaust iterates a range until it is exhausted. It also has the nice feature that if range.front is callable, exhaust will call it upon each iteration.
>
> Range exhaust(Range)(Range r)
> if (isInputRange!(Unqual!Range))
> {
>
>     while (!r.empty)
>     {
>         r.front();
>         r.popFront();
>     }
>
>     return r;
> }
>
> //Writes "www.dlang.org". x is an empty MapResult range.
> auto x = "www.dlang.org"
>          .map!((c) { c.write; return false; })
>          .exhaust;
>
> //Prints []
> [1, 2, 3].exhaust.writeln;

I's also like this in Phobos, for debugging purposes. But I'd like it to return nothing, so you are forced to use it only at the end of a chain.

(So I appreciate 2 of your 4 proposals. I have proposed both of them in D.learn time ago.)

---------------------

Brad Anderson:

> Andrei didn't care for the tap() you propose but loved the idea of a tap() function that works like unix tee.

Something like this Python itertool?

def tee(iterable, n=2):
    it = iter(iterable)
    deques = [collections.deque() for i in range(n)]
    def gen(mydeque):
        while True:
            if not mydeque:             # when the local deque is empty
                newval = next(it)       # fetch a new value and
                for d in deques:        # load it to all the deques
                    d.append(newval)
            yield mydeque.popleft()
    return tuple(gen(d) for d in deques)


Bye,
bearophile

On 6/2/13 1:58 AM, Meta wrote:
>> For reference type ranges and input ranges which are not forward
>> ranges, this
>> will consume the range and return nothing.
>
> I originally wrote it to accept forward ranges and use save, but I
> wanted to make it as inclusive as possible. I guess I overlooked the
> case of ref ranges.
[snip]

Thanks for sharing your ideas.

I think consuming all of a range evaluating front and doing nothing should be the role of reduce with only one parameter (the range). That overload would take the range to be "exhausted" and return void. Thus your example becomes:

[1, 2, 3, 4].map!(n => n.writeln).reduce;


Andrei

Andrei Alexandrescu:

> [1, 2, 3, 4].map!(n => n.writeln).reduce;

I have to shot this down for many reasons:

I think it's better to give that final function a different name (like "consume" or something like that) because it's used for very different purposes and it returns nothing. Re-using the name "reduce" doesn't reduce the amount of Phobos lines of code, it doesn't make the user code simpler to understand, it's more obscure because it's more semantically overloaded, and it's not more easy to find in the documentation by the future D users. Function names are not language keywords, packing different purposes in the same name as "static" doesn't give any advantage, and only disadvantages.

And using map with a lambda that returns nothing is not a style I like :-( It's probably better to encourage D programmers to give pure lambdas to map/filter, for several reasons (safety, cleanness, code style, future D front-end optimizations done on those higher order functions, to allow a better debuggability, and to avoid Phobos bugs like http://d.puremagic.com/issues/show_bug.cgi?id=9674 ). So I think it's better to introduce a new Phobos function like tap() that accepts a function/delegate with side effects that takes no input arguments.

Bye,
bearophile

> I think consuming all of a range evaluating front and doing nothing should be the role of reduce with only one parameter (the range). That overload would take the range to be "exhausted" and return void. Thus your example becomes:

Maybe, then, it would be best to have a template that calls reduce in such a way, that makes it perfectly clear what is happening.

On Sunday, 2 June 2013 at 13:07:18 UTC, Andrei Alexandrescu wrote:
> On 6/2/13 1:58 AM, Meta wrote:
>>> For reference type ranges and input ranges which are not forward
>>> ranges, this
>>> will consume the range and return nothing.
>>
>> I originally wrote it to accept forward ranges and use save, but I
>> wanted to make it as inclusive as possible. I guess I overlooked the
>> case of ref ranges.
> [snip]
>
> Thanks for sharing your ideas.
>
> I think consuming all of a range evaluating front and doing nothing should be the role of reduce with only one parameter (the range). That overload would take the range to be "exhausted" and return void. Thus your example becomes:
>
> [1, 2, 3, 4].map!(n => n.writeln).reduce;
>
>
> Andrei

One of the problems with using "map" for something such as this, is that the resulting object is not a range, since "front" now returns void, and a range *must* return a value. So that code will never compile (since reduce will ask for at least input range). Heck, I think we should make it so that map refuses to compile with an operator that returns void. It doesn't make much sense as-is.

Usage has to be something like:

map!((n) {n.writeln; return n;})

which is quite clunky. The idea of a "tee" range, that takes n, runs an operation on it, and then returns said n as is becomes really very useful (and more idiomatic). [1, 2, 3, 4].tee!(n => n.writeln). There! perfect :)

I've dabbled in implementing such a function, but there are conceptual problems: If the user calls "front" twice in a row, then should "fun" be called twice? If user popsFront without calling front, should "fun" be called at all?

Should it keep track of calls, to guarantee 1, and only 1, call on each element?

I'm not sure there is a correct answer to that, which is one of the reasons I haven't actually submitted anything.

--------

I don't think "argument-less reduce" should do what you describe, as it would be a bit confusing what the function does. 1-names; 1-operation, IMO. Users might accidentally think they are getting an additive reduction :(

I think a function called "walk", in line with "walkLength", would be much more appropriate, and make more sense to boot!

But we run into the same problem... Should "walk" call front between each element? Both answers are correct, IMO.