On Wednesday, 19 March 2014 at 23:00:08 UTC, Walter Bright wrote:
> I don't have a solid enough experience with this style of programming, but one opportunity of 'each' could be that it could be specialized and achieve greater speed for some arguments.

Speaking of which, it would be nice to have a compiler default iteration scheme that is optimal for any range.

The current status is:
"for ( ; !r.empty ; r.popFront )" is sub-optimal for string types, and sometimes arrays.

alias E = ElementEncodingType!Range;

"foreach(E e; range);" will create copies of elements

"foreach(ref E e; range);" will always work, but is arguably wrong for RValue ranges

"foreach(auto ref E e; range);" is not legal, but there is an ER requesting it, and I think it would make perfect sense to have this.

Finally: The hand written library code
while ( decode(s, i) < s.length )
{ ... }

Is faster for strings, than the compiler's/druntime's "foreach(E e; range);".

It would be awesome if:
foreach(auto ref E e; range);
Worked, *and* was always optimal.

The current scheme makes those who care jump through hoops, such as in `find`'s implementation.

'each' sounds like a good idea for supporting component programming a little more. As bearophile as already stated, you can do something like this...

someRange.filter!foo.frobulate.each!doSomethingWithIt;

For Walter's question about parallel execution, I imagine something like this.

// std.parallelism parallel function here.
someRange.whatever.parallel(numberOfUnits).each!doSomething

Basically it just removes a little foreach boilerplate. I don't think the implementation needs to be much more complicated than what Andrei wrote already, I would just pull that pretty much as-is.

I also don't think an argument like "you can already do this with foreach" is valid. You don't *have* to use it, and some people might like it. I know I would appreciate having it in std.algorithm. (I'm kind of a range fanboy.)

On Thursday, 20 March 2014 at 12:32:49 UTC, w0rp wrote:
> // std.parallelism parallel function here.
> someRange.whatever.parallel(numberOfUnits).each!doSomething

+1

This works:
foreach(i; [0,1,2,3,4,5].parallel) i.writeln;

This works:
[0,1,2,3,4,5].each!writeln;

This won't compile:
[0,1,2,3,4,5].parallel.each!writeln;

Error: template tmp.each cannot deduce function from argument types !(writeln)(ParallelForeach!(int[])), candidates are:
/tmp/tmp.d(9):        tmp.each(alias fun, Range)(Range range) if (isInputRange!Range)

On 2014-03-20 14:16, Andrea Fontana wrote:
> On Thursday, 20 March 2014 at 12:32:49 UTC, w0rp wrote:
>> // std.parallelism parallel function here.
>> someRange.whatever.parallel(numberOfUnits).each!doSomething
>
> +1
>
> This works:
> foreach(i; [0,1,2,3,4,5].parallel) i.writeln;
>
> This works:
> [0,1,2,3,4,5].each!writeln;
>
> This won't compile:
> [0,1,2,3,4,5].parallel.each!writeln;
>
> Error: template tmp.each cannot deduce function from argument types
> !(writeln)(ParallelForeach!(int[])), candidates are:
> /tmp/tmp.d(9):        tmp.each(alias fun, Range)(Range range) if
> (isInputRange!Range)


It could be made to work, though:

template isIterable(T) {
    enum isIterable = is(typeof((T t){foreach (e; t){}}));
}

template isRefIterable(T) {
    enum isRefIterable = is(typeof((T t){foreach (ref e; t){}}));
}

void each(alias fun, Range)(Range range)
    if (isInputRange!Range)
{
    while (!range.empty)
    {
        unaryFun!fun(range.front);
        range.popFront();
    }
}

void each(alias fun, Range)(Range range)
    if (!isInputRange!Range && isRefIterable!Range)
{
    foreach (ref e; range) {
        unaryFun!fun(e);
    }
}

void each(alias fun, Range)(Range range)
    if (!isInputRange!Range && isIterable!Range && !isRefIterable!Range)
{
    foreach (e; range) {
        unaryFun!fun(e);
    }
}

void main() {
    [0,1,2,3,4,5].parallel.each!writeln;
}

--
  Simen

On Thursday, 20 March 2014 at 15:02:46 UTC, Simen Kjærås wrote:
> On 2014-03-20 14:16, Andrea Fontana wrote:
>> On Thursday, 20 March 2014 at 12:32:49 UTC, w0rp wrote:
>>> // std.parallelism parallel function here.
>>> someRange.whatever.parallel(numberOfUnits).each!doSomething
>>
>> +1
>>
>> This works:
>> foreach(i; [0,1,2,3,4,5].parallel) i.writeln;
>>
>> This works:
>> [0,1,2,3,4,5].each!writeln;
>>
>> This won't compile:
>> [0,1,2,3,4,5].parallel.each!writeln;
>>
>> Error: template tmp.each cannot deduce function from argument types
>> !(writeln)(ParallelForeach!(int[])), candidates are:
>> /tmp/tmp.d(9):        tmp.each(alias fun, Range)(Range range) if
>> (isInputRange!Range)
>
>
> It could be made to work, though:
>
> template isIterable(T) {
>     enum isIterable = is(typeof((T t){foreach (e; t){}}));
> }
>
> template isRefIterable(T) {
>     enum isRefIterable = is(typeof((T t){foreach (ref e; t){}}));
> }
>
> void each(alias fun, Range)(Range range)
>     if (isInputRange!Range)
> {
>     while (!range.empty)
>     {
>         unaryFun!fun(range.front);
>         range.popFront();
>     }
> }
>
> void each(alias fun, Range)(Range range)
>     if (!isInputRange!Range && isRefIterable!Range)
> {
>     foreach (ref e; range) {
>         unaryFun!fun(e);
>     }
> }
>
> void each(alias fun, Range)(Range range)
>     if (!isInputRange!Range && isIterable!Range && !isRefIterable!Range)
> {
>     foreach (e; range) {
>         unaryFun!fun(e);
>     }
> }
>
> void main() {
>     [0,1,2,3,4,5].parallel.each!writeln;
> }
>
> --
>   Simen

I think that pull request should be updated.

Why isn't ParallelForEach implemented as Range, instead?

On Wednesday, 19 March 2014 at 15:06:40 UTC, Andrei Alexandrescu wrote:
> Pros and cons are already being discussed. Destroy!
>
> https://github.com/D-Programming-Language/phobos/pull/2024
>
>
> Andrei

User Ali Çehreli has posted in learn what I think is an interesting problem:
http://forum.dlang.org/thread/lgfmbf$v7c$1@digitalmars.com
//----
This is a somewhat common little exercise: Write a function that takes
the size of a diamond and produces a diamond of that size.

When printed, here is the output for size 11:

     *
    ***
   *****
  *******
 *********
***********
 *********
  *******
   *****
    ***
     *

What interesting, boring, efficient, slow, etc. ways are there?

Ali
//----

The reason I bring it up is that this usually leads to a competition of whoever thinks up the most creative/concise UFCS chain.

So here's what I'm thinking: Let's do this, but also using each/tee/tap/consume/walk (and others?)!

I think this "real world" scenario is a good bench for seeing the effects of mixing imperative-style statements into a functional-style wrapper.

Maybe the result will reveal that something is awesome, or that it is useless? That maybe something that looks like it works, is actually subtly buggy? That maybe each turns out to be useful past our wildest dreams?

Who knows?

monarch_dodra:

I have just given some solutions in D.learn.


> So here's what I'm thinking: Let's do this, but also using each/tee/tap/consume/walk (and others?)!
>
> I think this "real world" scenario is a good bench for seeing the effects of mixing imperative-style statements into a functional-style wrapper.

As I explained in a precedent post in this thread it's hard to design something if you don't try to use it, even something simple as a each(). So I approve such exercises.

Bye,
bearophile

On 3/20/2014 5:33 PM, bearophile wrote:
> As I explained in a precedent post in this thread it's hard to design something
> if you don't try to use it, even something simple as a each().

Right on.

On Friday, 21 March 2014 at 00:42:00 UTC, Walter Bright wrote:
> On 3/20/2014 5:33 PM, bearophile wrote:
>> As I explained in a precedent post in this thread it's hard to design something
>> if you don't try to use it, even something simple as a each().
>
> Right on.

So I toyed around a little. I played with mostly tee/tap and each. The first observation is that both can help make a "2D" ufcs chain, which is pretty nice. It makes more sense (IMO) to code it that way, than to use awkward "join", to concatenate the "2D" range into a continuous "1D" range, eg: join(lines, "\n");

First: "tee".

I wanted to try to experience the "multiple tees" approach. I made the code purposefully verbose, so as to better test it. Because I wanted to process each item once, and only once, I used the default policy of `Yes.pipeOnPop`. I did some tests with both "tee!pred" and "tee(range, output)", but they were functionally equivalent. Anyways, here was my first iteration:

//----
    size_t N = 5;
    chain(iota(0, N), only(N), iota(0, N).retro) //0 .. N .. 0
        .tee!((a)=>' '.repeat(N - a).write())()  //leading spaces
        .tee!((a)=>'*'.repeat(a).write())()      //first half of stars
        .tee!((a)=>'*'.write())()                //middle star
        .tee!((a)=>'*'.repeat(a).write())()      //second half
        .tee!((a)=>writeln())()                  //linefeed
        .reduce!"0"(0);                          //or consume/walk
//----

Surprise! It's wrong!

each "tee" triggers on the call to `popFront`. It does its job (calls pred(r.front)), and then push the "pop down the chain. What this means is that my "tee's" are actually executed right to left! Talk about counter intuitive.

So *this* is correct:
//----
    size_t N = 5;
    chain(iota(0, N), only(N), iota(0, N).retro) //0 .. N .. 0
        .tee!((a)=>writeln())()                  //linefeed
        .tee!((a)=>'*'.repeat(a).write())()      //second half
        .tee!((a)=>'*'.write())()                //middle star
        .tee!((a)=>'*'.repeat(a).write())()      //first half of stars
        .tee!((a)=>' '.repeat(N - a).write())()  //leading spaces
        .reduce!"0"(0);                          //or consume/walk/each()
//----

Odd!

Second:
Each tee pipes a call to front, and *then* calls front again when popped. Effectively, if you have *N* tee's in your range, you'll call "fun" roughly N*N/2 times. Not great.

This might be an unfair assesment of "tee", because I over-used it on purpose, but I think it *does* show that it's not scaling well, and that it is triggering in a confusing order.

Also:
//----
    size_t N = 5;
    foreach ( a ; chain(iota(0, N), only(N), iota(0, N).retro))
    {
        ' '.repeat(N - a).write();
        '*'.repeat(a).write();
        '*'.write();
        '*'.repeat(a).write();
        writeln();
    }
//----
Just saying. Why would I use "tee" when I have that?

But I think I'm using "tee" wrong: The idea is to "hook" it into a chain that actually does something. It shouldn't be the main "doer".

---------------------------------------------

What about each?

I wrote this:
//----
chain(iota(0, N), only(N), iota(0, N).retro)
    .each!(
        (a) => writeln(' '.repeat(N - a), '*'.repeat(a*2+1))
    )();
//----

I think this is a fair assesment of how "each" would be used? It looks nice, and is relatively short. But then I wrote this:

//----
foreach ( a ;
    chain(iota(0, N), only(N), iota(0, N).retro) )
{
    writeln(' '.repeat(N - a), '*'.repeat(a*2+1)))
}
//----

Hum... you still get the same functional initial declaration, but the "foreach" doesn't get in the way, while still keeping a clear functional/imperative distinction.

------------------------------------------

Si that's my initial assessment.

I'm not really sold on "each". I think it'll be abused by those that want a "1-liner" at all cost, leading to an image that "D is nothing but horrible 1 liners!", all that without providing any real functionality.

As for "tee": I'm convinced I didn't use it for its intended function, but I think it shows its function can easily be hijacked to do things it wasn't meant for.

monarch_dodra:

> chain(iota(0, N), only(N), iota(0, N).retro)
>     .each!(
>         (a) => writeln(' '.repeat(N - a), '*'.repeat(a*2+1))
>     )();
> //----

Something like this seems better (untested):

chain(N.iota, N.only, N.iota.retro)
.each!(a => '*'.replicate(a * 2 + 1).center(N).writeln);


> I think this is a fair assesment of how "each" would be used?

This UFCS chain is just 2 lines long, it's not the best usage example of each().


> I'm not really sold on "each".
> ...
> As for "tee": I'm convinced I didn't use it for its intended function, but I think it shows its function can easily be hijacked to do things it wasn't meant for.

You have used tee for something you think it's not very well designed, and you have given only one usage example of each(), and it's not the most common example.

So before taking any decision on the matter, more experiments and usage examples are necessary, where you have longer chains. If you have a longer chain:

items
.sort()
.group
.map!(g => g[1] / double(s.length))
.map!(p => -p * p.log2)
.sum
.each(...);

using a each() allows you to keep a nice column. If you use foreach your formatting and your logic has a hiccup, because you are mixing two different styles:

foreach (item; items
               .sort()
               .group
               .map!(g => g[1] / double(s.length))
               .map!(p => -p * p.log2)
               .sum) {
    // Do something imperative here.
}

But this is not a real example (it computes the entropy of items, and you usually don't need an each there), so more usage examples are needed. Taking a look at real world usages of foreach() in Scala could be useful.


> I think it'll be abused by those that want a "1-liner" at all cost, leading to an image that "D is nothing but horrible 1 liners!", all that without providing any real functionality.

I think each() is meant for longer chains as you see in F# languages.


Some possible usage examples for each():

import std.stdio, std.algorithm, std.string, std.exception, std.file;

void main() {
    string[][ubyte[]] an;
    foreach (w; "unixdict.txt".readText.splitLines)
        an[w.dup.representation.sort().release.assumeUnique] ~= w;
    immutable m = an.byValue.map!q{ a.length }.reduce!max;
    writefln("%(%s\n%)", an.byValue.filter!(ws => ws.length == m));
}



auto mode(T)(T[] items) pure nothrow {
    int[T] aa;
    foreach (item; items)
        aa[item]++;
    immutable m = aa.byValue.reduce!max;
    return aa.byKey.filter!(k => aa[k] == m);
}



double equalBirthdays(in uint nSharers, in uint groupSize,
                      in uint nRepetitions, ref Xorshift rng) {
    uint eq = 0;
    foreach (immutable j; 0 .. nRepetitions) {
        uint[365] group;
        foreach (immutable i; 0 .. groupSize)
            group[uniform(0, $, rng)]++;
        eq += group[].any!(c => c >= nSharers);
    }
    return (eq * 100.0) / nRepetitions;
}

Bye,
bearophile