View mode: basic / threaded / horizontal-split · Log in · Help
September 09, 2007
Empowering foreach with multiple aggregates
There are some things that iterators can do that foreach can't. I
/don't/ want to add iterators to D. Instead, I'd like to see foreach
enhanced, so that it can do the things it currently can't.

Here's my first example.

void tweak(int[] dst, int[] src)
{
    assert(dst.length == src.length);
    for (int i=0; i<dst.length; ++i)
    {
        dst[i] = src[i] + 1;
    }
}

How much nicer it would be to be able to say

void tweak(int[] dst, int[] src)
{
    foreach (d; dst; s; src)
    {
        d = s + 1;
    }
}

The ability to walk through two collections simultaneously is such a
common thing to want to do that I'm suprised there's no language
support for it. Of course, it needs some extra code if dst.length and
src.length are /not/ equal, but that would also be true of the
iterator way. Using the foreach way, you'd end up with something like:

void tweak(int[] dst, int[] src)
{
    int len = dst.length < src.length ? dst.length : src.length;
    foreach (d; dst[0..len]; s; src[0..len])
    {
        d = s + 1;
    }
    if (dst.length > src.length) foreach(d; dst[len..$])
    {
        d = 1;
    }
}

There might be some array operations planned that will do something
similar, but the point is that not all collections are arrays. You
might want to iterate through any arbitrary collections, and that's
where iterators show their strength. Why not let foreach have that
additional strength?
September 09, 2007
Re: Empowering foreach with multiple aggregates
Janice Caron wrote:
> [snip]
> 
>  void tweak(int[] dst, int[] src)
>  {
>      foreach (d; dst; s; src)
>      {
>          d = s + 1;
>      }
>  }

I believe Walter and Andrei have talked about this before.  The syntax
they proposed at the time was something like:

foreach(ref d ; dst)(s ; src)
{
   d = s + 1
}

I think the main problem with doing this is related to how foreach is
actually implemented.  If we assume that dst and src are actually user
types, then this loop:

foreach(ref T d ; dst)
 d = 1;

Becomes:

dst.opApply((ref T d) {
 d = 1;
 return 0;
});

This is a problem if you want to do iteration over multiple aggregates;
you'd essentially need to call two functions simultaneously, then have
those call into a third function passing half of the arguments each.

Adding this to D would probably require completely rethinking how custom
aggregates are created, which isn't necessarily a bad thing, but
certainly a difficult thing.

	-- Daniel
September 09, 2007
Re: Empowering foreach with multiple aggregates
> foreach(ref T d ; dst)
>   d = 1;
>
> Becomes:
>
> dst.opApply((ref T d) {
>   d = 1;
>   return 0;
> });
>
> This is a problem if you want to do iteration over multiple aggregates;

Yes, I see. I had not realised that. Thanks.

I guess you could do it by creating some sort of hybrid collection -
but I don't know if that could be done at a language level (and if it
can't there'd be no point). By this I mean, given a collection C c,
and collection D d, where (c.length == d.length), generate a third
collection A a such that a[n] is a Tuple!(ref c[n], d[n]). Then you
could do:

foreach(a; AggregatePair!(dst, src))
{
    a[0] = a[1] + 1;
}

But I don't have enough experience with Tuples to know if that's even
possible. I mean, AggregatePair!'s opApply() method would have to be
really clever.



> you'd essentially need to call two functions simultaneously, then have
> those call into a third function passing half of the arguments each.

Could this AggregatePair! idea do that? Or is that just too not possible?


> Adding this to D would probably require completely rethinking how custom
> aggregates are created, which isn't necessarily a bad thing, but
> certainly a difficult thing.

Oh well!
C'est la vie.
September 09, 2007
Re: Empowering foreach with multiple aggregates
This works, right now, but it requires a kind of abuse of the foreach
callback system. :D

xt4100 ~ $ cat foo2.d && gdc foo2.d -o foo2 && ./foo2
import std.stdio;

struct pairiter(T, U) {
 T[] first; U[] second;
 int opApply(int delegate(ref T, ref U) dg) {
   for (int i=0; i<first.length; ++i) {
     auto res=dg(first[i], second[i]);
     if (res) return res;
   }
   return 0;
 }
}

pairiter!(T, U) pair(T, U)(T[] a, U[] b) {
 assert(a.length==b.length);
 return pairiter!(T, U)(a, b);
}

void main() {
 char[] dst=new char[6]; char[] src="foobar";
 foreach (ref d, ref s; pair(dst, src)) {
   d=s+1;
 }
 writefln(dst);
}
gppcbs


Here's a version that works with arbitrary foreachable things (uses
scrapple.tools.stackthreads):

xt4100 ~ $ cat foo2.d && rebuild foo2 && ./foo2
module foo2;
import std.stdio;
import tools.stackthreads;

template isArray(T) { const bool isArray=false; }
template isArray(T: T[]) { const bool isArray=true; }

import std.traits;
template IT(T) { // itertype
 static if(isArray!(T)) alias typeof(T[0]) IT;
 else alias ParameterTypeTuple!(
   ParameterTypeTuple!(
     typeof(&T.init.opApply)
   )[0]
 )[$-1] IT;
}

struct pairiter(T, U) {
 T first; U second;
 int opApply(int delegate(ref IT!(T), ref IT!(U)) dg) {
   auto gen=new class Generator!(IT!(T)*) { void generate() {
     foreach (ref elem; first) yield (&elem);
   } };
   foreach (elem2; second) {
     auto elem1=gen();
     auto res=dg(*elem1, elem2);
     if (res) return res;
   }
   return 0;
 }
}

pairiter!(T, U) pair(T, U)(T a, U b) {
 return pairiter!(T, U)(a, b);
}

struct strng {
 string st;
 int opApply(int delegate(ref char) dg) {
   foreach (ref ch; st) {
     auto res=dg(ch); if (res) return res;
   }
   return 0;
 }
}

void main() {
 char[] dst=new char[6]; char[] src="foobar";
 foreach (ref d, ref s; pair(dst, strng(src))) {
   d=s+1;
 }
 writefln(dst);
}
gppcbs
September 09, 2007
Re: Empowering foreach with multiple aggregates
Wow! You're good!



>   static if(isArray!(T)) alias typeof(T[0]) IT;
>   else alias ParameterTypeTuple!(
>     ParameterTypeTuple!(
>       typeof(&T.init.opApply)
>     )[0]
>   )[$-1] IT;

Sorry to jump threads here, but this is a really good example of why
alias dst=src; would be a good thing. It took me a long time and a lot
of counting parantheses to figure out that IT was being defined!

Anyway - that is excellent.

I guess the next question would be, can this trick be built into the
language in an easy-to-use way. Or in Phobos?
September 09, 2007
Re: Empowering foreach with multiple aggregates
Janice Caron wrote:
> Wow! You're good!

He's downs.

>>   static if(isArray!(T)) alias typeof(T[0]) IT;
>>   else alias ParameterTypeTuple!(
>>     ParameterTypeTuple!(
>>       typeof(&T.init.opApply)
>>     )[0]
>>   )[$-1] IT;
> 
> Sorry to jump threads here, but this is a really good example of why
> alias dst=src; would be a good thing. It took me a long time and a lot
> of counting parantheses to figure out that IT was being defined!
> 
> Anyway - that is excellent.
> 
> I guess the next question would be, can this trick be built into the
> language in an easy-to-use way. Or in Phobos?

I suspect not unless Walter adds stackthreads/coroutines to it first.

Then again, since it's so clean to use:

 foreach (ref d, ref s; pair(dst, strng(src)))

It seems fine as a library method.

P.S. to downs: did you have that one lying around, or is this old work? :P

	-- Daniel
September 09, 2007
Re: Empowering foreach with multiple aggregates
Daniel Keep wrote:
> 
> Janice Caron wrote:
>> Wow! You're good!
> 
> He's downs.
> 
Hehe .. thanks :)

>>>   static if(isArray!(T)) alias typeof(T[0]) IT;
>>>   else alias ParameterTypeTuple!(
>>>     ParameterTypeTuple!(
>>>       typeof(&T.init.opApply)
>>>     )[0]
>>>   )[$-1] IT;
>> Sorry to jump threads here, but this is a really good example of why
>> alias dst=src; would be a good thing. It took me a long time and a lot
>> of counting parantheses to figure out that IT was being defined!
>>
>> Anyway - that is excellent.
>>
>> I guess the next question would be, can this trick be built into the
>> language in an easy-to-use way. Or in Phobos?
> 
> I suspect not unless Walter adds stackthreads/coroutines to it first.
> 
> Then again, since it's so clean to use:
> 
>   foreach (ref d, ref s; pair(dst, strng(src)))
> 
> It seems fine as a library method.
> 
> P.S. to downs: did you have that one lying around, or is this old work? :P
> 
> 	-- Daniel

The tools.stackthreads impl is relatively new (I wrote it a few days ago
with lots of help from #d), though there's an older (and better) one in
the stackthreads lib on http://assertfalse.com/projects.shtml (swhere I
got the idea).
The pair function was whipped up in about half an hour, most of it spent
hunting template bugs. I'm familiar with the pattern of functions that
return structs with extended behavior, because I use it a lot in
tools.iter. Also, I was looking for a good application of
tools.stackthreads for a while now :)

The problem with pair is that it's probably not extensible to triplets,
and there's no stackthreads in phobos. Maybe Walter could add them?
..
Hah. I wish. :)
--downs
September 09, 2007
Re: Empowering foreach with multiple aggregates
Reply to Daniel,

> This is a problem if you want to do iteration over multiple
> aggregates; you'd essentially need to call two functions
> simultaneously, then have those call into a third function passing
> half of the arguments each.
> 

stack threads?

> 
> -- Daniel
>
Top | Discussion index | About this forum | D home