On 6/14/17 11:42 AM, Luís Marques wrote:
> On Wednesday, 14 June 2017 at 14:35:32 UTC, Steven Schveighoffer wrote:
>> What I'd rather see is a better mechanism for ranges (or things that
>> provide ranges) to hook the foreach syntax to allow better control and
>> power from ranges without using traditional opApply.
>>
>> For example, if we had:
>>
>> foreach(size_t a, b; x)
>>
>> translates to:
>>
>> for(auto _context = x.opApply!(size_t, void); !context.empty;
>> _context.popFront)
>> {
>>    auto (a, b) = _context.front; // the way it works now with tuples,
>> don't think there's a real syntax for this.
>>    ...
>> }
>
> An approach like that seems fine to me. In any case, why do it in
> exclusion of a foreach-generated index? Imagine that I define an input
> range. Can we assume that range.front is *conceptually* equivalent to
> range[0], even if the range doesn't define random access?  Because if we
> can, then, if the range doesn't define the new opApply or whatever, why
> not allow foreach to generate the index?

That's when you use range.enumerate ;)

>> Yes, foreach could be augmented to do that. However, this is
>> essentially a way to implement enumerate without the .enumerate, and
>> nothing more. It's not a lot of added benefit IMO.
>
> Consider this case. You create a prototype where foo is a slice. Then
> you start improving the code and foo becomes a range. In your code you
> have several places that use foo with foreach, and several places where
> it is used directly. If you define the new foo as newRangeFoo.enumerate,
> then the several foreach work automatically, but the other uses don't
> because they have to be changed to foo.value; if you do it the other way
> around, the direct uses work automatically but the foreach all have to
> be changed from foo to foo.enumerate. Thus you have lost a lot of the
> code plasticity. If the foreach statement generated the indices then it
> would work automatically.

This is inherent of arrays working differently than ranges, but being considered ranges via std.array. People consider slices to be ranges, but they really aren't as far as the compiler is concerned.

I don't know if there is room to allow range-defined indexes and foreach-defined indexes. foreach(a, b; someRange) has to do one or the other. Otherwise, changes to how someRange operates can make this a completely different operation.

> This could also be solved by the range defining your new version of
> opApply and foreach understanding that. Assuming that we can wrap old
> ranges (e.g. from a library you don't want to change) with that opApply,
> I would be fine with that solution as long as this is something that
> actually goes forward. I'm just afraid this is something that will
> linger because it's a more difficult design decision, while having
> foreach support index generation seems like a more self contained
> solution that could be agreed upon and implemented quickly.

I think opApply is kind of a forgotten piece of the language since ranges have been introduced. We could resurrect it quite easily this way, as opApply with template parameters that return another range does not conflict with opApply that takes a delegate. Easy to see if one works and if not try something else. The huge benefit of this mechanism is to allow complete control to the type over how it should be iterated, and not have to fight the compiler.

-Steve

On Wednesday, 14 June 2017 at 16:17:50 UTC, Steven Schveighoffer wrote:
> I think opApply is kind of a forgotten piece of the language since ranges have been introduced. We could resurrect it quite easily this way, as opApply with template parameters that return another range does not conflict with opApply that takes a delegate. Easy to see if one works and if not try something else. The huge benefit of this mechanism is to allow complete control to the type over how it should be iterated, and not have to fight the compiler.

You say "quite easily". I'm afraid this will linger. Please let's not drop the ball on this, whatever the chosen solution is.

> This is inherent of arrays working differently than ranges, but being considered ranges via std.array. People consider slices to be ranges, but they really aren't as far as the compiler is concerned.

Sorry, I'm not seeing the connection. foreach is an abstract iteration statement and we are just discussion if it is reasonable for foreach to keep track of the iteration number.

> I don't know if there is room to allow range-defined indexes and foreach-defined indexes. foreach(a, b; someRange) has to do one or the other. Otherwise, changes to how someRange operates can make this a completely different operation.

Seems like exactly what we want. If you add a customization to someRange to change the iteration behavior then each relevant foreach automatically switches from the default to the requested behavior. That's exactly the point of programming to abstract interfaces.

On 6/14/17 12:35 PM, Luís Marques wrote:
> On Wednesday, 14 June 2017 at 16:17:50 UTC, Steven Schveighoffer wrote:
>> I don't know if there is room to allow range-defined indexes and
>> foreach-defined indexes. foreach(a, b; someRange) has to do one or the
>> other. Otherwise, changes to how someRange operates can make this a
>> completely different operation.
>
> Seems like exactly what we want. If you add a customization to someRange
> to change the iteration behavior then each relevant foreach
> automatically switches from the default to the requested behavior.
> That's exactly the point of programming to abstract interfaces.

For example:

foreach(i, v; hashmap) => i is counter, v is value

Later hashmap adds support for iterating key and value. Now i is key, v is value. Code means something completely different.

Compare with

foreach(i, v; hashmap.enumerate)

Intent is clear from the code.

-Steve

On 06/14/2017 12:22 PM, Steven Schveighoffer wrote:

> foreach(i, v; hashmap) => i is counter, v is value
>
> Later hashmap adds support for iterating key and value. Now i is key, v
> is value. Code means something completely different.
>
> Compare with
>
> foreach(i, v; hashmap.enumerate)
>
> Intent is clear from the code.
>
> -Steve

Then, perhaps we're arguing in favor of

* writing .enumerate even for slices (implying that automatic indexing for them has been a historical artifact and code that wants to be portable should always write .enumerate)

* making sure that enumerate() on arrays don't bring extra cost

Ali

On Wednesday, 14 June 2017 at 19:22:24 UTC, Steven Schveighoffer wrote:
> For example:
>
> foreach(i, v; hashmap) => i is counter, v is value
>
> Later hashmap adds support for iterating key and value. Now i is key, v is value. Code means something completely different.

If we take a step back, I think we are discussing the generalization of foreach. Let's assume we want to preserve the current foreach behavior where associative types iterate with key and value pairs. If the "hashmap" in your example was supposed to be an associative type then `i` wasn't a counter, it was the key already. If "hashmap" didn't define the iteration key (using a new kind of opApply or whatever) then it wasn't an associative type, and therefore your change wouldn't be valid.

So, we just have to be careful that when foreach(i, v; foo) is introduced the relevant types in phobos also implement the new behavior, so that this situation doesn't arise afterwards..

On Wednesday, 14 June 2017 at 22:02:30 UTC, Ali Çehreli wrote:

> Then, perhaps we're arguing in favor of
>
> * writing .enumerate even for slices (implying that automatic indexing for them has been a historical artifact and code that wants to be portable should always write .enumerate)
>
> * making sure that enumerate() on arrays don't bring extra cost
>
> Ali

I like this proposal.

Is there a reason you don't introduce .enumerate in the section of your book "The counter is automatic only for arrays"?

On 06/14/2017 03:48 PM, bachmeier wrote:
> On Wednesday, 14 June 2017 at 22:02:30 UTC, Ali Çehreli wrote:
>
>> Then, perhaps we're arguing in favor of
>>
>> * writing .enumerate even for slices (implying that automatic indexing
>> for them has been a historical artifact and code that wants to be
>> portable should always write .enumerate)
>>
>> * making sure that enumerate() on arrays don't bring extra cost
>>
>> Ali
>
> I like this proposal.
>
> Is there a reason you don't introduce .enumerate in the section of your
> book "The counter is automatic only for arrays"?

No reason other than ranges appear later in the book. But I took a note; I will add your suggestion soon.

Otherwise, .enumerate is mentioned here:


http://ddili.org/ders/d.en/foreach_opapply.html#ix_foreach_opapply.enumerate,%20std.range

Ali

On 6/14/17 6:46 PM, Luís Marques wrote:
> On Wednesday, 14 June 2017 at 19:22:24 UTC, Steven Schveighoffer wrote:
>> For example:
>>
>> foreach(i, v; hashmap) => i is counter, v is value
>>
>> Later hashmap adds support for iterating key and value. Now i is key,
>> v is value. Code means something completely different.
>
> If we take a step back, I think we are discussing the generalization of
> foreach. Let's assume we want to preserve the current foreach behavior
> where associative types iterate with key and value pairs. If the
> "hashmap" in your example was supposed to be an associative type then
> `i` wasn't a counter, it was the key already. If "hashmap" didn't define
> the iteration key (using a new kind of opApply or whatever) then it
> wasn't an associative type, and therefore your change wouldn't be valid.
>
> So, we just have to be careful that when foreach(i, v; foo) is
> introduced the relevant types in phobos also implement the new behavior,
> so that this situation doesn't arise afterwards..

The point I'm making is that we have mechanisms for a type to define what to do when you foreach with 2 parameters per loop iteration.

What you are proposing is to allow the compiler to define what to do in the case where the type doesn't. But what happens if it then defines the behavior later? code that was written expecting the first item to be an arbitrary index is now hijacked by the type to mean something else. This can't happen with arrays, because the language says so. The language doesn't have control over an arbitrary struct.

It could be any type, I just used a hypothetical hashmap that didn't already have key/value iteration. Probably an unlikely scenario, but it was what popped into my head as a demonstration.

And what if you DID want to iterate just an incrementing index with the values when the type defines 2-parameter iteration otherwise? That mechanism isn't available to you without some effort. So again, we still need an enumerate to do what you want.

-Steve

On 6/14/17 6:02 PM, Ali Çehreli wrote:
> On 06/14/2017 12:22 PM, Steven Schveighoffer wrote:
>
>> foreach(i, v; hashmap) => i is counter, v is value
>>
>> Later hashmap adds support for iterating key and value. Now i is key, v
>> is value. Code means something completely different.
>>
>> Compare with
>>
>> foreach(i, v; hashmap.enumerate)
>>
>> Intent is clear from the code.
>>
>> -Steve
>
> Then, perhaps we're arguing in favor of
>
> * writing .enumerate even for slices (implying that automatic indexing
> for them has been a historical artifact and code that wants to be
> portable should always write .enumerate)
>
> * making sure that enumerate() on arrays don't bring extra cost

I would say making enumerate on *any* range shouldn't bring any extra cost over how foreach works on an array.

One idea I had but haven't thought it through completely is a way to mark some parameter to foreach as always referencing the actual index, so you aren't making unnecessary copies for the loop. When you foreach a range, a copy is made just for the loop, and *then* a copy is made each loop iteration for the element itself.

Maybe tagging a parameter in foreach as lazy means "always use the range element".

-Steve