On Thursday, 6 October 2016 at 09:17:08 UTC, pineapple wrote:
> On Wednesday, 5 October 2016 at 19:30:01 UTC, Jonathan M Davis wrote:
>>> Would just like to point out that this is design weirdness on Phobos' part - the library I've been writing does not have this problem.
>>
>> It doesn't even make conceptual sense for a static array to be a range, because you can't remove elements from it.
>>
>> - Jonathan M Davis
>
> Just because the static array itself isn't a range doesn't mean that it should be necessary to do unintuitive gymnastics with it just to pass it to functions like `sort`.

`sort` takes a range. [] gets a range from anything.
It's a convention, not gymnastics.

On Wednesday, 5 October 2016 at 19:30:01 UTC, Jonathan M Davis wrote:
> It doesn't even make conceptual sense for a static array to be a range, because you can't remove elements from it.
>
> - Jonathan M Davis

Interestingly enough, I found that using .each() actually compiles without the [] but (as expected) creates a copy... So, these output different values:

  thing.each!((ref x) => writeln(&x));
  thing[].each!((ref x) => writeln(&x));

Should there me a more consistent behavior here? Even if the behavior is pretty undesired, why can the compiler consider it a range here but not .sort()?

On 10/06/2016 09:54 PM, TheGag96 wrote:
> Interestingly enough, I found that using .each() actually compiles
> without the []
[...]
> why can the compiler consider it a range here but not
> .sort()?

each is not restricted to ranges. It accepts other `foreach`-ables, too. The documentation says that it "also supports opApply-based iterators", but it's really anything that foreach accepts.

Relevant piece of the source:
https://github.com/dlang/phobos/blob/08c587ead2156c85c30a2bbc18028b5967010682/std/algorithm/iteration.d#L913-L914

On Thursday, 6 October 2016 at 09:23:19 UTC, pineapple wrote:
> On Thursday, 6 October 2016 at 09:17:08 UTC, pineapple wrote:
>> On Wednesday, 5 October 2016 at 19:30:01 UTC, Jonathan M Davis wrote:
>>>> Would just like to point out that this is design weirdness on Phobos' part - the library I've been writing does not have this problem.
>>>
>>> It doesn't even make conceptual sense for a static array to be a range, because you can't remove elements from it.
>>>
>>> - Jonathan M Davis
>>
>> Just because the static array itself isn't a range doesn't mean that it should be necessary to do unintuitive gymnastics with it just to pass it to functions like `sort`.
>
> I mean, people post here how often asking why static or dynamic arrays aren't being accepted by Phobos' range functions in their code?
>
> Maybe Phobos really ought to consider another approach. Accepting things that are _valid_ as ranges and not only things that are ranges themselves has proven to be an effective strategy in mach.

+1000

On Thursday, 6 October 2016 at 20:11:17 UTC, ag0aep6g wrote:
> On 10/06/2016 09:54 PM, TheGag96 wrote:
>> Interestingly enough, I found that using .each() actually compiles
>> without the []
> [...]
>> why can the compiler consider it a range here but not
>> .sort()?
>
> each is not restricted to ranges. It accepts other `foreach`-ables, too. The documentation says that it "also supports opApply-based iterators", but it's really anything that foreach accepts.
>   [snip]
> 
Thanks! Explains some things. I knew each! was callable in different circumstances than other functional operations, but hadn't quite figured it out. Looks like reduce! and fold! also take iterables.

There also appears to be a distinction between the iterator and range cases when a ref parameter is used. As it iterator, each! won't modify the reference. Example:

void main()
{
    import std.algorithm : each;

    int[] dynamicArray = [1, 2, 3, 4, 5];
    int[5] staticArray = [1, 2, 3, 4, 5];

    dynamicArray.each!((ref x) => x++);
    assert(dynamicArray == [2, 3, 4, 5, 6]); // modified

    staticArray.each!((ref x) => x++);
    assert(staticArray == [1, 2, 3, 4, 5]);  // not modified

    staticArray[].each!((ref x) => x++);
    assert(staticArray == [2, 3, 4, 5, 6]);  // modified
}

This distinction is a bit on the nuanced side. Is it behaving as it should?

--Jon

On Saturday, 8 October 2016 at 21:14:43 UTC, Jon Degenhardt wrote:
> This distinction is a bit on the nuanced side. Is it behaving as it should?
>
> --Jon

I think so? It's not being modified in the second case because the array is being passed by value... "x" there is a reference to an element of the copy created to be passed to each(). I assume there's a good reason why ranges in general are passed by value into these functions -- except in this one case, the stuff inside range types copied when passed by value won't be whole arrays, I'm guessing.

On Monday, October 10, 2016 16:29:41 TheGag96 via Digitalmars-d-learn wrote:
> On Saturday, 8 October 2016 at 21:14:43 UTC, Jon Degenhardt wrote:
> > This distinction is a bit on the nuanced side. Is it behaving as it should?
> >
> > --Jon
>
> I think so? It's not being modified in the second case because the array is being passed by value... "x" there is a reference to an element of the copy created to be passed to each(). I assume there's a good reason why ranges in general are passed by value into these functions -- except in this one case, the stuff inside range types copied when passed by value won't be whole arrays, I'm guessing.

Whether it's by value depends entirely on the type of the range. They're passed around, and copying them has whatever semantics it has. In most cases, it copies the state of the range but doesn't copy all of the elements (e.g. that's what happens with a dynamic array, since it gets sliced). But if a range is a class, then it's definitely a reference type.  The only way to properly save the state of a range is to call save.

But passing by ref would make no sense at all with input ranges. It would completely kill chaining them. Almost all range-based functions return rvalues.

- Jonathan M Davis

On Monday, 10 October 2016 at 16:46:55 UTC, Jonathan M Davis wrote:
> On Monday, October 10, 2016 16:29:41 TheGag96 via Digitalmars-d-learn wrote:
>> On Saturday, 8 October 2016 at 21:14:43 UTC, Jon Degenhardt wrote:
>> > This distinction is a bit on the nuanced side. Is it behaving as it should?
>> >
>> > --Jon
>>
>> I think so? It's not being modified in the second case because the array is being passed by value... "x" there is a reference to an element of the copy created to be passed to each(). I assume there's a good reason why ranges in general are passed by value into these functions -- except in this one case, the stuff inside range types copied when passed by value won't be whole arrays, I'm guessing.
>
> Whether it's by value depends entirely on the type of the range. They're passed around, and copying them has whatever semantics it has. In most cases, it copies the state of the range but doesn't copy all of the elements (e.g. that's what happens with a dynamic array, since it gets sliced). But if a range is a class, then it's definitely a reference type.  The only way to properly save the state of a range is to call save.
>
> But passing by ref would make no sense at all with input ranges. It would completely kill chaining them. Almost all range-based functions return rvalues.
>
> - Jonathan M Davis

The example I gave uses ref parameters. On the surface it would seem reasonable to that passing a static array by ref would allow it to be modified, without having to slice it first. The documentation says:

    // If the range supports it, the value can be mutated in place
   arr.each!((ref n) => n++);
   assert(arr == [1, 2, 3, 4, 5]);

but, 'arr' is a dynamic array, so technically it's not describing a static array (the opApply case).

Expanding the example, using foreach with ref parameters will modify the static array in place, without slicing it. I would have expected each! with a ref parameter to behave the same.

At a minimum this could be better documented, but it may also be a bug.

Example:

T increment(T)(ref T x) { return x++; }

void main()
{
    import std.algorithm : each;

    int[] dynamicArray = [1, 2, 3, 4, 5];
    int[5] staticArray = [1, 2, 3, 4, 5];

    dynamicArray.each!(x => x++);             // Dynamic array by value
    assert(dynamicArray == [1, 2, 3, 4, 5]);  // ==> Not modified

    dynamicArray.each!((ref x) => x++);       // Dynamic array by ref
    assert(dynamicArray == [2, 3, 4, 5, 6]);  // ==> Modified

    staticArray[].each!((ref x) => x++);      // Slice of static array, by ref
    assert(staticArray == [2, 3, 4, 5, 6]);   // ==> Modified

    staticArray.each!((ref x) => x++);        // Static array by ref
    assert(staticArray == [2, 3, 4, 5, 6]);   // ==> Not Modified

    /* Similar to above, using foreach and ref params. */
    foreach (ref x; dynamicArray) x.increment;
    assert(dynamicArray == [3, 4, 5, 6, 7]);  // Dynamic array => Modified

    foreach (ref x; staticArray[]) x.increment;
    assert(staticArray == [3, 4, 5, 6, 7]);   // Static array slice => Modified

    foreach (ref x; staticArray) x.increment;
    assert(staticArray == [4, 5, 6, 7, 8]);   // Static array => Modified
}

On 10/11/2016 06:24 AM, Jon Degenhardt wrote:
> The example I gave uses ref parameters. On the surface it would seem
> reasonable to that passing a static array by ref would allow it to be
> modified, without having to slice it first.

Your ref parameters are only for the per-element operations. You're not passing the array as a whole by reference. And you can't, because `each` itself takes the whole range by copy.

So, the by-ref increments themselves do work, but they're applied to a copy of your original static array.

Question is, should `each`

1) take all inputs (ranges, arrays, other foreachables) by reference, or
2) take some inputs (like static arrays) by reference, or
3) take all inputs by value (current behavior)?

#1 would break code. Would probably need some deprecating and name shuffling to be acceptable. Would also need to make sure that this is actually the most desirable behavior.

#2 would probably create surprising corner cases. I don't think we can tell for sure if a range needs to be passed by reference in order to see updates to its elements. I'd be against this.

#3 may be a little surprising in how it doesn't affect value types (like static arrays). However, before switching to #1, you'd need to make sure that that one doesn't have worse corner cases. I don't see any deal breakers, but that doesn't mean they're not there ;)

You also have to see if changing to #1 is worth the effort. It would be an effort not only for the implementer, but also for the users who have to update all their code.

On Tuesday, 11 October 2016 at 18:18:41 UTC, ag0aep6g wrote:
> On 10/11/2016 06:24 AM, Jon Degenhardt wrote:
>> The example I gave uses ref parameters. On the surface it would seem
>> reasonable to that passing a static array by ref would allow it to be
>> modified, without having to slice it first.
>
> Your ref parameters are only for the per-element operations. You're not passing the array as a whole by reference. And you can't, because `each` itself takes the whole range by copy.
>
> So, the by-ref increments themselves do work, but they're applied to a copy of your original static array.
>
I see. Thanks for the explanation. I wasn't thinking it through properly. Also, I guess I had assumed that the intent was that each! be able to modify the elements, and therefore the whole array it would be pass by reference, but didn't consider it properly.

I'm not going to make any suggestions about whether the behavior should be changed. At some point when I get a bit of time I'll try to submit a documentation change to make the current behavior clearer.

--Jon