On Thursday, October 24, 2013 12:14:27 Timothee Cour wrote:
> so far you've emphasized on the fact popFront should take care of all side effects and front should be logically pure and fast to compute, but you haven't suggested any alternative for what I'm trying to achieve.
> 
> In what I propose it's generic (works in all cases) and simple (just add
> ".cacheFront" after a lambda with side effect):
> 
> auto test(T)(T elements) if(isInputRange!T){
> return elements
> .map!( (a) {preaction(a); auto b=transform(a); postaction(a,b); return b;})
> .cacheFront
> .filter!foo; //or joiner or....
> }
> 
> So, how do you achieve the above without using cacheFront, without creating
> a _custom_ range or writing a lot of code ?
> I don't see how using foreach is a good idea (ForEachType!=ElementType, it
> doesn't reuse phobos functionality and doesn't play well with phobos range
> functions)

If you don't want to do this with foreach, then creating a custom range is exactly what you should be doing, and the logic for the pre and post operations should go in popFront. Having side effects in front is just begging for problems, and it's not at all what map is designed for. And honestly, if your popFront isn't pure as well, and the side effects that you're doing affect something outside of the range, then I would suggest that you not use ranges for what you're doing, because that violates their functional nature. You can do it if you want to, but that's not at all what they're designed for.

It has been proposed before that a function be added to Phobos which did an operation when iterating over an element, in which case that operation would go in popFront, and front would just return the wrapped front. That would probably get you what you want and be more generally applicable than cacheFront.

- Jonathan M Davis

On Thursday, October 24, 2013 21:10:58 deadalnix wrote:
> On Thursday, 24 October 2013 at 17:57:54 UTC, Jonathan M Davis
> 
> wrote:
> > Now, that being said, I'd argue that map should probably be
> > changed so that it
> > calls its function on popFront rather than front (though that
> > has the downside
> > of requiring that it then hold the current value, which it
> > doesn't currently
> > have to do), but regardless, you're using map for something
> > other than what it
> > was designed to do.
> 
> That would break random access.

No, it wouldn't. It would just require that the function be called directly in the case of random access, so you then have the problem with randomly accessed values that you really can't implement map in a way that avoids a calling its function every time you randomly access a variable. It's one more reason to consider not accessing the same element in a range multiple times, but as it's pretty much a guarantee that at least some range-based code will access front multiple times, it's not a good idea for the range to be designed such that it can't reasonably handle having each element accessed multiple times - which in the case of map would imply that the programmer using map should make sure that the function they give it can handle being called multiple times per element without changing the semantics of the code. It also would imply that doing something like

range.map!(a => new Aboject(a))

like you do in another post should be done with great care if at all, as there is a high risk that you will end up creating multiple objects per element as you traverse the range unless you specifically make sure that use foreach on it or immediately convert it to an array.

- Jonathan M Davis

On Thursday, October 24, 2013 21:09:25 deadalnix wrote:
> range.map!(a => new Aboject(a))
> 
> And here you go. The delegate passed is pure.

If you don't care about each call to front returning a completely new object, then that works, but it pretty much violates the idea that front is supposed to be emulating a member variable. front should be logically const and logically pure because those are the semantics of reading a member variable. And as front is generally treated just like member variable (including getting called multiple times between calls to popFront), doing anything with front that does not work when multiple calls to front are made between calls to popFront is going to be buggy.

Now, it's a good idea to try and minimize the number of calls to front inside a function in case the cost is higher than would be desirable (e.g. if map's lambda is not cheap), but the semantics of front are expected to be essentially those of a member variable, and that means that it's very precarious if multiple calls to front do not return the exact same object or if they have any side effects.

- Jonathan M Davis

On Thursday, 24 October 2013 at 17:52:45 UTC, Jonathan M Davis wrote:
> On Thursday, October 24, 2013 17:55:22 Chris wrote:
>> On Thursday, 24 October 2013 at 15:40:22 UTC, Dicebot wrote:
>> > On Thursday, 24 October 2013 at 15:36:59 UTC, Chris wrote:
>> >> How do you define side effects? After all ranges manipulate,
>> >> reformat or restructure data.
>> > 
>> > They should do it in popFront. I'd consider any `front` that
>> > does anything but accessing cached value suspicious.
>> 
>> So code like
>> 
>> auto front() {
>> doSomething();
>> return range[0];
>> }
>> 
>> should go into
>> 
>> void popFront() {
>> doSomething();
>> range = range[1..$];
>> }
>
> Yes. front _will_ be called multiple times by many range-based algorithms.
> There are no guarantees whatsoever that front will not be called more than
> once and relying on it being called only once between calls to popFront is
> just plain wrong.
>
> front is the logical equivalent to a public member variable and should be
> treated as such. In general, putting additional work inside of front is just
> asking for trouble. And if you do, it needs to be with the understanding
> that front stands a good chance of being called multiple times per call to
> popFront.
>
> It might make sense to add some debug stuff in there which has side effects
> (like printing something out), but the code's normal semantics should not
> rely on any kind of side effects on front. There should be no logical
> difference if you call front one time or twenty times after a single call to
> popFront.
>
> - Jonathan M Davis

Well, I just had a weird bug due to some stuff being in front. I moved everything into popFront and it worked as expected.

It was a linked list where PREVIOUS and CURRENT are checked and changed if certain conditions are met. Suddenly I got weird behaviour and PREVIOUS and CURRENT where the other way around or an old PREVIOUS showed up later. I guess I've learned my lesson.

On Thursday, 24 October 2013 at 19:50:44 UTC, Jonathan M Davis wrote:
> On Thursday, October 24, 2013 21:10:58 deadalnix wrote:
>> On Thursday, 24 October 2013 at 17:57:54 UTC, Jonathan M Davis
>> 
>> wrote:
>> > Now, that being said, I'd argue that map should probably be
>> > changed so that it
>> > calls its function on popFront rather than front (though that
>> > has the downside
>> > of requiring that it then hold the current value, which it
>> > doesn't currently
>> > have to do), but regardless, you're using map for something
>> > other than what it
>> > was designed to do.
>> 
>> That would break random access.
>
> No, it wouldn't. It would just require that the function be called directly in
> the case of random access, so you then have the problem with randomly accessed
> values that you really can't implement map in a way that avoids a calling its
> function every time you randomly access a variable. It's one more reason to
> consider not accessing the same element in a range multiple times, but as it's
> pretty much a guarantee that at least some range-based code will access front
> multiple times, it's not a good idea for the range to be designed such that it
> can't reasonably handle having each element accessed multiple times - which in
> the case of map would imply that the programmer using map should make sure
> that the function they give it can handle being called multiple times per
> element without changing the semantics of the code. It also would imply that
> doing something like
>
> range.map!(a => new Aboject(a))
>
> like you do in another post should be done with great care if at all, as there
> is a high risk that you will end up creating multiple objects per element as
> you traverse the range unless you specifically make sure that use foreach on it
> or immediately convert it to an array.
>
> - Jonathan M Davis

OK let me restate : it would give random access a different semantic than in order access. Which is undesirable.

On Friday, October 25, 2013 00:05:26 deadalnix wrote:
> OK let me restate : it would give random access a different semantic than in order access. Which is undesirable.

Yeah. It's a tradeoff, so it's debatable as to which way would be better. In general though, it's better to put the work in popFront. It's less likely to incur an efficiency hit and less bug-prone that way.

- Jonathan M Davis

On Thu, Oct 24, 2013 at 12:50 PM, Jonathan M Davis <jmdavisProg@gmx.com>wrote:

> On Thursday, October 24, 2013 12:14:27 Timothee Cour wrote:
> > so far you've emphasized on the fact popFront should take care of all
> side
> > effects and front should be logically pure and fast to compute, but you haven't suggested any alternative for what I'm trying to achieve.
> >
> > In what I propose it's generic (works in all cases) and simple (just add
> > ".cacheFront" after a lambda with side effect):
> >
> > auto test(T)(T elements) if(isInputRange!T){
> > return elements
> > .map!( (a) {preaction(a); auto b=transform(a); postaction(a,b); return
> b;})
> > .cacheFront
> > .filter!foo; //or joiner or....
> > }
> >
> > So, how do you achieve the above without using cacheFront, without
> creating
> > a _custom_ range or writing a lot of code ?
> > I don't see how using foreach is a good idea (ForEachType!=ElementType,
> it
> > doesn't reuse phobos functionality and doesn't play well with phobos
> range
> > functions)
>
> If you don't want to do this with foreach


foreach is not lazy, so not an option


> , then creating a custom range is exactly what you should be doing,


why make user reimplement the wheel each time with a custom range when a generic solution (such as above, maybe could be improved) works ?


> and the logic for the pre and post operations should go in popFront.


how can a *pre*-operation go in popFront, which occurs *after* front (without significant gymnastics) ?


> Having side effects in front is just begging for problems, and it's not at
> all what map is designed for. And honestly, if
> your popFront isn't pure as well, and the side effects that you're doing
> affect
> something outside of the range, then I would suggest that you not use
> ranges
> for what you're doing, because that violates their functional nature.


That's an arbitrary restriction. I've faced many situations where I could use the full power of range based algorithms while not restricting myself to purely functional code (necessary side effects). With proper care it can be done. Again, no need to reimplement standard algorithms just because of some impure lambdas.


> It has been proposed before that a function be added to Phobos which did an
> operation when iterating over an element, in which case that operation
> would
> go in popFront, and front would just return the wrapped front. That would
> probably get you what you want and be more generally applicable than
> cacheFront.
>

Are you referring to proposals for tap ? There were many proposed, so not sure which one you're mentioning. Or something else?

On Friday, 25 October 2013 at 06:38:00 UTC, Timothee Cour wrote:
> why make user reimplement the wheel each time with a custom range when a
> generic solution (such as above, maybe could be improved) works ?

Because it does not fit current range algorithm design and alien nature of such code should be as explicit as possible.

On Thursday, October 24, 2013 23:09:58 Timothee Cour wrote:
> On Thu, Oct 24, 2013 at 12:50 PM, Jonathan M Davis
> > , then creating a custom range is exactly what you should be doing,
> 
> why make user reimplement the wheel each time with a custom range when a generic solution (such as above, maybe could be improved) works ?

Because you're trying to do something that goes against the design of map and ranges. Sure, it might work with the current implementation, and you can do stlike cacheFront to try and guarantee that front is only called once so that your solution will work, but you're contorting the design in order to do something that goes against what it was designed for, and that's asking for problems.

Regardless, you're free to do what you want in your code, but that doesn't mean that it makes any sense for that to make it into the standard library. The standard library should have solutions which conform with its existing design rather than which try and work around it.

> > and the logic for the pre and post operations should go in popFront.
> 
> how can a *pre*-operation go in popFront, which occurs *after* front (without significant gymnastics) ?

You just make it so that it occurs when the range is constructed, and then it can occur after the previous front. You might have to look at what you're doing slightly differently to move operations from front to popFront, but in my experience, it really isn't all that hard, and you usually end up with better code that way.

> > Having side effects in front is just begging for problems, and it's not at
> > all what map is designed for. And honestly, if
> > your popFront isn't pure as well, and the side effects that you're doing
> > affect
> > something outside of the range, then I would suggest that you not use
> > ranges
> > for what you're doing, because that violates their functional nature.
> 
> That's an arbitrary restriction. I've faced many situations where I could use the full power of range based algorithms while not restricting myself to purely functional code (necessary side effects). With proper care it can be done. Again, no need to reimplement standard algorithms just because of some impure lambdas.

Again, what you're trying to do goes against the range design. You're free to try and work around the design in your own code to get done whatever you're trying to do, but we're not going to want that sort of thing in the standard library and are not going to encourage coding that way.

> > It has been proposed before that a function be added to Phobos which did
> > an
> > operation when iterating over an element, in which case that operation
> > would
> > go in popFront, and front would just return the wrapped front. That would
> > probably get you what you want and be more generally applicable than
> > cacheFront.
> 
> Are you referring to proposals for tap ? There were many proposed, so not sure which one you're mentioning. Or something else?

Probably. There were several names floating around. From what I recall, essentially what you'd get is that if you had something like

foreach(e; range.tap!tapFunc().map!mapFunc()) {...}

then tapFunc would be called for each range element as the range iterates prior to it being processed by map - probably when the element becomes the new front. I would think that you could get what you want with something like that. And if tap only worked for pre or post operations due to how it was put together, then it would probably be easy to create the opposite and have something like tapBefore and tapAfter where tapBefore's function was called before each element became front, and tapAfter's function was called after each element became front.

Though honestly, if I ever end up doing anything that doesn't quite fit into what the range-based functions in Phobos will already do for me, I'll just write my own, since it really isn't very hard and isn't a lot of code unless you trying to make it really generic (e.g. wrap any kind of range and take into account exactly what type of range you're trying to wrap and which of its functions can be forwarded). Simple forward ranges take very little code at all. So, I really don't find it to be a big deal to have to write my own ranges from time to time.

- Jonathan M Davis