I'm trying to put a bit of `pure` and `const`/`immutable` in my code, and I'm getting strange error messages.

Namely, I've a pure function, says `foo` that returns `std.algorithm.joiner(someValue)`:

import std.algorithm: joiner;
auto foo() pure
{
    // some internal calculation
    // creating `someValue`
    return joiner(someValue);
}

DMD tells me `foo` cannot use the impure `joiner`, due to `joiner`'s internal struct (Result) not having pure methods (`empty`/`front`/`popFront`).

Now, it seems obvious why these range methods are not pure (`popFront`, at least). But I don't use them in my function! I just return a lazy range to iterate on other ranges. My own function do not mutate anything, it just creates an object. A mutable value admittedly, but not one with global state.

I know Phobos is not using `pure` as much as it could right now, but I really don't see why it's causing trouble right there.

On Sunday, November 17, 2013 10:05:30 Philippe Sigaud wrote:
> I'm trying to put a bit of `pure` and `const`/`immutable` in my code, and I'm getting strange error messages.
> 
> Namely, I've a pure function, says `foo` that returns
> `std.algorithm.joiner(someValue)`:
> 
> import std.algorithm: joiner;
> auto foo() pure
> {
>      // some internal calculation
>      // creating `someValue`
>      return joiner(someValue);
> }
> 
> DMD tells me `foo` cannot use the impure `joiner`, due to
> `joiner`'s internal struct (Result) not having pure methods
> (`empty`/`front`/`popFront`).
> 
> Now, it seems obvious why these range methods are not pure
> (`popFront`, at least).

All of them should be able to be pure, as none of them access global or static variables.

> But I don't use them in my function! I
> just return a lazy range to iterate on other ranges. My own
> function do not mutate anything, it just creates an object. A
> mutable value admittedly, but not one with global state.
> 
> I know Phobos is not using `pure` as much as it could right now, but I really don't see why it's causing trouble right there.

Likely because it's calling an impure constructor. The compiler does not currently properly infer purity for Voldemort types.

https://d.puremagic.com/issues/show_bug.cgi?id=10329

Really, the compiler's attribute inferrence is pretty pathetic at this point. It only manages to infer attributes in the most basic of cases, and that's the number one reason that so little of Phobos works with pure right now.

- Jonathan M Davis

On Sun, Nov 17, 2013 at 11:17 AM, Jonathan M Davis <jmdavisProg@gmx.com> wrote:

>> DMD tells me `foo` cannot use the impure `joiner`, due to
>> `joiner`'s internal struct (Result) not having pure methods
>> (`empty`/`front`/`popFront`).
>>
>> Now, it seems obvious why these range methods are not pure
>> (`popFront`, at least).
>
> All of them should be able to be pure, as none of them access global or static variables.

But `popFront()` changes an internal state with visible, external
effects: it changes `front` return value and in the end, it will
affect `empty` return value.
So no, I don't consider `popFront` to be pure.
But my problem is: I don't use them! I just create a struct value that
happens to have them as methods. But that should not affect my own
`foo` purity, right?


>> But I don't use them in my function! I
>> just return a lazy range to iterate on other ranges. My own
>> function do not mutate anything, it just creates an object. A
>> mutable value admittedly, but not one with global state.
>>
>> I know Phobos is not using `pure` as much as it could right now, but I really don't see why it's causing trouble right there.
>
> Likely because it's calling an impure constructor. The compiler does not currently properly infer purity for Voldemort types.
>
> https://d.puremagic.com/issues/show_bug.cgi?id=10329
>
> Really, the compiler's attribute inferrence is pretty pathetic at this point. It only manages to infer attributes in the most basic of cases, and that's the number one reason that so little of Phobos works with pure right now.

Ah right, a Voldemort type, I did not know they could affect purity determination. I find they regularly cause problems. Whenever I create one, I then have to extract it from its enclosing father.

I'll create my own simplified version of joiner for this problem. My current code use an eager computation and I don't want that.

On Sunday, 17 November 2013 at 10:33:12 UTC, Philippe Sigaud wrote:
> But `popFront()` changes an internal state with visible, external
> effects: it changes `front` return value and in the end, it will
> affect `empty` return value.
> So no, I don't consider `popFront` to be pure.

That's actually not a reason for them to be impure (see e.g. http://klickverbot.at/blog/2012/05/purity-in-d#pure_member_functions for my attempt on explaining that).

David

On Sunday, November 17, 2013 11:25:33 Philippe Sigaud wrote:
> On Sun, Nov 17, 2013 at 11:17 AM, Jonathan M Davis <jmdavisProg@gmx.com>
wrote:
> >> DMD tells me `foo` cannot use the impure `joiner`, due to
> >> `joiner`'s internal struct (Result) not having pure methods
> >> (`empty`/`front`/`popFront`).
> >> 
> >> Now, it seems obvious why these range methods are not pure
> >> (`popFront`, at least).
> > 
> > All of them should be able to be pure, as none of them access global or static variables.
> 
> But `popFront()` changes an internal state with visible, external
> effects: it changes `front` return value and in the end, it will
> affect `empty` return value.
> So no, I don't consider `popFront` to be pure.

None of that matters for pure. _All_ that matters for pure is that the function does not access global, mutable state - so no mutable global or static variables. _Strong_ purity requires more, but you're not going to get that with a member function anyway, unless it's immutable, which is almost never the case.

If you're trying to think about functional purity when dealing with pure, you're definitely going to misunderstand it, because functional purity really has very little to do with D's pure. At this point, pure is all about indicating that the function does not access anything mutable that you don't pass to it. Anything approaching functional purity and strong purity is then merely an optimization that can be done under very limited circumstances. And since additional calls to strongly pure functions are only optimized out within a single expression (or maybe statement - I'm not sure which - but certainly not across multiple statements), it's not like calls to strongly pure functions can be optimized out very often anyway.

The other big gain from pure is the ability to change the mutability of the return type of a function under some circumstances, making it more flexible, but the whole idea that extra calls to it could be optimized out is more or less a fantasy outside of mathematical functions - and it definitely doesn't work with member functions, since they're almost never immutable like they'd have to be in order to be strongly pure.

> >> But I don't use them in my function! I
> >> just return a lazy range to iterate on other ranges. My own
> >> function do not mutate anything, it just creates an object. A
> >> mutable value admittedly, but not one with global state.
> >> 
> >> I know Phobos is not using `pure` as much as it could right now, but I really don't see why it's causing trouble right there.
> > 
> > Likely because it's calling an impure constructor. The compiler does not currently properly infer purity for Voldemort types.
> > 
> > https://d.puremagic.com/issues/show_bug.cgi?id=10329
> > 
> > Really, the compiler's attribute inferrence is pretty pathetic at this point. It only manages to infer attributes in the most basic of cases, and that's the number one reason that so little of Phobos works with pure right now.
> Ah right, a Voldemort type, I did not know they could affect purity determination. I find they regularly cause problems. Whenever I create one, I then have to extract it from its enclosing father.
> 
> I'll create my own simplified version of joiner for this problem. My current code use an eager computation and I don't want that.

I think that the typical approach at this point is to just drop purity for the moment, but if you want you really want it, you are indeed going to have to implement it yourself. But we'll get there with Phobos eventually. The primary holdup is some compiler improvements, and we'll get them. It's just a question of when.

- Jonathan M Davis

On Sunday, November 17, 2013 11:53:57 David Nadlinger wrote:
> On Sunday, 17 November 2013 at 10:33:12 UTC, Philippe Sigaud
> 
> wrote:
> > But `popFront()` changes an internal state with visible,
> > external
> > effects: it changes `front` return value and in the end, it will
> > affect `empty` return value.
> > So no, I don't consider `popFront` to be pure.
> 
> That's actually not a reason for them to be impure (see e.g. http://klickverbot.at/blog/2012/05/purity-in-d#pure_member_functions for my attempt on explaining that).

I should put that on my list of articles to link people to. I'd forgotten about it.

For that matter, I should probably write more articles like that myself so that I can point people to them instead of explaining the same thing over and over again.

- Jonathan M Davis

>> But `popFront()` changes an internal state with visible, external
>> effects: it changes `front` return value and in the end, it will
>> affect `empty` return value.
>> So no, I don't consider `popFront` to be pure.
>
> None of that matters for pure. _All_ that matters for pure is that the function does not access global, mutable state - so no mutable global or static variables. _Strong_ purity requires more, but you're not going to get that with a member function anyway, unless it's immutable, which is almost never the case.
>
> If you're trying to think about functional purity when dealing with pure, you're definitely going to misunderstand it, because functional purity really has very little to do with D's pure. At this point, pure is all about indicating that the function does not access anything mutable that you don't pass to it.

I know the difference between functional purity and D purity. It's just I consider the range internal state to *be* something mutable that I don't pass to the member.

I never considered that code like this:

auto a = range.front();
auto temp = a;
range.popFront(); // pure popFront()?
a = range.front();
assert(temp != a); // pure popFront() modified range.front() value!

could be authorized. Maybe that's because, until now, I only *read* about D purity and never tried to use it in my code.


> I think that the typical approach at this point is to just drop purity for the moment, but if you want you really want it, you are indeed going to have to implement it yourself. But we'll get there with Phobos eventually. The primary holdup is some compiler improvements, and we'll get them. It's just a question of when.

I implemented it myself without problem (~ 10 LoC). The trouble really
came from `joiner` using an internal struct.
I wanted to keep purity, if only to get a feeling of the kind of
constraints it adds to my coding style. I regularly try to 'limit'
myself to CTFE-compatible code, since I often use my code at
compile-time. I would like to also learn to write pure code, as fas as
possible (and drop it when it's not reasonable, of course).

> On Sunday, November 17, 2013 11:53:57 David Nadlinger wrote:

> That's actually not a reason for them to be impure (see e.g. http://klickverbot.at/blog/2012/05/purity-in-d#pure_member_functions for my attempt on explaining that).

I'll peruse it, thanks!

On Sunday, November 17, 2013 13:53:52 Philippe Sigaud wrote:
> >> But `popFront()` changes an internal state with visible, external
> >> effects: it changes `front` return value and in the end, it will
> >> affect `empty` return value.
> >> So no, I don't consider `popFront` to be pure.
> > 
> > None of that matters for pure. _All_ that matters for pure is that the function does not access global, mutable state - so no mutable global or static variables. _Strong_ purity requires more, but you're not going to get that with a member function anyway, unless it's immutable, which is almost never the case.
> > 
> > If you're trying to think about functional purity when dealing with pure, you're definitely going to misunderstand it, because functional purity really has very little to do with D's pure. At this point, pure is all about indicating that the function does not access anything mutable that you don't pass to it.
> 
> I know the difference between functional purity and D purity. It's just I consider the range internal state to *be* something mutable that I don't pass to the member.
> 
> I never considered that code like this:
> 
> auto a = range.front();
> auto temp = a;
> range.popFront(); // pure popFront()?
> a = range.front();
> assert(temp != a); // pure popFront() modified range.front() value!
> 
> could be authorized. Maybe that's because, until now, I only *read* about D purity and never tried to use it in my code.

If you just think about pure as meaning that the function doesn't access any mutable global state, then you'll have a much better grasp of pure. Essentially, it restricts the function to accessing what's passed to it directly and to global constants which can't possibly have changed their value once they were set, even through other references to them. As such, pure has nothing to do with mutation. Though it is true that it can seem a bit weird with member functions if you forget the fact that they're being passed the this pointer/reference as one of their arguments, which is the case with popFront. So, that's probably what was throwing you off.

- Jonathan M Davis

> If you just think about pure as meaning that the function doesn't access any mutable global state, then you'll have a much better grasp of pure.

I'll endeavour to keep this in mind :)

> Essentially, it restricts the function to accessing what's passed to it directly and to global constants which can't possibly have changed their value once they were set, even through other references to them. As such, pure has nothing to do with mutation. Though it is true that it can seem a bit weird with member functions if you forget the fact that they're being passed the this pointer/reference as one of their arguments, which is the case with popFront. So, that's probably what was throwing you off.

I considered that a hidden *this* was passed, but I forgot that a D pure function could modify its arguments (unless the parameters are specified as const or immutable in the function definition, of course).

I hope I get it now. I guess this is the kind of subject (like Unicode for example) that one only understands when one uses it in his code.