I've noticed that for some ranges in Phobos empty is marked const (e.g. iota) but for other ranges (e.g. multiwayMerge) it is not const. Is there a reason why? Isn't empty guaranteed not to alter the data of the range and so should be const?

This is causing me considerable headaches as I try to write my own ranges that accept other ranges and have it all work for the general case. Any advice would be welcome.

On Monday, 29 July 2019 at 17:32:58 UTC, Matt wrote:
> I've noticed that for some ranges in Phobos empty is marked const (e.g. iota) but for other ranges (e.g. multiwayMerge) it is not const. Is there a reason why? Isn't empty guaranteed not to alter the data of the range and so should be const?
>
> This is causing me considerable headaches as I try to write my own ranges that accept other ranges and have it all work for the general case. Any advice would be welcome.

You could use introspection (a static i) to check if the type defines a const/non-const version and write the appropriate empty declaration.

As for why it’s like this I’ll have to look at the code in Phobos, but I’m currently away from my pc (on the phone).

Cheers,
Edi

On Mon, Jul 29, 2019 at 05:32:58PM +0000, Matt via Digitalmars-d-learn wrote:
> I've noticed that for some ranges in Phobos empty is marked const (e.g.  iota) but for other ranges (e.g. multiwayMerge) it is not const. Is there a reason why? Isn't empty guaranteed not to alter the data of the range and so should be const?

Although .empty should be *logically* const, some ranges do non-const work in .empty for various reasons (e.g., caching, lazy evaluation, etc.).  Since D doesn't have logical const, it can't be used in every case.


> This is causing me considerable headaches as I try to write my own ranges that accept other ranges and have it all work for the general case. Any advice would be welcome.

Generally, the idiom is to let the compiler do attribute inference by templatizing your code and not writing any explicit attributes, then use unittests to ensure that instantiations of the range that ought to have certain attributes actually have those attributes.  For example, instead of writing:

	struct MyRange(R) {
		...
		@property bool empty() const { ... }
		...
	}

write instead:

	struct MyRange(R) {
		...
		// No attributes: compiler does inference
		@property bool empty() { ... }
		...
	}

	// unittest ensures .empty is callable with const object.
	unittest {
		const rr = MyRange!(const(Range))(...);
		assert(rr.empty); // will fail compilation if .empty is non-const
	}

The unittest tests that a specific instantiation of MyRange has const .empty. It's still possible to use MyRange with a range that has non-const .empty, but this unittest ensures that the non-const-ness wasn't introduced by the implementation of MyRange itself, but only comes from the template argument.


T

-- 
"Holy war is an oxymoron." -- Lazarus Long

On Monday, July 29, 2019 11:32:58 AM MDT Matt via Digitalmars-d-learn wrote:
> I've noticed that for some ranges in Phobos empty is marked const (e.g. iota) but for other ranges (e.g. multiwayMerge) it is not const. Is there a reason why? Isn't empty guaranteed not to alter the data of the range and so should be const?
>
> This is causing me considerable headaches as I try to write my own ranges that accept other ranges and have it all work for the general case. Any advice would be welcome.

empty is most definitely _not_ guaranteed to not mutate the range. No, it must not change whether the range is actually empty or not, consume elements, etc., but stuff like caching or delayed calculation can affect what empty does in ways that would require mutation. For instance, filter avoids doing any real work in its constructor, but that means that in order for empty to work if it's called before front or popFront (as would be typical), it has to do the work to get the range to its starting point so that it's known whether it's actually empty or not. So, filter's empty can't be const.

In general, generic code can't assume that a range-based function is const, because that varies wildly across ranges, because they're frequently wrapping other ranges. Even if a range that's being wrapped could theoretically be const, it's frequently the case that it isn't, because there isn't much point.

Ranges are pretty much useless if they're const. The best that you'd be able to do with any range API functions would be to call empty, front, back (if it's a bidirectional range), opIndex (if it's a random-access range), and maybe opSlice (if it has slicing). But pretty much the only range types that will give you a tail-const slice if you slice them is dynamic arrays, because the compiler understands them and knows that it's safe to const(T)[] instead of const(T[]), whereas with templated types, not only is there no way for it to know what the tail-const equivalent of const(R) or const(R!E) would be, but with how templates work, there's no guarantee that a different instantiation of the same template would be equivalent even if the only difference is const. Stuff like static if could be used to make them completely different. So, once you have a const range, your essentially stuck and can't mutate it or get a tail-const copy to mutate.

Because const ranges are basically useless, there really isn't much point in putting const on any range functions even if it would work for that particular range, and if a range is a wrapper range, the only way that it could do it would be if it used static if to make the code differ depending on whether the range it's wrapping will work if that function is const, which essentially means duplicating a bunch of code for little to no benefit.

- Jonathan M Davis

On Monday, July 29, 2019 1:35:18 PM MDT H. S. Teoh via Digitalmars-d-learn wrote:
> Generally, the idiom is to let the compiler do attribute inference by templatizing your code and not writing any explicit attributes, then use unittests to ensure that instantiations of the range that ought to have certain attributes actually have those attributes.  For example, instead of writing:
>
>   struct MyRange(R) {
>       ...
>       @property bool empty() const { ... }
>       ...
>   }
>
> write instead:
>
>   struct MyRange(R) {
>       ...
>       // No attributes: compiler does inference
>       @property bool empty() { ... }
>       ...
>   }
>
>   // unittest ensures .empty is callable with const object.
>   unittest {
>       const rr = MyRange!(const(Range))(...);
>       assert(rr.empty); // will fail compilation if .empty is non-const
>   }
>
> The unittest tests that a specific instantiation of MyRange has const .empty. It's still possible to use MyRange with a range that has non-const .empty, but this unittest ensures that the non-const-ness wasn't introduced by the implementation of MyRange itself, but only comes from the template argument.

Since when does const have anything to do with attribute inference? Unless something has changed recently, const is _never_ inferred for functions. Your unittest here should never compile regardless of whether empty could have been marked as const or not. If you want empty to be const or not based on the range being wrapped, you'd need to use two separate function definitions (one const and one not) and use static if to choose which got compiled in based on whether it could be const or not with the range type that it's wrapping.

- Jonathan M Davis

On 29.07.19 21:35, H. S. Teoh wrote:
> Generally, the idiom is to let the compiler do attribute inference by
> templatizing your code and not writing any explicit attributes, then use
> unittests to ensure that instantiations of the range that ought to have
> certain attributes actually have those attributes.  For example, instead
> of writing:
> 
> 	struct MyRange(R) {
> 		...
> 		@property bool empty() const { ... }
> 		...
> 	}
> 
> write instead:
> 
> 	struct MyRange(R) {
> 		...
> 		// No attributes: compiler does inference
> 		@property bool empty() { ... }
> 		...
> 	}
> 
> 	// unittest ensures .empty is callable with const object.
> 	unittest {
> 		const rr = MyRange!(const(Range))(...);
> 		assert(rr.empty); // will fail compilation if .empty is non-const
> 	}
> 
> The unittest tests that a specific instantiation of MyRange has const
> .empty. It's still possible to use MyRange with a range that has
> non-const .empty, but this unittest ensures that the non-const-ness
> wasn't introduced by the implementation of MyRange itself, but only
> comes from the template argument.

But const isn't inferred.

----
struct MyRange()
{
    @property bool empty() { return true; }
}
void main()
{
    pragma(msg, typeof(&MyRange!().empty));
        /* Prints: "bool function() pure nothrow @nogc @property @safe"
        Note: no const. */
    const MyRange!() r;
    assert(r.empty); /* fails compilation */
}
----

On Monday, 29 July 2019 at 19:38:34 UTC, Jonathan M Davis wrote:
> On Monday, July 29, 2019 11:32:58 AM MDT Matt via Digitalmars-d-learn wrote:
>

>
> Because const ranges are basically useless, there really isn't much point in putting const on any range functions even if it would work for that particular range, and if a range is a wrapper range, the only way that it could do it would be if it used static if to make the code differ depending on whether the range it's wrapping will work if that function is const, which essentially means duplicating a bunch of code for little to no benefit.
>
> - Jonathan M Davis

This was super helpful, thanks so much. I thought it was good practice to label member functions const if they didn't/couldn't modify any data. Now I see the reality is different for ranges. Not worrying about const for these seems simpler than code repetition.

Thanks again.

On Monday, July 29, 2019 3:11:45 PM MDT Matt via Digitalmars-d-learn wrote:
> On Monday, 29 July 2019 at 19:38:34 UTC, Jonathan M Davis wrote:
> > On Monday, July 29, 2019 11:32:58 AM MDT Matt via Digitalmars-d-learn wrote:
> >
> >
> >
> > Because const ranges are basically useless, there really isn't much point in putting const on any range functions even if it would work for that particular range, and if a range is a wrapper range, the only way that it could do it would be if it used static if to make the code differ depending on whether the range it's wrapping will work if that function is const, which essentially means duplicating a bunch of code for little to no benefit.
> >
> > - Jonathan M Davis
>
> This was super helpful, thanks so much. I thought it was good practice to label member functions const if they didn't/couldn't modify any data. Now I see the reality is different for ranges. Not worrying about const for these seems simpler than code repetition.
>
> Thanks again.

In principle, it's good to use const when you know that data isn't going to change, but that gets far more complicated when you're dealing with generic code or even with classes, since as soon as you use const, everything used with that template then needs to work with const, or in the case of classes, every derived class has to use const for their override of that function. Sometimes, that's fine, but that's usually when you're either requiring that const always work, or you're in control all of the code involved, so you know that you're not going to have to deal with issues like caching. It's issues like this that led us to decide a while ago that putting functions on Object was a mistake, since it locked all classes into a particular set of attributes (and even if we changed which attributes those were, it would still cause problems). The ProtoObject DIP (which would add a base class for Object that didn't have anything on it) will hopefully fix that, but that still hasn't been finalized yet.

In the case of ranges, on top of the general issues with const and generic code, their API just isn't designed with const in mind. Fundamentally, you need to be able to mutate a range to iterate through it. It would be different if we'd gone with more of a functional-style, head/tail solution where you have a function like head to get the first element, and a function like tail to return a range with the first element popped off, but for better or worse, that's not the direction we went. However, even if we had, issues like caching or delayed calculation would still come into play, and if you require that const work on something like empty, that prevents certain classes of solutions. Of course, on the flip side, without const, you don't know for sure that unwanted mutation isn't happening, but what would really be needed would be some kind of "logical" const rather than the full-on const we currently have, and that would be very difficult to implement. C++'s const _sort_ of allows that, because it has so many loopholes, but on the flip side, you lose most of the guarantees, and it mostly just becomes documentation of intent rather than actually enforcing logical constness. In practice, I find that D's const tends to not be terribly useful in generic code, but it's far more of a problem with libraries that are publicly available than with code where you control everything and can change stuff when you need to. This article I wrote goes into further detail about the issues with const in general:

http://jmdavisprog.com/articles/why-const-sucks.html

The situation with ranges would be improved if we had some kind of const or inout inference for templated code like we do with other attributes, but I don't know quite how that would work or what the downsides would be.

- Jonathan M Davis

On Tuesday, 30 July 2019 at 05:30:30 UTC, Jonathan M Davis wrote:
> In principle, it's good to use const when you know that data isn't going to change, but that gets far more complicated when you're dealing with generic code or even with classes, since as soon as you use const, everything used with that template then needs to work with const, or in the case of classes, every derived class has to use const for their override of that function. Sometimes, that's fine, but that's usually when you're either requiring that const always work, or you're in control all of the code involved, so you know that you're not going to have to deal with issues like caching. It's issues like this that led us to decide a while ago that putting functions on Object was a mistake, since it locked all classes into a particular set of attributes (and even if we changed which attributes those were, it would still cause problems). The ProtoObject DIP (which would add a base class for Object that didn't have anything on it) will hopefully fix that, but that still hasn't been finalized yet.
>
> In the case of ranges, on top of the general issues with const and generic code, their API just isn't designed with const in mind. Fundamentally, you need to be able to mutate a range to iterate through it. It would be different if we'd gone with more of a functional-style, head/tail solution where you have a function like head to get the first element, and a function like tail to return a range with the first element popped off, but for better or worse, that's not the direction we went. However, even if we had, issues like caching or delayed calculation would still come into play, and if you require that const work on something like empty, that prevents certain classes of solutions. Of course, on the flip side, without const, you don't know for sure that unwanted mutation isn't happening, but what would really be needed would be some kind of "logical" const rather than the full-on const we currently have, and that would be very difficult to implement. C++'s const _sort_ of allows that, because it has so many loopholes, but on the flip side, you lose most of the guarantees, and it mostly just becomes documentation of intent rather than actually enforcing logical constness. In practice, I find that D's const tends to not be terribly useful in generic code, but it's far more of a problem with libraries that are publicly available than with code where you control everything and can change stuff when you need to. This article I wrote goes into further detail about the issues with const in general:
>
> http://jmdavisprog.com/articles/why-const-sucks.html
>
> The situation with ranges would be improved if we had some kind of const or inout inference for templated code like we do with other attributes, but I don't know quite how that would work or what the downsides would be.
>
> - Jonathan M Davis

That was a great article you wrote. Const has been one of the more difficult concepts for me to grasp when I moved from python to C++ and then to D. I also never understood immutable and the difference with const. Your article makes that really clear. Thanks for sharing.

Matt