On 1/31/18 7:49 PM, Jonathan M Davis wrote:
> On Wednesday, January 31, 2018 11:58:38 Steven Schveighoffer via
> Digitalmars-d-learn wrote:
>> On 1/30/18 8:05 PM, Jonathan M Davis wrote:
>>> Except that unless front returns by ref, it really doesn't matter
>>> whether
>>> front is const unless it's violating the range API, since front is
>>> supposed to return the same value until popFront is called (or if it's
>>> assigned a new value via a front that returns by ref). So, in practice,
>>> putting const on front really doesn't help you any, and it actually
>>> hurts you for range composability.
>>
>> Right, but that is the difference between a convention ("front is
>> supposed to...") vs. a compiler-enforced guarantee (modifying data by
>> calling a const-tagged front is a compiler error).
>>
>> If you are OK with conventions, you don't need const at all.
> 
> Except that if you're the one writing the function and decided whether it's
> const or not, you're also the one deciding whether it returns by ref or not.
> Unless you're dealing with a reference type, and it doesn't return by ref,
> then const doesn't protect front at all. It just affects whether it can be
> called on a const range.

You are misunderstanding here. You don't put const on front for the purpose of allowing const ranges (which are useless), what it does is say that the compiler guarantees, *even if the range is mutable* that front won't modify it.

That is, code like the following is rejected by the compiler:

int front() const { return ++val; }

In other words, it's a contract that you can read without having to examine the code saying "this won't mutate the range".

Sure, you can document "front shouldn't modify the range", and use that convention, but without const, the compiler doesn't care.

> If you're dealing with generic code, then you have less control, and const
> starts mattering more, since you don't necessarily know what type is being
> returned, and if you're returning front from an underlying range, you the
> choice of eixther returning it by value or returning it by auto ref in case
> the underlying range returned by ref and passing that refness on is
> desirable. But const also interacts far more badly with generic code,
> because the odds are pretty high that it won't work in many cases. So, while
> in principle, using const to actually have the guarantees is valuable, in
> practice, it isn't very viable, because D's const is so restrictive.

Technically, wrapping requires introspection. If you don't care about forwarding the "guarantee" of constness, then you can just tag all your functions mutable, but if you do care, then you have to introspect.

> Personally, I avoid const in generic code like the plague, because unless
> you've restricted the types enough to know what you're dealing with and know
> that it will work with const, the odds are quite high that you're writing
> code that's going to fall flat on its face with many types.

Indeed, it's not straightforward, if you have to deal with types that aren't tagged the way they should be. In addition, const is not inferred for templates like other attributes, so you can't rely on that either.

-Steve

On Wednesday, 31 January 2018 at 01:45:57 UTC, H. S. Teoh wrote:
> I haven't thought through it carefully, but if .headConst is a viable solution to the head-const problem, then conceivably we could also extend it to deal with immutable payloads too.  Then we could go from, say, RefCounted!(immutable(T)) to RefCounted!(const(T)) generically, without any casting or breaking the type system.  We could potentially expand the scope of usefulness of immutable this way, if this approach turns out to be workable.

Going from RefCounted!(immutable(T)) to RefCounted!(const(T)) is nice, but I'd like to see a conversion from, const(RefCounted!T) to RefCounted!(const(T)). While this cannot be done without casts, the logic can be put inside .headMutable(), and include relevant checks. This will make it much safer than having the programmer cast manually.

--
  Simen

On Thu, Feb 01, 2018 at 07:52:32AM +0000, Simen Kjærås via Digitalmars-d-learn wrote:
> On Wednesday, 31 January 2018 at 01:45:57 UTC, H. S. Teoh wrote:
> > I haven't thought through it carefully, but if .headConst is a viable solution to the head-const problem, then conceivably we could also extend it to deal with immutable payloads too.  Then we could go from, say, RefCounted!(immutable(T)) to RefCounted!(const(T)) generically, without any casting or breaking the type system.  We could potentially expand the scope of usefulness of immutable this way, if this approach turns out to be workable.
> 
> Going from RefCounted!(immutable(T)) to RefCounted!(const(T)) is nice,
> but I'd like to see a conversion from, const(RefCounted!T) to
> RefCounted!(const(T)). While this cannot be done without casts, the
> logic can be put inside .headMutable(), and include relevant checks.
> This will make it much safer than having the programmer cast manually.
[...]

Hmm. I experimented with this for a bit, and found that if we limit ourselves to head-mutable, then these casts are inescapable.

The problem is that if we're handed a const object like
const(RefCounted!T), then there's no way we can back out to
RefCounted!(const(T)) without casting, because the latter contains a
mutable refcount whereas the former, due to const transitivity, must be
entirely unmodifiable.  If the refcount were stored in the RefCount
struct itself, then we could get away with a by-value copy into
RefCounted!(const(T)), but unfortunately we can't do that without
breaking the refcounting semantics; the refcount must be on the payload
itself, and RefCounted is basically just a smart pointer. So
const(RefCounted!T), by const transitivity, can only contain a const
pointer to the payload, so we're forced to use a cast to get a
RefCounted!(const(T)) out of it.

And on that note, this casting is NOT safe; for example, if you start with an immutable(RefCounted!T) and implicitly convert it to const(RefCounted!T), then if you cast the latter to RefCounted!(const(T)), you're now violating immutable.  And there's no way you can tell from inside .headMutable whether it's safe to cast, because by the time it gets to .headMutable, the original immutable type is already "forgotten".

However, if we go back to the idea of tail-const, we could potentially eliminate the need for casts and also avoid breaking immutable. Basically, the problem with writing const(RefCounted!T) is that it's a one-way street: on the scale of increasing restrictiveness, we have the series:

	RefCounted!T < RefCounted!(const(T)) < const(RefCounted!T)

Once you've gone all the way down to const(RefCounted!T), you can no
longer safely back out to RefCounted!(const(T)).

So that means we want to avoid const(RefCounted!T) completely. Instead, if we standardize on a way to produce a RefCounted!(const(T)) from a RefCounted!T, then we can stop halfway down the one-way street and never need to back up.  Let's say we standardize on an operation .tailConst that does these conversions:

	Container!T.tailConst		--> Container!(const(T))
	Container!(const(T)).tailConst	--> Container!(const(T))
	const(Container!T).tailConst	--> const(Container!T)

	// This is allowed because it's possible to implicitly convert
	// immutable(T) to const(T) internally:
	Container!(immutable(T)).tailConst --> Container!(const(T))

	immutable(Container!T).tailConst --> const(Container!T)

where Container can be any template that might want to support tail-const semantics.

Essentially, .tailConst becomes the mid-way stand-in for the language's
built-in implicit conversions from mutable/immutable to const. So
instead of passing around const(Container!T), we'd construct a
Container!(const(T)) by calling .tailConst on the original container,
and pass that around instead.

We can also generalize this via UFCS to built-in reference types:

	tailConst(T*)		--> const(T)*
	tailConst((const(T))*)	--> const(T)*
	tailConst(const(T*))	--> const(T*)

	tailConst(immutable(T)*) --> const(T)*
	tailConst(immutable(T*)) --> const(T*)

Then .tailConst becomes a standard way of constructing a tail-const type in the language.  No explicit language support is needed.


T

-- 
Claiming that your operating system is the best in the world because more people use it is like saying McDonalds makes the best food in the world. -- Carl B. Constantine

On Thursday, 1 February 2018 at 18:58:15 UTC, H. S. Teoh wrote:
> However, if we go back to the idea of tail-const, we could potentially eliminate the need for casts and also avoid breaking immutable. Basically, the problem with writing const(RefCounted!T) is that it's a one-way street: on the scale of increasing restrictiveness, we have the series:
>
> 	RefCounted!T < RefCounted!(const(T)) < const(RefCounted!T)
>
> Once you've gone all the way down to const(RefCounted!T), you can no
> longer safely back out to RefCounted!(const(T)).
>
> So that means we want to avoid const(RefCounted!T) completely.

I'm not really saying I disagree with that, but it's just not realistic. Which code would you rather write?

void foo(T)(const T t) {}
foo(myValue);

or:

void foo(T)(T t) if (isTailConst!T) {}
foo(myValue.tailConst);

The beauty of .headMutable is it generally doesn't affect user code. If we have to tell people not to use const(T) because its semantics are broken, we've failed.

Now, if at any point in your program you have an immutable(RefCounted!T), something's gone horribly wrong - basically all of the RefCounted's semantics break down when it's immutable, and any attempt at fixing it is undefined behavior. I think we can safely disregard the problems of immutable(RefCounted!T).

Once we've defined immutable(RefCounted!T) to be undefined behavior, suddenly casting from const(RefCounted!T) to RefCounted!(const(T)) is OK again.

--
  Simen

On Fri, Feb 02, 2018 at 07:06:56AM +0000, Simen Kjærås via Digitalmars-d-learn wrote:
> On Thursday, 1 February 2018 at 18:58:15 UTC, H. S. Teoh wrote:
> > However, if we go back to the idea of tail-const, we could potentially eliminate the need for casts and also avoid breaking immutable.  Basically, the problem with writing const(RefCounted!T) is that it's a one-way street: on the scale of increasing restrictiveness, we have the series:
> > 
> > 	RefCounted!T < RefCounted!(const(T)) < const(RefCounted!T)
> > 
> > Once you've gone all the way down to const(RefCounted!T), you can no
> > longer safely back out to RefCounted!(const(T)).
> > 
> > So that means we want to avoid const(RefCounted!T) completely.
> 
> I'm not really saying I disagree with that, but it's just not realistic.  Which code would you rather write?
> 
> void foo(T)(const T t) {}
> foo(myValue);
> 
> or:
> 
> void foo(T)(T t) if (isTailConst!T) {}
> foo(myValue.tailConst);
> 
> The beauty of .headMutable is it generally doesn't affect user code. If we have to tell people not to use const(T) because its semantics are broken, we've failed.

Its semantics are not broken; it's just harder to use. Due to const transitivity, it's an all-or-nothing deal.  .tailConst gives us the middle ground.


> Now, if at any point in your program you have an immutable(RefCounted!T), something's gone horribly wrong - basically all of the RefCounted's semantics break down when it's immutable, and any attempt at fixing it is undefined behavior. I think we can safely disregard the problems of immutable(RefCounted!T).

Immutable may be useless for RefCounted, but I'm thinking of containers and wrapper types in general. Today, due to ranges being basically useless when you can't mutate them, you're forced to choose between a completely mutable range, or no range at all. Having .tailConst as a standard construction lets you create a usable range that provides a compiler-verified guarantee that nobody will be able to modify range elements. Today we don't have a standard way of doing this, and so people have given up on using const with ranges. We're missing out on the guarantees that const provides. .tailConst lets us get some of those guarantees back without requiring us to tie our hands behind our backs.


> Once we've defined immutable(RefCounted!T) to be undefined behavior,
> suddenly casting from const(RefCounted!T) to RefCounted!(const(T)) is
> OK again.
[...]

The problem with this is that we're now relying on convention rather than something that can be statically verified by the compiler. Once you allow casting away const, there's no longer a guarantee that somebody didn't pass in an immutable, whether by mistake or otherwise. We know from C/C++ where programming by convention leads us. :-P


T

-- 
Having a smoking section in a restaurant is like having a peeing section in a swimming pool. -- Edward Burr

On Fri, Feb 02, 2018 at 07:06:56AM +0000, Simen Kjærås via Digitalmars-d-learn wrote: [...]
> Which code would you rather write?
> 
> void foo(T)(const T t) {}
> foo(myValue);
> 
> or:
> 
> void foo(T)(T t) if (isTailConst!T) {}
> foo(myValue.tailConst);
[...]

More thoughts on this: what if we made it so that Wrapper!T is implicitly convertible to Wrapper!(const T)? Something like this:

	auto foo(Wrapper,T)(Wrapper!(const(T)) t) ...

	struct Wrapper(T) {
		auto tailConst() {
			return Wrapper!(const(T))(...);
		}
		alias tailConst this;
	}

	void main() {
		Wrapper!int w;
		foo(w);
	}

It will interact badly if Wrapper is already using alias this for something else, but this lets us keep the convenience of implicitly decaying to tail-const without requiring an explicit call to .tailConst.

Though the above currently doesn't compile, it seems because the compiler doesn't know how to resolve Wrapper!(const(T)) given a Wrapper!T despite the alias this. However, removing `Wrapper` from the template arguments of foo() does work; and seems to have the right semantics.  Seems to be just a limitation of IFTI.  So calling a function that expects, say, RefCounted!(const T), with an argument of type RefCounted!T can already be made to work today, if we tie .tailConst to alias this.

The fully general solution will have to wait until we can improve IFTI to support the generic case where the wrapper type is also a template parameter.  Might be worth filing an enhancement against the compiler to support this?

If this can be made to work, we may not even need a standard name for .tailConst, as long as the wrapper type can somehow make itself decay into its tail-const version implicitly.


T

-- 
In theory, there is no difference between theory and practice.

On Friday, 2 February 2018 at 14:29:34 UTC, H. S. Teoh wrote:
> Its semantics are not broken; it's just harder to use. Due to const transitivity, it's an all-or-nothing deal.  .tailConst gives us the middle ground.

If the semantics of const means that users will have to write .tailConst all over the place, it's broken. If it means that users can't use const(T) because they actually want TailConst!T, it's broken.

TailConst seems like the logical solution to the problem, but it isn't. It directly impacts user code and it leads to lots of boilerplate. In addition to .tailConst, we also need .tailImmutable. And to top it off, it just doesn't mix with const at all - if you pass it as a const parameter, it's broken. If it's part of a struct or class with const methods, it's broken. It infects every part of your codebase that touches it, it forces you to basically implement your own const system in templates, and it makes const even harder to use than it currently is.

Tail-const is the more intuitive way to think of it, so if I'm wrong, please show me.


>> Once we've defined immutable(RefCounted!T) to be undefined behavior,
>> suddenly casting from const(RefCounted!T) to RefCounted!(const(T)) is
>> OK again.
> [...]
>
> The problem with this is that we're now relying on convention rather than something that can be statically verified by the compiler. Once you allow casting away const, there's no longer a guarantee that somebody didn't pass in an immutable, whether by mistake or otherwise. We know from C/C++ where programming by convention leads us. :-P

True. Sadly, there's no way to tell the type system 'this type should never be immutable'. Maybe such a thing should be in the language. Meanwhile, if RefCounted!T implements .headMutable, it can check at runtime that the refcount is in writable memory.


> Though the above currently doesn't compile, it seems because the compiler doesn't know how to resolve Wrapper!(const(T)) given a Wrapper!T despite the alias this.

Yeah, I found the same bug when playing with .headMutable:
https://issues.dlang.org/show_bug.cgi?id=18260

--
  Simen

First of all, thank you all for the replies. It has taken me some time to learn a bit more to be able to understand at least some parts of them. I have a further question below the quotes.

On Tuesday, 30 January 2018 at 01:20:09 UTC, Jonathan M Davis wrote:
> On Tuesday, January 30, 2018 01:05:54 Drone1h via Digitalmars-d-learn wrote:

>> [...]
>> I am trying to implement a ("struct template" ? what is the
>> correct word ?) range that just forwards its primitives ("empty",
>> "front", "popFront") to another range, possibly with some very
>> limited filtering/alteration, as std.range.Take (just to learn).
>>
>> Initially, the "front" member function (property) used to be
>> declared "const", but that was not accepted when the underlying
>> range (denoted "R" in the code below) was std.stdio.File.ByChunk
>> ("Error: mutable method std.stdio.File.ByChunk.front is not
>> callable using a const object").
>> [...]

> If you want to put an attribute on it, inout is better, because then it will work with any constness

I am not sure whether I can make it work with "inout" instead of "const". Perhaps I am missing something.

Here is my code:

    auto Take2 (R) (R r, size_t n)
    {
        import std.range.primitives;

        struct Result
        {
        private:
            R      _r;
            size_t _n;
            size_t _i;

        public:
            @property bool empty () const
            {
                return _i >= _n || _r.empty;
            }

            @property auto ref front () const // Replace "const" with "inout" here ?
            {
                return _r.front;
            }

            void popFront ()
            {
                ++_i;
                if (_i < _n)
                    _r.popFront ();
            }

            this (R r, size_t n)
            {
                _r = r;
                _n = n;
                _i = 0;
            }
        }

        return Result (r, n);
    }

    unittest
    {
        import std.stdio;

        auto file = File ("Example", "rb");
        foreach (c; file.byChunk (0x100).Take2 (5))
            stdout.rawWrite (c);
    }

I get compile error:

    Error: mutable method `std.stdio.File.ByChunkImpl.front` is not callable using a `const` object

If I replace "const" with "inout" for the "front" function, i.e. "@property auto ref front () inout", I get similar error:

    Error: mutable method `std.stdio.File.ByChunkImpl.front` is not callable using a `inout` object

May I ask that you confirm that this is what you suggested ? Thank you.



> [...] const ranges are utterly useless, because they can't be mutated and thus can't be iterated. [...]

I am considering a function that takes as parameter a (reference to) const range. It should be able to check whether the range is "empty" and, if not empty, it should be able to access the "front" element.

Now the caller of that function can rest assured that the function is not going to modify the range by "popFront".

The function might be more useful than a function that just takes as parameter a value (the result of "front") because it may be called with an empty range and it can detect that. This is similar to getting as parameter a (possibly null) pointer to a value instead of getting as parameter a value.

Therefore, it seems to me that a const range might be useful.

If you have already considered this and have actually seen one-step ahead of me, may I ask that you confirm, please ?



Thank you respectfully.

On Monday, March 19, 2018 00:14:11 Drone1h via Digitalmars-d-learn wrote:
> I am not sure whether I can make it work with "inout" instead of "const". Perhaps I am missing something.
...
> May I ask that you confirm that this is what you suggested ? Thank you.

Marking a empty or front with inout has most of the problems that const has. The difference is that if the range is mutable, then the return value will be treated as mutable. The internals of the function, however, must still work with const or inout, and that's not true for most ranges. It can work just fine to mark empty or front as inout or const if you're in full control of the element types and aren't wrapping other ranges, but as soon as you start wrapping other ranges, you pretty much must give up on inout and const, because most ranges won't work with them - even to just call empty or front. And in many cases, they can't be made to work with const or inout, because that often causes serious problems with the return type. The range API simply does not include const or inout, so you can't assume that any range will compile with them, meaning that ranges that wrap other ranges can only use const or inout when they're intended to wrap a very specific set of ranges that do work with const or inout. Ranges that generically wrap other ranges cannot use const or inout, or they will not compile with many (most?) ranges.

> > [...] const ranges are utterly useless, because they can't be mutated and thus can't be iterated. [...]
>
> I am considering a function that takes as parameter a (reference to) const range. It should be able to check whether the range is "empty" and, if not empty, it should be able to access the "front" element.
>
> Now the caller of that function can rest assured that the function is not going to modify the range by "popFront".
>
> The function might be more useful than a function that just takes as parameter a value (the result of "front") because it may be called with an empty range and it can detect that. This is similar to getting as parameter a (possibly null) pointer to a value instead of getting as parameter a value.
>
> Therefore, it seems to me that a const range might be useful.
>
> If you have already considered this and have actually seen one-step ahead of me, may I ask that you confirm, please ?

Most ranges do not work with const in any way shape or form. _Some_ will work if all you're doing is looking at is empty or front. But if all you're doing is looking at the front, in general, why pass a range? Just call front and pass it, and then the function will work with more than just ranges. Yes, if you want a function that does something like

auto frontOrInit(R)(R range)
{
    return range.empty ? ElementType!R.init : range.front;
}

then you can make the range const, but in general, a function is either going to be iterating over the range (so the range can't be const), or what the function is doing really has nothing to do with ranges and would be far more flexible if it just took the range's element type. Functions like frontOrInit where it would make sense to only call front or empty on a range are rare. And honestly, making a function like frontOrInit accept the range by const doesn't buy you much if a range's front and empty are const, and it makes the function useless with most ranges, because most ranges don't - and many can't - mark front or empty as const.

Honestly, I think that you will be far better off if you just don't try and use const or inout with ranges. You can make it work in very restricted circumstances, but you will constantly be fighting problems where code does not compile. I'd suggest that you read this:

http://jmdavisprog.com/articles/why-const-sucks.html

- Jonathan M Davis

On Friday, 2 February 2018 at 14:29:34 UTC, H. S. Teoh wrote:
> On Fri, Feb 02, 2018 at 07:06:56AM +0000, Simen Kjærås via Digitalmars-d-learn wrote:
>> [...]
>
> Its semantics are not broken; it's just harder to use. Due to const transitivity, it's an all-or-nothing deal.  .tailConst gives us the middle ground.
>
> [...]

FWIW there's also head const in phobos since more than one year - known as Final:

https://dlang.org/phobos/std_experimental_typecons.html