On 11/15/2015 09:34 AM, Dicebot wrote:
> On Sunday, 15 November 2015 at 14:23:05 UTC, Jonathan M Davis wrote:
>>> As I mentioned, he's okay with changing the language to make the
>>> casts well defined. -- Andrei
>>
>> Well, that's a big change, since it pretty much means that D's const
>> isn't physical const anymore, and Walter has been _very_ insistent
>> about that in the past - to the point that he's argued that C++'s
>> const is outright useless because it isn't physical const. If casting
>> away const and mutating is well-defined behavior, then we basically
>> have C++'s const except that it's transitive ...
>
> Casting away _const_ is already legal if programmer can himself
> guarantee underlying object has mutable origin (i.e. not available via
> immutable reference), by the very definition of const. It is casting
> away immutable and mutating that is strictly UB.

Correct. I'm not sure whether that's clarified in the language documentation yet. -- Andrei

On Sunday, 15 November 2015 at 14:42:25 UTC, Jonathan M Davis wrote:
>> Casting away _const_ is already legal if programmer can himself guarantee underlying object has mutable origin (i.e. not available via immutable reference), by the very definition of const. It is casting away immutable and mutating that is strictly UB.
>
> No. As it stands, casting away const and mutating is _always_ considered undefined behavior, regardless of whether the object being referred to is actually mutable, const, or immutable. In fact, there was a discussion on that not long ago, and the spec was updated to be clearer on that count - with approval from Walter. AFAIK, it has never been the case that casting away const and mutating was defined behavior in D2 (I have no idea what the deal with D1 and const is other than the fact that it was quite different).

How was it justified to fit definition of `const`? It is in direct contradiction, because const is supposed to mean "can be either mutable or immutable, can't make any assumptions about it".

My understanding from previous discussions was that mutating const is UB is general because mutating immutable is UB and const _may_ be a view to immutable data. And if this can be proved to not be the case, const is not really const anymore. In fact, major chunk of our ported code relies on this.

(D1 is irrelevant here as const isn't a type qualifier there and this cant be part of cast)

On Sunday, 15 November 2015 at 14:55:36 UTC, Dicebot wrote:
> On Sunday, 15 November 2015 at 14:42:25 UTC, Jonathan M Davis wrote:
>>> Casting away _const_ is already legal if programmer can himself guarantee underlying object has mutable origin (i.e. not available via immutable reference), by the very definition of const. It is casting away immutable and mutating that is strictly UB.
>>
>> No. As it stands, casting away const and mutating is _always_ considered undefined behavior, regardless of whether the object being referred to is actually mutable, const, or immutable. In fact, there was a discussion on that not long ago, and the spec was updated to be clearer on that count - with approval from Walter. AFAIK, it has never been the case that casting away const and mutating was defined behavior in D2 (I have no idea what the deal with D1 and const is other than the fact that it was quite different).
>
> How was it justified to fit definition of `const`? It is in direct contradiction, because const is supposed to mean "can be either mutable or immutable, can't make any assumptions about it".
>
> My understanding from previous discussions was that mutating const is UB is general because mutating immutable is UB and const _may_ be a view to immutable data. And if this can be proved to not be the case, const is not really const anymore. In fact, major chunk of our ported code relies on this.

const in D guarantees that the object will not be mutated via that reference regardless of what that reference refers to. And there _are_ cases where the compiler can make assumptions based purely on const, though they're not common. e.g.

auto bar(const Foo) pure;

auto foo = new Foo;
auto result = bar(foo);

After the call to bar, the compiler can guarantee that the value of foo has not changed, because it's not possible for bar to access foo except via a const reference, and it can clearly see that, because it can see that it's not possible for bar to access foo except via the reference that's passed to it. In fact, with the following code

auto bar(const Foo) pure;

auto foo = new Foo;
auto result1 = bar(foo);
auto result2 = bar(foo);

it would be possible for the compiler to know that result1 and result2 are identical and elide the function call like it could if bar were given an immutable object. AFAIK, the compiler doesn't do anything like that currently, but const is enough for it to do so in this case. So, const my itself can provide guarantees, even though usually the compiler doesn't have enough information to do much with const alone (since it has to be able to know that the object is not mutated via another, mutable reference, which it usually can't).

Whenever this has come up, Walter has been adamant that D's const is physical const and that it's guaranteed by the compiler that you an object is not mutated via a  const reference - regardless of whether the object referred to is actually mutable, const, or immutable.

Now, realistically, you're going to get away with casting away const and mutating almost all of the time, because the assumptions that the compiler can make on const alone are pretty limited, but it's still undefined behavior.

- Jonathan M Davis

On Sunday, 15 November 2015 at 14:54:51 UTC, Andrei Alexandrescu wrote:
> On 11/15/2015 09:34 AM, Dicebot wrote:
>> On Sunday, 15 November 2015 at 14:23:05 UTC, Jonathan M Davis wrote:
>>>> As I mentioned, he's okay with changing the language to make the
>>>> casts well defined. -- Andrei
>>>
>>> Well, that's a big change, since it pretty much means that D's const
>>> isn't physical const anymore, and Walter has been _very_ insistent
>>> about that in the past - to the point that he's argued that C++'s
>>> const is outright useless because it isn't physical const. If casting
>>> away const and mutating is well-defined behavior, then we basically
>>> have C++'s const except that it's transitive ...
>>
>> Casting away _const_ is already legal if programmer can himself
>> guarantee underlying object has mutable origin (i.e. not available via
>> immutable reference), by the very definition of const. It is casting
>> away immutable and mutating that is strictly UB.
>
> Correct. I'm not sure whether that's clarified in the language documentation yet. -- Andrei

Quite the opposite in fact. Walter recently approved an update to the spec which clarified that it _is_ undefined behavior to cast away const and mutate even if the object being referred to isn't immutable underneath the hood:

https://github.com/D-Programming-Language/dlang.org/pull/1047

And from what I've seen, Walter has always been adamant that D's const is physical const and that casting away const and mutating is undefined behavior, arguing that if it weren't, it would be like C++'s const and that C++'s const is pretty much useless, because it doesn't provide actual guarantees. So, if he's now willing to have casting away const and mutating be well-defined behavior, that's a big shift.

- Jonathan M Davis

On Sunday, 15 November 2015 at 15:09:25 UTC, Jonathan M Davis wrote:
> After the call to bar, the compiler can guarantee that the value of foo has not changed, because it's not possible for bar to access foo except via a const reference, and it can clearly see that, because it can see that it's not possible for bar to access foo except via the reference that's passed to it.

Note that it can only do so if escape analysis is able to prove no other reference escapes to the same data. Otherwise it can be mutated even during that function call in other thread. All together it makes benefits of such deduction absolutely not worth UB limitation in my opinion.

For pure functions it is quite different story though and I agree that demanding it to be UB there is totally legitimate.

On Sunday, 15 November 2015 at 15:20:24 UTC, Dicebot wrote:
> On Sunday, 15 November 2015 at 15:09:25 UTC, Jonathan M Davis wrote:
>> After the call to bar, the compiler can guarantee that the value of foo has not changed, because it's not possible for bar to access foo except via a const reference, and it can clearly see that, because it can see that it's not possible for bar to access foo except via the reference that's passed to it.
>
> Note that it can only do so if escape analysis is able to prove no other reference escapes to the same data. Otherwise it can be mutated even during that function call in other thread. All together it makes benefits of such deduction absolutely not worth UB limitation in my opinion.
>
> For pure functions it is quite different story though and I agree that demanding it to be UB there is totally legitimate.

Probably also worth mentioning that I don't consider use case like in Andrei example (casting away const to mutate allocator/refcount of const argument) a legitimate example as it is public API and guarantees of mutability of underlying object are actually impossible.

Use case I had myself it to cast away const inside invariants but this was only ok because we know for sure to never use immutable objects _within out own code_. In Phobos such casting is very unlikely to be future-proof.

On 11/15/2015 10:09 AM, Jonathan M Davis wrote:
> const in D guarantees that the object will not be mutated via that
> reference regardless of what that reference refers to.

We need to change that if we want things like composable containers that work with const. And I think it's a good thing to want. -- Andrei

On Sunday, 15 November 2015 at 15:20:24 UTC, Dicebot wrote:
> On Sunday, 15 November 2015 at 15:09:25 UTC, Jonathan M Davis wrote:
>> After the call to bar, the compiler can guarantee that the value of foo has not changed, because it's not possible for bar to access foo except via a const reference, and it can clearly see that, because it can see that it's not possible for bar to access foo except via the reference that's passed to it.
>
> Note that it can only do so if escape analysis is able to prove no other reference escapes to the same data. Otherwise it can be mutated even during that function call in other thread.

Except that shared isn't involved here. The objects are thread-local, so other threads don't matter. The combination of TLS and pure does make it so that the compiler could theoretically do optimizations based on const. immutable is not actually required, though it obviously provides far better guarantees.

However, even with full-on escape analysis, the compiler is pretty limited in where it can guarantee that a const object isn't mutated by another reference. So, it really doesn't mean much. But my point was that there _are_ cases when the compiler can do optimizations based on const without immutable being involved, even if they're rare.

>  All together it makes benefits of such deduction absolutely not worth UB limitation in my opinion.

The primary benefit of making it undefined behavior to cast away const and mutate is that then you actually have the guarantee that an object will not be mutated via a const reference. You get actual, physical const. As long as there's a backdoor out of const, const provides no real guarantees. Rather, it just prevents accidental mutation and serves as documentation for which functions are supposed to be able to mutate the logical state on of an object - which is why (as I understand it at least) Walter has typically argued that C++'s const is useless.

Now, given how incredibly limiting physical const is, maybe we should be more lax with D's const, but if we are, we lose out on the compiler guarantees that we currently get and basically end up with a transitive version of C++'s const (except that we have the added risk of mutating an immutable object if we're not careful).

- Jonathan M Davis

On Sunday, 15 November 2015 at 17:38:10 UTC, Andrei Alexandrescu wrote:
> On 11/15/2015 10:09 AM, Jonathan M Davis wrote:
>> const in D guarantees that the object will not be mutated via that
>> reference regardless of what that reference refers to.
>
> We need to change that if we want things like composable containers that work with const. And I think it's a good thing to want. -- Andrei

Basically, we have to decide between having physical const with the guarantees that it provides (but losing out on being able to use const in a _lot_ of places) and having a transitive version of C++'s const (which means losing out on the solid compiler guarantees).

The guarantees that we get with physical const are really nice, but yes, they do come at the cost of not being able to use const in a _lot_ cases. So, it could very well be that we'd better off losing physical const. And one thing to consider is that there are plenty of folks who mistakenly think that it's okay to cast away const and mutate an object as long as the object isn't actually immutable, so there's plenty of code out there that already does that. It almost certainly works, because the optimizations that the compiler can make based on const alone are extremely limited, but it _is_ undefined behavior, and if it's that easy for folks to misunderstand and end up relying on UB, maybe we should just change it so that it's defined behavior.

I don't know. I like physical const as it is, but it so often seems too limiting to actually be useful, which ultimately probably means that we need to change it (in order to take into account human behavior if nothing else, since const is clearly being misunderstood and misused as-is).

That being said, even if we make it defined behavior to mutate a mutable object via a const reference via casting, I think that having @mutable like you suggested would still be a good idea, because it would cover one of the primary use cases where casting would be required and eliminate the need for that cast and disallowing immutable where it would be violated if it were used, making such idioms safer.

The other major use case that I can think of would be lazy initialization, and @mutable doesn't seem like the appropriate thing for that, but maybe we can add something else for it if we're already loosening the restrictions/guarantees on const (I think that someone just created a DIP for that too, though I haven't looked at it yet).

Regardless, we need to be very careful about what we do with casts involving const because of how easy it is to screw up and mutate an immutable objects in the process.

- Jonathan M Davis

On 11/15/2015 01:00 PM, Jonathan M Davis wrote:
> Basically, we have to decide between having physical const with the
> guarantees that it provides

We have that - it's immutable. -- Andrei