On Wed, Aug 29, 2018 at 06:58:16PM +0200, Timon Gehr via Digitalmars-d wrote:
> On 28.08.2018 19:02, H. S. Teoh wrote:
> > On Tue, Aug 28, 2018 at 08:18:57AM +0000, Eugene Wissner via Digitalmars-d wrote: [...]
> > > There are still valid use cases where const should be "broken". One of them is mutex (another one caching). I have very little experiance in multi-threaded programming, but what do you think about "mutable" members, despite the object is const?
> > The problem with compromising const is that it would invalidate any guarantees const may have provided.
> 
> No. You start with the set of allowed program rewrites, then require code with __mutable to not break under them. Code using __mutable is unsafe.

Currently, immutable implicitly converts to const. If const is allowed to be overridden, then you could violate immutable, which is UB.


> > Const in D is not the same as const in languages like C++; const in D means*physical*  const, as in, the data might reside in ROM where it's physically impossible to modify.  Allowing the user to bypass this means UB if the data exists in ROM.
> > 
> > Plus, the whole point of const in D is that it is machine-verifiable, i.e., the compiler checks that the code does not break const in any way and therefore you are guaranteed (barring compiler bugs) that the data does not change.  If const were not machine-verifiable, it would be nothing more than programming by convention, since it would guarantee nothing.  Allowing const to be "broken" somewhere would mean it's no longer machine-verifiable (you need a human to verify whether the semantics are still correct).
> 
> It is not unusual to need a human to verify that your code does what it was intended to do.

And it is not unusual for humans to make mistakes and certify code that is not actually correct.  Automation provides much stronger guarantees than human verification.

Besides, this is missing the point.  What I meant was that if const could be arbitrarily overridden anywhere down the call chain, then the compiler could no longer feasibly verify that a particular piece of code doesn't violate const. The code could be calling a function for which the compiler has no source code, and who knows what that function might do. It could override const and modify the data willy-nilly, and if the const reference is pointing to an immutable object, you're in UB land.

Not allowing const to be overridden (without the user deliberately treading into UB land by casting it away) allows the compiler to statically check that the code doesn't actually modify a const object.

You appear to be thinking I was making a statement about verifying program correctness in general, which is taking what I said out of context.


T

-- 
It is not the employer who pays the wages. Employers only handle the money. It is the customer who pays the wages. -- Henry Ford

On Wednesday, August 29, 2018 10:38:48 AM MDT Ali Çehreli via Digitalmars-d wrote:
> On 08/22/2018 10:20 PM, Nicholas Wilson wrote:
> > That reminds me, what happened to our conversation with Ali Çehreli about splitting general [newsgroup/forum] into Technical and less technical?
> Even I remember that conversation. :) I don't remember who were involved but as soon as I opened the discussion at the next dinner table, I was discouraged; I think people were against the idea.

As it is, we're lucky if we can even get folks to post in D.Learn when their post should be there instead of the main newsgroup...

Also, dlang-study was already basically an attempt to get more focused, technical discussions separate from the main newsgroup, and it didn't work.

- Jonathan M Davis

On 30/08/2018 5:20 AM, Jonathan M Davis wrote:
> Also, dlang-study was already basically an attempt to get more focused,
> technical discussions separate from the main newsgroup, and it didn't work.

It wasn't as simple as posting to D.General. It required subscription (based upon what I did) and it couldn't just be posted to like D.General even from e.g. Thunderbird. So there was a barrier to actually posting there.

On Wednesday, August 29, 2018 11:15:15 AM MDT H. S. Teoh via Digitalmars-d wrote:
> On Wed, Aug 29, 2018 at 06:58:16PM +0200, Timon Gehr via Digitalmars-d
wrote:
> > On 28.08.2018 19:02, H. S. Teoh wrote:
> > > On Tue, Aug 28, 2018 at 08:18:57AM +0000, Eugene Wissner via Digitalmars-d wrote: [...]
> > >
> > > > There are still valid use cases where const should be "broken". One of them is mutex (another one caching). I have very little experiance in multi-threaded programming, but what do you think about "mutable" members, despite the object is const?
> > >
> > > The problem with compromising const is that it would invalidate any guarantees const may have provided.
> >
> > No. You start with the set of allowed program rewrites, then require code with __mutable to not break under them. Code using __mutable is unsafe.
>
> Currently, immutable implicitly converts to const. If const is allowed to be overridden, then you could violate immutable, which is UB.

If I understand correctly, the main reason behind looking to add __mutable is to be able to do stuff with containers that can't currently be done where you have a way of knowing whether the actual data is truly immutable or not and thus can avoid mutating data that's actually immutable. That's still undefined behavior right now, but presumably, if __mutable were added, it would then be defined behavior that was highly @system and not intended for normal code. However, even allowing that much does make it so that the compiler can't then do any optimizations based on const, since while it may be possible in some cases to avoid mutating immutable data when casting away const and mutating, I don't see how it would be possible to guarantee that it would be done in a way that could not possibly be screwed by up optimizations made at a higher level based on the fact that the objects in question are typed as const.

Basically, it seems that Andrei really wants a backdoor in const for certain uses cases, so he's looking to find a way to enable it without really putting a backdoor in const (IIRC it was discussed as part of this year's dconf talk talking about the new containers that one of the students is working on). I guess that he's managed to talk Timon into working on the issue for him given Timon's excellent knowledge about the type system and about the related computer science concepts. We'll see what they come up with, but it's going to be _very_ difficult to make it so that you can actually rely on const's guarantees if it has any kind of backdoors at all. However, given some of the technical issues that they've run into with allocators and containers, Andrei has been rather motivated to change the status quo. We'll see what happens.

- Jonathan M Davis

On 28.08.2018 03:11, Walter Bright wrote:
> On 8/27/2018 10:08 AM, H. S. Teoh wrote:
>> Const in D makes sense as-is.  Though, granted, its infectiousness means
>> its scope is actually very narrow, and as a result, we ironically can't
>> use it in very many places, and so its touted benefits only rarely
>> apply. :-(  Which also means that it's taking up a lot of language
>> design real estate with not many benefits to show for it.
> 
> D const is of great utility if you're interested in functional programming.

D const/immutable is stronger than immutability in Haskell (which is usually _lazy_).

On 29.08.2018 19:15, H. S. Teoh wrote:
> On Wed, Aug 29, 2018 at 06:58:16PM +0200, Timon Gehr via Digitalmars-d wrote:
>> On 28.08.2018 19:02, H. S. Teoh wrote:
>>> On Tue, Aug 28, 2018 at 08:18:57AM +0000, Eugene Wissner via Digitalmars-d wrote:
>>> [...]
>>>> There are still valid use cases where const should be "broken".
>>>> One of them is mutex (another one caching). I have very little
>>>> experiance in multi-threaded programming, but what do you think
>>>> about "mutable" members, despite the object is const?
>>> The problem with compromising const is that it would invalidate any
>>> guarantees const may have provided.
>>
>> No. You start with the set of allowed program rewrites, then require
>> code with __mutable to not break under them. Code using __mutable is
>> unsafe.
> 
> Currently, immutable implicitly converts to const. If const is allowed
> to be overridden, then you could violate immutable, which is UB.
> ...

__mutable fields are __mutable also in the immutable instance. You might get into trouble with shared if you are not careful because of the unfortunate "implicit shared" semantics of immutable, but it is up to the programmer to get this right.

> 
>>> Const in D is not the same as const in languages like C++; const in
>>> D means*physical*  const, as in, the data might reside in ROM where
>>> it's physically impossible to modify.  Allowing the user to bypass
>>> this means UB if the data exists in ROM.
>>>
>>> Plus, the whole point of const in D is that it is
>>> machine-verifiable, i.e., the compiler checks that the code does not
>>> break const in any way and therefore you are guaranteed (barring
>>> compiler bugs) that the data does not change.  If const were not
>>> machine-verifiable, it would be nothing more than programming by
>>> convention, since it would guarantee nothing.  Allowing const to be
>>> "broken" somewhere would mean it's no longer machine-verifiable (you
>>> need a human to verify whether the semantics are still correct).
>>
>> It is not unusual to need a human to verify that your code does what
>> it was intended to do.
> 
> And it is not unusual for humans to make mistakes and certify code that
> is not actually correct.  Automation provides much stronger guarantees
> than human verification.
> ...

Absolutely. But D only strives to provide such automation in @safe code. For @system code, we need a formal specification of what is allowed. (And it needs to include all things that the GC and language do; no magic.) Note that such a formal specification is a prerequisite for any (possibly language-external) automated verification approaches.

> Besides, this is missing the point.  What I meant was that if const
> could be arbitrarily overridden anywhere down the call chain, then the
> compiler could no longer feasibly verify that a particular piece of code
> doesn't violate const. The code could be calling a function for which
> the compiler has no source code, and who knows what that function might
> do. It could override const and modify the data willy-nilly, and if the
> const reference is pointing to an immutable object, you're in UB land.
> 
> Not allowing const to be overridden (without the user deliberately
> treading into UB land by casting it away) allows the compiler to
> statically check that the code doesn't actually modify a const object.
> 
> You appear to be thinking I was making a statement about verifying
> program correctness in general, which is taking what I said out of
> context.
> 
> 
> T
> 

I was thinking you were making a statement about __mutable fields.

On Wednesday, 29 August 2018 at 18:02:16 UTC, Timon Gehr wrote:
> On 29.08.2018 19:15, H. S. Teoh wrote:
>> On Wed, Aug 29, 2018 at 06:58:16PM +0200, Timon Gehr via Digitalmars-d wrote:
>>> On 28.08.2018 19:02, H. S. Teoh wrote:
>>>> On Tue, Aug 28, 2018 at 08:18:57AM +0000, Eugene Wissner via Digitalmars-d wrote:
>>
>> 
>> Currently, immutable implicitly converts to const. If const is allowed
>> to be overridden, then you could violate immutable, which is UB.
>> ...
>
> __mutable fields are __mutable also in the immutable instance. You might get into trouble with shared if you are not careful because of the unfortunate "implicit shared" semantics of immutable, but it is up to the programmer to get this right.

So you cant cast away const but you can specify a field stays mutable even if the aggregate is const or immutable?

On Wed, Aug 29, 2018 at 07:50:57PM +0200, Timon Gehr via Digitalmars-d wrote:
> On 28.08.2018 03:11, Walter Bright wrote:
> > On 8/27/2018 10:08 AM, H. S. Teoh wrote:
> > > Const in D makes sense as-is.  Though, granted, its infectiousness means its scope is actually very narrow, and as a result, we ironically can't use it in very many places, and so its touted benefits only rarely apply. :-(  Which also means that it's taking up a lot of language design real estate with not many benefits to show for it.
> > 
> > D const is of great utility if you're interested in functional programming.
> 
> D const/immutable is stronger than immutability in Haskell (which is
> usually _lazy_).

This makes me wonder: is it possible to model a lazy immutable value in D?

Likely not, if we were to take the immutability literally, since once the variable is marked immutable and initialized, you couldn't change it afterwards (without casting and UB).

We *might* be able to get away with a head-mutable reference to the data, though. Say something like this:

	struct LazyImmutable(T, alias initializer)
	{
		immutable(T)* impl;
		@property T get() {
			if (impl is null)
				impl = initializer();
			return *impl;
		}
		alias get this;
	}

Seems rather cumbersome to use in practice, though.  And adds indirection overhead to by-value types. One could possibly use emplace to alleviate that, but still, the variable itself cannot be marked immutable without breaking its functionality.

Which means you couldn't rely on such a wrapper type to work transitively in complex types, unlike how immutable applies transitively to all aggregate members: if T was an aggregate type, they couldn't be LazyImmutable, but must be actually immutable.  Maybe this can be made to work, but at the sacrifice of being unable to use built-in type qualifiers like const/immutable.


T

-- 
Fact is stranger than fiction.

On Wed, Aug 29, 2018 at 07:02:42PM +0000, Dave Jones via Digitalmars-d wrote:
> On Wednesday, 29 August 2018 at 18:02:16 UTC, Timon Gehr wrote:
> > On 29.08.2018 19:15, H. S. Teoh wrote:
> > > On Wed, Aug 29, 2018 at 06:58:16PM +0200, Timon Gehr via Digitalmars-d wrote:
> > > > On 28.08.2018 19:02, H. S. Teoh wrote:
> > > > > On Tue, Aug 28, 2018 at 08:18:57AM +0000, Eugene Wissner via Digitalmars-d wrote:
> > > 
> > > Currently, immutable implicitly converts to const. If const is allowed to be overridden, then you could violate immutable, which is UB.  ...
> > 
> > __mutable fields are __mutable also in the immutable instance. You might get into trouble with shared if you are not careful because of the unfortunate "implicit shared" semantics of immutable, but it is up to the programmer to get this right.
> 
> So you cant cast away const but you can specify a field stays mutable even if the aggregate is const or immutable?

That appears to be the case.  But it scares me that const(T) would no longer guarantee you can't modify anything in T.  I fear it will break some subtle assumptions about how const/immutable works, and introduce hidden bugs to existing code.


T

-- 
Doubt is a self-fulfilling prophecy.

On Wednesday, 29 August 2018 at 17:15:15 UTC, H. S. Teoh wrote:
> Besides, this is missing the point.  What I meant was that if const could be arbitrarily overridden anywhere down the call chain, then the compiler could no longer feasibly verify that a particular piece of code doesn't violate const. The code could be calling a function for which the compiler has no source code, and who knows what that function might do. It could override const and modify the data willy-nilly, and if the const reference is pointing to an immutable object, you're in UB land.
>
> Not allowing const to be overridden (without the user deliberately treading into UB land by casting it away) allows the compiler to statically check that the code doesn't actually modify a const object.
>
> You appear to be thinking I was making a statement about verifying program correctness in general, which is taking what I said out of context.
>
>
> T

You keep saying that, it has to be machine verifiable, but honestly I don't see the benefit to being machine verifiable. As in the case it can't verify the object doesn't change in scope at all, it can only verify that the code in scope doesn't modify it. I'd rather have C++ const and be useful than avoiding const almost completely.