Razvan Nitu is working on the DIP initiated by Timon Gehr, known colloquially as the one that introduces __mutable - i.e. a mechanism for allowing controlled changes to immutable data. Here's a draft:

https://github.com/RazvanN7/DIPs/blob/Mutable_Dip/DIPs/DIP1xxx-rn.md

We figured that we're dealing a misnomer - we don't want __mutable, but instead __metadata - information that is nominally part of the object but needs certain leeway from the type system. Typical use cases are:

* reference counting of immutable data structures
* caching
* lazy evaluation

We got stuck at the interaction of __mutable with const parent objects (unclear whether the parent object originated as immutable or unqualified), and how pure functions should deal with __mutable. The few solutions we are toying with are either incomplete or too complicated (or both).

The help of a few PL and compiler specialists would be very valuable here. I'm cc'ing a few, if anyone wants to help please let us know.

I'm concerned that it will be used as an escape from the type system arbitrarily. Right now there is nothing to discourage the abuse of this storage class.

Is there something we can do to discourage abuse?

On Saturday, 13 April 2019 at 01:17:55 UTC, rikki cattermole wrote:
> I'm concerned that it will be used as an escape from the type system arbitrarily. Right now there is nothing to discourage the abuse of this storage class.
>
> Is there something we can do to discourage abuse?

From skimping through the DIP it doesn't look too dangerous it seems like just a fancy 'shared' escape for immutable fields, the rest looks the same as casting immutable away in @system code.

On Saturday, 13 April 2019 at 00:45:07 UTC, Andrei Alexandrescu wrote:
> ...

The problems you encountered are normal but the question is whether this feature is useful at all if the field is not explicitly shared? If so the a solution to the const case is to introduce an '@unshared' attribute instead which would protect against the worst cases of both being shared and being thread local. this attribute would disallow unsychronized read/write and at the same time prevent implicit conversion to 'shared' since it may just be thread local hence mixing it with normal 'shared' may not bring good results.

example:
```
struct S
{
    private shared int* p; // or @unshared

    void inc() inout pure @system
    {
        import core.atomic;
        atomicOp!"+="(*cast(shared(int*))p, 1); // or @unshared
    }
}

void foo(ref immutable S a) pure
{
    a.inc();
}

void foo(ref const S a) pure
{
    a.inc();
}

int x; // thread local
shared int y;

void main()
{
    auto i = immutable S(cast(immutable)&x);
    auto c = const S(&y);
    //auto d = const S(cast(@unshared)&x);
    foo(i);
    foo(c);
}
```

On 13/04/2019 3:45 PM, Suleyman wrote:
> On Saturday, 13 April 2019 at 01:17:55 UTC, rikki cattermole wrote:
>> I'm concerned that it will be used as an escape from the type system arbitrarily. Right now there is nothing to discourage the abuse of this storage class.
>>
>> Is there something we can do to discourage abuse?
> 
>  From skimping through the DIP it doesn't look too dangerous it seems like just a fancy 'shared' escape for immutable fields, the rest looks the same as casting immutable away in @system code.

What you described is a feature that escapes the type system which can lead to program crashes. Read only memory is what I'm concerned about.

On Saturday, 13 April 2019 at 04:49:01 UTC, rikki cattermole wrote:
> ...

The DIP says you can edit a `__mutable` only in `@system` mode however if this is just a sugar around an unsafe cast I don't like it I like unsafe code to be ugly and scary.

On 13.04.19 02:45, Andrei Alexandrescu wrote:
> Razvan Nitu is working on the DIP initiated by Timon Gehr,

(Disclaimer: Note that the current state of the DIP is significantly different from my original proposal, and I would not argue for acceptance of the current state of the DIP even though I am listed as an author.)

> known colloquially as the one that introduces __mutable - i.e. a mechanism for allowing controlled changes to immutable data. Here's a draft:
> 
> https://github.com/RazvanN7/DIPs/blob/Mutable_Dip/DIPs/DIP1xxx-rn.md
> 
> We figured that we're dealing a misnomer - we don't want __mutable, but instead __metadata - information that is nominally part of the object but needs certain leeway from the type system. Typical use cases are:
> 
> * reference counting of immutable data structures

For that you need the more general (originally supported using __mutable functions):

* manual allocation and deallocation of immutable data

> * caching
> * lazy evaluation
> 
> We got stuck at the interaction of __mutable with const parent objects (unclear whether the parent object originated as immutable or unqualified),

I think we got that one sorted out.

> and how pure functions should deal with __mutable. The few solutions we are toying with are either incomplete or too complicated (or both).
> 
> The help of a few PL and compiler specialists would be very valuable here. I'm cc'ing a few, if anyone wants to help please let us know.

If the principled approach is too complicated, the feature is not worth it. My original proposal was to define a set of rewrites that is based on the function signature alone which is semantics-preserving if there is no __metadata (except possibly reference identity of immutable data), and then say that this set of rewrites is still permissible even if there are __metadata annotations. If complexity of the DIP is a concern, I guess we could simplify this by just saying that any rewrite based on the function signature alone that is sound without __metadata will remain permissible with it.

On 13.04.19 06:49, rikki cattermole wrote:
> On 13/04/2019 3:45 PM, Suleyman wrote:
>> On Saturday, 13 April 2019 at 01:17:55 UTC, rikki cattermole wrote:
>>> I'm concerned that it will be used as an escape from the type system arbitrarily. Right now there is nothing to discourage the abuse of this storage class.
>>>
>>> Is there something we can do to discourage abuse?
>>
>>  From skimping through the DIP it doesn't look too dangerous it seems like just a fancy 'shared' escape for immutable fields, the rest looks the same as casting immutable away in @system code.
> 
> What you described is a feature that escapes the type system which can lead to program crashes. Read only memory is what I'm concerned about.

Won't happen, because data with __metadata annotations inside will not be put in read-only memory. (This can be known statically.)

On Saturday, 13 April 2019 at 08:58:06 UTC, Timon Gehr wrote:
> Won't happen, because data with __metadata annotations inside will not be put in read-only memory. (This can be known statically.)

If it's adding these kinds of protections then I think it would be worth it and a step forward in making D more safe. you might also consider the 'shared' protection I mentioned earlier then maybe you can lift the restriction on modification in @safe code.

On 14.04.19 02:19, Suleyman wrote:
> On Saturday, 13 April 2019 at 08:58:06 UTC, Timon Gehr wrote:
>> Won't happen, because data with __metadata annotations inside will not be put in read-only memory. (This can be known statically.)
> 
> If it's adding these kinds of protections then I think it would be worth it and a step forward in making D more safe. you might also consider the 'shared' protection I mentioned earlier then maybe you can lift the restriction on modification in @safe code.

I don't think it should be @safe. Rather, `pure` and `immutable` should retain their meanings, which implies that there are wrong ways to use `__mutable` (hence unsafe), and there are still non-cosmetic reasons to use `immutable` even if there are `__mutable` fields.