In the recent discussion about reference counting, a question that's repeatedly come up is, how can we store a pointer to immutable data in immutable memory?

So far, suggested solutions have included:

• Add a __mutable qualifier that functions as a "back door" to immutable and const.
• Store the mutable data at some offset outside the object's declared layout, and access it with pointer arithmetic.
• Store the mutable data in a global associative array.

All of these solutions have unsatisfying tradeoffs: some are incompatible with pure; others require language changes to work without causing undefined behavior. It seems inevitable that we will have to accept some kind of compromise--our only choice is which one.

Except...what if we don't? What if the One Weird Trick that will give us everything we want has been sitting under our noses all along?

I present: TailUnqual!

import tail_unqual;

void main() @safe pure
{
    // Qualifiers on a TailUnqual variable are not transitive--that is,
    // they apply only to the variable itself, not the data it points to.
    immutable TailUnqual!(int*) p = new int(123);
    assert(*p == 123);

    // We can't mutate `p` itself, because it's immutable...
    assert(!__traits(compiles,
        p = immutable(TailUnqual!(int*))(new int(789))
    ));

    // ...but we can mutate the data it points to!
    *p = 456;
    assert(*p == 456);
}

How does it work? How can I possibly break such a fundamental language rule and get away with it? As it turns out, the answer is almost embarrassingly simple:

• You're allowed to cast a pointer to an integer, and then cast that integer back to the same pointer.
• You're allowed to convert integer values freely between mutable, const, and immutable.

So, all we have to do is (a) store the pointer as an integer, and (b) convert it back to a pointer every time we access it. Add a little bit of union seasoning to satisfy the GC, and you get the embarrassingly simple implementation behind this magic trick:

module tail_unqual;

import std.traits: isPointer;

private union HiddenPointer
{
    // Stores a pointer, cast to an integer
    size_t address;

    // Enables GC scanning (and prevents some other UB)
    void* unused;
}

struct TailUnqual(P)
    if (isPointer!P)
{
    private HiddenPointer data;

    this(P ptr) inout
    {
        data.address = cast(size_t) ptr;
    }

    @trusted @property
    P ptr() inout
    {
        // This is safe because:
        //  - `address` is always the active field of `data`
        //  - `data.address` was originally cast from a `P`
        return cast(P) data.address;
    }

    alias ptr this;
}

Try it yourself on run.dlang.io.

What do you think? Is it just crazy enough to work, or just plain crazy? Is there some fatal safety violation I've overlooked? Let me know in your replies!

On Saturday, 13 November 2021 at 06:50:48 UTC, Paul Backus wrote:

This technique is worth utilizing.
The pointer remains immutable and the pointee data is mutable.

On Saturday, 13 November 2021 at 07:20:37 UTC, Timon Gehr wrote:

void main()@safe{
    immutable TailUnqual!(int*) p = new int(123);
    auto mut = p.ptr;
    immutable immut = p.ptr;
    import std.stdio;
    writeln(*immut); // 123
    *mut=0;
    assert(mut is immut);
    assert(*mut == *immut);
    writeln(*mut," ",*immut); // 0 123
}

That's a compiler bug. The explicit return type of ptr is being ignored when assigning to immutable immut. Stupid fix:

    private __gshared int dummy = int.min;

    @trusted @property
    P ptr() inout
    {
        P ret = & dummy; // Persuade the frontend to actually use the explicit return type.
        ret = cast(P) data.address;
        return ret;
    }

Now the compiler will correctly identify your main as illegal:

onlineapp.d(43): Error: cannot implicitly convert expression `p.ptr()` of type `int*` to `immutable(int*)`

On Saturday, 13 November 2021 at 06:50:48 UTC, Paul Backus wrote:

It might be crazy 😅, but would be nice if some variant of this is made to work, because it's a very simple solution