On 11/13/2015 06:41 PM, Timon Gehr wrote:
> On 11/14/2015 12:10 AM, Andrei Alexandrescu wrote:
>>
>> * Lines 6: By construction the list doesn't work with mutable objects.
>
> =(

I've decided persistent lists with mutable elements just too weird to endorse. Lisp allows them but every time it mentions that it very strongly advises against it.

Other container semantics are better, I think, for mutable elements. Let's leave persistent lists have their nice value semantics.


Andrei

On 11/13/2015 07:28 PM, Jakob Ovrum wrote:
> On Friday, 13 November 2015 at 23:10:04 UTC, Andrei Alexandrescu wrote:
>> * Lines 11-12: I came to terms with the notion that some types cannot
>> be made immutable. We've been trying to do reference counting on
>> immutable objects for a long time. It's time to acknowledge that true
>> immutability (which the immutable keyword models) and reference
>> counting don't mix.
>
> External reference counting should work fine with immutable, i.e.
> RefCounted!(immutable T).

I've been asking this same question myself, vaguely, many times. Whenever I want to get work done using that mindset, I can't. Then I bring it up to Walter, and he asks again the same. Then I don't know how to explain it. Time to nip it in the bud once and for all. Clarify it forever so no need to bring it up again, ever.

The general pattern goes like this: "So okay, understood, immutable doesn't work with reference counting. Then, clearly what we need is an object with an immutable part and a mutable part! Instead of immutable(RC!T), we use RC!(immutable T) throughout. It has a mutable reference count and an immutable Problem solved!"

Technically, that clearly works. There's a problem with scaling it up:

COMPOSITION

Disallowing immutable(RC!T) in favor of RC!(immutable T) effectively disables composition of larger immutable objects from smaller ones.

One obvious thing to do is with a reference counted object it to make it a field of a larger object. That effectively makes that larger object impossible to be used with immutable, transitively.

That's not a bad thing; it's merely a factual acknowledgment that in the D programming language, immutable models (transitively and in a way that makes composition possible) true immutability of bits, which makes sharing possible at no cost. Reference counting is mutating, therefore it does not and cannot work with immutability as defined by D.

So disabling immutable(RC!T) in favor of RC!(immutable T) is a fine way to approach things, but not a solution to the general problem of making general reference counted D objects mutable. That problem does not have a solution.


Andrei

I don't think immutable should be something casual and widespread in most designs at all. It is a very restrictive qualifier with an extremely narrow use case, trying to retro-fit it to be easy to use and compose tends to do more harm than actually going with mutable instead.

All trouble comes from trying to use physical immutable as logical one while still pretending it gives physical guarantees. Even if existing immutability is not widely applicable, I'd prefer to have narrow applicability over wide false confidence. Right now I know for sure that if I can use immutable data without any thread locking and it is not possible to screw it up. It is rarely important, but when it is, it is priceless.

P.S. another problem with `RC!(immutable T)` is that it actually must be `shared(RC!(immutable T)` to be provide @safe API.

On 11/14/2015 04:45 PM, Andrei Alexandrescu wrote:
> On 11/13/2015 06:41 PM, Timon Gehr wrote:
>> On 11/14/2015 12:10 AM, Andrei Alexandrescu wrote:
>>>
>>> * Lines 6: By construction the list doesn't work with mutable objects.
>>
>> =(
>
> I've decided persistent lists with mutable elements just too weird to
> endorse.

Not /enforcing/ _transitive_ immutability is not the same as endorsing arbitrary mutation patterns as reasonable.

> Lisp allows them but every time it mentions that it very
> strongly advises against it.
> ...

This is just not true. E.g. lazy thunks are considered fine.

> Other container semantics are better, I think, for mutable elements.

Right, because e.g. whether the elements are reference counted or traced should totally completely change the required container semantics for the problem at hand! This does not make any sense, unless you are saying that nobody should actually use the persistent containers, in which case it seems like a waste of time to concentrate efforts on them.

You have not provided a technical argument, so I am going to assume you have an irrational fear of bad PR. Forcing transitive immutable on people who want persistent container semantics is ultimately a very poor choice both from a technical and a PR perspective.

> Let's leave persistent lists have their nice value semantics.
> ...

It does not make sense to conflate transitive immutability and value semantics, unless one considers it reasonable to make all struct fields transitively tail-immutable.

On 11/14/2015 05:05 PM, Andrei Alexandrescu wrote:
>
> Technically, that clearly works. There's a problem with scaling it up:
>
> COMPOSITION
> ...

Composition matters.

> Disallowing immutable(RC!T) in favor of RC!(immutable T) effectively
> disables composition of larger immutable objects from smaller ones.
>
> One obvious thing to do is with a reference counted object it to make it
> a field of a larger object. That effectively makes that larger object
> impossible to be used with immutable, transitively.
>
> That's not a bad thing; it's merely a factual acknowledgment that in the
> D programming language, immutable models (transitively and in a way that
> makes composition possible) true immutability of bits, which makes
> sharing possible at no cost. Reference counting is mutating, therefore
> it does not and cannot work with immutability as defined by D.

Obviously. But RC /does/ work with persistent containers! (Unless the author of the container has added a gratuitous compile-time check to make it impossible.)

On 11/14/2015 03:36 PM, Timon Gehr wrote:
> Right, because e.g. whether the elements are reference counted or traced
> should totally completely change the required container semantics for
> the problem at hand! This does not make any sense, unless you are saying
> that nobody should actually use the persistent containers, in which case
> it seems like a waste of time to concentrate efforts on them.

I'm just as weary of persistent containers of mutable elements using GC. It doesn't make a difference.

> You have not provided a technical argument, so I am going to assume you
> have an irrational fear of bad PR. Forcing transitive immutable on
> people who want persistent container semantics is ultimately a very poor
> choice both from a technical and a PR perspective.

The English language has a word I like a lot: "unassuming". I'm not sure about its etymology, but a nice theory is "a person who doesn't assume bad things about others".

I'm glad to hear about various pros and cons regarding these containers. Slinging this kind of stuff is unlikely to further the dialog.


Andrei

On 11/14/2015 12:10 AM, Andrei Alexandrescu wrote:
> ...
> * Lines 11-12: I came to terms with the notion that some types cannot be
> made immutable. We've been trying to do reference counting on immutable
> objects for a long time. It's time to acknowledge that true immutability
> (which the immutable keyword models) and reference counting don't mix.
> Const does work (more below). But overall I'm at peace with the notion
> that if you can't live without immutable, I'll refer you to the garbage
> collector.
> ...

I.e. you think it is fine that there can be no list of lists.
Anyway, the static assert does not actually do what you intend it to do.

> * Lines 26-29: The allocator is fundamentally a mutable part of the
> container. This is an insufficiency of our type system - we can't say
> "this object may be constant, but this reference is to a mutable part of
> it". We can't explain that necessity out of existence, and List is part
> of the proof. So we need to make that cast legal.
> ...

Just don't make the method const. (This is not C++.)

On Saturday, 14 November 2015 at 16:27:17 UTC, Dicebot wrote:
> All trouble comes from trying to use physical immutable as logical one while still pretending it gives physical guarantees. Even if existing immutability is not widely applicable, I'd prefer to have narrow applicability over wide false confidence. Right now I know for sure that if I can use immutable data without any thread locking and it is not possible to screw it up. It is rarely important, but when it is, it is priceless.

I can't say I'm following this discussion in great detail, but
one thing strikes me.  "const"-ness and "immutable"-ility are
at some abstract level flavors of value stability.  And that
suggests to me that perhaps we should not be looking so much
for backdoors as for new terms, leaving the old terms alone.
Perhaps a given old keyword won't work well with RC; so be it.
Perhaps a new keyword such as "stable" could be used to describe
a storage category where the payload fields are unchanging but
any metadata fields are potentially mutable.  Then perhaps const
could mean physically immutable, while stable means logically
immutable.  Or something like that.

On 11/14/2015 03:55 PM, Timon Gehr wrote:
> On 11/14/2015 12:10 AM, Andrei Alexandrescu wrote:
>> ...
>> * Lines 11-12: I came to terms with the notion that some types cannot be
>> made immutable. We've been trying to do reference counting on immutable
>> objects for a long time. It's time to acknowledge that true immutability
>> (which the immutable keyword models) and reference counting don't mix.
>> Const does work (more below). But overall I'm at peace with the notion
>> that if you can't live without immutable, I'll refer you to the garbage
>> collector.
>> ...
>
> I.e. you think it is fine that there can be no list of lists.

T may be const in List!T, so composing with const should work fine.

> Anyway, the static assert does not actually do what you intend it to do.

The odd thing is it does as far as I can tell, but the error message is less than informative. Try it!

>> * Lines 26-29: The allocator is fundamentally a mutable part of the
>> container. This is an insufficiency of our type system - we can't say
>> "this object may be constant, but this reference is to a mutable part of
>> it". We can't explain that necessity out of existence, and List is part
>> of the proof. So we need to make that cast legal.
>> ...
>
> Just don't make the method const. (This is not C++.)

I want to allow mutable lists and const lists, just not immutable lists. It seems to me const has a value even if it doesn't originate from immutable.


Andrei