On 11/17/2015 04:43 AM, Marc Schütz wrote:
> On Monday, 16 November 2015 at 14:45:35 UTC, Andrei Alexandrescu wrote:
>> The challenge is proving that a mutation is not observable. Got an
>> attack on that? -- Andrei
>
> Here's what I wrote a few days ago, without having read the current
> discussion, so it's not a perfect fit:
> http://wiki.dlang.org/DIP85
>
> For the case of lazy initialization, assigning a member exactly once is
> safe, if it hasn't been read and returned before. The compiler can check
> this by automatically inserting a few asserts. The rules in the DIP
> aren't completely water-tight, but they are a good approximation.
>
> But for members that have to be mutated several times (e.g. refcount),
> this obviously isn't useful. I think we can make this @system, thereby
> forcing implementors to write @trusted code and (hopefully) consider the
> consequences thoroughly. But the compiler can also assist with that. For
> example, it could detect whether a "privately mutable" value, or
> something depending on it, is returned from the function, and reject that.
>
> Maybe the implementation for the two kinds of use-cases (write once vs
> write multiple times) can be unified, too.

Thanks for the DIP, I saw it since a couple days ago. Dynamically-verified lazy initialization is difficult, but doesn't cover:

1. Reference count updates (as you mention)

2. The reference to the allocator is essentially a mutable part of an object that's otherwise constant.

So it seems to be DIP85 is solving a quite narrow problem, and one that doesn't address the issue at hand.


Andrei

On 11/17/2015 04:47 AM, Marc Schütz wrote:
> Refcount also needs to change when you get an additional reference. But
> it is non-observable, because the caller can't access the refcount
> directly (or we could allow it, but as @system). The only potentially
> observable side-effect would be if the object is destroyed, but this
> only happens if there are no references left. Then the only way this can
> be achieved is if the destructor modifies state that is otherwise
> accessible (global, or by references), which however is not a problem,
> because the normal rules of const-ness apply to it.

I think it would be very difficult to prove at the type system level that the reference count is not observable. Humanly, sure, what you say makes sense. For a static checker, I don't think that's an approachable angle. -- Andrei

On Tuesday, 17 November 2015 at 09:47:24 UTC, Marc Schütz wrote:
> On Monday, 16 November 2015 at 16:58:24 UTC, Lionello Lunesu
>> If it's RC we want, then @mutable is an axe when what we need is a scalpel.
>
> It has additional uses, though, e.g. memoizing and  lazy initialization of members.

Hiding memoization behind fake const is one of more unpleasant C++ multi-threading abominations I have encountered during my C++ says. Resulting races are so subtle (because only happen once) and so inevitable (because it is subconsciously reasoned about as physical const) that debugging them becomes real pain.

This thread is really scary because of that. It is pretty much saying that D attempt to rethink multi-threading data model that caused trouble in C++ has failed completely and we back to square one. Even if it isn't technical disaster, it is marketing one for sure.

On Tuesday, 17 November 2015 at 14:29:53 UTC, Andrei Alexandrescu wrote:
> Thanks for the DIP, I saw it since a couple days ago. Dynamically-verified lazy initialization is difficult, but doesn't cover:
>
> 1. Reference count updates (as you mention)
>
> 2. The reference to the allocator is essentially a mutable part of an object that's otherwise constant.
>
> So it seems to be DIP85 is solving a quite narrow problem, and one that doesn't address the issue at hand.

I know, I wrote the DIP before I read this thread, and it was originally only supposed to address lazy initialization specifically. Maybe some of the ideas can be reused for a more general un-const concept, though.

On Tuesday, 17 November 2015 at 14:40:13 UTC, Dicebot wrote:
> On Tuesday, 17 November 2015 at 09:47:24 UTC, Marc Schütz wrote:
>> On Monday, 16 November 2015 at 16:58:24 UTC, Lionello Lunesu
>>> If it's RC we want, then @mutable is an axe when what we need is a scalpel.
>>
>> It has additional uses, though, e.g. memoizing and  lazy initialization of members.
>
> Hiding memoization behind fake const is one of more unpleasant C++ multi-threading abominations I have encountered during my C++ says. Resulting races are so subtle (because only happen once) and so inevitable (because it is subconsciously reasoned about as physical const) that debugging them becomes real pain.
>
> This thread is really scary because of that. It is pretty much saying that D attempt to rethink multi-threading data model that caused trouble in C++ has failed completely and we back to square one. Even if it isn't technical disaster, it is marketing one for sure.

I don't think this is a concern for D, because we still have thread-awareness in the type system. We can distinguish between `@mutable` and `shared @mutable`, just as we distinguish between `const` and `shared const`.

On 11/16/15 8:11 PM, Meta wrote:

> http://stackoverflow.com/questions/31516713/escaping-from-inout-hell

BTW, I cannot post there, but this statement in the second answer is wrong:

"Just remove inout. The compiler infers attributes like const automatically for templates"

The compiler does NOT infer const automatically.

Someone should correct that for those looking at these things.

-Steve

On 11/17/15 1:01 PM, Marc Schütz wrote:
> On Tuesday, 17 November 2015 at 14:29:53 UTC, Andrei Alexandrescu wrote:
>> Thanks for the DIP, I saw it since a couple days ago.
>> Dynamically-verified lazy initialization is difficult, but doesn't cover:
>>
>> 1. Reference count updates (as you mention)
>>
>> 2. The reference to the allocator is essentially a mutable part of an
>> object that's otherwise constant.
>>
>> So it seems to be DIP85 is solving a quite narrow problem, and one
>> that doesn't address the issue at hand.
>
> I know, I wrote the DIP before I read this thread, and it was originally
> only supposed to address lazy initialization specifically. Maybe some of
> the ideas can be reused for a more general un-const concept, though.

Yes, it's a great building block for something more general. It is well done. -- Andrei

On Tuesday, 17 November 2015 at 01:58:54 UTC, deadalnix wrote:
> On Sunday, 15 November 2015 at 12:56:27 UTC, Andrei Alexandrescu wrote:
>> On 11/14/2015 05:49 PM, Timon Gehr wrote:
>>> List uses RC internally. I don't think the UB casts will stay for the
>>> final version, unless you are willing to completely dilute the meaning
>>> of const in ways that Walter has explicitly expressed will not be done
>>> in the past.
>>
>> As I mentioned, he's okay with changing the language to make the casts well defined. -- Andrei
>
> Ok Here is what I propose as a spec. This is rough brain dump, but that is something I think can be a good start.
>
> There are mutable data that are thread local and immutable data that are shared. const is a wildcard type qualifier that can refers to mutable or immutable data.
>
> immutable is supposed to be effectively immutable. This means that the compiler is allowed to store these data in ro segments and optimize redundant loads without having to prove that no aliasing write occur in between.
>
> One can cast away const or immutable. It is a @system operation and this cast needs to be explicit.
>
> If the underlying data is in ro segment, any write is effectively undefined behavior (most likely a fault).
> If the underlying data is allocated on the head (using the GC or malloc for instance) then writing to it will effectively update the underlying memory.
>
> Any read from a const or immutable reference after these write can is undefined.
> Granted the write was not UB :
> 1/ Any read from a mutable reference in another thread is undefined behavior unless the writing thread release after write and the reading one acquire (or any stronger memory barrier).
> 2/ Any read from a mutable reference in the same thread will see the updates in a sequentially consistent manner.
>
> Sounds good ? This definitively allow to do RC for const/immutable without throwing away optimization opportunities.

Just quoting myself. In case someone is interested...

Also, on a related topic - is there any reason why you are trying to put rc in container itself here instead of making it part of dedicated allocator?

On 11/17/2015 02:38 PM, Dicebot wrote:
>
>> Anyway, any design that does not allow e.g. nested lists is inadequate
>> IMO.
>
> But here I have to disagree. There simply isn't anything generic in
> immutable containers with RC, each requires own tweaked solution. It all
> works nice with GC because all memory tracking becomes exclusively
> external - trying to ignore and hide that fact makes no sense to me.
>
> Main use case for such containers is efficient thread sharing and that
> totally justifies specialized container (or even allocator) for each
> dataset.

Persistent containers are useful even if only a single thread is involved, as they can speed up some algorithms.

Having the library provide List!T, but not List!(List!T) is a ridiculous situation. More generally, types shouldn't need to satisfy any additional constraints (beyond abstract container specific ones) to be allowed to place their instances in containers.

I don't think the cases where writing a new specialized version from scratch is justified are what the new container module will be about.