On Sun, Mar 01, 2015 at 01:43:44PM +0000, via Digitalmars-d wrote:
> On Sunday, 1 March 2015 at 06:42:02 UTC, H. S. Teoh wrote:
[...]
> >So then Const!MyClass is a modified version of MyClass where the data fields are const (similarly, we can define Immutable for the analogous purpose) but the fields marked as metadata will remain mutable.
> >
> >Of course, this is just a crude first stab at the problem; I'm sure there's plenty of room for refinement to make it more usable, and to address some obvious roadblocks, like how to make MyClass implicitly convertible to Const!MyClass, etc.. But it seems likely that D's template machinery can actually express this in a way that does not violate the guarantee of physical const.
> 
> You still cannot access it through a const reference, though.

The whole point is to use Const!T instead of const(T).


T

-- 
Be in denial for long enough, and one day you'll deny yourself of things you wish you hadn't.

On Sunday, 1 March 2015 at 01:40:40 UTC, Andrei Alexandrescu wrote:
> We have a few candidates for solutions, but wanted to open with a good discussion first. So, how do you envision a way to define and access mutable metadata for objects (including immutable ones)?
>
>
> Andrei

I don't think const or immutable intrusive-reference-counted classes is a feasible idea.

I understand the motivation: we want to dynamically allocate a class instance, initialize it and never mutate it again, and pass it around freely; *without* using tracing GC. Having it typed as immutable helps code readability and whatnot.

However, AFAICS, it comes with a serious problem. Immutable objects are freely passable between threads, so surely an immutable RC object would need atomic counting just like a shared RC object, but unlike shared, immutable does not necessarily express the intent of sharing between threads; the immutable RC object could easily be counting atomically for nothing.

There might be other problems, such as problems regarding ROM.

This is not a problem with reference-counting as a whole but with intrusive reference-counting. With RefCounted, immutable(RefCounted!T) makes no sense, but RefCounted!(immutable T) does. It's also neatly composable; shared(RefCounted!(immutable T)) makes sense too.

I wish we had external, composable reference-counting for class instances. I know why that's problematic, so sorry for posting without any suggestion on how to proceed...

On Sunday, 1 March 2015 at 02:53:34 UTC, Manu wrote:
> On 1 March 2015 at 12:48, Andrei Alexandrescu via Digitalmars-d
> <digitalmars-d@puremagic.com> wrote:
>> On 2/28/15 6:33 PM, Manu via Digitalmars-d wrote:
>>>
>>> one of the biggest recurring
>>> complaints relating to the D type system
>>
>>
>> That didn't get talked about in I don't remember. -- Andrei
>
> So, what's the solution?
> I've never complained about it, but I still have this problem
> regularly. The solution in my experience is; 'everything is always
> mutable'.

I've lost count now of how many times I've had to downgrade to auto despite always wanting immutable or const. This doesn't work:

 auto reg = regex(`(foo)`);
 const match = "foo".matchAll(reg);
 writeln(match.captures); //oops, captures isn't const

It should, but it doesn't. Maxime talked about it here as well:

http://pointersgonewild.com/2014/07/11/the-constness-problem/

Atila

On Sunday, 1 March 2015 at 15:08:47 UTC, H. S. Teoh wrote:
> On Sun, Mar 01, 2015 at 01:43:44PM +0000, via Digitalmars-d wrote:
>> On Sunday, 1 March 2015 at 06:42:02 UTC, H. S. Teoh wrote:
> [...]
>> >So then Const!MyClass is a modified version of MyClass where the data
>> >fields are const (similarly, we can define Immutable for the
>> >analogous purpose) but the fields marked as metadata will remain
>> >mutable.
>> >
>> >Of course, this is just a crude first stab at the problem; I'm sure
>> >there's plenty of room for refinement to make it more usable, and to
>> >address some obvious roadblocks, like how to make MyClass implicitly
>> >convertible to Const!MyClass, etc.. But it seems likely that D's
>> >template machinery can actually express this in a way that does not
>> >violate the guarantee of physical const.
>> 
>> You still cannot access it through a const reference, though.
>
> The whole point is to use Const!T instead of const(T).

But that's intrusive! You can only apply it code you control, or at least you have to convince everyone to use it.

On Sunday, 1 March 2015 at 15:40:06 UTC, Atila Neves wrote:
> I've lost count now of how many times I've had to downgrade to auto despite always wanting immutable or const. This doesn't work:
>
>  auto reg = regex(`(foo)`);
>  const match = "foo".matchAll(reg);
>  writeln(match.captures); //oops, captures isn't const
>
> It should, but it doesn't. Maxime talked about it here as well:
>
> http://pointersgonewild.com/2014/07/11/the-constness-problem/
>
> Atila

`match.captures` is a range; it's only natural for a range to have mutable state to be iterable.

D's const is a bridge between immutable and mutable. const has to be transitive because immutable is transitive. Don't use it as if it was C++ const: there's no logical const in D, and if there ever will be, it can't use the same `const` type qualifier.

On Sunday, 1 March 2015 at 15:28:56 UTC, Jakob Ovrum wrote:
> On Sunday, 1 March 2015 at 01:40:40 UTC, Andrei Alexandrescu wrote:
>> We have a few candidates for solutions, but wanted to open with a good discussion first. So, how do you envision a way to define and access mutable metadata for objects (including immutable ones)?
>>
>>
>> Andrei
>
> I don't think const or immutable intrusive-reference-counted classes is a feasible idea.
>
> I understand the motivation: we want to dynamically allocate a class instance, initialize it and never mutate it again, and pass it around freely; *without* using tracing GC. Having it typed as immutable helps code readability and whatnot.
>
> However, AFAICS, it comes with a serious problem. Immutable objects are freely passable between threads, so surely an immutable RC object would need atomic counting just like a shared RC object, but unlike shared, immutable does not necessarily express the intent of sharing between threads; the immutable RC object could easily be counting atomically for nothing.

Argh! I didn't think about this. Any chance we can deprecate this behaviour? It's also an obstacle for the implementation of thread-local heaps.

>
> There might be other problems, such as problems regarding ROM.

Not if the RC wrapper allocated the memory in the first place. It knows that it can't be in ROM.

I haven't put much thought into this, but here's an idea:

struct MetadataWrapper(T, M) {
    immutable T payload; //or const via static if?
    M _metadata;
    alias payload this;
    ref M metadata() { return _metadata;}
}

auto withMetadata(T, Ts...)(Ts args) {
    return MetadataWrapper!(T, int)(cast(immutable)(new T(args)), 0);
}

It's short and stupid but works:

void main() {
    import std.stdio;

    static class MyClass {
        this(string s) {
            this.s = s;
        }
        string s;
        string stuff() const pure nothrow @safe { return s ~ " and stuff";}
    }

    auto foo = withMetadata!MyClass("a string");
    writeln("calling stuff on foo: ", foo.stuff());
    writeln("foo's metadata is ", foo.metadata);
    foo.metadata++;
    writeln("foo's metadata is ", foo.metadata);
}



On Sunday, 1 March 2015 at 01:40:40 UTC, Andrei Alexandrescu wrote:
> Tracing garbage collection can afford the luxury of e.g. mutating data that was immutable during its lifetime.
>
> Reference counting needs to make minute mutations to data while references to that data are created. In fact, it's not mutation of the "useful" data, the payload of a data structure; it's mutation of metadata, additional information about the data (i.e. a reference count integral).
>
> The RCOs described in DIP74 and also RCSlice discussed in this forum need to work properly with const and immutable. Therefore, they need a way to reliably define and access metadata for a data structure.
>
> One possible solution is to add a "@mutable" or "@metadata" attribute similar to C++'s keyword "mutable". Walter and I both dislike that solution because it's hamfisted and leaves too much opportunity for abuse - people can essentially create unbounded amounts of mutable payload for an object claimed to be immutable. That makes it impossible (or unsafe) to optimize code based on algebraic assumptions.
>
> We have a few candidates for solutions, but wanted to open with a good discussion first. So, how do you envision a way to define and access mutable metadata for objects (including immutable ones)?
>
>
> Andrei

On Sunday, 1 March 2015 at 01:40:40 UTC, Andrei Alexandrescu wrote:
> Tracing garbage collection can afford the luxury of e.g. mutating data that was immutable during its lifetime.
>
> Reference counting needs to make minute mutations to data while references to that data are created. In fact, it's not mutation of the "useful" data, the payload of a data structure; it's mutation of metadata, additional information about the data (i.e. a reference count integral).
>
> The RCOs described in DIP74 and also RCSlice discussed in this forum need to work properly with const and immutable. Therefore, they need a way to reliably define and access metadata for a data structure.
>
> One possible solution is to add a "@mutable" or "@metadata" attribute similar to C++'s keyword "mutable". Walter and I both dislike that solution because it's hamfisted and leaves too much opportunity for abuse - people can essentially create unbounded amounts of mutable payload for an object claimed to be immutable. That makes it impossible (or unsafe) to optimize code based on algebraic assumptions.
>
> We have a few candidates for solutions, but wanted to open with a good discussion first. So, how do you envision a way to define and access mutable metadata for objects (including immutable ones)?

I need to get educated on this issue. First suggestion: Just break the type system by encouraging the idiom of using casts in opAddRef and opRelease. It's too easy, but I don't know why.

On Sun, 01 Mar 2015 15:53:06 +0000, Marc Schütz wrote:

>> However, AFAICS, it comes with a serious problem. Immutable objects are freely passable between threads, so surely an immutable RC object would need atomic counting just like a shared RC object, but unlike shared, immutable does not necessarily express the intent of sharing between threads; the immutable RC object could easily be counting atomically for nothing.
> 
> Argh! I didn't think about this. Any chance we can deprecate this behaviour? It's also an obstacle for the implementation of thread-local heaps.

but deprecating this is... illogical at best. immutable objects are immutable, so why can't i pass 'em to any thread? do you mean that i can't use `immutable string = "mystr";` in different threads without copying anymore?

On Sunday, 1 March 2015 at 17:03:46 UTC, ketmar wrote:
> On Sun, 01 Mar 2015 15:53:06 +0000, Marc Schütz wrote:
>
>>> However, AFAICS, it comes with a serious problem. Immutable objects are
>>> freely passable between threads, so surely an immutable RC object would
>>> need atomic counting just like a shared RC object, but unlike shared,
>>> immutable does not necessarily express the intent of sharing between
>>> threads; the immutable RC object could easily be counting atomically
>>> for nothing.
>> 
>> Argh! I didn't think about this. Any chance we can deprecate this
>> behaviour? It's also an obstacle for the implementation of thread-local
>> heaps.
>
> but deprecating this is... illogical at best. immutable objects are
> immutable, so why can't i pass 'em to any thread? do you mean that i
> can't use `immutable string = "mystr";` in different threads without
> copying anymore?

Don't know. It was more of a question than a recommendation. It might be impractical.

OTOH, I don't think it's illogical, because thread-locality and immutability are independent concepts. There can be shared mutable data (`shared`)  as well as thread-local immutable data. Hmm... that would be equivalent to `const`, no? But that's getting off-topic.