On 12/27/11 2:24 AM, Jonathan M Davis wrote:
> On Monday, December 26, 2011 20:47:50 Andrei Alexandrescu wrote:
>> 1. "Basic" containers - reference semantics, using classic garbage
>> collection
>>
>> 2. Reference counted containers - still reference semantics, using
>> reference counting
>>
>> 3. COW containers - value semantics, using reference counting to make
>> copying O(1).
>>
>> How to we provide an Apple-style nice wrapping to all of the above?
>
> I'd be inclined to argue that reference counting and COW should be kept
> separate as opposed to being different versions of the same type simply to make
> it easier to understand. Not to mention, how often is COW needed for
> containers? It's useful for many things, but it's only useful for value-type
> containers, and the use cases for value-type containers are fairly limited I
> should think. I'm sure that they're very useful under certain circumstances,
> but as has been pointed out in the past, the fact that C++ defaults to having
> value-type containers is a big problem. So, I don't think that it's a bad
> thing to make it possible to have value-type containers and/or COW containers,
> but if they're going to seriously complicate things, then they should be
> separate IMHO. They're more specialized as opposed to being the normal use
> case.

Someone coming from C++ would instantly recognize value containers, and might prefer using them in spite our judgment that reference containers are preferable. COW is a great bridge because it keeps the copy constructor cheap.

> By the way, I'm surprised to see the use of delete in that code, since delete
> is supposed to be going away.

It's just a prototype.


Andrei

On 27/12/11 3:10 PM, Andrei Alexandrescu wrote:
> On 12/27/11 4:32 AM, Peter Alexander wrote:
>> On 27/12/11 3:05 AM, Andrei Alexandrescu wrote:
>>> On 12/26/11 8:23 PM, Peter Alexander wrote:
>>>> I am
>>>> not convinced that it can be used seamlessly without some relatively
>>>> large changes to the language.
>>>
>>> I repeat that opDispatch and auto ref were invented for this, so
>>> anything that doesn't work now is a bug. There are no changes needed to
>>> the language, only fix the bugs :o).
>>
>> How do you call template member functions of the held object without
>> changing opDispatch?
>
> The idea is to pass the entire template instantiation as the string.
>
> obj.foo!(bar, baz)(a, b);
>
> ->
>
> obj.opDispatch!("foo!(bar, baz)")(a, b);
>
>
>
> Andrei

I don't believe this will work in general when the template parameter passed in requires name look-up in the local scope.


struct Foo
{
    int bar(alias f)() { return f(); }
}

void main()
{
    static int fun() { return 1; }

    RefCounted!Foo foo;
    writeln(foo.bar!fun()); // "fun" isn't in scope when mixed in.
}

Am 27.12.2011 02:14, schrieb Robert Jacques:
> On Mon, 26 Dec 2011 17:09:02 -0800, Peter Alexander
> <peter.alexander.au@gmail.com> wrote:
>
>> On 26/12/11 5:25 PM, Andrei Alexandrescu wrote:
>>> (a) All interaction with the held object is done via opDispatch. In fact
>>> opDispatch can be engineered to statically enforce no reference to the
>>> held object escapes.
>>
>> I have a separate, but very much related concern:
>>
>> If the held object has a method with the same name as RefCounted (e.g.
>> asConst) then how do you call the held object's method instead of
>> RefCounted's method?
>
> You, can't.

You can:

container.opDispatch!"asConst"();

On 2011-12-27 14:57:18 +0000, Andrei Alexandrescu <SeeWebsiteForEmail@erdani.org> said:

> On 12/27/11 8:21 AM, Michel Fortin wrote:
>> On 2011-12-27 02:47:50 +0000, Andrei Alexandrescu
>>> 1. "Basic" containers - reference semantics, using classic garbage
>>> collection
>>> 
>>> 2. Reference counted containers - still reference semantics, using
>>> reference counting
>>> 
>>> 3. COW containers - value semantics, using reference counting to make
>>> copying O(1).
>> 
>> I like the idea, but...
>> 
>> I worry about fragmentation.
>> 
>> Say we have those three variant of each container and you write a
>> function accepting a container, which one do you use for the parameter?
>> The only type common to all three is "DListImpl", except it shouldn't
>> work because:
>> 
>> 1. you barred access to the underlying DListImpl in RefCounted (using
>> opDispatch instead of alias this)
>> 2. you want to spare users of writing "ref" every time they write a
>> function prototype
>> 
>> Those two design constrains makes it impossible to have a common type
>> independent of the "packaging" policy.
> 
> Things could be arranged to make that happen (e.g. by making "cow" a runtime parameter), but I think that would be a wrong turn. These options are important design choices, not happenstance details; one should _not_ naively mix them. For example, say we arrange things such that "cow" is opaque. Then code using a DList would have dramatically different semantics depending on what DList you pass in, or would need to guard everything with lst.isCow vs. not.

Indeed, COW must not be a runtime parameter.


>> I worry also about const and immutable.
>> 
>> It's nice to have reference counting, but it works poorly with const and
>> immutable. If a function declares it takes a const container, you'll
>> have to pass the reference counted container by ref to avoid mutating
>> the reference count, which goes contrary one of your goals.
> 
> I talked a lot about this with Walter and we concluded that the story with const and immutable goes like this: in a RefCounted!T object, the T is constant, not the RefCounted. So it would be RefCounted!(immutable T).

That'll work, but I'm not sure where you're getting at with this.

Instead of writing "const DList!T" as the parameter type in your function you have to write "RefCounted!(const DListImpl!T)"? Isn't that worse than "ref const DList!T"?


>> - - -
>> 
>> Let me make a language suggestion that will address both of these problems.
>> 
>> Allow structs to be flagged so they are always passed by reference to a
>> function, like this:
>> 
>> ref struct DListImpl {…}
>> 
>> Now a function accepting a DListImpl will implicitly have it passed by
>> "ref" with no unnecessary copy, and no need to mutate a reference count:
>> 
>> void func(DListImpl list) {…}
>> 
>> Then you can use alias this in RefCounted giving access to a "ref
>> DListImpl" which can be passed to functions always by ref, regardless of
>> the packaging policy, and without the need to remember to write "ref"
>> every time you write a function accepting a container (because it is a
>> ref struct which is always implicitly passed by ref).
>> 
>> Functions that don't need to store a reference to the container won't
>> have to care about whether the container they get is maintained using a
>> reference counter, COW, or the GC. And those functions can accept a
>> const container, or even a value-type container (which gets implicitly
>> passed by ref).
>> 
>> So with implicit ref parameters for "ref struct" types you solve the
>> issue of fragmentation and of passing const containers to functions.
>> 
>> The underlying principle is that how to efficiently pass a type to
>> functions should be decided at the type level. You're trying to figure
>> out how it should work for containers, but I think it should apply more
>> broadly to the language as a whole.
>> 
>> But there is one more thing. ;-)
>> 
>> Implicit ref parameters could also solve elegantly the more general
>> problem of passing rvalues by reference if you make them accept rvalues.
>> Only parameters explicitly annotated with "ref" would refuse rvalues,
>> those implicitly passed by ref would accept them.
> 
> Not to seem dismissive, but this sounds quite complicated for what it does. Could we implement it as a library feature? And assuming your motivating argument wasn't as strong, is it needed?

It can't be a library feature because it requires the compiler to implicitly add a "ref" to functions parameters when they are of a "ref struct" type. That's pretty much all it does: add a flag to struct types, implicitly add "ref" to parameters based on that flag.

I'm not sure why you think it's complicated. Perhaps I should just not have talked about rvalues: that's a separate benefit you can built on top of it by adding one or two lines of code in DMD, but it's a separate thing.

-- 
Michel Fortin
michel.fortin@michelf.com
http://michelf.com/

On 12/27/11 9:27 AM, Michel Fortin wrote:
> On 2011-12-27 14:57:18 +0000, Andrei Alexandrescu
> <SeeWebsiteForEmail@erdani.org> said:
>> I talked a lot about this with Walter and we concluded that the story
>> with const and immutable goes like this: in a RefCounted!T object, the
>> T is constant, not the RefCounted. So it would be
>> RefCounted!(immutable T).
>
> That'll work, but I'm not sure where you're getting at with this.
>
> Instead of writing "const DList!T" as the parameter type in your
> function you have to write "RefCounted!(const DListImpl!T)"? Isn't that
> worse than "ref const DList!T"?

That's an issue we're thinking about how to address.

>> Not to seem dismissive, but this sounds quite complicated for what it
>> does. Could we implement it as a library feature? And assuming your
>> motivating argument wasn't as strong, is it needed?
>
> It can't be a library feature because it requires the compiler to
> implicitly add a "ref" to functions parameters when they are of a "ref
> struct" type. That's pretty much all it does: add a flag to struct
> types, implicitly add "ref" to parameters based on that flag.

Anyhow, I think a feature would need to be tremendously justified and with a huge power/weight ratio. This one I actually find ill-designed because now I can't look at the body of a function to figure pass-by-value vs. pass-by-reference - I also need to look at the definition (what if it was a ref struct?). One more non-modular thing to keep in mind. This is not going to fly.

> I'm not sure why you think it's complicated. Perhaps I should just not
> have talked about rvalues: that's a separate benefit you can built on
> top of it by adding one or two lines of code in DMD, but it's a separate
> thing.

I think the "But there's one more thing" did it to me.


Andrei

On 12/27/11 9:34 AM, Peter Alexander wrote:
> I don't believe this will work in general when the template parameter
> passed in requires name look-up in the local scope.
>
>
> struct Foo
> {
> int bar(alias f)() { return f(); }
> }
>
> void main()
> {
> static int fun() { return 1; }
>
> RefCounted!Foo foo;
> writeln(foo.bar!fun()); // "fun" isn't in scope when mixed in.
> }

That is a problem.

Andrei

Le 27/12/2011 16:06, Andrei Alexandrescu a écrit :
> On 12/27/11 5:43 AM, deadalnix wrote:
>>> BTW, the plan is to have the same thread that constructed objects to
>>> call their destructors; each thread would have a worklist of garbage
>>> objects to destroy. The list can be reduced during calls to allocation
>>> functions within each thread, and of course when the thread terminates.
>>>
>>>
>>> Andrei
>>
>> This is nice for thread local data, but does it make sense for
>> immuables/shared ?
>
> Yes - when the GC reaps objects, they are past their useful lifetime so
> their qualifiers are gone.

I'm 100% convinced that the GC should be aware of type qualifier. this lead to many possible optimizations, and can be simply ignored when not required.

>> I'm not sure this design is secure regrading GC. What happen if the
>> destructor of the RefCounted is called, (and will call dispose) and then
>> the payload is destructed. We have a risk of double destruction.
>
> I'm not sure I understand the scenario.
>

OK, let's use an exemple, this is usually easier to understand.

You have something of type T with a destructor, and I'll call it Patrick. And a RefCount struct pointing to it.

Let's consider the RefCount struct is on the heap and garbage collected. Yes, I know, this is useless but possible so this have to be considered.

Now, the garbage collector run, and will mark both memory location to be deleted : Patrick and the RefCount struct.

The RefCount, will call dispose on the something and the GC will call dtor on Patrick. You'll never know in which order and so you can ends up calling dispose on a destructed object. Or destruct something used in the destructor in dispose. This is confusing.

Let's put in the fact that you cant to run thoses in the thread that created them and you possibly ends-up with something quite confusing.

I think the dtor should be the only way to destruct something and not the dispose method, and we have to ensure that the dtor is called by the GC.

Le 27/12/2011 16:09, Andrei Alexandrescu a écrit :
> On 12/27/11 5:27 AM, kenji hara wrote:
>> I've already posted a dmd patch to fix all of opDispatch problems:
>>
>> https://github.com/D-Programming-Language/dmd/pull/280
>
> As far a I understand you pass the whole template as a string inside
> opDispatch? For example, say obj defines opDispatch:
>
> obj.foo!(bar, baz)(a, b);
>
> Would that be rewritten as
>
> obj.opDispatch!("foo!(bar, baz)")(a, b);
>
> ?
>
> That would be great.
>

I think a variadoic template is a better option because of scope issues. You may not know anymore what is bar or baz within opDispatch.

Le 27/12/2011 16:04, Andrei Alexandrescu a écrit :
> On 12/27/11 5:32 AM, deadalnix wrote:
>> The first thing that I see looking at the source code is how many null
>> check you'll find in RefCounted . As the RefCounted struct is useless
>> without a payload, we should ensure that the payload exists, in all
>> cases, and not null check at every operation.
>
> D's take on default constructor makes it impossible to initialize the
> object 100% automatically. What can be done is a test inside opDispatch
> followed by default construction if the object is null. That's not
> terribly useful and intuitive. Consider:
>

That is just moving the problem around. It is even worse I think.

>
> This is the behavior of any reference type that can be null, because
> null doesn't refer to any object. IMHO this, and not null pointer
> exceptions, is the largest liability of null.
>
>
> Andrei

Yes, the problem is way larger. I think null is a very poor solution to a real instanciation problem. This cause more trouble than solution. It is error prone (forget a null check and you are doomed) and have a runtime cost.

On 27/12/11 4:32 PM, deadalnix wrote:
> Le 27/12/2011 16:09, Andrei Alexandrescu a écrit :
>> On 12/27/11 5:27 AM, kenji hara wrote:
>>> I've already posted a dmd patch to fix all of opDispatch problems:
>>>
>>> https://github.com/D-Programming-Language/dmd/pull/280
>>
>> As far a I understand you pass the whole template as a string inside
>> opDispatch? For example, say obj defines opDispatch:
>>
>> obj.foo!(bar, baz)(a, b);
>>
>> Would that be rewritten as
>>
>> obj.opDispatch!("foo!(bar, baz)")(a, b);
>>
>> ?
>>
>> That would be great.
>>
>
> I think a variadoic template is a better option because of scope issues.
> You may not know anymore what is bar or baz within opDispatch.

How would that work? opDispatch already has variadic templates for the normal function parameters. Make opDispatch a template inside a template?