When I go to that link it just says

Unknown Paste ID!

Don't see any code anywhere.. hum




 What is wrong with value containers?  They work great in C++, a container is such a basic thing that ref counting and whatnot is rarely if ever needed, and in the unlikely event you need to share a container, wrapping it with a smart pointer of some sort is easy enough in C++, let alone D with its better support for type aliasing.

On 2011-12-27 02:47:50 +0000, Andrei Alexandrescu <SeeWebsiteForEmail@erdani.org> said:

> The perspectives here are extremely exciting. The question remains how to best package this awesome array of options to maximize coherence. So we have:
> 
> 1. "Basic" containers - reference semantics, using classic garbage collection
> 
> 2. Reference counted containers - still reference semantics, using reference counting
> 
> 3. COW containers - value semantics, using reference counting to make copying O(1).

I like the idea, but...

I worry about fragmentation.

Say we have those three variant of each container and you write a function accepting a container, which one do you use for the parameter? The only type common to all three is "DListImpl", except it shouldn't work because:

1. you barred access to the underlying DListImpl in RefCounted (using opDispatch instead of alias this)
2. you want to spare users of writing "ref" every time they write a function prototype

Those two design constrains makes it impossible to have a common type independent of the "packaging" policy.

I worry also about const and immutable.

It's nice to have reference counting, but it works poorly with const and immutable. If a function declares it takes a const container, you'll have to pass the reference counted container by ref to avoid mutating the reference count, which goes contrary one of your goals.

- - -

Let me make a language suggestion that will address both of these problems.

Allow structs to be flagged so they are always passed by reference to a function, like this:

	ref struct DListImpl {…}

Now a function accepting a DListImpl will implicitly have it passed by "ref" with no unnecessary copy, and no need to mutate a reference count:

	void func(DListImpl list) {…}

Then you can use alias this in RefCounted giving access to a "ref DListImpl" which can be passed to functions always by ref, regardless of the packaging policy, and without the need to remember to write "ref" every time you write a function accepting a container (because it is a ref struct which is always implicitly passed by ref).

Functions that don't need to store a reference to the container won't have to care about whether the container they get is maintained using a reference counter, COW, or the GC. And those functions can accept a const container, or even a value-type container (which gets implicitly passed by ref).

So with implicit ref parameters for "ref struct" types you solve the issue of fragmentation and of passing const containers to functions.

The underlying principle is that how to efficiently pass a type to functions should be decided at the type level. You're trying to figure out how it should work for containers, but I think it should apply more broadly to the language as a whole.

But there is one more thing. ;-)

Implicit ref parameters could also solve elegantly the more general problem of passing rvalues by reference if you make them accept rvalues. Only parameters explicitly annotated with "ref" would refuse rvalues, those implicitly passed by ref would accept them.


-- 
Michel Fortin
michel.fortin@michelf.com
http://michelf.com/

On 12/27/11 12:08 AM, Andrej Mitrovic wrote:
> On 12/27/11, Andrei Alexandrescu<SeeWebsiteForEmail@erdani.org>  wrote:
>> It actually does, as per the unittests. Even if it currently does by
>> @property being too lax, it should continue to work.
>
> It actually doesn't:
>
> struct FooImpl
> {
>      int _x;
>      @property int x() { return _x; }
>      @property void x(int val) { _x = val; }
>
>      auto dup() { return FooImpl(); }
>      auto dispose() { }
> }
>
> void main()
> {
>      RefCounted!FooImpl foo;
>      foo.x = 1;  // NG
> }
>
> Sure you could make a property return ref, but that's circumventing
> the setter and allowing direct access to the field from client-code.
> I've ran into this issue before when playing with opDispatch but I
> couldn't figure out a workaround.

Sorry for being unclear - when I wrote "should work" I meant "it's a bug if it doesn't".

>> So construction of objects
>> needs to be handled by RefCounted itself - e.g. by forwarding
>> constructor parameters.
>
> But it doesn't do that currently?

The prototype does not define a variadic forwarding constructor.


Andrei

On 27/12/11 12:15 PM, Froglegs wrote:
> What is wrong with value containers? They work great in C++, a container
> is such a basic thing that ref counting and whatnot is rarely if ever
> needed, and in the unlikely event you need to share a container,
> wrapping it with a smart pointer of some sort is easy enough in C++, let
> alone D with its better support for type aliasing.

There's not much point in having value containers if you just pass them around by reference all the time (as you do in C++). If you treat them like reference types then they should be reference types.

On 12/27/11 8:21 AM, Michel Fortin wrote:
> On 2011-12-27 02:47:50 +0000, Andrei Alexandrescu
>> 1. "Basic" containers - reference semantics, using classic garbage
>> collection
>>
>> 2. Reference counted containers - still reference semantics, using
>> reference counting
>>
>> 3. COW containers - value semantics, using reference counting to make
>> copying O(1).
>
> I like the idea, but...
>
> I worry about fragmentation.
>
> Say we have those three variant of each container and you write a
> function accepting a container, which one do you use for the parameter?
> The only type common to all three is "DListImpl", except it shouldn't
> work because:
>
> 1. you barred access to the underlying DListImpl in RefCounted (using
> opDispatch instead of alias this)
> 2. you want to spare users of writing "ref" every time they write a
> function prototype
>
> Those two design constrains makes it impossible to have a common type
> independent of the "packaging" policy.

Things could be arranged to make that happen (e.g. by making "cow" a runtime parameter), but I think that would be a wrong turn. These options are important design choices, not happenstance details; one should _not_ naively mix them. For example, say we arrange things such that "cow" is opaque. Then code using a DList would have dramatically different semantics depending on what DList you pass in, or would need to guard everything with lst.isCow vs. not.

> I worry also about const and immutable.
>
> It's nice to have reference counting, but it works poorly with const and
> immutable. If a function declares it takes a const container, you'll
> have to pass the reference counted container by ref to avoid mutating
> the reference count, which goes contrary one of your goals.

I talked a lot about this with Walter and we concluded that the story with const and immutable goes like this: in a RefCounted!T object, the T is constant, not the RefCounted. So it would be RefCounted!(immutable T).

> - - -
>
> Let me make a language suggestion that will address both of these problems.
>
> Allow structs to be flagged so they are always passed by reference to a
> function, like this:
>
> ref struct DListImpl {…}
>
> Now a function accepting a DListImpl will implicitly have it passed by
> "ref" with no unnecessary copy, and no need to mutate a reference count:
>
> void func(DListImpl list) {…}
>
> Then you can use alias this in RefCounted giving access to a "ref
> DListImpl" which can be passed to functions always by ref, regardless of
> the packaging policy, and without the need to remember to write "ref"
> every time you write a function accepting a container (because it is a
> ref struct which is always implicitly passed by ref).
>
> Functions that don't need to store a reference to the container won't
> have to care about whether the container they get is maintained using a
> reference counter, COW, or the GC. And those functions can accept a
> const container, or even a value-type container (which gets implicitly
> passed by ref).
>
> So with implicit ref parameters for "ref struct" types you solve the
> issue of fragmentation and of passing const containers to functions.
>
> The underlying principle is that how to efficiently pass a type to
> functions should be decided at the type level. You're trying to figure
> out how it should work for containers, but I think it should apply more
> broadly to the language as a whole.
>
> But there is one more thing. ;-)
>
> Implicit ref parameters could also solve elegantly the more general
> problem of passing rvalues by reference if you make them accept rvalues.
> Only parameters explicitly annotated with "ref" would refuse rvalues,
> those implicitly passed by ref would accept them.

Not to seem dismissive, but this sounds quite complicated for what it does. Could we implement it as a library feature? And assuming your motivating argument wasn't as strong, is it needed?


Andrei

On 12/27/11 6:15 AM, Froglegs wrote:
> When I go to that link it just says
>
> Unknown Paste ID!
>
> Don't see any code anywhere.. hum
>
>
>
>
> What is wrong with value containers? They work great in C++, a container
> is such a basic thing that ref counting and whatnot is rarely if ever
> needed, and in the unlikely event you need to share a container,
> wrapping it with a smart pointer of some sort is easy enough in C++, let
> alone D with its better support for type aliasing.

Value containers are almost never used as values. Code that does manipulate them as values, e.g.

void fun(vector<int> v);

is technically correct but automatically suspicious and raises questions. (It only passes a code review at Facebook if the implementation of fun does need in fact a mutable temporary vector inside, which is rare.)

There is only one place where C++ value containers work well, and that's as members inside value objects. But then those objects in turn must be effectively manipulated as references...


Andrei

On 12/27/11 5:32 AM, deadalnix wrote:
> The first thing that I see looking at the source code is how many null
> check you'll find in RefCounted . As the RefCounted struct is useless
> without a payload, we should ensure that the payload exists, in all
> cases, and not null check at every operation.

D's take on default constructor makes it impossible to initialize the object 100% automatically. What can be done is a test inside opDispatch followed by default construction if the object is null. That's not terribly useful and intuitive. Consider:

void fun(DList!int lst)
{
   lst.insertFront(42);
}

void main()
{
    DList!int lst1;
    lst1.insertFront(43);
    fun(lst1);
    assert(lst1.front == 42); // fine

    DList!int lst2;
    fun(lst2);
    assert(lst2.front == 42); // gaaaaaaa!
}

This is the behavior of any reference type that can be null, because null doesn't refer to any object. IMHO this, and not null pointer exceptions, is the largest liability of null.


Andrei

On 12/27/11 5:43 AM, deadalnix wrote:
>> BTW, the plan is to have the same thread that constructed objects to
>> call their destructors; each thread would have a worklist of garbage
>> objects to destroy. The list can be reduced during calls to allocation
>> functions within each thread, and of course when the thread terminates.
>>
>>
>> Andrei
>
> This is nice for thread local data, but does it make sense for
> immuables/shared ?

Yes - when the GC reaps objects, they are past their useful lifetime so their qualifiers are gone.

> I'm not sure this design is secure regrading GC. What happen if the
> destructor of the RefCounted is called, (and will call dispose) and then
> the payload is destructed. We have a risk of double destruction.

I'm not sure I understand the scenario.


Andrei

On 12/27/11 5:27 AM, kenji hara wrote:
> I've already posted a dmd patch to fix all of opDispatch problems:
>
> https://github.com/D-Programming-Language/dmd/pull/280

As far a I understand you pass the whole template as a string inside opDispatch? For example, say obj defines opDispatch:

obj.foo!(bar, baz)(a, b);

Would that be rewritten as

obj.opDispatch!("foo!(bar, baz)")(a, b);

?

That would be great.

> With it, I've posted a useful library utility to implement
> 'super-type' like D1 typedef:
>
> https://github.com/D-Programming-Language/phobos/pull/300
>
> I'm naming it 'ProxyOf', and it supports various of forwardings,
> function call, property access, and specialized template member
> function.
>
> Kenji Hara

Thanks! Sorry I didn't look over ProxyOf first, it might have saved me some work. I'll do so soon.

Andrei

On 12/27/11 4:32 AM, Peter Alexander wrote:
> On 27/12/11 3:05 AM, Andrei Alexandrescu wrote:
>> On 12/26/11 8:23 PM, Peter Alexander wrote:
>>> I am
>>> not convinced that it can be used seamlessly without some relatively
>>> large changes to the language.
>>
>> I repeat that opDispatch and auto ref were invented for this, so
>> anything that doesn't work now is a bug. There are no changes needed to
>> the language, only fix the bugs :o).
>
> How do you call template member functions of the held object without
> changing opDispatch?

The idea is to pass the entire template instantiation as the string.

obj.foo!(bar, baz)(a, b);

->

obj.opDispatch!("foo!(bar, baz)")(a, b);



Andrei