Michel Fortin wrote:

Le 28-juin-2013 à 4:55, Rainer Schuetze <r.sagitario@gmx.de> a écrit :

> What happens if the class also implements interfaces? A reference of the interface type must do reference counting as well. So the interface must also define AddRef and Release. This is currently true for COM-interfaces derived from IUnknown, but not for other interfaces.


I would assume if an object of a reference-counted class cannot be cast to its base non-reference-counted class that it'd be the same for casting to non-reference-counted interfaces.

Rainer Schuetze wrote:

On 28.06.2013 13:37, Michel Fortin wrote:
> Le 28-juin-2013 à 4:55, Rainer Schuetze <r.sagitario@gmx.de> a écrit
> :
>
>> What happens if the class also implements interfaces? A reference
>> of the interface type must do reference counting as well. So the
>> interface must also define AddRef and Release. This is currently
>> true for COM-interfaces derived from IUnknown, but not for other
>> interfaces.
>
>
> I would assume if an object of a reference-counted class cannot be
> cast to its base non-reference-counted class that it'd be the same
> for casting to non-reference-counted interfaces.
>

Yes, that is probably the way to go. But that makes using protocols like COM difficult in safe mode. We have already talked Walter into not linking reference counting to AddRef and Release, but if it is implemented with other methods, these cannot be added to the already existing COM interfaces. Passing or getting interface pointers to/from external code being unsafe sounds ok, but passing around these interface references in D code would be unsafe as well.

Adding aliases or non-virtual wrappers to the interface declaration to forward reference counting to AddRef/Release might help but could also introduce ambiguities in a class that derives both from a reference counted base class and an interface like this.

Michel Fortin wrote:

Le 25-juin-2013 à 17:00, Walter Bright  a écrit :

> If a class contains the following methods, in either itself or a base class, it is
> an RC class:
>
>
>    T AddRef();
>    T Release();
>
> An RC class is like a regular D class with these additional semantics:
>
> 1. In @safe code, casting (implicit or explicit) to a base class that does not
> have both AddRef() and Release() is an error.

I'm just realizing that this means @safe code cannot call any member function of the non-reference-counted base class. This would require an implicit conversion of "this" to the base class.

@system code could, but it'd be extremely uneasy doing such calls unless I am the one in charge of that code and can make sure the base function will never store the (unretained) pointer somewhere it shouldn't now and in the future. An misstep here and you get memory corruption. Seriously, I don't think @system code should allow implicit conversions to the base class, it should be explicit.

I am starting to doubt there is any value in inheriting the base ref-counted-class from another class.

On 6/28/2013 1:47 AM, Rainer Schuetze wrote:
> On 28.06.2013 00:35, Michel Fortin wrote:
>> So if we return to the core of it, here's the problems that still
>> need solving:
>>
>> 1. Depending on the reference counting scheme implemented, it might
>> be more efficient to have a single operation for an assignment
>> (retain a/release b) operation. I think that should be allowed.
>
>> 2. If the pointer variable is shared assignment must be atomic (done
>> under a lock, and it must always be the same lock for a given
>> pointer, obviously).
>
>> 3. If the pointer variable is shared, reading its value must be done
>> atomically with a retain too.
>
> I just had an idea, maybe it is obvious and just distracts, but I thought it might be worth sharing:
>
> Instead of defining methods on the class type, we could also redefine the reference type. The compiler detects a type declaration "reference_type" in the class declaration and replaces all references to that class with that type.
>
> class C
> {
>     alias shared_ptr!C reference_type;
> }
>
> C c = new C;
>
> is lowered to
>
> shared_ptr!C c = new C;
>
> "new C" returns a shared_ptr!C aswell.
>
> It is then up to the implementation of shared_ptr to define what member functions to call for reference counting and to deal with proper shared semantics in assignments. It can also define whether opCall should increment the reference count or not. For most of the needed functionality, struct semantics work out-of-the-box.
>
> 2 immediate gotchas
>
> - In a class hierarchy, you would want to define the reference_type in the base class only, so maybe it has to be a template. I'm not sure implicite casting to base class reference type and interfaces can be implemented.
>
> - the implementation of the shared_ptr template will have to be able to deal with the "raw" reference, so that might need some type modifier/annotation. I think this might also be true for the addRef/release version, if the implementation is not just working on the refcount, but is also calling other functions.
>
> - To elide redundant reference counting, the compiler will need annotations here, too. Move semantics of structs might reduce the number of reference count operations already, though.
>
>

The main problem with this is the decay of a shared_ptr!C to a C. Once that happens, all the memory safety goes out the window.

On 6/28/2013 7:14 AM, Michel Fortin wrote:
> Le 25-juin-2013 à 17:00, Walter Bright  a écrit :
>
>> If a class contains the following methods, in either itself or a base class, it is
>> an RC class:
>>
>>
>>     T AddRef();
>>     T Release();
>>
>> An RC class is like a regular D class with these additional semantics:
>>
>> 1. In @safe code, casting (implicit or explicit) to a base class that does not
>> have both AddRef() and Release() is an error.
> I'm just realizing that this means @safe code cannot call any member function of the non-reference-counted base class. This would require an implicit conversion of "this" to the base class.

That's right.

>
> @system code could, but it'd be extremely uneasy doing such calls unless I am the one in charge of that code and can make sure the base function will never store the (unretained) pointer somewhere it shouldn't now and in the future. An misstep here and you get memory corruption. Seriously, I don't think @system code should allow implicit conversions to the base class, it should be explicit.

It's a worthy point.

>
> I am starting to doubt there is any value in inheriting the base ref-counted-class from another class.
>

Rainer Schuetze wrote:

On 28.06.2013 21:50, Walter Bright wrote:
> The main problem with this is the decay of a shared_ptr!C to a C. Once
> that happens, all the memory safety goes out the window.

By "decay", do mean the lowering or something else?

There is no stray C reference in user code, it always gets lowered to shared_ptr!C. Only @trusted code in shared_ptr will have to deal with "raw" references. It is shared_ptr's responsibilty to maintain memory safety, just the same as for AddRef and Release.

On 6/28/2013 1:11 PM, Rainer Schuetze wrote:
> On 28.06.2013 21:50, Walter Bright wrote:
>> The main problem with this is the decay of a shared_ptr!C to a C. Once
>> that happens, all the memory safety goes out the window.
>
> By "decay", do mean the lowering or something else?
>
> There is no stray C reference in user code, it always gets lowered to shared_ptr!C. Only @trusted code in shared_ptr will have to deal with "raw" references. It is shared_ptr's responsibilty to maintain memory safety, just the same as for AddRef and Release.
>

"Decay" means it is converted to type C in order to call functions that take C as the 'this' pointer or C as a parameter. The problem is both type C and type shared_ptr!C will exist.

Rainer Schuetze wrote:

On 28.06.2013 22:29, Walter Bright wrote:
>
> On 6/28/2013 1:11 PM, Rainer Schuetze wrote:
>> On 28.06.2013 21:50, Walter Bright wrote:
>>> The main problem with this is the decay of a shared_ptr!C to a C. Once
>>> that happens, all the memory safety goes out the window.
>>
>> By "decay", do mean the lowering or something else?
>>
>> There is no stray C reference in user code, it always gets lowered to
>> shared_ptr!C. Only @trusted code in shared_ptr will have to deal with
>> "raw" references. It is shared_ptr's responsibilty to maintain memory
>> safety, just the same as for AddRef and Release.
>>
>
> "Decay" means it is converted to type C in order to call functions that
> take C as the 'this' pointer or C as a parameter. The problem is both
> type C and type shared_ptr!C will exist.
>

Any parameter of type C is also lowered to shared_ptr!C. Calling a member function would go through opDot, which could also do reference counting for safety. Treating every explicite or implicite usage of "this" as a temporary shared_ptr!C might be overkill, so it could be restricted to assigning "this" to another reference (this includes passing it as an argument to another function or returning it from a function). My current adhoc rule: if "this" is not followed by a '.', it has to be lowered to construct shared_ptr!C(this).

Assuming the reference count is updated by shared_ptr!C.opDot, there will always be a thread local reference while inside a member function (it must have been called through an external reference at least once). Other member functions of the same object can always be called without ref-counting assuming that the object never gets destroyed through changing other references.

On 6/28/2013 2:03 PM, Rainer Schuetze wrote:
> On 28.06.2013 22:29, Walter Bright wrote:
>>
>> On 6/28/2013 1:11 PM, Rainer Schuetze wrote:
>>> On 28.06.2013 21:50, Walter Bright wrote:
>>>> The main problem with this is the decay of a shared_ptr!C to a C. Once
>>>> that happens, all the memory safety goes out the window.
>>>
>>> By "decay", do mean the lowering or something else?
>>>
>>> There is no stray C reference in user code, it always gets lowered to
>>> shared_ptr!C. Only @trusted code in shared_ptr will have to deal with
>>> "raw" references. It is shared_ptr's responsibilty to maintain memory
>>> safety, just the same as for AddRef and Release.
>>>
>>
>> "Decay" means it is converted to type C in order to call functions that
>> take C as the 'this' pointer or C as a parameter. The problem is both
>> type C and type shared_ptr!C will exist.
>>
>
> Any parameter of type C is also lowered to shared_ptr!C.

I don't see how lowering C to shared_ptr!C and lowering share_ptr!C to C can work?

> Calling a member function would go through opDot, which could also do reference counting for safety. Treating every explicite or implicite usage of "this" as a temporary shared_ptr!C might be overkill, so it could be restricted to assigning "this" to another reference (this includes passing it as an argument to another function or returning it from a function). My current adhoc rule: if "this" is not followed by a '.', it has to be lowered to construct shared_ptr!C(this).
>
> Assuming the reference count is updated by shared_ptr!C.opDot, there will always be a thread local reference while inside a member function (it must have been called through an external reference at least once). Other member functions of the same object can always be called without ref-counting assuming that the object never gets destroyed through changing other references.
>

Michel Fortin wrote:

Le 28-juin-2013 à 17:03, Rainer Schuetze <r.sagitario@gmx.de> a écrit :

> Any parameter of type C is also lowered to shared_ptr!C.

class C {}

People still constantly forget that C used as a type represents a *reference* to an object of class C, not the object itself. If you replace type C with shared_ptr!C, you must then replace it with shared_ptr!(shared_ptr!C) and so on; there's no end to it.

Also, I strongly doubt the compiler will be able to elide redundant calls to retain/release made within shared_ptr!C while still respecting normal struct semantics.