(Title is borrowed from Hans Boehm's famous "Threads cannot be implemented as a library", http://www.hpl.hp.com/techreports/2004/HPL-2004-209.pdf.)

We want to implement safe reference counting in D, and the sentiment that it can be done in a library (with no or minimal changes to the language) is often echoed in the newsgroup. This post explains that reference counting can be implemented as a library _only_ if safety is forgone. If safety is desired, reference counting must be built into language semantics.

The crux of the matter is modular typechecking. Consider the following example:

// module widget.d
@safe class Widget {
  void fun() {
    g_widget = this;
  }
}
static Widget g_widget;
// end of module widget.d

This is a perfect valid D class, and safe too. The typechecker assumes infinite lifetime for all Widget objects, and allows escaping the reference to this from foo() into the global.

Numerous similar examples can be constructed involving more elaborate escaping patterns than this simple global assignment.

Now, once the typechecker OKs module widget.d, the summary that all other typechecking "sees" is:

@safe class Widget {
  void fun();
}

A library reference counting wrapper a la RC!Widget needs to allow calls to fun(). Once that happens, the global will hold an alias to the respective Widget object indefinitely, which means as soon as the RC!Widget object is released by the reference counting protocol, g_widget will become a dangling reference.

It follows that if we want safe reference counting, there must be language support for it. One possibility is to attach an attribute to the class definition:

@safe @rc class Widget {
  ...
}

Then the compiler is able to enforce more stringent typechecking on Widget (for example, address of fields are not allowed to escape) and also insert the appropriate reference counting logic. This will be a topic discussed on the lifetime list.


Andrei

On 28/10/15 12:41 AM, Andrei Alexandrescu wrote:
> (Title is borrowed from Hans Boehm's famous "Threads cannot be
> implemented as a library",
> http://www.hpl.hp.com/techreports/2004/HPL-2004-209.pdf.)
>
> We want to implement safe reference counting in D, and the sentiment
> that it can be done in a library (with no or minimal changes to the
> language) is often echoed in the newsgroup. This post explains that
> reference counting can be implemented as a library _only_ if safety is
> forgone. If safety is desired, reference counting must be built into
> language semantics.
>
> The crux of the matter is modular typechecking. Consider the following
> example:
>
> // module widget.d
> @safe class Widget {
>    void fun() {
>      g_widget = this;
>    }
> }
> static Widget g_widget;
> // end of module widget.d
>
> This is a perfect valid D class, and safe too. The typechecker assumes
> infinite lifetime for all Widget objects, and allows escaping the
> reference to this from foo() into the global.
>
> Numerous similar examples can be constructed involving more elaborate
> escaping patterns than this simple global assignment.
>
> Now, once the typechecker OKs module widget.d, the summary that all
> other typechecking "sees" is:
>
> @safe class Widget {
>    void fun();
> }
>
> A library reference counting wrapper a la RC!Widget needs to allow calls
> to fun(). Once that happens, the global will hold an alias to the
> respective Widget object indefinitely, which means as soon as the
> RC!Widget object is released by the reference counting protocol,
> g_widget will become a dangling reference.
>
> It follows that if we want safe reference counting, there must be
> language support for it. One possibility is to attach an attribute to
> the class definition:
>
> @safe @rc class Widget {
>    ...
> }
>
> Then the compiler is able to enforce more stringent typechecking on
> Widget (for example, address of fields are not allowed to escape) and
> also insert the appropriate reference counting logic. This will be a
> topic discussed on the lifetime list.
>
>
> Andrei

Thought: we have RTInfo where we can place anything we want for a type.
Perhaps we could explore this avenue a bit?

A library solution, but perhaps in druntime instead of e.g. Phobos.
Add in compiler hooks and wala. Language support for it.

Other random thoughts, a storage attribute cast(@rc)myRef;
On a class (as you shown) @rc class Widget {}

I think there is no one solution here. But similar behavior with slightly different semantics.

Anyway, the only wrong solution involves:

RefCount!T func(IAllocator alloc=theAllocator) {
	return RefCount!T(alloc.make!T, alloc);
}

Which I'm already doing.
And now I want to join that group... great.

On 27 October 2015 at 21:41, Andrei Alexandrescu via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>
> It follows that if we want safe reference counting, there must be language support for it. One possibility is to attach an attribute to the class definition:
>
> @safe @rc class Widget {
>   ...
> }

An attribute? Is presence of opInc()/opDec() insufficient? Would the
attribute signal fabrication of some default opInc/opDec operators
applicable for general use?

On 10/27/2015 07:57 AM, Manu via Digitalmars-d wrote:
> On 27 October 2015 at 21:41, Andrei Alexandrescu via Digitalmars-d
> <digitalmars-d@puremagic.com> wrote:
>>
>> It follows that if we want safe reference counting, there must be language
>> support for it. One possibility is to attach an attribute to the class
>> definition:
>>
>> @safe @rc class Widget {
>>    ...
>> }
>
> An attribute? Is presence of opInc()/opDec() insufficient? Would the
> attribute signal fabrication of some default opInc/opDec operators
> applicable for general use?

You're right, opInc/opDec detection would suffice.

Unrelated, and a foreshadowing of the discussion on the lifetime mailing list: the compiler has ample opportunity to fuse incs/decs together, so the signatures of these functions is:

void opInc(uint delta);
void opDec(uint delta);

For example, consider:

class Widget {
  void fun(Widget, Widget);
}
...
auto w = new Widget;
w.fun(w, w);

In this case the compiler may insert opInc with a value larger than 1 prior to entering the call.


Andrei

On 27 October 2015 at 22:27, Andrei Alexandrescu via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
> On 10/27/2015 07:57 AM, Manu via Digitalmars-d wrote:
>>
>> On 27 October 2015 at 21:41, Andrei Alexandrescu via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>>>
>>>
>>> It follows that if we want safe reference counting, there must be
>>> language
>>> support for it. One possibility is to attach an attribute to the class
>>> definition:
>>>
>>> @safe @rc class Widget {
>>>    ...
>>> }
>>
>>
>> An attribute? Is presence of opInc()/opDec() insufficient? Would the
>> attribute signal fabrication of some default opInc/opDec operators
>> applicable for general use?
>
>
> You're right, opInc/opDec detection would suffice.
>
> Unrelated, and a foreshadowing of the discussion on the lifetime mailing list: the compiler has ample opportunity to fuse incs/decs together, so the signatures of these functions is:
>
> void opInc(uint delta);
> void opDec(uint delta);
>
> For example, consider:
>
> class Widget {
>   void fun(Widget, Widget);
> }
> ...
> auto w = new Widget;
> w.fun(w, w);
>
> In this case the compiler may insert opInc with a value larger than 1 prior to entering the call.

Awesome. Hadn't thought of that. Nice catch!
Perhaps add me to the list. I don't think I have anything to add that
I haven't said before, but I have very high interest in this topic.

On Tuesday, 27 October 2015 at 12:27:29 UTC, Andrei Alexandrescu wrote:
> Unrelated, and a foreshadowing of the discussion on the lifetime mailing list:

What's the lifetime mailing list?

- Jonathan M Davis

On 10/27/15 11:59 AM, Jonathan M Davis wrote:
> On Tuesday, 27 October 2015 at 12:27:29 UTC, Andrei Alexandrescu wrote:
>> Unrelated, and a foreshadowing of the discussion on the lifetime
>> mailing list:
>
> What's the lifetime mailing list?

To be created. -- Andrei

I've made the claim that we should implement reference counting as a library many time, so I think I should explicit my position. Indeed, RC require some level a compiler support to be safe. That being said, the support does not need to be specific to RC. On fact, my position is that the language should provide some basic mechanism on top of which safe RC can be implemented, as a library.

The problem at hand here is escape analysis. The compiler must be able to ensure that a reference doesn't escape the RC mechanism in an uncontrolled manner. I'd like to add such mechanism to the language rather than bake in reference counting, as it can be used to solve other problem we are facing today (@nogc exception for instance).

 Why care about this?

 Even Rust doesn't try to solve this problem...because it isn't really a problem in practice.

 In c++/rust code you use value/unique types 99.9% of the time, and reference counting is only for shared/aliased objects.


 Reference counting == rare and unimportant

On Tuesday, 27 October 2015 at 20:19:42 UTC, PuglyWUgly wrote:
>  Why care about this?
>
>  Even Rust doesn't try to solve this problem...because it isn't really a problem in practice.
>
>  In c++/rust code you use value/unique types 99.9% of the time, and reference counting is only for shared/aliased objects.
>
>
>  Reference counting == rare and unimportant

+1