Jump to page: 1 24  
Page
Thread overview
[Dlang-study] [lifetime] Initial thoughts on lifetime management
Oct 28, 2015
Marc Schütz
Oct 28, 2015
Marc Schütz
Oct 28, 2015
Timon Gehr
Oct 28, 2015
Timon Gehr
Oct 29, 2015
Timon Gehr
Oct 29, 2015
Michel Fortin
Oct 29, 2015
Timon Gehr
Oct 29, 2015
Michel Fortin
Oct 29, 2015
Timon Gehr
Oct 29, 2015
Michel Fortin
Oct 28, 2015
Walter Bright
Oct 28, 2015
Walter Bright
Oct 28, 2015
Walter Bright
Oct 28, 2015
Walter Bright
Oct 28, 2015
Jacob Carlborg
Oct 28, 2015
Dmitry Olshansky
Oct 28, 2015
Walter Bright
Oct 28, 2015
Dmitry Olshansky
Oct 28, 2015
Jonathan M Davis
Oct 28, 2015
Dmitry Olshansky
Oct 28, 2015
Michel Fortin
Oct 28, 2015
Michel Fortin
Oct 28, 2015
Timon Gehr
Oct 28, 2015
Timon Gehr
October 27, 2015
Hello all and thanks for your interest in the topic of lifetime management of D objects. This post outlines a starting point for the discussion.

We need for D a form of shared ownership that is not backed up by a tracing garbage collector, and one obvious choice is reference counting.

I have argued with simple examples (http://forum.dlang.org/thread/n0nnu0$1tth$1 at digitalmars.com) that safe reference counting cannot be implemented as a library. So we're looking at a language-backed solution.

Here there are two options we discussed - an @rc attribute, or simple detection of a pair such as opInc/opDec. I've argued for the latter but Walter made a good argument for @rc: it gives the compiler complete control over "as-if" style optimizations. So I'll proceed under the assumption that we go with @rc. Namely, we're looking at:

@rc class Widget {
    ...
}

Such classes:

* do not inherit Object. (Do we want a root of all @rc classes called RCObject?)

* cannot be converted to interfaces (we may later add @rc interfaces)

* embed an implementation-defined reference count as a hidden member

* have a vptr only if they define non-final methods (in particular, I think we should disable "synchronized" for them - it seems an unnecessary complication at least for v1)

* should they have a hidden pointer to the parent scope if nested?

Each reference to the class behaves as-if there is a corresponding bump in reference count for it. This is maximally conservative, meaning even method calls need to make sure there's a refcount bump for "this". Consider:

Widget global;
@rc class Widget {
   int x;
   void fun() {
     global = null;
     ++x;
   }
}

void main() {
   global = new Widget;
   global.fun();
}

In this example, if global has a refcount==1 upon entering fun(), the assignment "global = null" deletes the Widget object and ++x accesses dangling memory.

So my plan is to start maximally conservative - i.e. bump the reference count appropriately for every use of the object. Then, we define a static analysis that eliminates or fuses together most of these increments and decrements. There is much research in this area. I am counting on the folks in this group to define this analysis rigorously.


Thanks,

Andrei
October 27, 2015
I should add here another pattern that turned problematic for our older attempts in DIP74:

C c = new C();
foo(c);

int foo(scope C d) {
     c = new C();    // c's old instance gets deleted
     return d.i;        // oops! d is invalid
}


Andrei


October 28, 2015
On Wednesday, 28 October 2015 at 00:45:00 UTC, Andrei Alexandrescu wrote:
> Hello all and thanks for your interest in the topic of lifetime management of D objects. This post outlines a starting point for the discussion.

I would like to mention a previous email conversation that has already been posted on the newsgroups for reference [1]

> Here there are two options we discussed - an @rc attribute, or simple detection of a pair such as opInc/opDec. I've argued for the latter but Walter made a good argument for @rc: it gives the compiler complete control over "as-if" style optimizations. So I'll proceed under the assumption that we go with @rc. Namely, we're looking at:
>
> @rc class Widget {
>     ...
> }

Are there plans to allow the implementor of a @rc class to decide the implementation of the reference counting? I'm thinking from the point of view of Objective-C interoperability.

> Such classes:
>
> * do not inherit Object. (Do we want a root of all @rc classes called RCObject?)

Not sure if this has any implications for the Objective-C interoperability. In Objective-C there's one root (kind of), that is "NSObject". Not sure if there would be a problem if NSObject in D inherited from RCObject.

> * cannot be converted to interfaces (we may later add @rc interfaces)

I'm pretty sure this will be needed when proper support for Objective-C interface are added.

> * embed an implementation-defined reference count as a hidden member

I guess this answers my first question.

All the concerns mentioned above about Objective-C interoperability can of course be special cased in the compiler, if needed.

[1] http://forum.dlang.org/thread/l34lei$255v$1@digitalmars.com

--
/Jacob Carlborg
_______________________________________________
Dlang-study mailing list
Dlang-study@puremagic.com
http://lists.puremagic.com/cgi-bin/mailman/listinfo/dlang-study

October 28, 2015
On 28-Oct-2015 10:44, Jacob Carlborg wrote:
> On Wednesday, 28 October 2015 at 00:45:00 UTC, Andrei Alexandrescu wrote:
>
>> * embed an implementation-defined reference count as a hidden member
>
> I guess this answers my first question.

Just a short note on this: COM (which is the backbone of all new WinRT) would be nice to support as well and that means being flexible with where and how reference count is stored. Even putting aside COM & ObjectiveC - in some cases storing ref-counts in a separate block is advantageous and limiting ourselves to hidden embedded ref-count is not necessary.

In fact, compiler being aware of ref-counting helps these more complicated (and sometimes opaque) methods of re-counting b/c it would not be able to optimize then away otherwise.
>
> All the concerns mentioned above about Objective-C interoperability can of course be special cased in the compiler, if needed.
>
> [1] http://forum.dlang.org/thread/l34lei$255v$1@digitalmars.com
>
> -- 
> /Jacob Carlborg
> _______________________________________________
> Dlang-study mailing list
> Dlang-study@puremagic.com
> http://lists.puremagic.com/cgi-bin/mailman/listinfo/dlang-study


-- 
Dmitry Olshansky




October 28, 2015

On 10/28/2015 1:40 AM, Dmitry Olshansky wrote:
>
> Just a short note on this: COM (which is the backbone of all new WinRT) would be nice to support as well and that means being flexible with where and how reference count is stored. Even putting aside COM & ObjectiveC - in some cases storing ref-counts in a separate block is advantageous and limiting ourselves to hidden embedded ref-count is not necessary.

D already supports COM - just derive from IUnknown.

I don't think it'll be a good idea to base a native rc mechanism on COM, though. COM relies on virtual functions to do the ref counting (slow), and of course it would interfere with any code that called AddRef or Release directly. I'm not sure how it would interact with the large body of existing COM code.
_______________________________________________
Dlang-study mailing list
Dlang-study@puremagic.com
http://lists.puremagic.com/cgi-bin/mailman/listinfo/dlang-study

October 28, 2015
On 28-Oct-2015 12:03, Walter Bright wrote:
>
>
> On 10/28/2015 1:40 AM, Dmitry Olshansky wrote:
>>
>> Just a short note on this: COM (which is the backbone of all new WinRT) would be nice to support as well and that means being flexible with where and how reference count is stored. Even putting aside COM & ObjectiveC - in some cases storing ref-counts in a separate block is advantageous and limiting ourselves to hidden embedded ref-count is not necessary.
>
> D already supports COM - just derive from IUnknown.
>
> I don't think it'll be a good idea to base a native rc mechanism on COM, though.
Nor do I see below.
> COM relies on virtual functions to do the ref counting (slow), and of course it would interfere with any code that called AddRef or Release directly. I'm not sure how it would interact with the large body of existing COM code.

I mean our @rc stuff should automatically work for ref-counting of COM objects and beyond. Manual ref-counting of COM is error prone, library solution is suboptimal see Andrei initial post.
> _______________________________________________
> Dlang-study mailing list
> Dlang-study@puremagic.com
> http://lists.puremagic.com/cgi-bin/mailman/listinfo/dlang-study


-- 
Dmitry Olshansky




October 28, 2015
On Tuesday, October 27, 2015 20:45:00 Andrei Alexandrescu wrote:
> * do not inherit Object. (Do we want a root of all @rc classes called
> RCObject?)

Honestly, I think that having Object was a mistake. We already decided previously that we wanted to remove pretty much all of the functions from Object in order to fix the problem with have with forcing specific attributes on functions like toString or opEquals (though unfortunately, we haven't gotten very far with actually implementing such changes). And if we do that, Object becomes borderline useless. Also, because we have proper templates rather than Java-style generics, we've eliminated what's probably the primary use case for having an Object reference rather than the reference of the actual type of the object (or one of its base classes whose API actually gets used as opposed to just storing it as a reference toa base type).

That being the case, I see no reason to have an RCObject. It's just repeating our past mistakes.

> * cannot be converted to interfaces (we may later add @rc interfaces)

That seems like a potentially painful limitation, though I can see why it would be necessary. I'd expect it to be pretty much a forgeone conclusion that we're going to want to add @rc interfaces later, particularly when you consider that at least a portion of the D community will inevitably switch to using ref-counted classes and avoiding GC classes as much as possible.

> * have a vptr only if they define non-final methods (in particular, I think we should disable "synchronized" for them - it seems an unnecessary complication at least for v1)

Seems reasonable.

> * should they have a hidden pointer to the parent scope if nested?

I'm afraid that I don't follow what you mean by this - though we already have problems with templated code not compiling due to there being two or more scopes involved, forcing us to do stuff like make a voldemort type static in case the predicate has an associated scope. So, I'd be leery of adding more scopes along those lines if we don't have to (though I don't know if that really relates to what you're talking about).

> So my plan is to start maximally conservative - i.e. bump the reference count appropriately for every use of the object. Then, we define a static analysis that eliminates or fuses together most of these increments and decrements. There is much research in this area. I am counting on the folks in this group to define this analysis rigorously.

Sounds like a plan.

- Jonathan M Davis

_______________________________________________
Dlang-study mailing list
Dlang-study@puremagic.com
http://lists.puremagic.com/cgi-bin/mailman/listinfo/dlang-study

October 28, 2015
On 28-Oct-2015 13:34, Jonathan M Davis wrote:
> On Tuesday, October 27, 2015 20:45:00 Andrei Alexandrescu wrote:
>> * do not inherit Object. (Do we want a root of all @rc classes called
>> RCObject?)
> Honestly, I think that having Object was a mistake. We already decided previously that we wanted to remove pretty much all of the functions from Object in order to fix the problem with have with forcing specific attributes on functions like toString or opEquals (though unfortunately, we haven't gotten very far with actually implementing such changes). And if we do that, Object becomes borderline useless. Also, because we have proper templates rather than Java-style generics, we've eliminated what's probably the primary use case for having an Object reference rather than the reference of the actual type of the object (or one of its base classes whose API actually gets used as opposed to just storing it as a reference toa base type).
>
>

Cutting to the chase - there is big reason for even nominal super type -
(manual) type erasure.
Simply put - Queue!T where T is a class is trivially implemented as
Queue!Object, same for most containers.
Some might need ordering or hashing, these are easily added as
interfaces Hashable and/or Comparable. Now we might argue that
containers must be templates, but code bloat in case of objects is HUGE
and unjustified.

Even if there is nothing special to Object, there must be a way to express reference to any class instance in the language, let us not lose that.

-- 
Dmitry Olshansky




October 28, 2015
Le 27 oct. 2015 à 20:45, Andrei Alexandrescu <andrei@erdani.com> a écrit :

> * do not inherit Object. (Do we want a root of all @rc classes called RCObject?)
> 
> * cannot be converted to interfaces (we may later add @rc interfaces)

This is actually how things work for D/Objective-C. And it does work. Assuming you want to allow multiple implementations of reference counting, you could have a different root object for each.


> * embed an implementation-defined reference count as a hidden member

Language-wise, you don't even need to bother about that. Since you have a root class, how the counting mechanism works is a root-class implementation detail. (IMHO)


> * have a vptr only if they define non-final methods (in particular, I think we should disable "synchronized" for them - it seems an unnecessary complication at least for v1)

If you want the root-class implementation to be able to jettison all the bagage, I'm sure that can be done. But this has nothing to do with reference counting and lifetime management, does it?


> * should they have a hidden pointer to the parent scope if nested?

The question is who owns who? Should the object retain its parent scope, or should the parent scope retain its object? One thing is sure: if you create a cycle, one of the two must be a weak (non-retaining) pointer. Which one probably depends on the situation. Perhaps just disallow non-static nested @rc classes.


> Each reference to the class behaves as-if there is a corresponding bump in reference count for it. This is maximally conservative, meaning even method calls need to make sure there's a refcount bump for "this".

True. Note that "each reference" must also counts temporaries. So if you have this:

	foo(getObject());

what you really have is this:

	foo(getObject(){-1});

where {-1} is a notation I made up to denote that the counter of the object returned by getObject() is decremented at the end of the statement. (The returned value is already {+1} by the caller.)

And if you have this:

	c = getObject();

it becomes this:

	c{-1}{+1} = getObject(){-1};

where the value assigned for c gets incremented during assignment, and both the old value for c and the temporary counters get decremented afterwards. (To be safe, the increments must be done before the decrements.) Since we have both a {+1} and a {-1} for the same object, they can be elided:

	c{-1}{} = getObject(){};

It's actually easier to reason about how it works with temporaries because temporaries do not mutate. When you elide pairs, you have to follow the object reference, which is the value and not the variable. If we're to discuss this, we need a proper notation to annotate values in code like this:

	void swap(ref C a, ref C b) {
		C tmp = a;
		a = b;
		b = tmp;
	}


-- 
Michel Fortin
michel.fortin@michelf.ca
https://michelf.ca


_______________________________________________
Dlang-study mailing list
Dlang-study@puremagic.com
http://lists.puremagic.com/cgi-bin/mailman/listinfo/dlang-study

October 28, 2015
On 10/28/2015 05:05 AM, Dmitry Olshansky wrote:
> I mean our @rc stuff should automatically work for ref-counting of COM
> objects and beyond. Manual ref-counting of COM is error prone, library
> solution is suboptimal see Andrei initial post.

That would be on the "nice to have" list but let's not make compromises for its sake. Code using COM probably does not have safety as an essential concern, and also does not passes COM objects as casually as regular class references. -- Andrei
_______________________________________________
Dlang-study mailing list
Dlang-study@puremagic.com
http://lists.puremagic.com/cgi-bin/mailman/listinfo/dlang-study

« First   ‹ Prev
1 2 3 4