Is there any plan to add support for weak references to the D garbage collector?

Weak references are references which don't prevent the object from being garbage collected.  If it has been garbage collected then the weak reference returns null.

This is useful for things like resource caches, maintaining references to all instances of a specific class (ex: to implement an operation such as Window.minimizeAllWindows()), or to register a callback/listener with an object such that the registration won't prevent that object from being garbage collected.

A weak reference library exists but doesn't always work...

PJP Wrote:

> Is there any plan to add support for weak references to the D garbage collector?
> 
> Weak references are references which don't prevent the object from being garbage collected.  If it has been garbage collected then the weak reference returns null.
> 
> This is useful for things like resource caches, maintaining references to all instances of a specific class (ex: to implement an operation such as Window.minimizeAllWindows()), or to register a callback/listener with an object such that the registration won't prevent that object from being garbage collected.

On Thu, Aug 7, 2008 at 8:12 AM, Jason House <jason.james.house@gmail.com> wrote:
> A weak reference library exists but doesn't always work...

By doesn't always work you mean that it seems to you that sometimes it prevents the target object from being collected, right?

But I think you said your only test case for this was something that is not reproducible, correct?

Very frustrating to not be able to say for sure.  It would be a huge help if the GC exposed some API for querying where a particular pointer is being referenced.  Without that it's really hard to say where the reference actually is coming from.

One possible gotcha that just crossed my mind:  I think the current GC allocates chunks in an all or nothing manner -- hasPointers is just a single bit.  So could it be that the fact that all Objects have a vtable pointer mean that everything in an Object is always treated as having pointers?

--bb

> PJP Wrote:
>
>> Is there any plan to add support for weak references to the D garbage collector?
>>
>> Weak references are references which don't prevent the object from being garbage collected.  If it has been garbage collected then the weak reference returns null.
>>
>> This is useful for things like resource caches, maintaining references to all instances of a specific class (ex: to implement an operation such as Window.minimizeAllWindows()), or to register a callback/listener with an object such that the registration won't prevent that object from being garbage collected.

PJP schrieb:
> Is there any plan to add support for weak references to the D garbage
> collector?
> 
> Weak references are references which don't prevent the object from
> being garbage collected.  If it has been garbage collected then the
> weak reference returns null.
> 
> This is useful for things like resource caches, maintaining
> references to all instances of a specific class (ex: to implement an
> operation such as Window.minimizeAllWindows()), or to register a
> callback/listener with an object such that the registration won't
> prevent that object from being garbage collected.

And please, put a WeakHashMap on the TODO list too :)
http://java.sun.com/j2se/1.4.2/docs/api/java/util/WeakHashMap.html

Bill Baxter Wrote:

> On Thu, Aug 7, 2008 at 8:12 AM, Jason House <jason.james.house@gmail.com> wrote:
> > A weak reference library exists but doesn't always work...
> 
> By doesn't always work you mean that it seems to you that sometimes it prevents the target object from being collected, right?

It causes crashes under the right circumstances. I posted test cases to reproduce the behavior when I was experimenting with it...




> 
> But I think you said your only test case for this was something that is not reproducible, correct?
> 
> Very frustrating to not be able to say for sure.  It would be a huge help if the GC exposed some API for querying where a particular pointer is being referenced.  Without that it's really hard to say where the reference actually is coming from.
> 
> One possible gotcha that just crossed my mind:  I think the current GC allocates chunks in an all or nothing manner -- hasPointers is just a single bit.  So could it be that the fact that all Objects have a vtable pointer mean that everything in an Object is always treated as having pointers?
> 
> --bb
> 
> > PJP Wrote:
> >
> >> Is there any plan to add support for weak references to the D garbage collector?
> >>
> >> Weak references are references which don't prevent the object from being garbage collected.  If it has been garbage collected then the weak reference returns null.
> >>
> >> This is useful for things like resource caches, maintaining references to all instances of a specific class (ex: to implement an operation such as Window.minimizeAllWindows()), or to register a callback/listener with an object such that the registration won't prevent that object from being garbage collected.

Frank Benoit Wrote:

> And please, put a WeakHashMap on the TODO list too :) http://java.sun.com/j2se/1.4.2/docs/api/java/util/WeakHashMap.html

I wrote one of those, but it does not work. When investigating, I found issues with the basic weak reference.

Jason House Wrote:

> Bill Baxter Wrote:
> 
> > On Thu, Aug 7, 2008 at 8:12 AM, Jason House <jason.james.house@gmail.com> wrote:
> > > A weak reference library exists but doesn't always work...
> > 
> > By doesn't always work you mean that it seems to you that sometimes it prevents the target object from being collected, right?
> 
> It causes crashes under the right circumstances. I posted test cases to reproduce the behavior when I was experimenting with it...
> 
> 
> 
> 
> > 
> > But I think you said your only test case for this was something that is not reproducible, correct?
> > 
> > Very frustrating to not be able to say for sure.  It would be a huge help if the GC exposed some API for querying where a particular pointer is being referenced.  Without that it's really hard to say where the reference actually is coming from.
> > 
> > One possible gotcha that just crossed my mind:  I think the current GC allocates chunks in an all or nothing manner -- hasPointers is just a single bit.  So could it be that the fact that all Objects have a vtable pointer mean that everything in an Object is always treated as having pointers?
> > 
> > --bb
> > 
> > > PJP Wrote:
> > >
> > >> Is there any plan to add support for weak references to the D garbage collector?
> > >>
> > >> Weak references are references which don't prevent the object from being garbage collected.  If it has been garbage collected then the weak reference returns null.
> > >>
> > >> This is useful for things like resource caches, maintaining references to all instances of a specific class (ex: to implement an operation such as Window.minimizeAllWindows()), or to register a callback/listener with an object such that the registration won't prevent that object from being garbage collected.
> 

"Bill Baxter" wrote
> On Thu, Aug 7, 2008 at 8:12 AM, Jason House wrote:
>> A weak reference library exists but doesn't always work...
>
> By doesn't always work you mean that it seems to you that sometimes it prevents the target object from being collected, right?
>
> But I think you said your only test case for this was something that is not reproducible, correct?
>
> Very frustrating to not be able to say for sure.  It would be a huge help if the GC exposed some API for querying where a particular pointer is being referenced.  Without that it's really hard to say where the reference actually is coming from.
>
> One possible gotcha that just crossed my mind:  I think the current GC allocates chunks in an all or nothing manner -- hasPointers is just a single bit.  So could it be that the fact that all Objects have a vtable pointer mean that everything in an Object is always treated as having pointers?

At least on Tango, this is not the case:

import tango.core.Memory;

class X
{
    int x;
}

class Y
{
    int *y;
}

void main()
{
    auto n = new X;
    assert(GC.getAttr(cast(void*)n) & GC.BlkAttr.NO_SCAN);
    auto m = new Y;
    assert(!(GC.getAttr(cast(void*)m) & GC.BlkAttr.NO_SCAN));
}

-Steve

On Thu, Aug 7, 2008 at 11:38 PM, Steven Schveighoffer <schveiguy@yahoo.com> wrote:
> "Bill Baxter" wrote
>> On Thu, Aug 7, 2008 at 8:12 AM, Jason House wrote:
>>
>> One possible gotcha that just crossed my mind:  I think the current GC allocates chunks in an all or nothing manner -- hasPointers is just a single bit.  So could it be that the fact that all Objects have a vtable pointer mean that everything in an Object is always treated as having pointers?
>
> At least on Tango, this is not the case:
>
> import tango.core.Memory;
>
> class X
> {
>    int x;
> }
>
> class Y
> {
>    int *y;
> }
>
> void main()
> {
>    auto n = new X;
>    assert(GC.getAttr(cast(void*)n) & GC.BlkAttr.NO_SCAN);
>    auto m = new Y;
>    assert(!(GC.getAttr(cast(void*)m) & GC.BlkAttr.NO_SCAN));
> }

Hmm, yeh I also went looking for the previous discussion on this topic and found there that someone had already checked that the GC flag on the WeakRef class was getting set properly.

The only other potential problem I can see is the manipulation of the member variable that goes on in the destructor of the WeakRef.  It makes the assumption that if the referred object has been collected then the notification function will have been called, and thus the pointer-as-size_t variable will be null.  But that code is very similar to what std.signals does in its destructor.  So if that's the source of the bug then it's probably a bug that's in std.signal too.

--bb

"Bill Baxter" wrote
> On Thu, Aug 7, 2008 at 11:38 PM, Steven Schveighoffer <schveiguy@yahoo.com> wrote:
>> "Bill Baxter" wrote
>>> On Thu, Aug 7, 2008 at 8:12 AM, Jason House wrote:
>>>
>>> One possible gotcha that just crossed my mind:  I think the current GC allocates chunks in an all or nothing manner -- hasPointers is just a single bit.  So could it be that the fact that all Objects have a vtable pointer mean that everything in an Object is always treated as having pointers?
>>
>> At least on Tango, this is not the case:
>>
>> import tango.core.Memory;
>>
>> class X
>> {
>>    int x;
>> }
>>
>> class Y
>> {
>>    int *y;
>> }
>>
>> void main()
>> {
>>    auto n = new X;
>>    assert(GC.getAttr(cast(void*)n) & GC.BlkAttr.NO_SCAN);
>>    auto m = new Y;
>>    assert(!(GC.getAttr(cast(void*)m) & GC.BlkAttr.NO_SCAN));
>> }
>
> Hmm, yeh I also went looking for the previous discussion on this topic and found there that someone had already checked that the GC flag on the WeakRef class was getting set properly.
>
> The only other potential problem I can see is the manipulation of the member variable that goes on in the destructor of the WeakRef.  It makes the assumption that if the referred object has been collected then the notification function will have been called, and thus the pointer-as-size_t variable will be null.  But that code is very similar to what std.signals does in its destructor.  So if that's the source of the bug then it's probably a bug that's in std.signal too.

I don't think that's possible.  In order for this to cause a problem, the GC would have to collect the memory from the referred object before calling your hook, or else that it doesn't call your hook at all.  Neither of these seem likely.

Looking through your code with thread-oriented eyes, and assuming that any thread can cause a collect cycle at any time, it seems that it is impossible for a race to occur.

-Steve