Hello,

I have a class "A" and I'd like to keep a list of all the created instances of this class. To do that, I have a static List!(A) in the A class and, in the constructor, I add each new instance to this list. This gives me the following code:

class A {
   private static List!(A) s_instances;

   public this() {
      s_instances.add(this);
   }

   public ~this() {
      s_instances.remove(this);
   }

   public static void printAll() {
      foreach (A instance; s_instances)
         print(instance.toString());
   }
}

But then, since all the instances are referenced by the static list, they are never garbage-collected, which could be a problem. In some other languages, this can be solved using weak references, but I haven't found any informations about using weak references in D. Is there any way to solve this problem?

Thanks,
Simon

Simon TRENY, el 29 de marzo a las 16:33 me escribiste:
> Hello,
> 
> I have a class "A" and I'd like to keep a list of all the created instances of this class. To do that, I have a static List!(A) in the A class and, in the constructor, I add each new instance to this list. This gives me the following code:
> 
> class A {
>    private static List!(A) s_instances;
> 
>    public this() {
>       s_instances.add(this);
>    }
> 
>    public ~this() {
>       s_instances.remove(this);
>    }
> 
>    public static void printAll() {
>       foreach (A instance; s_instances)
>          print(instance.toString());
>    }
> }
> 
> But then, since all the instances are referenced by the static list, they are never garbage-collected, which could be a problem. In some other languages, this can be solved using weak references, but I haven't found any informations about using weak references in D. Is there any way to solve this problem?

This was discussed several times in the past. For example:
http://www.digitalmars.com/d/archives/digitalmars/D/learn/weak_references_13301.html
http://www.digitalmars.com/d/archives/digitalmars/D/learn/Soft_weak_references_8264.html
http://www.digitalmars.com/d/archives/digitalmars/D/announce/ANN_WeakObjectReference_-_class_to_hold_weak_references_9103.html
etc.

I hope it helps.

-- 
Leandro Lucarella (luca) | Blog colectivo: http://www.mazziblog.com.ar/blog/
----------------------------------------------------------------------------
GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145  104C 949E BFB6 5F5A 8D05)
----------------------------------------------------------------------------
Que el viento y la lluvia sean dos hermanos y corran furiosos por los
terraplenes de Víctor Heredia.
	-- Ricardo Vaporeso. Lanús, 1912.

On Sun, Mar 29, 2009 at 4:42 PM, Leandro Lucarella <llucax@gmail.com> wrote:
>
> This was discussed several times in the past. For example:
> http://www.digitalmars.com/d/archives/digitalmars/D/learn/weak_references_13301.html
> http://www.digitalmars.com/d/archives/digitalmars/D/learn/Soft_weak_references_8264.html
> http://www.digitalmars.com/d/archives/digitalmars/D/announce/ANN_WeakObjectReference_-_class_to_hold_weak_references_9103.html
> etc.
>
> I hope it helps.

The one provided by Bill:

http://www.dsource.org/projects/scrapple/browser/trunk/weakref

seems to work fine, and has the advantage of working in both Phobos and Tango.

Simon TRENY wrote:
> Hello,
> 
> I have a class "A" and I'd like to keep a list of all the created instances of this class. To do that, I have a static List!(A) in the A class and, in the constructor, I add each new instance to this list. This gives me the following code:
> 
> class A {
>    private static List!(A) s_instances;
> 
>    public this() {
>       s_instances.add(this);
>    }
> 
>    public ~this() {
>       s_instances.remove(this);
>    }
> 
>    public static void printAll() {
>       foreach (A instance; s_instances)
>          print(instance.toString());
>    }
> }
> 
> But then, since all the instances are referenced by the static list, they are never garbage-collected, which could be a problem. In some other languages, this can be solved using weak references, but I haven't found any informations about using weak references in D. Is there any way to solve this problem?
> 
> Thanks,
> Simon
> 

Maybe what you are looking for are the GC.removeRoot or GC.removeRange functions which are available in both Phobos and Tango? http://www.dsource.org/projects/tango/docs/current/tango.core.Memory.html http://www.digitalmars.com/d/2.0/phobos/std_gc.html http://www.digitalmars.com/d/1.0/phobos/std_gc.html

Jarrett Billingsley wrote:
> On Sun, Mar 29, 2009 at 4:42 PM, Leandro Lucarella <llucax@gmail.com> wrote:
>> This was discussed several times in the past. For example:
>> http://www.digitalmars.com/d/archives/digitalmars/D/learn/weak_references_13301.html
>> http://www.digitalmars.com/d/archives/digitalmars/D/learn/Soft_weak_references_8264.html
>> http://www.digitalmars.com/d/archives/digitalmars/D/announce/ANN_WeakObjectReference_-_class_to_hold_weak_references_9103.html
>> etc.
>>
>> I hope it helps.
> 
> The one provided by Bill:
> 
> http://www.dsource.org/projects/scrapple/browser/trunk/weakref
> 
> seems to work fine, and has the advantage of working in both Phobos and Tango.

First, I doubt this actually works. The WeakRef stores the pointer as size_t, but the GC is conservative and will still recognize the size_t as a pointer. The unittest in the existing code only works, because he uses an explicit delete on the referenced object.

To actually hide the pointer from the GC, you could XOR the size_t value with a constant. Note that you need to be very careful with the WeakRef.ptr() function: what happens, if the GC invalidates the object, and then the user calls ptr() in parallel, before the GC calls rt_detachDisposeEvent()? The user will get an invalid pointer. As far as I remember, rt_detachDisposeEvent() is supposed to be called when all threads have been resumed (after a collect() run). This is to avoid deadlocks if the dispose handler locks something.

Secondly, this should be extended by a ReferenceQueue (like in Java). As soon as the object referenced by the WeakRef is collected, it is added to the ReferenceQueue associated with the WeakRef. (In Java, ReferenceQueue.remove() also provides a roundabout way to notify the program when a reference has been collected.)

And finally: why is this thing not in Tango?

Maybe a Tango dev could comment on this and the correctness issues mentioned above?

grauzone wrote:
> Jarrett Billingsley wrote:
>> On Sun, Mar 29, 2009 at 4:42 PM, Leandro Lucarella <llucax@gmail.com> wrote:
>>> This was discussed several times in the past. For example:
>>> http://www.digitalmars.com/d/archives/digitalmars/D/learn/weak_references_13301.html 
>>>
>>> http://www.digitalmars.com/d/archives/digitalmars/D/learn/Soft_weak_references_8264.html 
>>>
>>> http://www.digitalmars.com/d/archives/digitalmars/D/announce/ANN_WeakObjectReference_-_class_to_hold_weak_references_9103.html 
>>>
>>> etc.
>>>
>>> I hope it helps.
>>
>> The one provided by Bill:
>>
>> http://www.dsource.org/projects/scrapple/browser/trunk/weakref
>>
>> seems to work fine, and has the advantage of working in both Phobos and Tango.
> 
> First, I doubt this actually works. The WeakRef stores the pointer as size_t, but the GC is conservative and will still recognize the size_t as a pointer. The unittest in the existing code only works, because he uses an explicit delete on the referenced object.

If the WeakRef is on the stack, this is true.

If the WeakRef is part of an aggregate type that contains pointers, this is true.

Otherwise, the GC will see that the relevant block is marked as having no pointers.

> To actually hide the pointer from the GC, you could XOR the size_t value with a constant. Note that you need to be very careful with the WeakRef.ptr() function: what happens, if the GC invalidates the object, and then the user calls ptr() in parallel, before the GC calls rt_detachDisposeEvent()? The user will get an invalid pointer. As far as I remember, rt_detachDisposeEvent() is supposed to be called when all threads have been resumed (after a collect() run). This is to avoid deadlocks if the dispose handler locks something.

True -- weakref is a difficult thing to make thread-safe.

Part of the reason I wrote it and made it available was to serve as a focal point for such critiques.   If you think it doesn't work and can fix it, please do so!

--bb

On Mon, Mar 30, 2009 at 7:00 AM, grauzone <none@example.net> wrote:
> Jarrett Billingsley wrote:
>>
>> On Sun, Mar 29, 2009 at 4:42 PM, Leandro Lucarella <llucax@gmail.com> wrote:
>>>
>>> This was discussed several times in the past. For example:
>>>
>>> http://www.digitalmars.com/d/archives/digitalmars/D/learn/weak_references_13301.html
>>>
>>> http://www.digitalmars.com/d/archives/digitalmars/D/learn/Soft_weak_references_8264.html
>>>
>>> http://www.digitalmars.com/d/archives/digitalmars/D/announce/ANN_WeakObjectReference_-_class_to_hold_weak_references_9103.html
>>> etc.
>>>
>>> I hope it helps.
>>
>> The one provided by Bill:
>>
>> http://www.dsource.org/projects/scrapple/browser/trunk/weakref
>>
>> seems to work fine, and has the advantage of working in both Phobos and Tango.
>
> First, I doubt this actually works. The WeakRef stores the pointer as size_t, but the GC is conservative and will still recognize the size_t as a pointer. The unittest in the existing code only works, because he uses an explicit delete on the referenced object.
>
> To actually hide the pointer from the GC, you could XOR the size_t value
> with a constant. Note that you need to be very careful with the
> WeakRef.ptr() function: what happens, if the GC invalidates the object, and
> then the user calls ptr() in parallel, before the GC calls
> rt_detachDisposeEvent()? The user will get an invalid pointer. As far as I
> remember, rt_detachDisposeEvent() is supposed to be called when all threads
> have been resumed (after a collect() run). This is to avoid deadlocks if the
> dispose handler locks something.
>
> Secondly, this should be extended by a ReferenceQueue (like in Java). As soon as the object referenced by the WeakRef is collected, it is added to the ReferenceQueue associated with the WeakRef. (In Java, ReferenceQueue.remove() also provides a roundabout way to notify the program when a reference has been collected.)
>
> And finally: why is this thing not in Tango?
>
> Maybe a Tango dev could comment on this and the correctness issues mentioned above?
>

Christopher Wright wrote:
> grauzone wrote:
>> Jarrett Billingsley wrote:
>>> On Sun, Mar 29, 2009 at 4:42 PM, Leandro Lucarella <llucax@gmail.com> wrote:
>>>> This was discussed several times in the past. For example:
>>>> http://www.digitalmars.com/d/archives/digitalmars/D/learn/weak_references_13301.html 
>>>>
>>>> http://www.digitalmars.com/d/archives/digitalmars/D/learn/Soft_weak_references_8264.html 
>>>>
>>>> http://www.digitalmars.com/d/archives/digitalmars/D/announce/ANN_WeakObjectReference_-_class_to_hold_weak_references_9103.html 
>>>>
>>>> etc.
>>>>
>>>> I hope it helps.
>>>
>>> The one provided by Bill:
>>>
>>> http://www.dsource.org/projects/scrapple/browser/trunk/weakref
>>>
>>> seems to work fine, and has the advantage of working in both Phobos and Tango.
>>
>> First, I doubt this actually works. The WeakRef stores the pointer as size_t, but the GC is conservative and will still recognize the size_t as a pointer. The unittest in the existing code only works, because he uses an explicit delete on the referenced object.
> 
> If the WeakRef is on the stack, this is true.
> 
> If the WeakRef is part of an aggregate type that contains pointers, this is true.

If WeakRef is a class, this is also true. Because all objects contain a hidden monitor pointer, and the monitor is subject to garbage collection AFAIK.

> Otherwise, the GC will see that the relevant block is marked as having no pointers.
> 
> True -- weakref is a difficult thing to make thread-safe.

It seems there's still work to do, and a thread-safe WeakRef can't be created with the current interfaces. Is this true?

I'm thinking rt_attachDisposeEvent() should take a *pointer* to the reference instead of the reference itself (effectively a double pointer), and clear this pointer during garbage collection, when all threads are still globally locked.

grauzone:
> First, I doubt this actually works. [...] To actually hide the pointer from the GC, you could XOR the size_t value with a constant.<

This is may be a stupid idea: Can't the OP just allocate with std.c.stdlib.malloc a block of void* pointers (plus keep an int length too), fill them with the object references and and then cast one of them back to object reference when necessary? Objects of such class can keep a similarly C-heap pointer to the cell of the block that contains its reference, and set it to null when they are removed.
It's not a general solution yet and it looks a bit messy.

Weak references may just need to be added to Phobos/Tango GC, if not present.

Bye,
bearophile

On Mon, Mar 30, 2009 at 8:04 AM, bearophile <bearophileHUGS@lycos.com> wrote:
> grauzone:
>> First, I doubt this actually works. [...] To actually hide the pointer from the GC, you could XOR the size_t value with a constant.<
>
> This is may be a stupid idea: Can't the OP just allocate with std.c.stdlib.malloc a block of void* pointers (plus keep an int length too), fill them with the object references and and then cast one of them back to object reference when necessary? Objects of such class can keep a similarly C-heap pointer to the cell of the block that contains its reference, and set it to null when they are removed.
> It's not a general solution yet and it looks a bit messy.
>
> Weak references may just need to be added to Phobos/Tango GC, if not present.


Yes, please!   Weak references absolutely should be part of the standard distributions of a GC'ed language.

--bb