View mode: basic / threaded / horizontal-split · Log in · Help
April 15, 2009
D2 weak references
Does anyone have a weak reference library for D2?

Without that, hash tables and search trees don't mix. I'm hoping that a weak reference library can be merged into druntime.
April 15, 2009
Re: D2 weak references
Jason House wrote:
> Does anyone have a weak reference library for D2?
> 
> Without that, hash tables and search trees don't mix. I'm hoping that 
> a weak reference library can be merged into druntime.

Weak references don't fit well into D at the moment.  It's been talked 
about before (albeit not in reference to D2 in particular):

http://www.digitalmars.com/d/archives/digitalmars/D/713.html
http://www.digitalmars.com/d/archives/digitalmars/D/Weak_references._69761.html

and probably others.

But I think that, if D gained weak references as a built-in feature it 
would work well and be useful.  A decent library implementation may 
otherwise be possible if D's GC API gains:
- an explicit pinning mechanism
- a means of creating listeners on the GC

Stewart.
April 15, 2009
Re: D2 weak references
Stewart Gordon Wrote:

> Jason House wrote:
> > Does anyone have a weak reference library for D2?
> > 
> > Without that, hash tables and search trees don't mix. I'm hoping that 
> > a weak reference library can be merged into druntime.
> 
> Weak references don't fit well into D at the moment.  It's been talked 
> about before (albeit not in reference to D2 in particular):
> 
> http://www.digitalmars.com/d/archives/digitalmars/D/713.html
> http://www.digitalmars.com/d/archives/digitalmars/D/Weak_references._69761.html
> 
> and probably others.

I participated in the more recent of those threads.

I feel like I have a great application for weak pointers. The best manual memmory management of search trees and hashtables is to do mark and sweep. That should sound awfully familiar...


> But I think that, if D gained weak references as a built-in feature it 
> would work well and be useful.  A decent library implementation may 
> otherwise be possible if D's GC API gains:
> - an explicit pinning mechanism
> - a means of creating listeners on the GC
> 
> Stewart.

Everything is pinned at the moment and Tango's GC (and therefore druntime's GC) has an explicit notification mechanism. The big advantage to having a weak ref implementation in the runtime is that it'll evolve with the GC as well as pool people's bug fixes / ports.
April 19, 2009
Re: D2 weak references
Jason House wrote:

> Tango's GC (and therefore
> druntime's GC) has an explicit notification mechanism.

I guess I shouldn't have assumed that the features of tango became part of 
druntime.  I don't see any notification mechanism :(

The best design I've come up with is to make a helper class 
weakReferencable(T) that will add a finalizer to fix any outstanding weak 
references.  I'd then use a double linked list of structs as weak 
references.  As long as collections are are single threaded, it should be 
pretty easy to make work.  Introducing a new type besides just a weak 
reference seems like a bit of a hack.  I'm hoping others will have 
suggestions on how to do that better.
April 19, 2009
Re: D2 weak references
Jason House wrote:
> Jason House wrote:
> 
>> Tango's GC (and therefore
>> druntime's GC) has an explicit notification mechanism.
> 
> I guess I shouldn't have assumed that the features of tango became part of 
> druntime.  I don't see any notification mechanism :(

Same as Tango:

alias void delegate(Object) DEvent;
extern (C) void rt_attachDisposeEvent(Object h, DEvent e);
extern (C) void rt_detachDisposeEvent(Object h, DEvent e);

I should probably expose these in core.runtime.
April 20, 2009
Re: D2 weak references
Sean Kelly wrote:
> Jason House wrote:
>> Jason House wrote:
>>
>>> Tango's GC (and therefore
>>> druntime's GC) has an explicit notification mechanism.
>>
>> I guess I shouldn't have assumed that the features of tango became
>> part of druntime.  I don't see any notification mechanism :(
> 
> Same as Tango:
> 
> alias void delegate(Object) DEvent;
> extern (C) void rt_attachDisposeEvent(Object h, DEvent e);
> extern (C) void rt_detachDisposeEvent(Object h, DEvent e);
> 
> I should probably expose these in core.runtime.

How are these events dispatched?  I remember a discussion about race
conditions if these events were fired off after all threads had been
resumed; you could have the event zero out the weak ref AFTER something
else had dereferenced it.

If that's still a possibility, maybe you could add a case where if the
delegate's funcptr is null, it just assumes the context pointer points
to a void*.sizeof word and zeroes it out.  You could do that without
resuming threads.

 -- Daniel
April 20, 2009
Re: D2 weak references
Daniel Keep wrote:
> 
> Sean Kelly wrote:
>> Jason House wrote:
>>> Jason House wrote:
>>>
>>>> Tango's GC (and therefore
>>>> druntime's GC) has an explicit notification mechanism.
>>> I guess I shouldn't have assumed that the features of tango became
>>> part of druntime.  I don't see any notification mechanism :(
>> Same as Tango:
>>
>> alias void delegate(Object) DEvent;
>> extern (C) void rt_attachDisposeEvent(Object h, DEvent e);
>> extern (C) void rt_detachDisposeEvent(Object h, DEvent e);
>>
>> I should probably expose these in core.runtime.
> 
> How are these events dispatched?  I remember a discussion about race
> conditions if these events were fired off after all threads had been
> resumed; you could have the event zero out the weak ref AFTER something
> else had dereferenced it.

It's the responsibility of the person making the weak ref to make it 
thread-safe if that's a design consideration.  If the cleanup is 
performed before the threads are restarted and the weak ref (or 
signal/slot mechanism) uses mutexes then a deadlock is possible.  At 
least this approach provides the option for correct code to actually work.

> If that's still a possibility, maybe you could add a case where if the
> delegate's funcptr is null, it just assumes the context pointer points
> to a void*.sizeof word and zeroes it out.  You could do that without
> resuming threads.

This is all tied into the finalizer.  To do what you suggest the GC 
would have to perform two sweeps--one before resuming threads and one after.
April 20, 2009
Re: D2 weak references
Sean Kelly Wrote:

> Daniel Keep wrote:
> > 
> > Sean Kelly wrote:
> >> Jason House wrote:
> >>> Jason House wrote:
> >>>
> >>>> Tango's GC (and therefore
> >>>> druntime's GC) has an explicit notification mechanism.
> >>> I guess I shouldn't have assumed that the features of tango became
> >>> part of druntime.  I don't see any notification mechanism :(
> >> Same as Tango:
> >>
> >> alias void delegate(Object) DEvent;
> >> extern (C) void rt_attachDisposeEvent(Object h, DEvent e);
> >> extern (C) void rt_detachDisposeEvent(Object h, DEvent e);
> >>
> >> I should probably expose these in core.runtime.
> > 
> > How are these events dispatched?  I remember a discussion about race
> > conditions if these events were fired off after all threads had been
> > resumed; you could have the event zero out the weak ref AFTER something
> > else had dereferenced it.
> 
> It's the responsibility of the person making the weak ref to make it 
> thread-safe if that's a design consideration.  If the cleanup is 
> performed before the threads are restarted and the weak ref (or 
> signal/slot mechanism) uses mutexes then a deadlock is possible.  At 
> least this approach provides the option for correct code to actually work.
> 
> > If that's still a possibility, maybe you could add a case where if the
> > delegate's funcptr is null, it just assumes the context pointer points
> > to a void*.sizeof word and zeroes it out.  You could do that without
> > resuming threads.
> 
> This is all tied into the finalizer.  To do what you suggest the GC 
> would have to perform two sweeps--one before resuming threads and one after.

If I understand you correctly, a weak reference library needs to query the GC to see if an object is marked for collection. How do I do this?
April 20, 2009
Re: D2 weak references
Jason House Wrote:

> Sean Kelly Wrote:
> 
> > Daniel Keep wrote:
> > > 
> > > Sean Kelly wrote:
> > >> Jason House wrote:
> > >>> Jason House wrote:
> > >>>
> > >>>> Tango's GC (and therefore
> > >>>> druntime's GC) has an explicit notification mechanism.
> > >>> I guess I shouldn't have assumed that the features of tango became
> > >>> part of druntime.  I don't see any notification mechanism :(
> > >> Same as Tango:
> > >>
> > >> alias void delegate(Object) DEvent;
> > >> extern (C) void rt_attachDisposeEvent(Object h, DEvent e);
> > >> extern (C) void rt_detachDisposeEvent(Object h, DEvent e);
> > >>
> > >> I should probably expose these in core.runtime.
> > > 
> > > How are these events dispatched?  I remember a discussion about race
> > > conditions if these events were fired off after all threads had been
> > > resumed; you could have the event zero out the weak ref AFTER something
> > > else had dereferenced it.
> > 
> > It's the responsibility of the person making the weak ref to make it 
> > thread-safe if that's a design consideration.  If the cleanup is 
> > performed before the threads are restarted and the weak ref (or 
> > signal/slot mechanism) uses mutexes then a deadlock is possible.  At 
> > least this approach provides the option for correct code to actually work.
> > 
> > > If that's still a possibility, maybe you could add a case where if the
> > > delegate's funcptr is null, it just assumes the context pointer points
> > > to a void*.sizeof word and zeroes it out.  You could do that without
> > > resuming threads.
> > 
> > This is all tied into the finalizer.  To do what you suggest the GC 
> > would have to perform two sweeps--one before resuming threads and one after.
> 
> If I understand you correctly, a weak reference library needs to query the GC to see if an object is marked for collection. How do I do this?

Here's the implementation I'm currently thinking of:

weak ref construction:
Copy pointer into struct without casting
Call rt_attach_dispose_event

weak ref dereference:
copy value into temp var
** query gc to see if garbage **
If not garbage, Recopy value and return
If garbage set value = null

dispose of weakly referenced obj:
set value = null

The emphasized line is impossible according to the publicized gc interface. Given the delayed finalization, I don't see any way around it.
April 20, 2009
Re: D2 weak references
== Quote from Jason House (jason.james.house@gmail.com)'s article
> Sean Kelly Wrote:
> > Daniel Keep wrote:
> > >
> > > Sean Kelly wrote:
> > >> Jason House wrote:
> > >>> Jason House wrote:
> > >>>
> > >>>> Tango's GC (and therefore
> > >>>> druntime's GC) has an explicit notification mechanism.
> > >>> I guess I shouldn't have assumed that the features of tango became
> > >>> part of druntime.  I don't see any notification mechanism :(
> > >> Same as Tango:
> > >>
> > >> alias void delegate(Object) DEvent;
> > >> extern (C) void rt_attachDisposeEvent(Object h, DEvent e);
> > >> extern (C) void rt_detachDisposeEvent(Object h, DEvent e);
> > >>
> > >> I should probably expose these in core.runtime.
> > >
> > > How are these events dispatched?  I remember a discussion about race
> > > conditions if these events were fired off after all threads had been
> > > resumed; you could have the event zero out the weak ref AFTER something
> > > else had dereferenced it.
> >
> > It's the responsibility of the person making the weak ref to make it
> > thread-safe if that's a design consideration.  If the cleanup is
> > performed before the threads are restarted and the weak ref (or
> > signal/slot mechanism) uses mutexes then a deadlock is possible.  At
> > least this approach provides the option for correct code to actually work.
> >
> > > If that's still a possibility, maybe you could add a case where if the
> > > delegate's funcptr is null, it just assumes the context pointer points
> > > to a void*.sizeof word and zeroes it out.  You could do that without
> > > resuming threads.
> >
> > This is all tied into the finalizer.  To do what you suggest the GC
> > would have to perform two sweeps--one before resuming threads and one after.
> If I understand you correctly, a weak reference library needs to query the GC to see if an object is marked for collection.
How do I do this?

I think the WeakRef just has to either wrap the deref operation in a synchronized
block if it's intended for multithreaded use or perform an atomic read and update
of the reference.  From my understanding, the problem Daniel Keep describes above
is simply an issue of the referenced object vanishing at the same instant someone
tries to access it through the WeakRef.  The same thing could happen if someone
explicitly deletes the object instead of letting the GC clean it up.
« First   ‹ Prev
1 2 3 4 5
Top | Discussion index | About this forum | D home