Sean Kelly wrote:
> == Quote from Jason House (jason.james.house@gmail.com)'s article
>> Sean Kelly wrote:
>>> == Quote from Jarrett Billingsley (kb3ctd2@yahoo.com)'s article
>>>> "Sean Kelly" <sean@invisibleduck.org> wrote in message
>>>> news:fttd02$1mp1$1@digitalmars.com...
>>>>> Are these tables AAs?  And how do the weak references store their
>>>>> reference?
>>>>> The type information provided to AAs is a bit weak, and so some
>>>>> non-pointer
>>>>> data is currently still scanned by the GC.  I'm going to improve this a
>>>>> bit in the
>>>>> next Tango release, but it still won't be perfect (because the compiler
>>>>> doesn't
>>>>> provide enough info for it to be).
>>>> In my tests I've just been doing
>>>> auto weakref = new WeakRef!(A)(new A);
>>>> and then checking that after doing god-knows-what to try to get it to
>>>> collect that instance of A, so it's not even that I'm doing anything that
>>>> crazy.
>>> Just for kicks, try querying the GC to determine whether the GC thinks
>>> weakref contains pointers.
>>>
>>>
>>> Sean
>> How would one do that?
> 
> (from memory)
> 
>     if( GC.getAttr( weakref ) & GC.BlkAttr.NO_SCAN )
>         printf( "The GC will not scan weakref\n" );

Is that how it's supposed to work?  I mean WeakRef is basically

class WeakRef { size_t hidden_pointer; }

Does that mean Tango shouldn't be trying to scan instances of it?  I just ask because I was thinking the GC *would* scan the WeakRef's memory just not find anything that looks like a pointer there.


Also (for Jason, in particular) it occurs to me that this code:

  assert(wa.ptr !is null);
  collect();
  assert(wa.ptr is null);

Has to store wa.ptr somewhere (register or stack) when you ask to check if it is null or not.  So that act of checking itself will likely make a reference that prevents the following collect() from cleaning it up.

--bb

"Sean Kelly" <sean@invisibleduck.org> wrote in message news:ftubv9$kph$1@digitalmars.com...
> == Quote from Jarrett Billingsley (kb3ctd2@yahoo.com)'s article
>> "Sean Kelly" <sean@invisibleduck.org> wrote in message news:fttd02$1mp1$1@digitalmars.com...
>> >
>> > Are these tables AAs?  And how do the weak references store their
>> > reference?
>> > The type information provided to AAs is a bit weak, and so some
>> > non-pointer
>> > data is currently still scanned by the GC.  I'm going to improve this a
>> > bit in the
>> > next Tango release, but it still won't be perfect (because the compiler
>> > doesn't
>> > provide enough info for it to be).
>> In my tests I've just been doing
>> auto weakref = new WeakRef!(A)(new A);
>> and then checking that after doing god-knows-what to try to get it to
>> collect that instance of A, so it's not even that I'm doing anything that
>> crazy.
>
> Just for kicks, try querying the GC to determine whether the GC thinks weakref contains pointers.

Already tried it.  The GC sees the weakref instance's memory as NO_SCAN and FINALIZE, both correct.

Bill Baxter Wrote:
> >     if( GC.getAttr( weakref ) & GC.BlkAttr.NO_SCAN )
> >         printf( "The GC will not scan weakref\n" );
> 
> Is that how it's supposed to work?  I mean WeakRef is basically
> 
> class WeakRef { size_t hidden_pointer; }
> 
> Does that mean Tango shouldn't be trying to scan instances of it?  I just ask because I was thinking the GC *would* scan the WeakRef's memory just not find anything that looks like a pointer there.

That's the intent.  It keeps a reference without the GC detecting it.  The only extra wrinkle to the implementation is that it needs to null out its internal pointer when the object gets collected.


> Also (for Jason, in particular) it occurs to me that this code:
> 
>    assert(wa.ptr !is null);
>    collect();
>    assert(wa.ptr is null);
> 
> Has to store wa.ptr somewhere (register or stack) when you ask to check if it is null or not.  So that act of checking itself will likely make a reference that prevents the following collect() from cleaning it up.

That's true that that might cause the last assert to fail, oops.  I don't know what the correct way to clear everything out is (including from the preceding function call).  Any more tips on making a more correct unit test are appreciated.

Jarret was having issues with non-collection (ever) when using weak ref's and I had written this test to try and exercise it (his problem is windows + tango + dmd).  An assert failure doesn't mean it won't ever collect but will narrow the search down and allow closer examination... and the crash on linux+tango+gdc is definitely not correct behavior.

== Quote from Jarrett Billingsley (kb3ctd2@yahoo.com)'s article
> "Sean Kelly" <sean@invisibleduck.org> wrote in message news:ftubv9$kph$1@digitalmars.com...
> > == Quote from Jarrett Billingsley (kb3ctd2@yahoo.com)'s article
> >> "Sean Kelly" <sean@invisibleduck.org> wrote in message news:fttd02$1mp1$1@digitalmars.com...
> >> >
> >> > Are these tables AAs?  And how do the weak references store their
> >> > reference?
> >> > The type information provided to AAs is a bit weak, and so some
> >> > non-pointer
> >> > data is currently still scanned by the GC.  I'm going to improve this a
> >> > bit in the
> >> > next Tango release, but it still won't be perfect (because the compiler
> >> > doesn't
> >> > provide enough info for it to be).
> >> In my tests I've just been doing
> >> auto weakref = new WeakRef!(A)(new A);
> >> and then checking that after doing god-knows-what to try to get it to
> >> collect that instance of A, so it's not even that I'm doing anything that
> >> crazy.
> >
> > Just for kicks, try querying the GC to determine whether the GC thinks weakref contains pointers.
> Already tried it.  The GC sees the weakref instance's memory as NO_SCAN and FINALIZE, both correct.

Huh.  Afraid I'm at a loss as to what's going on then.  Even stranger that it works on linux but not Win32, since the code is basically the same for both. I suppose I could add some profiling code to try and figure out where the lingering reference is, assuming there is one.  But not sure when I'll have the time to do this.


Sean

== Quote from Jason House (jason.james.house@gmail.com)'s article
> Bill Baxter Wrote:
> > >     if( GC.getAttr( weakref ) & GC.BlkAttr.NO_SCAN )
> > >         printf( "The GC will not scan weakref\n" );
> >
> > Is that how it's supposed to work?  I mean WeakRef is basically
> >
> > class WeakRef { size_t hidden_pointer; }
> >
> > Does that mean Tango shouldn't be trying to scan instances of it?  I just ask because I was thinking the GC *would* scan the WeakRef's memory just not find anything that looks like a pointer there.
> That's the intent.  It keeps a reference without the GC detecting it.  The only extra wrinkle to the implementation is that it needs to
null out its internal pointer when the object gets collected.
> > Also (for Jason, in particular) it occurs to me that this code:
> >
> >    assert(wa.ptr !is null);
> >    collect();
> >    assert(wa.ptr is null);
> >
> > Has to store wa.ptr somewhere (register or stack) when you ask to check if it is null or not.  So that act of checking itself will likely make a reference that prevents the following collect() from cleaning it up.
> That's true that that might cause the last assert to fail, oops.  I don't know what the correct way to clear everything out is (including
from the preceding function call).  Any more tips on making a more correct unit test are appreciated.

The easiest thing I can think of would be to do an access of a dummy weakptr after each access of the real weakptr:

auto dummy = new WeakPtr!(void);
assert(wa.ptr !is null);
assert(dummy.ptr is null);
collect();
assert(wa.ptr is null);

Since the dummy op is identical to the real op, it should hopefully hit the same registers.  Best way to be sure would be to test with -O turned off.

> Jarret was having issues with non-collection (ever) when using weak ref's and I had written this test to try and exercise it (his problem
is windows + tango + dmd).  An assert failure doesn't mean it won't ever collect but will narrow the search down and allow closer
examination... and the crash on linux+tango+gdc is definitely not correct behavior.

Thanks for any investigation you do for this.  If you can find a bug in Tango, I'd be happy to fix it, but I'm too busy to do the research myself at the moment.


Sean

Sean Kelly Wrote:
> Thanks for any investigation you do for this.  If you can find a bug in Tango, I'd be happy to fix it, but I'm too busy to do the research myself at the moment.

I have a vested interest in it... I want my code to work ;)

Just to be clear, what is enough to proclaim stuff as a bug in Tango?

For the failed assert: prior to collect, show a lack of lingering references through disassembly?

For a runtime crash: Isn't any crash in core libraries used by Tango a Tango bug?

For whatever I can prove to be a Tango bug, I'll submit a ticket.

Jason House wrote:

> Sean Kelly Wrote:
>> Thanks for any investigation you do for this.  If you can find a bug in Tango, I'd be happy to fix it, but I'm too busy to do the research myself at the moment.
> 
> I have a vested interest in it... I want my code to work ;)
> 
> Just to be clear, what is enough to proclaim stuff as a bug in Tango?
> 
> For the failed assert: prior to collect, show a lack of lingering references through disassembly?
> 
> For a runtime crash: Isn't any crash in core libraries used by Tango a Tango bug?
> 
> For whatever I can prove to be a Tango bug, I'll submit a ticket.

You can create a Tango ticket for anything directly related to Tango that doesn't work. In this particular case, a compiler bug is at least as likely (due to what is working with what where). We have many tickets for problems that have been verified to be toolchain problems, as it is easier for us to track how it affect us that way.

-- 
Lars Ivar Igesund
blog at http://larsivi.net
DSource, #d.tango & #D: larsivi
Dancing the Tango

== Quote from Jason House (jason.james.house@gmail.com)'s article
> Sean Kelly Wrote:
> > Thanks for any investigation you do for this.  If you can find a bug in Tango, I'd be happy to fix it, but I'm too busy to do the research myself at the moment.
> I have a vested interest in it... I want my code to work ;)
> Just to be clear, what is enough to proclaim stuff as a bug in Tango?

Ideally, any bug report will contain a simple repro and/or mention of where the actual problem is in Tango.  But we see ambiguous bug reports or things that, after investigation, turn out to be compiler problems and the like.  But the easier you can make things on our end, the faster the problem is likely to be fixed :-)

> For the failed assert: prior to collect, show a lack of lingering references through disassembly?

What I'd do to diagnose this is rebuild the Tango runtime with debug reporting turned on for the GC, possibly adding something to print the address of all blocks being collected (if that doesn't exist already).  We know that weakptr data isn't being collected on Win32... the question is why.


> For a runtime crash: Isn't any crash in core libraries used by Tango a Tango bug?

Not necessarily.  We've seen a bunch of crashes that GDB reported were in the runtime that actually turned out to be compiler bugs in GDC, or in some cases errors in the user program that ended up surfacing as runtime code exploding.  But in theory you're right.

> For whatever I can prove to be a Tango bug, I'll submit a ticket.

Thank you :-)


Sean

Bill Baxter schrieb:
> of weak refs.  In addition to null pointers polluting the table, there's also just the basic overhead of needing to create an extra object for each entry.  Ideally there would be WeakArray, WeakSet, WeakHeap... etc.  I think I've seen such things in some language.  Python maybe?
> 
> --bb

weakHashMap, weakTreeMap ???

implemented in Java.... ? OpenJDK/ GNU Classpath
Well, Frank is looking for a D weakHashMap implementation. Due to the fact that I am actually doing plombering, plastering, electric, establish tons of cement concrete  ... I am not able to offer a solution. (Beeing allready happy that I am  able to write a msg)

Nevertheless I did like to suggest to use Bill B's weakRef!  as  inner adaptor class of WeakHashMap.  (pretty much the same as in GNU classpath)  as static array of WeakRef objects. Further the dCollection : see dSource / collection classes are <IMO> fitting better than Tango Collection.


Bjoern

Bjoern wrote:
> Bill Baxter schrieb:
>> of weak refs.  In addition to null pointers polluting the table, there's also just the basic overhead of needing to create an extra object for each entry.  Ideally there would be WeakArray, WeakSet, WeakHeap... etc.  I think I've seen such things in some language.  Python maybe?
>>
>> --bb
> 
> weakHashMap, weakTreeMap ???
> 
> implemented in Java.... ? OpenJDK/ GNU Classpath
> Well, Frank is looking for a D weakHashMap implementation. Due to the fact that I am actually doing plombering, plastering, electric, establish tons of cement concrete  ... I am not able to offer a solution. (Beeing allready happy that I am  able to write a msg)
> 
> Nevertheless I did like to suggest to use Bill B's weakRef!  as  inner adaptor class of WeakHashMap.  (pretty much the same as in GNU classpath)  as static array of WeakRef objects. Further the dCollection : see dSource / collection classes are <IMO> fitting better than Tango Collection.

Using the wrappers might have too much overhead. But perhaps MinTL with an allocator using malloc would do the trick?


-- 
Tomasz Stachowiak
http://h3.team0xf.com/
h3/h3r3tic on #D freenode