DMD v100.00
OPTLINK v7.50B1
Win2k SP4

The following creates a general protection fault.  I've traced it down to phobos\internal\invariant.d (_d_invariant).

import std.stdio;

class Foo{
public void notify(){
writefln("notified");
}
}
class Bar{
public void delegate() event;
~this(){
event();
}
}
void main(){
Bar b = new Bar();
Foo f = new Foo();
b.event = &(f.notify);
}

It looks the instance of Foo is being collected before the instance of Bar, thus invalidating the delegate.

- Pragma

[[ EricAnderton at (don't spam me anymore) yahoo.com ]]

pragma wrote:

> DMD v100.00
> OPTLINK v7.50B1
> Win2k SP4
> 
> The following creates a general protection fault.  I've traced it down to phobos\internal\invariant.d (_d_invariant).
> 
> import std.stdio;
> 
> class Foo{
> public void notify(){
> writefln("notified");
> }
> }
> class Bar{
> public void delegate() event;
> ~this(){
> event();
> }
> }
> void main(){
> Bar b = new Bar();
> Foo f = new Foo();
> b.event = &(f.notify);
> }
> 
> It looks the instance of Foo is being collected before the instance of Bar, thus invalidating the delegate.
> 
> - Pragma
> 
> [[ EricAnderton at (don't spam me anymore) yahoo.com ]]

I think it will be quite hard getting this kind of thing to work. A destructor should never try to reference another object - since as you found out it might be a bogus reference. I'd like to see a sentance or two in the D destructor doc to avoid referencing other objects - or anything that belongs to the GC. See my post on the main newsgroup http://www.digitalmars.com/drn-bin/wwwnews?digitalmars.D/9023

-Ben

"Ben Hinkle" <bhinkle4@juno.com> wrote in message news:cggsr7$1i9o$1@digitaldaemon.com...
> pragma wrote:
>
> > DMD v100.00
> > OPTLINK v7.50B1
> > Win2k SP4
> >
> > The following creates a general protection fault.  I've traced it down
to
> > phobos\internal\invariant.d (_d_invariant).
> >
> > import std.stdio;
> >
> > class Foo{
> > public void notify(){
> > writefln("notified");
> > }
> > }
> > class Bar{
> > public void delegate() event;
> > ~this(){
> > event();
> > }
> > }
> > void main(){
> > Bar b = new Bar();
> > Foo f = new Foo();
> > b.event = &(f.notify);
> > }
> >
> > It looks the instance of Foo is being collected before the instance of Bar, thus invalidating the delegate.
> >
> > - Pragma
> >
> > [[ EricAnderton at (don't spam me anymore) yahoo.com ]]
>
> I think it will be quite hard getting this kind of thing to work. A destructor should never try to reference another object - since as you found out it might be a bogus reference. I'd like to see a sentance or two in the D destructor doc to avoid referencing other objects - or anything that belongs to the GC. See my post on the main newsgroup http://www.digitalmars.com/drn-bin/wwwnews?digitalmars.D/9023
>
> -Ben


Yes, that would be helpful. But it's not /always/ practical to avoid touching other objects during destruction, though perhaps 'desirable'.

And it's not an obvious error either, since one could hardly be blamed for thinking the delegate holds a reference to its enclosing class instance (it's a delegate; not a function). If so, then the instance should still be live as long as the delegate is referenced ... yes? Apparently that's not how it works. Sure looks like a bug.

Do you know what the delegate holds a reference to, Ben?

antiAlias wrote:

> "Ben Hinkle" <bhinkle4@juno.com> wrote in message news:cggsr7$1i9o$1@digitaldaemon.com...
>> pragma wrote:
>>
>> > DMD v100.00
>> > OPTLINK v7.50B1
>> > Win2k SP4
>> >
>> > The following creates a general protection fault.  I've traced it down
> to
>> > phobos\internal\invariant.d (_d_invariant).
>> >
>> > import std.stdio;
>> >
>> > class Foo{
>> > public void notify(){
>> > writefln("notified");
>> > }
>> > }
>> > class Bar{
>> > public void delegate() event;
>> > ~this(){
>> > event();
>> > }
>> > }
>> > void main(){
>> > Bar b = new Bar();
>> > Foo f = new Foo();
>> > b.event = &(f.notify);
>> > }
>> >
>> > It looks the instance of Foo is being collected before the instance of Bar, thus invalidating the delegate.
>> >
>> > - Pragma
>> >
>> > [[ EricAnderton at (don't spam me anymore) yahoo.com ]]
>>
>> I think it will be quite hard getting this kind of thing to work. A destructor should never try to reference another object - since as you found out it might be a bogus reference. I'd like to see a sentance or two in the D destructor doc to avoid referencing other objects - or anything that belongs to the GC. See my post on the main newsgroup http://www.digitalmars.com/drn-bin/wwwnews?digitalmars.D/9023
>>
>> -Ben
> 
> 
> Yes, that would be helpful. But it's not /always/ practical to avoid touching other objects during destruction, though perhaps 'desirable'.

well... just as long as none of those objects need collecting at program exit. Otherwise it's a game of chance which one gets GC'ed first.

> And it's not an obvious error either, since one could hardly be blamed for thinking the delegate holds a reference to its enclosing class instance (it's a delegate; not a function). If so, then the instance should still be live as long as the delegate is referenced ... yes? Apparently that's not how it works. Sure looks like a bug.

True - it should be considered a bug. Java (I think) allows other objects to be referenced in finalizers. I don't know the details, though, since finalizers are frowned upon I've never really gotten into it.

> Do you know what the delegate holds a reference to, Ben?

in the case above, f.

>> 
>> Yes, that would be helpful. But it's not /always/ practical to avoid touching other objects during destruction, though perhaps 'desirable'.
>
>well... just as long as none of those objects need collecting at program exit. Otherwise it's a game of chance which one gets GC'ed first.

It gets even more frustrating than that.  Similar to what you ran into with streams, I have a .dll Library class that has delegate array (event) that functions as a multicast callback to other interested parties when it is unloaded.  This is a much needed runtime behavior, and is especially useful to fire this event when the library is unloaded during finalization.

>
>> And it's not an obvious error either, since one could hardly be blamed for thinking the delegate holds a reference to its enclosing class instance (it's a delegate; not a function). If so, then the instance should still be live as long as the delegate is referenced ... yes? Apparently that's not how it works. Sure looks like a bug.

Yep, that's basically where I was when I filed the report.  The delegate has an outstanding reference, yet it's object was destroyed.

>
>True - it should be considered a bug. Java (I think) allows other objects to be referenced in finalizers. I don't know the details, though, since finalizers are frowned upon I've never really gotten into it.

It creates a lot of problems, especially when you're relying on the GC to perform collections during the execution of the program.  Its nice to know when something is completely destroyed, and releases its resources:

# class Foo{
#   int value;
#   ~this(){  Logger.getLogger.put("Foo Destroyed (%d).",value); }
# }

Something like this becomes a problem with the current revision of D: it works great during runtime and then (very likely) fails miserably once you exit the program.  The problem also gets worse when expecting to be able to free a resource on termination, but can't since some random destructor causes a GPF.

Honestly, my expectation was that during the final collection (before program exit) all the finalizers would be run *first* before any memory is freed.

# // pseudocode for GC termination
# // (multithreading issues not withstanding)
# alias void* function(void*) Finalizer;
# void*[] gc_objects;
# Finalizer[void*] gc_finalizers;
#
# void shutdown(){
#   while(gc_finalizers.length > 0){
#     foreach(void* obj,Finalizer fin; gc_finalizers){
#       fin(obj);
#     }
#   }
#   foreach(void* obj; gc_objects) free(obj);
# }

I get the impression that it's doing something like this instead:

# void shutdown(){
#   foreach(void* obj,Finalizer fin; gc_finalizers){
#     fin(obj);
#     free(obj);
#   }
# }

- Pragma
[[ EricAnderton at (pounding away on code) yahoo.com ]]

> Honestly, my expectation was that during the final collection (before
program
> exit) all the finalizers would be run *first* before any memory is freed.

That makes sense - though it would cause two loops over the garbage instead of one. Also no-one has brought up the cases when garbage can be brought back to life. That's one reason why Java's finalizers are frowned upon - they can bring back dead references and generally cause the GC to jump through hoops to make sure it isn't cleaning up live data. What if one of the destructors assigns a reference to some garbage to a static variable and thus brings it back to life? The GC needs to check all the program's references after the destructors are run to make sure no garbage was brought to life. That would be a nasty performance hit.

It occured to me that this problem of having destructors reference other objects isn't limited to program exit. It can happen any time there is a collection. The reason it shows up at exit often is that D *always* collects at exit and our programs that have the problem don't run long enough to force collections during normal run-time (or the object being referenced isn't garbage during runtime).

Or we can keep the current architecture and just document the fact that destructors can't reference any other GC managed resources (or anything that has managed GC resources). Nasty but it lets the GC do whatever it wants.

In article <cgia4c$273m$1@digitaldaemon.com>, Ben Hinkle says...
>
>
>> Honestly, my expectation was that during the final collection (before
>program
>> exit) all the finalizers would be run *first* before any memory is freed.
>
>That makes sense - though it would cause two loops over the garbage instead of one. Also no-one has brought up the cases when garbage can be brought back to life. That's one reason why Java's finalizers are frowned upon - they can bring back dead references and generally cause the GC to jump through hoops to make sure it isn't cleaning up live data.

I see what you mean.  Granted, there's nothing keeping a destructor from creating *more* data on the heap in a growing spiral of allocations.  But I really don't see that as any more a problem as infinite recursion or mutually-referenced functions (like in the invariant 'bug' menioned earlier in this NG).  But I strongly feel that reducing destructors to playing with only scalar types (as we've discovered that object references are all "weak" in the present D implementation during the final collect) reduces their capability to near uselessness.

I also just noticed that my pseudocode on the previous post was also a little flawed, and hence misleading.  If one were to loop over a /copy/ of the current set of finalizers, after clearing out the old list, the outer loop stands a good chance of actually terminating.

>What if one of
>the destructors assigns a reference to some garbage to a static variable and
>thus brings it back to life? The GC needs to check all the program's
>references after the destructors are run to make sure no garbage was brought
>to life. That would be a nasty performance hit.

Well, "ressurecting" an object properly would require the GC to register that object's finalizer, in which case it looks like a normal object.  That /would/ thwart the present behavior as well as my informal proposal in my previous post.

Personally, I'd rather have all my objects a /chance/ of cleaning up propertly on program exit rather than have unpredictable behavior inside of destructors. I know what I'm asking for is deterministic destruction of objects, in all cases, which has been something of a holy war on the D NG.

I think its possible to finialize in the proposed method and abort to a full free of the heap if the loop runs for too long, or if there's no change in the number of finalizers for a number of iterations.  At least that way, you have given the program a fighting chance of being well behaved on exit without bending over backwards in your code.

Another way to go would be to allow the GC to track a set of privileged references that are guaranteed to be finalized before a dealloc on exit.  These would be explicitly set in the client code and would be understood to delay program termination by ever so much.

At the very least, it'd be nice to have a way to check if a reference is still valid, and hasn't been collected.  That way one could treat object references as 'weak' if you're using code inside of a destructor.

-Pragma
[[ EricAnderton at (let my objects go) yahoo.com ]]

If this behavior is a trade-off between correct program-execution and "performance" during program termination, then there's something very seriously wrong with the foundations of this language.

As Pragma points out, if you can't reliably release resources during a destructor then their value is diminished to the level of worthless. Actually better not to have destructors at all. Now where would that leave us?

What's the point of tracking references and dependencies if they're then simply ignored during Object destruction? Utter lunacy.


"pragma" <pragma_member@pathlink.com> wrote in message news:cgif07$29ev$1@digitaldaemon.com...
> In article <cgia4c$273m$1@digitaldaemon.com>, Ben Hinkle says...
> >
> >
> >> Honestly, my expectation was that during the final collection (before
> >program
> >> exit) all the finalizers would be run *first* before any memory is
freed.
> >
> >That makes sense - though it would cause two loops over the garbage
instead
> >of one. Also no-one has brought up the cases when garbage can be brought back to life. That's one reason why Java's finalizers are frowned upon - they can bring back dead references and generally cause the GC to jump through hoops to make sure it isn't cleaning up live data.
>
> I see what you mean.  Granted, there's nothing keeping a destructor from creating *more* data on the heap in a growing spiral of allocations.  But
I
> really don't see that as any more a problem as infinite recursion or mutually-referenced functions (like in the invariant 'bug' menioned
earlier in
> this NG).  But I strongly feel that reducing destructors to playing with
only
> scalar types (as we've discovered that object references are all "weak" in
the
> present D implementation during the final collect) reduces their
capability to
> near uselessness.
>
> I also just noticed that my pseudocode on the previous post was also a
little
> flawed, and hence misleading.  If one were to loop over a /copy/ of the
current
> set of finalizers, after clearing out the old list, the outer loop stands
a good
> chance of actually terminating.
>
> >What if one of
> >the destructors assigns a reference to some garbage to a static variable
and
> >thus brings it back to life? The GC needs to check all the program's references after the destructors are run to make sure no garbage was
brought
> >to life. That would be a nasty performance hit.
>
> Well, "ressurecting" an object properly would require the GC to register
that
> object's finalizer, in which case it looks like a normal object.  That
/would/
> thwart the present behavior as well as my informal proposal in my previous
post.
>
> Personally, I'd rather have all my objects a /chance/ of cleaning up
propertly
> on program exit rather than have unpredictable behavior inside of
destructors.
> I know what I'm asking for is deterministic destruction of objects, in all cases, which has been something of a holy war on the D NG.
>
> I think its possible to finialize in the proposed method and abort to a
full
> free of the heap if the loop runs for too long, or if there's no change in
the
> number of finalizers for a number of iterations.  At least that way, you
have
> given the program a fighting chance of being well behaved on exit without bending over backwards in your code.
>
> Another way to go would be to allow the GC to track a set of privileged references that are guaranteed to be finalized before a dealloc on exit.
These
> would be explicitly set in the client code and would be understood to
delay
> program termination by ever so much.
>
> At the very least, it'd be nice to have a way to check if a reference is
still
> valid, and hasn't been collected.  That way one could treat object
references as
> 'weak' if you're using code inside of a destructor.
>
> -Pragma
> [[ EricAnderton at (let my objects go) yahoo.com ]]

In article <cgiiff$2bio$1@digitaldaemon.com>, antiAlias says...
>
>If this behavior is a trade-off between correct program-execution and "performance" during program termination, then there's something very seriously wrong with the foundations of this language.
>
>As Pragma points out, if you can't reliably release resources during a destructor then their value is diminished to the level of worthless. Actually better not to have destructors at all. Now where would that leave us?

I'll stay off my soapbox on this one.  However, I'll be happy to provide any support to the cause should there be a weighing-in of pros and cons.

All I want is to write good software in D, and through that have D succeed.

>What's the point of tracking references and dependencies if they're then simply ignored during Object destruction? Utter lunacy.

Well let's not jump to conclusions... I would like to contend that this seems like a mistake in implementation and is still likely a "bug" until proven otherwise. :)

(Now, if it's been said before that the current behavior is quite deliberate, then I'll stand corrected.)

-Pragma
[[ EricAnderton at (making D better) yahoo.com ]]

> Well let's not jump to conclusions... I would like to contend that this
seems
> like a mistake in implementation and is still likely a "bug" until proven otherwise. :)

Fair point Eric. Having come-a-cropper over so many fundamental "no ... surely not!" type issues, I'm afraid my resilience against more of same has been frayed to tatters. Part of the whole "library development issues: what to do?" thread.

> All I want is to write good software in D, and through that have D
succeed.

Hear hear. There's no other reason why I've spent four months 80-hours a week on Mango. Unfortunately, it sometimes feels as though the language, and /particularly/ the maturation 'process' behind it, is actually working against that goal (note that the "what to do?" thread all but disappeared down the black-hole). More's the pity.