If you remove the allocator it works ok. The garbage collector does not scan
the C heap for objects to collect so an instance of class A (when it has the
custom allocator) will never get collected and ones has to call "delete a"
explicitly.
I think the point of custom allocators/deallocators is to allow classes to
avoid the GC memory pool. If you want GC then use the default GC allocator
or have your custom allocator allocate from the GC pool and not the C heap.

"Arcane Jill" <Arcane_member@pathlink.com> wrote in message news:ca6rl9$22p9$1@digitaldaemon.com...
> Destruction - Definitive test results.
>
> Faced with uncertainty, I did what any scientist would do. I did an
experiment.
> Here's the source code:
>
> //======== BEGIN =========
> >   import std.c.stdio;
> >   import std.c.stdlib;
> >   import std.gc;
> >
> >   FILE * f;
> >
> >   class A
> >   {
> >       new(uint n)
> >       {
> >           f = fopen("C:\\gctest.txt","wt");
> >           fprintf(f,"Allocator called\n");
> >           return malloc(n);
> >       }
> >
> >       this()
> >       {
> >           fprintf(f,"Constructor called\n");
> >       }
> >
> >       ~this()
> >       {
> >           fprintf(f,"Destructor called\n");
> >       }
> >
> >       delete(void* p)
> >       {
> >           fprintf(f,"Deallocator called\n");
> >           fclose(f);
> >           free(p);
> >       }
> >
> >       int t;
> >   }
> >
> >   int main()
> >   {
> >       A a = new A();
> >       a = null;
> >       fullCollect();
> >       return 0;
> >   }
> //========= END ==========
>
> And here's the contents of the file after the program is run:
>
> //======= BEGIN ========
> Allocator called
> Constructor called
> //======== END =========
>
> Conclusions:
> (1) The destructor was not called.
> (2) The class deallocator was not called.
>
> Observe that the garbage collector WAS EXPLICITLY INVOKED, and that the
variable
> 'a' was not reachable at the time the gc was run. This "destruction" is
not
> merely lazy, it seems to be actually on vacation.
>
> Now, there are two possibilities. Either
> (a) This is correct behavior
> (b) This is a bug in the compiler
>
> If it's (a) I will need to think very hard to try to find a workaround. If
I
> can't find one I will have to abandon my D project and write the thing in
C++.
> If it's (b) then it's critical to me that the bug be fixed asap, because,
again,
> this has to work, or D becomes useless for my purposes.
>
>
>

In article <ca7it0$5b6$1@digitaldaemon.com>, Ben Hinkle says...
>
>If you remove the allocator it works ok. The garbage collector does not scan the C heap for objects to collect so an instance of class A (when it has the custom allocator) will never get collected and ones has to call "delete a" explicitly.

Thank you very much. That does work, and that's excellent.

I guess I just hadn't understood the relationship between custom allocators and the GC. I'm really glad you pointed that out. Thanks again.



>I think the point of custom allocators/deallocators is to allow classes to avoid the GC memory pool. If you want GC then use the default GC allocator or have your custom allocator allocate from the GC pool and not the C heap.

Yeah, I got that now.

It would seem a minor deficiency in the D language that a custom allocator can only realistically be used with an auto class (unless it doesn't need a destructor). But that's not a problem for me, so I'm not going to worry about it.

I think the manual should probably mention this, but you've solved my problem so I'm happy now. Thanks again,

Jill

When you overload operators new and delete for your class, then YOU are managing the memory for that class, not the GC. The GC only manages memory allocated by new when new is NOT overloaded.

Overloading new and delete is saying "I want to manage the memory myself; GC keep your hands off of it."


"Arcane Jill" <Arcane_member@pathlink.com> wrote in message news:ca6rl9$22p9$1@digitaldaemon.com...
> Destruction - Definitive test results.
>
> Faced with uncertainty, I did what any scientist would do. I did an
experiment.
> Here's the source code:
>
> //======== BEGIN =========
> >   import std.c.stdio;
> >   import std.c.stdlib;
> >   import std.gc;
> >
> >   FILE * f;
> >
> >   class A
> >   {
> >       new(uint n)
> >       {
> >           f = fopen("C:\\gctest.txt","wt");
> >           fprintf(f,"Allocator called\n");
> >           return malloc(n);
> >       }
> >
> >       this()
> >       {
> >           fprintf(f,"Constructor called\n");
> >       }
> >
> >       ~this()
> >       {
> >           fprintf(f,"Destructor called\n");
> >       }
> >
> >       delete(void* p)
> >       {
> >           fprintf(f,"Deallocator called\n");
> >           fclose(f);
> >           free(p);
> >       }
> >
> >       int t;
> >   }
> >
> >   int main()
> >   {
> >       A a = new A();
> >       a = null;
> >       fullCollect();
> >       return 0;
> >   }
> //========= END ==========
>
> And here's the contents of the file after the program is run:
>
> //======= BEGIN ========
> Allocator called
> Constructor called
> //======== END =========
>
> Conclusions:
> (1) The destructor was not called.
> (2) The class deallocator was not called.
>
> Observe that the garbage collector WAS EXPLICITLY INVOKED, and that the
variable
> 'a' was not reachable at the time the gc was run. This "destruction" is
not
> merely lazy, it seems to be actually on vacation.
>
> Now, there are two possibilities. Either
> (a) This is correct behavior
> (b) This is a bug in the compiler
>
> If it's (a) I will need to think very hard to try to find a workaround. If
I
> can't find one I will have to abandon my D project and write the thing in
C++.
> If it's (b) then it's critical to me that the bug be fixed asap, because,
again,
> this has to work, or D becomes useless for my purposes.
>
>
>

In article <ca7mkb$b4b$1@digitaldaemon.com>, Walter says...
>
>When you overload operators new and delete for your class, then YOU are managing the memory for that class, not the GC. The GC only manages memory allocated by new when new is NOT overloaded.
>
>Overloading new and delete is saying "I want to manage the memory myself; GC keep your hands off of it."

Gotcha. Ben explained that to me earlier. Thanks for being so helpful. :} Jill

In article <ca6rl9$22p9$1@digitaldaemon.com>, Arcane Jill says...
>
>Destruction - Definitive test results.
>
>Faced with uncertainty, I did what any scientist would do. I did an experiment.
> [snip]
>Conclusions:
>(1) The destructor was not called.
>(2) The class deallocator was not called.

Jill,
One thing ocurred to me with your test case here, that may have some hidden
dangers lurking in it.  I'm sure you've already thought of something like this,
but I thought I'd at least share it here anyway for completeness' sake (as well
as for my own education, since I'm drawing a huge blank here).

If this class were to hold a reference to a gc'd object, would that reference be scanned by the collector even though this class is allocated "manually"?

I know next to nothing about security programming, so I haven't a clue if allowing the GC to scan your objects is even remotely acceptable.  But since this class lives outside the GC, knowing when and how a GC'd object reference is still valid keeps coming to my mind as a rather large hurdle.

So I guess the big question here is:
Is any product of a user-defined new() added to the GC automatically for root
scanning, or should it be added with a call to std.gc.AddRange()?

- Eric

In article <ca7ohq$ee3$1@digitaldaemon.com>, EricAnderton at yahoo dot com says...

>So I guess the big question here is:
>Is any product of a user-defined new() added to the GC automatically for root
>scanning, or should it be added with a call to std.gc.AddRange()?

If you had have asked me that question yesterday I would not have known the answer, but now, thanks to Ben and Walter...

Yup. You have to call addRange() if it's going to contain referenced to GC'ed
stuff. (If I've understood correctly).

Arcane Jill

"Arcane Jill" <Arcane_member@pathlink.com> wrote in message news:ca7pjd$g0k$1@digitaldaemon.com...
> Yup. You have to call addRange() if it's going to contain referenced to
GC'ed
> stuff. (If I've understood correctly).

Call addRange() for any memory that will contain references to GC'd objects that is not already in the GC pool, or the stack, registers, or static data.

In article <ca7la5$90l$1@digitaldaemon.com>, Arcane Jill says...
>
>In article <ca7it0$5b6$1@digitaldaemon.com>, Ben Hinkle says...
>>
>>If you remove the allocator it works ok. The garbage collector does not scan the C heap for objects to collect so an instance of class A (when it has the custom allocator) will never get collected and ones has to call "delete a" explicitly.
>
>Thank you very much. That does work, and that's excellent.
>
>I guess I just hadn't understood the relationship between custom allocators and the GC. I'm really glad you pointed that out. Thanks again.
>
>
>
>>I think the point of custom allocators/deallocators is to allow classes to avoid the GC memory pool. If you want GC then use the default GC allocator or have your custom allocator allocate from the GC pool and not the C heap.
>
>Yeah, I got that now.
>
>It would seem a minor deficiency in the D language that a custom allocator can only realistically be used with an auto class (unless it doesn't need a destructor). But that's not a problem for me, so I'm not going to worry about it.
>
>I think the manual should probably mention this, but you've solved my problem so I'm happy now. Thanks again,
>
>Jill

I would caution you that GC will still miss objects where a stack pointer happens to point to the object.  If you need absolute guarantees (as it seems you do), a more forceful approach may still be required.  Maybe the object could add itself to some kind of global "set" when constructed, and remove itself when destructed.  At program end, the set would need to be iterated and destructed.

If the objects should be deleted by the GC during the program run, the pointer would need to be masked somehow.  Also, it would be a mistake to add and remove the object itself - the pointer must be a pointer to the exact allocation you are concerned with.  If object A points to object B, the destructor for A cannot be sure that B has not already be collected.  So the global set would need a list of B pointers, where B is a struct or array.

Kevin

In article <cake5h$1629$1@digitaldaemon.com>, Kevin Bealer says...
>
>In article <ca7la5$90l$1@digitaldaemon.com>, Arcane Jill says...
>>
>>In article <ca7it0$5b6$1@digitaldaemon.com>, Ben Hinkle says...
>>>
>>>If you remove the allocator it works ok. The garbage collector does not scan the C heap for objects to collect so an instance of class A (when it has the custom allocator) will never get collected and ones has to call "delete a" explicitly.
>>
>>Thank you very much. That does work, and that's excellent.
>>
>>I guess I just hadn't understood the relationship between custom allocators and the GC. I'm really glad you pointed that out. Thanks again.
>>
>>
>>
>>>I think the point of custom allocators/deallocators is to allow classes to avoid the GC memory pool. If you want GC then use the default GC allocator or have your custom allocator allocate from the GC pool and not the C heap.
>>
>>Yeah, I got that now.
>>
>>It would seem a minor deficiency in the D language that a custom allocator can only realistically be used with an auto class (unless it doesn't need a destructor). But that's not a problem for me, so I'm not going to worry about it.
>>
>>I think the manual should probably mention this, but you've solved my problem so I'm happy now. Thanks again,
>>
>>Jill
>
>I would caution you that GC will still miss objects where a stack pointer happens to point to the object.  If you need absolute guarantees (as it seems you do), a more forceful approach may still be required.  Maybe the object could add itself to some kind of global "set" when constructed, and remove itself when destructed.  At program end, the set would need to be iterated and destructed.
>
>If the objects should be deleted by the GC during the program run, the pointer would need to be masked somehow.  Also, it would be a mistake to add and remove the object itself - the pointer must be a pointer to the exact allocation you are concerned with.  If object A points to object B, the destructor for A cannot be sure that B has not already be collected.  So the global set would need a list of B pointers, where B is a struct or array.
>
>Kevin

Another thought: it would not be hard for an attacker to determine likely addresses that malloc may produce (by downloading the source and running similar programs).  Also, some malloc() implementations have very predictable patterns such as alignment to power of two for most allocations.

The attacker could then use input (to a web server for example) which contains these addresses.  One technique would be to specify the memory address as a spoofed IP address or find unicode characters that match these addresses. Assuming a conservative GC, this will pin your secure data object in memory, for whatever nefarious purpose, because the integer will look a lot like a pointer.

(Walter's technique of timing out the secure data will work against this.)

Kevin