There is a bug regarding unordered object collection in the GC. My finalizer accesses another GC-allocated object and the application *sometimes* crashes here:

void _d_invariant(Object o)
{   ClassInfo c;

    //printf("__d_invariant(%p)\n", o);

    // BUG: needs to be filename/line of caller, not library routine
    assert(o !is null); // just do null check, not invariant check

    c = typeid(o);

        ^--------- this is the crash location

The culprit seems to be these operations:

00007ff6`881f324b 488b4510        mov     rax,qword ptr [rbp+10h]
00007ff6`881f324f 488b10          mov     rdx,qword ptr [rax]
00007ff6`881f3252 488b1a          mov     rbx,qword ptr [rdx] ds:00000000`00000000=????????????????

The vtable lookup wants to dereference a null entry. Not sure how I can fix this, but in the meantime I think typeid could actually add a small check on RDX and return null if that's what it is. Any input?

On 6/16/15 10:00 AM, Etienne wrote:
> There is a bug regarding unordered object collection in the GC. My
> finalizer accesses another GC-allocated object

Don't do that. Nothing is guaranteed in order of destruction. Note that after calling the destructor of an object, the vtable pointer is nulled.

-Steve

On Tuesday, 16 June 2015 at 14:32:45 UTC, Steven Schveighoffer wrote:
> On 6/16/15 10:00 AM, Etienne wrote:
>> There is a bug regarding unordered object collection in the GC. My
>> finalizer accesses another GC-allocated object
>
> Don't do that. Nothing is guaranteed in order of destruction. Note that after calling the destructor of an object, the vtable pointer is nulled.
>
> -Steve

Obviously the debugger shows that the object's destructor wasn't called. I have the class set as final and all the locals are there. Somehow its vtable is destroyed before though.

When I comment out the _d_invariant part at this point until the end, I can do a crazy amount of requests (this is a webserver) and there will be no crash at all.

So with the proper flags, its obviously possible to access a "final class" object locals during destruction, isn't this why the GC isn't re-entrant in the first place?

On Tuesday, 16 June 2015 at 14:00:55 UTC, Etienne wrote:
> There is a bug regarding unordered object collection in the GC. My finalizer accesses another GC-allocated object and the application *sometimes* crashes here:
>
> void _d_invariant(Object o)
> {   ClassInfo c;
>
>     //printf("__d_invariant(%p)\n", o);
>
>     // BUG: needs to be filename/line of caller, not library routine
>     assert(o !is null); // just do null check, not invariant check
>
>     c = typeid(o);
>
>         ^--------- this is the crash location
>
> The culprit seems to be these operations:
>
> 00007ff6`881f324b 488b4510        mov     rax,qword ptr [rbp+10h]
> 00007ff6`881f324f 488b10          mov     rdx,qword ptr [rax]
> 00007ff6`881f3252 488b1a          mov     rbx,qword ptr [rdx] ds:00000000`00000000=????????????????
>
> The vtable lookup wants to dereference a null entry. Not sure how I can fix this, but in the meantime I think typeid could actually add a small check on RDX and return null if that's what it is. Any input?

This is undefined behavior, the only solution is "don't do it"
see my thread
http://forum.dlang.org/post/vcpcjujvkbuoswyzycat@forum.dlang.org

destructors as they are shouldn't exist at all, they are incredibly bug prone.

Bye.

On Tuesday, 16 June 2015 at 14:00:55 UTC, Etienne wrote:
> There is a bug regarding unordered object collection in the GC. My finalizer accesses another GC-allocated object

Well, theoretically you can replace GC, which will allow you do that. But that's hardly practical, the behavior you're asking for is unsound anyway.

On Tuesday, 16 June 2015 at 15:39:06 UTC, rsw0x wrote:
> destructors as they are shouldn't exist at all, they are incredibly bug prone.
>
> Bye.

To be fair, everything is bug prone until you understand them. GC finalization is done in a single lock, none of the memory is re-used, so objects can have their own "destroyed" flags and destroy eachother fine if the typeinfo issue isn't there.

On the other hand, by keeping the GC destructors, we must agree that all the destructors are declared this way:

shared @nogc nothrow ~this()

Only then can we count it in as having predictable behavior.

I can't really agree on removing finalizers, I'd really love to master GC destructors in D so that I can return plain class objects from my functions with data allocated elsewhere, like in any managed language out there. It's really the only way. Don't get me wrong, I allocate and free on a freelist or pool everywhere I can, but the GC has its advantages and I'd really like to put it to work (safely) to build more convenient APIs.

On Tuesday, 16 June 2015 at 16:28:54 UTC, Etienne wrote:
> On Tuesday, 16 June 2015 at 15:39:06 UTC, rsw0x wrote:
>> destructors as they are shouldn't exist at all, they are incredibly bug prone.
>>
>> Bye.
>
> To be fair, everything is bug prone until you understand them.

No, they are just bug prone.

> GC finalization is done in a single lock, none of the memory is re-used, so objects can have their own "destroyed" flags and destroy eachother fine if the typeinfo issue isn't there.

Implementation-defined behavior.

>
> On the other hand, by keeping the GC destructors, we must agree that all the destructors are declared this way:
>
> shared @nogc nothrow ~this()
>
> Only then can we count it in as having predictable behavior.

Still not predictable, as they can be ran in any thread, in any order, and be ran parallel.

>
> I can't really agree on removing finalizers, I'd really love to master GC destructors in D so that I can return plain class objects from my functions with data allocated elsewhere, like in any managed language out there. It's really the only way. Don't get me wrong, I allocate and free on a freelist or pool everywhere I can, but the GC has its advantages and I'd really like to put it to work (safely) to build more convenient APIs.

You're attempting to use GC for a problem that they don't solve because you don't have other tools to fix it. When all you have is a hammer, everything looks like a nail.

On Tuesday, 16 June 2015 at 20:08:36 UTC, rsw0x wrote:
> You're attempting to use GC for a problem that they don't solve because you don't have other tools to fix it. When all you have is a hammer, everything looks like a nail.

Well no, I have plenty of tools. I use a memory library that does a wide range of everything I need https://github.com/etcimon/memutils .. It's not like I'm reliant only on it, like it's my hammer and I only see nails..

It's simply easier to manage the lifetime of objects through the GC because sometimes I have them referred to as delegates at a lower level. I like having finalizers because those objects may hold thread-local containers.

Isn't that what we'd want? To use the GC only when it's important and manual memory management at other times? What happens when you need the two and your manually allocated objects are tracked by a GC-allocated object? Well I'm experimenting a solution here, or am I wrong to do so? I know the behavior is undefined in the docs, but I didn't post this in D.learn did I?

I think you're the one who's actually looking at everything like a nail wrt improving performance in benchmarks.

On Tuesday, 16 June 2015 at 20:30:50 UTC, Etienne wrote:
> On Tuesday, 16 June 2015 at 20:08:36 UTC, rsw0x wrote:
>> You're attempting to use GC for a problem that they don't solve because you don't have other tools to fix it. When all you have is a hammer, everything looks like a nail.
>
> Well no, I have plenty of tools. I use a memory library that does a wide range of everything I need https://github.com/etcimon/memutils .. It's not like I'm reliant only on it, like it's my hammer and I only see nails..
>
> It's simply easier to manage the lifetime of objects through the GC because sometimes I have them referred to as delegates at a lower level. I like having finalizers because those objects may hold thread-local containers.

A GC is not for managing an object's lifetime. There are no guarantees from the GC.

>
> Isn't that what we'd want? To use the GC only when it's important and manual memory management at other times? What happens when you need the two and your manually allocated objects are tracked by a GC-allocated object?

The correct answer would be for the GC object to be upgraded to an RCO because the resource's lifetime is now bound to the GC object.

>
> I think you're the one who's actually looking at everything like a nail wrt improving performance in benchmarks.

No, I'm looking at correctness and not piling hack upon hack that will fall apart at the first sneeze to the GC codebase. Performance just happens to be a side effect of correctness.

On 6/16/15 4:30 PM, Etienne wrote:
> On Tuesday, 16 June 2015 at 20:08:36 UTC, rsw0x wrote:
>> You're attempting to use GC for a problem that they don't solve
>> because you don't have other tools to fix it. When all you have is a
>> hammer, everything looks like a nail.
>
> Well no, I have plenty of tools. I use a memory library that does a wide
> range of everything I need https://github.com/etcimon/memutils .. It's
> not like I'm reliant only on it, like it's my hammer and I only see nails..

Finalizers cannot access external GC memory. period. It's just not supported.

http://dlang.org/class.html#destructors

"This means that when the garbage collector calls a destructor for an object of a class that has members that are references to garbage collected objects, those references may no longer be valid. This means that destructors cannot reference sub objects."

If you want to manage memory from a GC'd object, you can use C malloc and C free.

Or, you can write your own GC that solves this problem that Sun/Oracle couldn't :)

In all seriousness, you could potentially have exceptions to this rule, but it would take a lot of cajoling of druntime and some funky @UDA magic.

-Steve