I seem to have run into a heisenbug involving destructors and the GC. I'm kind of stuck at this point and need help tracking down the issue.

I put the broken code in a branch called heisenbug on github:
https://github.com/higgsjs/Higgs/tree/heisenbug

The problem manifests itself on runs of `make test` (my unittests), but only some of the time. I wrote a script to run `make test` repeatedly to try and find a solution:
https://github.com/higgsjs/Higgs/blob/heisenbug/source/repeatmaketest.py

The problem usually manifests itself after 5 to 15 runs on my machine. I get a segmentation fault, not always in the same place. The randomness seems to stem from address space randomization.

It seems the issue is caused by my freeing/reinitializing the VM during unit tests. More specifically, commenting out this line makes the problem go away:
https://github.com/higgsjs/Higgs/blob/heisenbug/source/runtime/vm.d#L741

Higgs can run all of my benchmarks without ever failing, but restarting the VM during `make test` seems to be causing this problem to happen. It's not impossible that there could be another underlying issue, such as the JITted code I generate corrupting some memory location, but it would seem that if this were the case, the issue would likely show up outside of unit tests. Any help would be appreciated.

On Friday, 26 June 2015 at 18:27:34 UTC, Maxime Chevalier-Boisvert wrote:
> I seem to have run into a heisenbug involving destructors and the GC. I'm kind of stuck at this point and need help tracking down the issue.
>
> [...]

I should add that I'm running Ubuntu 12.04, 64-bit, and using DMD 2.067.1

On 2015-06-26 14:27, Maxime Chevalier-Boisvert wrote:
> I seem to have run into a heisenbug involving destructors and the GC.
> I'm kind of stuck at this point and need help tracking down the issue.
>
> I put the broken code in a branch called heisenbug on github:
> https://github.com/higgsjs/Higgs/tree/heisenbug
>
> The problem manifests itself on runs of `make test` (my unittests), but
> only some of the time. I wrote a script to run `make test` repeatedly to
> try and find a solution:
> https://github.com/higgsjs/Higgs/blob/heisenbug/source/repeatmaketest.py
>
> The problem usually manifests itself after 5 to 15 runs on my machine. I
> get a segmentation fault, not always in the same place. The randomness
> seems to stem from address space randomization.
>
> It seems the issue is caused by my freeing/reinitializing the VM during
> unit tests. More specifically, commenting out this line makes the
> problem go away:
> https://github.com/higgsjs/Higgs/blob/heisenbug/source/runtime/vm.d#L741
>
> Higgs can run all of my benchmarks without ever failing, but restarting
> the VM during `make test` seems to be causing this problem to happen.
> It's not impossible that there could be another underlying issue, such
> as the JITted code I generate corrupting some memory location, but it
> would seem that if this were the case, the issue would likely show up
> outside of unit tests. Any help would be appreciated.

This might come as a surprise to you as much as it did to me at the time, but when you have GCRoot* root; where GCRoot is a struct, if you destroy(root), you're setting your local pointer to null. You're not actually calling the destructor on the struct.

Also, I would avoid throwing of any type in a destructor.

https://github.com/higgsjs/Higgs/blob/0b48477120c4acce46a01b05a1d4b035aa432550/source/jit/codeblock.d#L157

On Saturday, 27 June 2015 at 02:53:42 UTC, Etienne Cimon wrote:
> On 2015-06-26 14:27, Maxime Chevalier-Boisvert wrote:
>
> This might come as a surprise to you as much as it did to me at the time, but when you have GCRoot* root; where GCRoot is a struct, if you destroy(root), you're setting your local pointer to null. You're not actually calling the destructor on the struct.
>
> Also, I would avoid throwing of any type in a destructor.
>
> https://github.com/higgsjs/Higgs/blob/0b48477120c4acce46a01b05a1d4b035aa432550/source/jit/codeblock.d#L157

calling destroy on a pointer should either be fixed or be an error, that should not be allowed to happen.

On Saturday, 27 June 2015 at 03:16:35 UTC, rsw0x wrote:
> calling destroy on a pointer should either be fixed or be an error, that should not be allowed to happen.

Completely agreed. Calling destroy() on a pointer has been incorrect EVERY time I've seen it done.

Disagree. Destroy on a pointer calls dtor of a struct.
Why it should be an error ?

On 06/27/2015 05:44 AM, Temtaime wrote:
> Disagree. Destroy on a pointer calls dtor of a struct.
> Why it should be an error ?

import std.stdio;
bool destroyed=false;
struct S{
    ~this(){ destroyed=true; }
}

void main(){
    auto s=new S;
    destroy(s);
    writeln(destroyed); // false
}

On 2015-06-26 23:44, Temtaime wrote:
> Disagree. Destroy on a pointer calls dtor of a struct.
> Why it should be an error ?

Exactly what I assumed too. Can you imagine all of the random errors that stem from such a basic, low-level assumption? I carry a scar for every day I've spent in the debugger, and this bug has given me its load of torments. Of course, I'm the only one to blame, I didn't know:

import std.stdio;

void main() {
        struct A { ~this() { writeln("Dtor"); } }
        A* a = new A;
        destroy(a);
        writeln("Done");
}

output: Done

On Saturday, 27 June 2015 at 03:44:54 UTC, Temtaime wrote:
> Disagree. Destroy on a pointer calls dtor of a struct.

This is false. The fact that you think that this is true is the reason that we want it to be an error

import std.stdio : writeln;
import core.stdc.stdlib : malloc;

private struct S
{
	~this()
	{
		writeln("Destructor call");
	}
}

void main()
{
	S* s1 = cast(S*) malloc(S.sizeof);
	destroy(s1);
	S* s2 = cast(S*) malloc(S.sizeof);
	typeid(S).destroy(s2);
}

Run this program. You'll only see one destructor call.

On Friday, 26 June 2015 at 18:27:34 UTC, Maxime Chevalier-Boisvert wrote:
> I seem to have run into a heisenbug involving destructors and the GC. I'm kind of stuck at this point and need help tracking down the issue.

BTW, how GC performance is doing? Etienne is experimenting with thread local GC: http://forum.dlang.org/post/nfuzudyoatryapcwxquu@forum.dlang.org which also proved to be faster. But only if single-threaded GC is good enough for you, which normally should be given single-threaded model of javascript.