Thread overview
Debugging InvalidMemoryOperationError
Jul 11, 2016
Sam
Jul 11, 2016
Adam D. Ruppe
Jul 11, 2016
Sam
Jul 11, 2016
ag0aep6g
July 11, 2016
I am building the open source project Sambamba with LDC and running its tests. If I build it in debug configuration (-g in place of -O2) then I get an InvalidMemoryOperationError. This does not occur in release configuration.

Various different platforms and debuggers (Linux, OS X, GDB, LLDB) give different quality de-mangling and stack trace output, but none seem to give local variables. Versions:

LDC - the LLVM D compiler (1.0.0):
  based on DMD v2.070.2 and LLVM 3.8.0
  built with DMD64 D Compiler v2.071.1
  Default target: x86_64-unknown-linux-gnu

Here is the stack from GDB (7.6.1-80.el7) on Linux:

#0  0x0000000000670a50 in onInvalidMemoryOperationError ()
#1  0x000000000068657b in gc.gc.GC.malloc() ()
#2  0x000000000067a4d2 in _d_newclass ()
#3  0x0000000000670d53 in _d_assert ()
#4  0x000000000053d8e2 in std.typecons.__T10RefCountedTS4cram8wrappers54__T5RcPtrTS4cram6htslib10cram_sliceS15cram_free_sliceZ5RcPtr7PayloadVE3std8typecons24RefCountedAutoInitializei0Z.RefCounted.__dtor() ()
#5  0x00000000005d39e6 in cram.wrappers.__T5RcPtrTS4cram6htslib10cram_sliceS15cram_free_sliceZ.RcPtr.__fieldDtor() ()
#6  0x000000000043a5e4 in cram.wrappers.CramSlice.__fieldDtor() ()
#7  0x00000000005d4ce2 in cram.wrappers.UndecodedSliceRange.__fieldDtor() ()
#8  0x000000000067c20e in rt_finalize2 ()
#9  0x000000000068935a in gc.gc.Gcx.sweep() ()
#10 0x0000000000687a1b in gc.gc.Gcx.fullcollect() ()
#11 0x0000000000687bae in gc.gc.GC.fullCollectNoStack() ()
#12 0x000000000067922b in gc_term ()
#13 0x0000000000679efa in rt_term ()
#14 0x000000000067a170 in _d_run_main ()
#15 0x0000000000414278 in main ()

Questions:

* does this suggest that an assert in the destructor of std.typecons.RefCounted is trying to allocate memory?
* if so, is this considered a bug?
* the wiki suggests that you need a debug build of the standard library to get accurate stack traces. is this just for line numbers or could the stack be completely wrong?
* any further suggestions as to finding the cause of this problem?

July 11, 2016
On Monday, 11 July 2016 at 12:42:26 UTC, Sam wrote:
> #3  0x0000000000670d53 in _d_assert ()
> #4  0x000000000053d8e2 in std.typecons.__T10RefCountedTS4cram8wrappers54__T5RcPtrTS4cram6htslib10cram_sliceS15cram_free_sliceZ5RcPtr7PayloadVE3std8typecons24RefCountedAutoInitializei0Z.RefCounted.__dtor() ()


I'm not 100% sure but this is saying an assert in RefCounted's destructor and looking at Phobos source, there's just one:

assert(_refCounted._store._count > 0);


The stack trace also says this is being triggered from the garbage collector. Do you have a dynamic array of RefCounted objects, or one inside a class object?


I'm not sure as to the nature of the bug but refcounted inside a GC object isn't going to work well anyway.

> * the wiki suggests that you need a debug build of the standard library to get accurate stack traces. is this just for line numbers or could the stack be completely wrong?

just line numbers and I don't think it even applies here since it is a template.
July 11, 2016
On 07/11/2016 02:42 PM, Sam wrote:
> #0  0x0000000000670a50 in onInvalidMemoryOperationError ()
> #1  0x000000000068657b in gc.gc.GC.malloc() ()
> #2  0x000000000067a4d2 in _d_newclass ()
> #3  0x0000000000670d53 in _d_assert ()
> #4  0x000000000053d8e2 in
> std.typecons.__T10RefCountedTS4cram8wrappers54__T5RcPtrTS4cram6htslib10cram_sliceS15cram_free_sliceZ5RcPtr7PayloadVE3std8typecons24RefCountedAutoInitializei0Z.RefCounted.__dtor()
> ()
> #5  0x00000000005d39e6 in
> cram.wrappers.__T5RcPtrTS4cram6htslib10cram_sliceS15cram_free_sliceZ.RcPtr.__fieldDtor()
> ()
> #6  0x000000000043a5e4 in cram.wrappers.CramSlice.__fieldDtor() ()
> #7  0x00000000005d4ce2 in
> cram.wrappers.UndecodedSliceRange.__fieldDtor() ()
> #8  0x000000000067c20e in rt_finalize2 ()
> #9  0x000000000068935a in gc.gc.Gcx.sweep() ()
> #10 0x0000000000687a1b in gc.gc.Gcx.fullcollect() ()
> #11 0x0000000000687bae in gc.gc.GC.fullCollectNoStack() ()
> #12 0x000000000067922b in gc_term ()
> #13 0x0000000000679efa in rt_term ()
> #14 0x000000000067a170 in _d_run_main ()
> #15 0x0000000000414278 in main ()
>
> Questions:
>
> * does this suggest that an assert in the destructor of
> std.typecons.RefCounted is trying to allocate memory?

Looks like it. I suppose that means that the assert has failed. Otherwise it shouldn't allocate. Allocating in a destructor is what usually raises InvalidMemoryOperationError.

The only assert there is this:

https://github.com/dlang/phobos/blob/61a7f2a250962a677aa01495cb342234ddac3a7f/std/typecons.d#L4864

> * if so, is this considered a bug?

Possibly. Would need a smaller test case to make sense of it.
July 11, 2016
On Monday, 11 July 2016 at 13:01:31 UTC, Adam D. Ruppe wrote:
> The stack trace also says this is being triggered from the garbage collector. Do you have a dynamic array of RefCounted objects, or one inside a class object?
>
> I'm not sure as to the nature of the bug but refcounted inside a GC object isn't going to work well anyway.

I find this code:

	// FIXME: D arrays don't call element destructors when GC-d :(
        RoundBuf!CramSlice _input_queue;

Where CramSlice contains RcPtr!(cram_slice, cram_free_slice) and RcPtr contains a RefCounted (allowing a custom free function, wrapping C API - see https://github.com/lomereiter/sambamba/blob/master/cram/wrappers.d#L14). What exactly are the two failure modes you are alluding to?