Hi,

A while ago, I've tracked down the cause of an insidious memory corruption problem in one of my D programs. The problem was caused by an inadvertent allocation in a destructor (called by the GC). The current GC implementation is completely unprepared to handle such a situation - an allocation during a GC run will break the GC's invariants, and will ultimately result in memory corruption. I've filed this problem as issue 5653.

I've created a simple test case which illustrates the problem:

//////////////////////////////////////////////////////////////////////////////
const message = "Hello, world!";

char[] s = null;

class C
{
	~this()
	{
		s = message.dup;
	}
}

version(D_Version2)
	import core.memory;
else
	import std.gc;

void main()
{
	C c;
	c = new C();
	c = new C(); // clobber any references to first instance

	version(D_Version2)
		GC.collect();
	else
		fullCollect();

	assert(s !is null, "Destructor wasn't called");
	assert(s == message, "Memory was corrupted");
}
//////////////////////////////////////////////////////////////////////////////

The exact reason the above program corrupts memory is that .dup will allocate memory by taking an item from a free list. However, after the destructor returns, the GC continues on to rebuild the free list with the information it had before the .dup allocation. The first machine word of the allocated region will be overwritten with a pointer to the next item in the free list.

I wrote a patch to the D1 GC to forbid allocations from destructors, and was considering to port it to D2 and wrap it in a pull request, but realized that my patch breaks the GC in case a destructor throws. However, looking at the GC code it doesn't look like the GC is prepared to handle that situation either... while I haven't noticed any ways in which it could lead to memory corruption, if the program would catch exceptions propagated through a GC run, it could lead to persistent memory leaks (inconsistency between flags and freelists) and destructors of other objects being called several times (due to free lists not being rebuilt).

Thus, my question is: what's the expected behavior of D programs when a destructor throws?

-- 
Best regards,
 Vladimir                            mailto:vladimir@thecybershadow.net

On 13.05.2011 06:53, Vladimir Panteleev wrote:

> Thus, my question is: what's the expected behavior of D programs when a destructor throws?

  I would say, the only expected (and correct, IMHO) behavior should be termination of the program because of unhandled exception.

/Alexander

Am 13.05.2011 09:57, schrieb Alexander:
> On 13.05.2011 06:53, Vladimir Panteleev wrote:
> 
>> Thus, my question is: what's the expected behavior of D programs when a destructor throws?
> 
>   I would say, the only expected (and correct, IMHO) behavior should be termination of the program because of unhandled exception.
> 
> /Alexander

That sucks if there's no way to handle that exception (other than by try { ... } catch {} in the destructor itself)..

But probably the destructor should generally be forbidden to throw, so if it does and it crashes the program it may be fine.

http://www.digitalmars.com/d/archives/digitalmars/D/What_is_nothrow_for_70451.html and http://www.digitalmars.com/d/archives/digitalmars/D/What_is_throwable_86055.html

discussed throwing in destructors a little bit.

Cheers,
- Daniel

On Fri, 13 May 2011 10:57:37 +0300, Alexander <aldem+dmars@nk7.net> wrote:

> On 13.05.2011 06:53, Vladimir Panteleev wrote:
>
>> Thus, my question is: what's the expected behavior of D programs when a destructor throws?
>
>   I would say, the only expected (and correct, IMHO) behavior should be termination of the program because of unhandled exception.

Yes, but what if it's handled (there's a try/catch block around the allocation or fullCollect call that invoked the GC)?

-- 
Best regards,
 Vladimir                            mailto:vladimir@thecybershadow.net

On Fri, 13 May 2011 11:11:47 +0300, Daniel Gibson <metalcaedes@gmail.com> wrote:

> http://www.digitalmars.com/d/archives/digitalmars/D/What_is_nothrow_for_70451.html
> and
> http://www.digitalmars.com/d/archives/digitalmars/D/What_is_throwable_86055.html
>
> discussed throwing in destructors a little bit.

Both threads refer to destructors in RAII, though.

-- 
Best regards,
 Vladimir                            mailto:vladimir@thecybershadow.net

Am 13.05.2011 10:12, schrieb Vladimir Panteleev:
> On Fri, 13 May 2011 10:57:37 +0300, Alexander <aldem+dmars@nk7.net> wrote:
> 
>> On 13.05.2011 06:53, Vladimir Panteleev wrote:
>>
>>> Thus, my question is: what's the expected behavior of D programs when a destructor throws?
>>
>>   I would say, the only expected (and correct, IMHO) behavior should
>> be termination of the program because of unhandled exception.
> 
> Yes, but what if it's handled (there's a try/catch block around the allocation or fullCollect call that invoked the GC)?
> 

I don't think the exception from a destructor should be thrown to an
allocation.. and probably not fullCollect either.
For clear() or delete it /may/ make sense to get exceptions thrown in
the destructor, but it'd be inconsistent if the exceptions would just
vanish or terminate the program otherwise.

It may be sane to just define that destructors are nothrow and if they throw anyway to terminate the program.

What is the current behaviour anyway? ;)

Alexander wrote:
> On 13.05.2011 06:53, Vladimir Panteleev wrote:
> 
>> Thus, my question is: what's the expected behavior of D programs when a destructor throws?
> 
>   I would say, the only expected (and correct, IMHO) behavior should be termination of the program because of unhandled exception.
> 
> /Alexander

Are you talking about *finalizers* or *destructors* ?

Throwing from inside a destructor should definitely work (unlike C++).
But finalizers should probably be nothrow.

On Fri, 13 May 2011 11:25:01 +0300, Don <nospam@nospam.com> wrote:

> Alexander wrote:
>> On 13.05.2011 06:53, Vladimir Panteleev wrote:
>>
>>> Thus, my question is: what's the expected behavior of D programs when a destructor throws?
>>    I would say, the only expected (and correct, IMHO) behavior should be termination of the program because of unhandled exception.
>>  /Alexander
>
> Are you talking about *finalizers* or *destructors* ?
>
> Throwing from inside a destructor should definitely work (unlike C++).
> But finalizers should probably be nothrow.

How would you distinguish the two in a language? Class destructors = finalizers?

Come to think of it, SafeD shouldn't allow accessing anything on the heap in destructors as well...

-- 
Best regards,
 Vladimir                            mailto:vladimir@thecybershadow.net

On Fri, 13 May 2011 11:22:11 +0300, Daniel Gibson <metalcaedes@gmail.com> wrote:

> It may be sane to just define that destructors are nothrow and if they
> throw anyway to terminate the program.
>
> What is the current behaviour anyway? ;)

D2 throws a FinalizeError (which is an Error, so "not recoverable").
D1 just allows the exception to propagate through the GC to whatever caused a GC to run.
Both leave the GC in an indeterminate state, as far as I can tell.

-- 
Best regards,
 Vladimir                            mailto:vladimir@thecybershadow.net

On Fri, 13 May 2011 11:25:01 +0300, Don <nospam@nospam.com> wrote:

> Alexander wrote:
>> On 13.05.2011 06:53, Vladimir Panteleev wrote:
>>
>>> Thus, my question is: what's the expected behavior of D programs when a destructor throws?
>>    I would say, the only expected (and correct, IMHO) behavior should be termination of the program because of unhandled exception.
>>  /Alexander
>
> Are you talking about *finalizers* or *destructors* ?
>
> Throwing from inside a destructor should definitely work (unlike C++).
> But finalizers should probably be nothrow.

How would you distinguish the two in the language? Class destructors = finalizers?

Come to think of it, SafeD shouldn't allow accessing anything on the heap in finalizers as well...

-- 
Best regards,
 Vladimir                            mailto:vladimir@thecybershadow.net