Hello,

Sorry for the long subject line. Hopefully it will help people searching for these topics later on.

This is my first post after a month or so of lurking. I'm trying out D after years of using C++, C and Java for home projects and in work (I'm an ex full-time game developer).

I'm experimenting with creating a thin wrapper around SDL and OpenGL using DerelictSDL, and DerelictGL, and my own Surface, Texture, etc objects.

I'd like not to have to manually call SDL_FreeSurface, SDL_CloseFont, etc before my Surface and Font objects get GC'd, so I put my cleanup calls in my destructors.

From what I've read here (I searched for 'thread', 'destructor' and '~this' using a subject search (the only useful kind of search available) in my news client), the GC is non-deterministic. Walter said at some point that destructors may not even be called for non auto objects (which makes things hard).

I have a few questions to help me handle this. I'd be really grateful if anyone has some insights they could offer me.

(a) What thread will destructors be called from? Java calls finalizers from a separate thread. Is this behaviour specified in D? Wrapping OpenGL will be awkward if I call glDeleteTextures in a different thread to my GL context.

(b) Can I rely on destructors being called at all? If not, what is their purpose for non auto objects? I can't use auto classes because then I can't embed references inside other classes.

(c) Just calling SDL_FreeSurface in my destructor causes a crash. I'm guessing that the memory I'm referencing is garbage since SDL has been shut down elsewhere. I currently maintain a static associative array per class to track them, in the hope that they will then get collected in the module cleanup. Is this wise? It currently seems not to crash, but it could be leaking memory and I just can't tell.

(d) Would I be better off just wrapping everything in try...finally blocks and manually calling my own finalize() and close() methods? Do people have to do this for files anyway?

Thanks in advance for any hints anyone may be able to give me.

Peter

"Peter Mackay" <a_pointy_stick.NoJunkMail@yahoo.co.uk> wrote in message news:d5ksit$15bc$1@digitaldaemon.com...
> Hello,
>
> Sorry for the long subject line. Hopefully it will help people searching for these topics later on.
>
> This is my first post after a month or so of lurking. I'm trying out D after years of using C++, C and Java for home projects and in work (I'm an ex full-time game developer).
>
> I'm experimenting with creating a thin wrapper around SDL and OpenGL using DerelictSDL, and DerelictGL, and my own Surface, Texture, etc objects.
>
> I'd like not to have to manually call SDL_FreeSurface, SDL_CloseFont, etc before my Surface and Font objects get GC'd, so I put my cleanup calls in my destructors.
>
> From what I've read here (I searched for 'thread', 'destructor' and '~this' using a subject search (the only useful kind of search available) in my news client), the GC is non-deterministic. Walter said at some point that destructors may not even be called for non auto objects (which makes things hard).

correct. In fact a dtor might not even get called today if the GC finds an "ambiguous pointer" to the object. An ambiguous pointer is a number scanned by the GC that looks like a pointer (ie points to something legit) but without the type information the GC can't actually tell if it is a pointer or not. In those cases the GC must assume it is a pointer. It is also unclear if the GC is run at program exit or not.

> I have a few questions to help me handle this. I'd be really grateful if anyone has some insights they could offer me.
>
> (a) What thread will destructors be called from? Java calls finalizers from a separate thread. Is this behaviour specified in D? Wrapping OpenGL will be awkward if I call glDeleteTextures in a different thread to my GL context.

The thread is unspecified. Right now it is on the same thread as the allocation request that triggered the garbage collect. Different collectors might run in another thread, though.

> (b) Can I rely on destructors being called at all? If not, what is their purpose for non auto objects? I can't use auto classes because then I can't embed references inside other classes.

You can't rely on dtors ever being run. They are run with high probability,
though. Are your resources cleaned up automatically by the OS on program
exit? If so you might not need to worry about those rare instances. One can
also have auto instances without declaring the entire class as auto. The
downside of that, though, is auto variables can't be passed to functions or
used outside of the current function. Here's a simple guard that will always
free an object at scope exit:
  auto class Guard(T) {
    T obj; // obj will never gets GC'ed while we exist
    this(T obj) { this.obj = obj; }
    ~this() { delete obj; }
  }
  ...
  foo() {
    Surface s = ...;
    auto Guard!(Surface) sg = new Guard!(Surface)(s); // ensure s gets
cleaned up.
    ...
  }
Note you might want to use a "finalize" or "close" method instead of a dtor
since you can control which threads call finalize but you can't control
which threads call the dtor.

> (c) Just calling SDL_FreeSurface in my destructor causes a crash. I'm guessing that the memory I'm referencing is garbage since SDL has been shut down elsewhere. I currently maintain a static associative array per class to track them, in the hope that they will then get collected in the module cleanup. Is this wise? It currently seems not to crash, but it could be leaking memory and I just can't tell.

Are you shutting down SDL right before returning from main()? If so then probably the GC run that happens after main() is done is the culprit. Be aware the static assoc array won't get collected. Also note that the references from the static array will prevent the surfaces from every getting GC'ed. If there is only one thread that can release the resources you might want to use a global free list (malloc'ed so that it doesn't get scanned by the GC) that objects append themselves to in their dtor and then the thread that can free resources can free the list when it can. The tricky part is that a dtor can't reference any other GC-managed resource since order in which dtors are run is not deterministic.

> (d) Would I be better off just wrapping everything in try...finally blocks and manually calling my own finalize() and close() methods? Do people have to do this for files anyway?

Files get closed automatically at program exit so the rare ambiguous pointer doesn't hurt in practice. Also files can be closed from any thread so it is much easier.

> Thanks in advance for any hints anyone may be able to give me.
>
> Peter

Hello,

Thank you very much for your reply Ben, it helped me out a lot.

>>the GC is non-deterministic. Walter said at some point that destructors may not even be called for non auto objects (which makes things hard).
> 
> 
> correct. In fact a dtor might not even get called today if the GC finds an "ambiguous pointer" to the object. An ambiguous pointer is a number scanned by the GC that looks like a pointer (ie points to something legit) but without the type information the GC can't actually tell if it is a pointer or not. In those cases the GC must assume it is a pointer. It is also unclear if the GC is run at program exit or not.

Once when trying to call SDL_FreeSurface in my destructor, my app crashed in ~this which was called from inside a compiler-generated function called "fullcollect", which was called from mainCRTStartup, or similar.

So, the GC is called, but whether it's in the spec and could be relied upon, I don't know. I'm not going to bank on it.

>>(a) What thread will destructors be called from?
>
> The thread is unspecified. Right now it is on the same thread as the allocation request that triggered the garbage collect. Different collectors might run in another thread, though.

That makes sense, thanks. :-)

> You can't rely on dtors ever being run. They are run with high probability, though. Are your resources cleaned up automatically by the OS on program exit? If so you might not need to worry about those rare instances. One can also have auto instances without declaring the entire class as auto. The downside of that, though, is auto variables can't be passed to functions or used outside of the current function. Here's a simple guard that will always free an object at scope exit:
>   auto class Guard(T) {
>     T obj; // obj will never gets GC'ed while we exist
>     this(T obj) { this.obj = obj; }
>     ~this() { delete obj; }
>   }
>   ...
>   foo() {
>     Surface s = ...;
>     auto Guard!(Surface) sg = new Guard!(Surface)(s); // ensure s gets cleaned up.
>     ...
>   }
> Note you might want to use a "finalize" or "close" method instead of a dtor since you can control which threads call finalize but you can't control which threads call the dtor.

I think this is what I'm going to do.

>>(c) Just calling SDL_FreeSurface in my destructor causes a crash. I'm guessing that the memory I'm referencing is garbage since SDL has been shut down elsewhere. I currently maintain a static associative array per class to track them, in the hope that they will then get collected in the module cleanup. Is this wise? It currently seems not to crash, but it could be leaking memory and I just can't tell.
> 
> 
> Are you shutting down SDL right before returning from main()?

I'm shutting it down in my sdl wrapper module static destructor. Here's the solution I came up with, hopefully I'm doing things the 'right way'. Sorry about the lack of comments. I don't yet know the proper way to add document comments in D. The code should be easily readable though.

Here's the rundown, for people who would prefer not to read the code:

1. Every SDL object inherits from SDLObject, which has an abstract finalize method.
2. SDLObject has a static AA of refs to it's instances.
3. In my module dtor, I ask SDLObject to go through all the instances and finalize them, if they weren't finalized elsewhere.

Problems:

1. If the app creates lots of SDL objects, the AA will keep growing and growing...
2. I'm still not sure what will get GC'd and what won't. Maybe that's normal.
3. Threading is completely ignored. I usually don't use extra threads for external API calls, only for loading files into buffers, streaming sounds, etc.

--- code follows ---

module saurus.sdl;

import derelict.sdl.sdl;
private import derelict.sdl.image;
private import derelict.sdl.ttf;
private import saurus.drawing;
private import std.stdio;

private abstract class SDLObject
{
	private static SDLObject[SDLObject] objects;
	protected bool finalized;

	this()
	{
		objects[this] = this;
		finalized = false;
	}

	static void finalizeAll()
	{
		foreach (SDLObject object; objects)
		{
			if (!object.finalized)
			{
				object.finalize();
				assert(object.finalized == true);
			}
			delete objects[object];
		}
	}

	abstract void finalize();
}

static this()
{
	DerelictSDL_Load();
	if (SDL_Init(SDL_INIT_EVERYTHING) < 0)
	{
		throw new Error("SDL_Init failed.");
	}

	DerelictSDLImage_Load();

	DerelictSDLttf_Load();
	if (TTF_Init() < 0)
	{
		throw new Error("TTF_Init failed.");
	}
}

static ~this()
{
	SDLObject.finalizeAll();

	TTF_Quit();
	SDL_Quit();
}

class Surface : SDLObject
{
	private SDL_Surface* surface;

	this(SDL_Surface* surface)
	{
		this.surface = surface;
	}

	this(char[] name)
	{
		this(IMG_Load(name ~ "\0"));
	}

	...

	protected void finalize()
	{
		if (!finalized)
		{
			SDL_FreeSurface(surface);
			surface = null;
			finalized = true;
		}
	}

	invariant
	{
		if (!finalized)
		{
			assert(surface != null);
			assert(surface.refcount == 1);
		}
	}
}

--- end of code ---

I hope this thread helps anyone who is trying to solve a similar problem.

Peter