View mode: basic / threaded / horizontal-split · Log in · Help
October 28, 2009
Need some help with this...
Following code will freeze app on std.gc.fullCollect(), when sqlite3_close() in destructor is called. If destructor is called manualy, everything goes ok.

Is it a bug, and if is, with what? It behaves same on winxp64 and centos5.2 using dmd 1.30 and sqlite 3.6.5 or 3.6.19 statically import lib. Libraries are tested so I do not suspect problem lies in them (they are compiled with dmc/gcc using full threading support).

Is this some problem with GC or, more likely, my knowledge? I would appreciate some clarification, this thing took me a lot of hours to track.

Thanks, 
Bane

==========================================


import std.stdio;
import std.gc;
import std.string;
import std.thread;

pragma(lib, "sqlite3.lib");
const int SQLITE_OK = 0;	// Successful result.
struct sqlite3 {}
extern(C) int sqlite3_open (char* filename, sqlite3** database);
extern(C) int sqlite3_close(sqlite3* database);

class SQLite {
 sqlite3* h;
 this(){
   assert(sqlite3_open(toStringz(":memory:"), &h) == SQLITE_OK);
 }
 ~this(){
   writefln("~this start"); // to help debug
   assert(sqlite3_close(h) == SQLITE_OK);
   writefln("~this stop"); // to help debug
 }
}

class T : Thread {
 int run(){
   SQLite s = new SQLite;
   // if next line is uncommented then app wont freeze
   // delete s;
   return 0;
 }
}

void main(){
 while(true){
   T t = new T;
   t.start;
   writefln(Thread.nthreads);
   if(Thread.nthreads > 10)
     fullCollect; // this will freeze app
 }
}
October 28, 2009
Re: Need some help with this...
Object destructors can be tricky in a GC'd language. It looks like you're accessing a deallocated pointer in your destructor. Order of collection/destruction is not guaranteed.

Bane Wrote:

> Following code will freeze app on std.gc.fullCollect(), when sqlite3_close() in destructor is called. If destructor is called manualy, everything goes ok.
> 
> Is it a bug, and if is, with what? It behaves same on winxp64 and centos5.2 using dmd 1.30 and sqlite 3.6.5 or 3.6.19 statically import lib. Libraries are tested so I do not suspect problem lies in them (they are compiled with dmc/gcc using full threading support).
> 
> Is this some problem with GC or, more likely, my knowledge? I would appreciate some clarification, this thing took me a lot of hours to track.
> 
> Thanks, 
> Bane
> 
> ==========================================
> 
> 
> import std.stdio;
> import std.gc;
> import std.string;
> import std.thread;
> 
> pragma(lib, "sqlite3.lib");
> const int SQLITE_OK = 0;	// Successful result.
> struct sqlite3 {}
> extern(C) int sqlite3_open (char* filename, sqlite3** database);
> extern(C) int sqlite3_close(sqlite3* database);
> 
> class SQLite {
>   sqlite3* h;
>   this(){
>     assert(sqlite3_open(toStringz(":memory:"), &h) == SQLITE_OK);
>   }
>   ~this(){
>     writefln("~this start"); // to help debug
>     assert(sqlite3_close(h) == SQLITE_OK);
>     writefln("~this stop"); // to help debug
>   }
> }
> 
> class T : Thread {
>   int run(){
>     SQLite s = new SQLite;
>     // if next line is uncommented then app wont freeze
>     // delete s;
>     return 0;
>   }
> }
> 
> void main(){
>   while(true){
>     T t = new T;
>     t.start;
>     writefln(Thread.nthreads);
>     if(Thread.nthreads > 10)
>       fullCollect; // this will freeze app
>   }
> }
October 28, 2009
Re: Need some help with this...
Bane wrote:
> Following code will freeze app on std.gc.fullCollect(), when sqlite3_close() in destructor is called. If destructor is called manualy, everything goes ok.
> 
> Is it a bug, and if is, with what? It behaves same on winxp64 and centos5.2 using dmd 1.30 and sqlite 3.6.5 or 3.6.19 statically import lib. Libraries are tested so I do not suspect problem lies in them (they are compiled with dmc/gcc using full threading support).

It's not your fault, it's a well known bug. The following is what happens:

- in thread 1, a C function (e.g. malloc()) enters an internal lock
- while thread 1 holds the lock, thread 2 triggers a D garbage 
collection cycle
- thread 2 pauses all threads forcibly, including thread 1
- thread 2 collects some objects and calls finalizers on it
- your finalizer calls a C function, which tries to enter the same lock 
that is held by thread 1
- but thread 1 has been paused
- the GC won't resume the other threads until your function returns, and 
you have a deadlock

As a solution, switch to D2 or Tango. These resume all suspended threads 
before running the finalizers.
October 28, 2009
Re: Need some help with this...
grauzone Wrote:

> Bane wrote:
> > Following code will freeze app on std.gc.fullCollect(), when sqlite3_close() in destructor is called. If destructor is called manualy, everything goes ok.
> > 
> > Is it a bug, and if is, with what? It behaves same on winxp64 and centos5.2 using dmd 1.30 and sqlite 3.6.5 or 3.6.19 statically import lib. Libraries are tested so I do not suspect problem lies in them (they are compiled with dmc/gcc using full threading support).
> 
> It's not your fault, it's a well known bug. The following is what happens:
> 
> - in thread 1, a C function (e.g. malloc()) enters an internal lock
> - while thread 1 holds the lock, thread 2 triggers a D garbage 
> collection cycle
> - thread 2 pauses all threads forcibly, including thread 1
> - thread 2 collects some objects and calls finalizers on it
> - your finalizer calls a C function, which tries to enter the same lock 
> that is held by thread 1
> - but thread 1 has been paused
> - the GC won't resume the other threads until your function returns, and 
> you have a deadlock
> 
> As a solution, switch to D2 or Tango. These resume all suspended threads 
> before running the finalizers.

Thank you, grauzone. That clears it. Switching to D2 or Tango is a bit overkill as existing codebase is big and fairly tested (not including this issue :).

I assume this happens pretty rarely, because I haven't noticed this bug so far. Manual delete seems to be workaround, so it is not a critical issue.

Thanks again :) Love D.
Top | Discussion index | About this forum | D home