Permalink
Reply
Consider this program:
module test;
import core.memory : GC;
import core.sync.semaphore;
import core.thread;
import std;
enum keys = [
"foo", "bar", "baz", "whee",
"foo1", "foo2", "foo3", "foo4", "foo5", "foo6", "foo7", "foo8",
"bar1", "bar2", "bar3", "bar4", "bar5", "bar6", "bar7", "bar8",
"baz1", "baz2", "baz3", "baz4", "baz5", "baz6", "baz7", "baz8"];
void spamGC(Semaphore sema) {
void recursionTest(int depth) {
if (depth < 1000) {
recursionTest(depth + 1);
if (depth % 300 == 0)
recursionTest(depth + 1);
}
string[string] assocArray;
static foreach (a; keys)
{
assocArray[a] = a;
}
// ?????
assocArray.clear;
assocArray = null;
}
recursionTest(0);
sema.notify;
Thread.sleep(3600.seconds);
}
void main() {
Thread[] threads;
auto sema = new Semaphore(0);
enum threadCount = 100;
threadCount.iota.each!((_) {
auto thread = new Thread({ spamGC(sema); });
thread.start;
threads ~= thread;
});
// this clears the leak up!
// threads.each!((thread) { thread.join; });
threadCount.iota.each!((i) {
sema.wait;
});
writefln!"Done.";
100.iota.each!((_) {
GC.collect;
GC.minimize;
});
writefln!"Collected.";
// Now look at residential memory for the process.
Thread.sleep(3600.seconds);
}
We've had problems with excessive memory leaks in JSON decoding in production. So I've put together this test program.
The spamGC recursive call represents recursive JSON decoding.
The Thread.sleep call represents a thread idling in a threadpool.
After completion, the program sits at ~600MB residential. I believe, from looking at it, it should be obvious that this memory is entirely dead.
If I alloca 10KB of stack at the beginning of the recursion, the residential RAM drops to ~180MB. I don't think this completely solves the issue, but it sure is interesting.
-gx does nothing.
