Thread overview
GC.collect inflating memory usage?
Dec 07, 2019
cc
Dec 07, 2019
Rainer Schuetze
Dec 08, 2019
Rainer Schuetze
Dec 09, 2019
cc
December 07, 2019
Given the following program:

	//version=FREE;
	//version=COLLECT;
	import std.stdio;
	import std.datetime.stopwatch;
	import core.memory;
	immutable int[] intZ = [1,2,3,4,4,6,6,8,8,65,8,23,76,2,57,264,23,4,4,6,6,8,8,65,8,23,76,2,57,264,23,4,4,6,6,8,8,65,8,23,76,2,57,264,23,4,4,6,6,8,8,65,8,23,76,2,57,264,23,4,4,6,6,8,8,65,8,23,76,2,57,264,23,4,4,6,6,8,8,65,8,23,76,2,57,264,23,4,4,6,6,8,8,65,8,23,76,2,57,264,23,4,4,6,6,8,8,65,8,23,76,2,57,264,23,4,4,6,6,8,8,65,8,23,76,2,57,264,23];
	void main() {
		writeln(GC.stats);
		enum max = 100000;
		StopWatch sw;
		sw.start();
		foreach (i; 0 .. max) {
			bool doprint = !(i % (max/10));
			int[] z = intZ.dup;
			if (doprint) writef("%7d  ", GC.stats.usedSize);
			version(FREE) GC.free(cast(void*) z.ptr);
			version(COLLECT) GC.collect();
			if (doprint) writefln("%7d", GC.stats.usedSize);
		}
		sw.stop();
		writefln("Elapsed: %d ms", sw.peek.total!"msecs");
	}

When compiled with neither the FREE or COLLECT versions, I get results like this typically:
	Stats(16, 1048560, 16)
	    848      848
	 883104   883104
	 711072   711072
	 539040   539040
	 367008   367008
	 191696   191696
	  19664    19664
	 887200   887200
	 715168   715168
	 540672   540672
	Elapsed: 11 ms

When only the FREE line is enabled, I see results like this:
	// FREE
	Stats(16, 1048560, 16)
	    848       32
	    848       32
	    848       32
	    848       32
	    848       32
	    848       32
	    848       32
	    848       32
	    848       32
	    848       32
	Elapsed: 12 ms

When only the COLLECT line is enabled, I see results like this:
	// COLLECT
	Stats(16, 1048560, 16)
	    848     4096
	   4928     4096
	   4928     4096
	   4928     4096
	   9024     8192
	   4928     4096
	   4928     4096
	   4928     4096
	   4928     4096
	   4928     4096
	Elapsed: 1130 ms

But when both FREE and COLLECT are enabled, things seem to spiral out of control:
	// FREE, COLLECT
	Stats(16, 1048560, 16)
	    848     4096
	40960832  40964096
	81920832  81924096
	122880832  122884096
	163840832  163844096
	204800832  204804096
	245760832  245764096
	286720832  286724096
	327680832  327684096
	368640832  368644096
	Elapsed: 29143 ms

I wouldn't normally call GC.collect on every frame in my application, but I'm curious why this is happening and if there is unnecessary bloat being added somehow when I do choose to call GC.free manually and garbage collection later occurs in a long-running program.  Ideally I'd like to free up as many objects and arrays as soon as they become unused to avoid lengthy collections reducing performance.  I know that std.container.array is one alternative to using D's GC-managed dynamic arrays, but could I run into the same issue when trying to manually deallocate class objects as well?

Using DMD32 D Compiler v2.089.0-dirty


December 07, 2019
On 07/12/2019 12:20, cc wrote:
> Given the following program:
[...]
> But when both FREE and COLLECT are enabled, things seem to spiral out of
> control:
>     // FREE, COLLECT
>     Stats(16, 1048560, 16)
>         848     4096
>     40960832  40964096
>     81920832  81924096
>     122880832  122884096
>     163840832  163844096
>     204800832  204804096
>     245760832  245764096
>     286720832  286724096
>     327680832  327684096
>     368640832  368644096
>     Elapsed: 29143 ms
> 
> I wouldn't normally call GC.collect on every frame in my application, but I'm curious why this is happening and if there is unnecessary bloat being added somehow when I do choose to call GC.free manually and garbage collection later occurs in a long-running program.  Ideally I'd like to free up as many objects and arrays as soon as they become unused to avoid lengthy collections reducing performance.  I know that std.container.array is one alternative to using D's GC-managed dynamic arrays, but could I run into the same issue when trying to manually deallocate class objects as well?
> 
> Using DMD32 D Compiler v2.089.0-dirty
> 

Seems like a bug introduced in dmd 2.086, I've created a bugzilla issue: https://issues.dlang.org/show_bug.cgi?id=20438

I suspect there is something broken with respect to the free-lists inside the GC when manually freeing memory :-/
December 08, 2019
On 07/12/2019 21:05, Rainer Schuetze wrote:
> 
> On 07/12/2019 12:20, cc wrote:
>> Given the following program:
> [...]
>>
>> Using DMD32 D Compiler v2.089.0-dirty
>>
> 
> Seems like a bug introduced in dmd 2.086, I've created a bugzilla issue: https://issues.dlang.org/show_bug.cgi?id=20438
> 
> I suspect there is something broken with respect to the free-lists inside the GC when manually freeing memory :-/
> 

Fixed in stable for the next point-release.
December 09, 2019
On Sunday, 8 December 2019 at 17:49:09 UTC, Rainer Schuetze wrote:
>
>> Seems like a bug introduced in dmd 2.086, I've created a bugzilla issue: https://issues.dlang.org/show_bug.cgi?id=20438
>> 
>> I suspect there is something broken with respect to the free-lists inside the GC when manually freeing memory :-/
>> 
>
> Fixed in stable for the next point-release.

Cool, thanks for the update.