Hello everyone,
I've got a problem with the following (very simple) code:

void foo()
{
      void[] dat=new void[50_000_000]; // allocate 50 MByte of dummy-data
      delete dat;
}

after I called foo() and watched the memory usage in the windows taskmanager, the program blowed up to 50 MBytes although I deleted the allocated memory...

Why can't the GC remove that data and how CAN I remove it?

Thanks in advance!

Reply to AxelS,

> Hello everyone,
> I've got a problem with the following (very simple) code:
> void foo()
> {
> void[] dat=new void[50_000_000]; // allocate 50 MByte of
> dummy-data
> delete dat;
> }
> after I called foo() and watched the memory usage in the windows
> taskmanager, the program blowed up to 50 MBytes although I deleted the
> allocated memory...
> 
> Why can't the GC remove that data and how CAN I remove it?
> 
> Thanks in advance!
> 

You can't. The D runtime (and most other runtimes) don't ever reduce the amount of memory they keep in the heap. If you where to allocate another 25MB right after that function you would see no change in the memory usage. The good news is that with virtual memory, all of that has almost zero cost. What matters is how much ram you are actively using.

On Fri, Jul 3, 2009 at 4:30 PM, AxelS<a_bothe@gmx.net> wrote:
> Hello everyone,
> I've got a problem with the following (very simple) code:
>
> void foo()
> {
>      void[] dat=new void[50_000_000]; // allocate 50 MByte of dummy-data
>      delete dat;
> }
>
> after I called foo() and watched the memory usage in the windows taskmanager, the program blowed up to 50 MBytes although I deleted the allocated memory...
>
> Why can't the GC remove that data and how CAN I remove it?
>
> Thanks in advance!

If you're using Tango or D2, you can use the GC.minimize() function (in tango.core.Memory or core.memory) to release memory back to the OS, but as BCS said, the amount of virtual memory is not really a useful indicator anyway.

AxelS:
>       void[] dat=new void[50_000_000]; // allocate 50 MByte of dummy-data

I am not sure, but the GC may scan that chunk of voids. If you need a raw block it may be better to work with an array of uint.


> Why can't the GC remove that data and how CAN I remove it?

If you really need to remove it, then you may not allocate it from the GC heap in the first place, and use the C heap, with a malloc and then free. But that requires a full manual management, and for safety it's then usually much better to put in such memory chunk only data that doesn't contain references managed by GC.

Bye,
bearophile

BCS Wrote:

> You can't. The D runtime (and most other runtimes) don't ever reduce the amount of memory they keep in the heap. If you where to allocate another 25MB right after that function you would see no change in the memory usage. The good news is that with virtual memory, all of that has almost zero cost. What matters is how much ram you are actively using.
> 
> 

I want to load and send a file via network...when I load the entire file into memory it's very stupid that I can't release that memory again...

OK I'll try it with the C API but thanks for your help!

BCS wrote:
> ... The good news is that with virtual memory, all of that has almost zero cost. What matters is how much ram you are actively using.

You've obviously never used a netbook with no swap file.  :)

AxelS escribió:
> BCS Wrote:
> 
>> You can't. The D runtime (and most other runtimes) don't ever reduce the amount of memory they keep in the heap. If you where to allocate another 25MB right after that function you would see no change in the memory usage. The good news is that with virtual memory, all of that has almost zero cost. What matters is how much ram you are actively using.
>>
>>
> 
> I want to load and send a file via network...when I load the entire file into memory it's very stupid that I can't release that memory again...

Why not pipe it?

@Ary Borenszweig: Good idea but I can't pipe data to a HTTP-server located somewhere in the internet...


OK, I tried it with C's malloc and free - but everytime I access my array, D puts the memory into its heap...I'm getting crazy because of this!

ubyte[] data=cast(ubyte[])malloc(50_000_000)[0 .. 50_000_000];

foreach(ref d;data)
{
   d=4; // Simulate data access
}

free(data.ptr);

Can't the D compiler handle such problems in future versions?
Of course it's right to release unused memory...but not just release it into the program-internal heap...

AxelS wrote:
> @Ary Borenszweig: Good idea but I can't pipe data to a HTTP-server located somewhere in the internet...

I believe he means to read the file in chunks, sending them across the network as you get them.

> OK, I tried it with C's malloc and free - but everytime I access my array, D puts the memory into its heap...I'm getting crazy because of this!

D doesn't copy data like that; something else must be going on.

Hello Daniel,

> BCS wrote:
> 
>> ... The good news is that with virtual memory, all of that has almost
>> zero cost. What matters is how much ram you are actively using.
>> 
> You've obviously never used a netbook with no swap file.  :)
> 

Nope, my netbook has a swap file, and I've never heard of one befor (you would be better referring to embedded systems). And even if it didn't, unless you use that whole array, the system doesn't need to actually map ram for what isn't used.

As it happens, I ran across this discussion on swapfiles a few days back.

http://blog.stackoverflow.com/2009/06/podcast-59/
http://serverfault.com/questions/23621/any-benefit-or-detriment-from-removing-a-pagefile-on-an-8gb-ram-machine