Hello AxelS,

> BCS Wrote:
> 
>> You can't. The D runtime (and most other runtimes) don't ever reduce
>> the amount of memory they keep in the heap. If you where to allocate
>> another 25MB right after that function you would see no change in the
>> memory usage. The good news is that with virtual memory, all of that
>> has almost zero cost. What matters is how much ram you are actively
>> using.
>> 
> I want to load and send a file via network...when I load the entire
> file into memory it's very stupid that I can't release that memory
> again...
> 
> OK I'll try it with the C API but thanks for your help!
> 

C's runtime (malloc/free) doesn't return memory to the OS either. Unless your system starts complaining about wanting to make the swap file bigger, quit worrying about it.

OTOH another way to work this is to memory map the file and unmap it when you are done. That way the original file is used as the backing store rather than the swap file and the memory gets unmapped when you dump the file.

BCS wrote:
> Hello AxelS,
> 
>> BCS Wrote:
>>
>>> You can't. The D runtime (and most other runtimes) don't ever reduce the amount of memory they keep in the heap. If you where to allocate another 25MB right after that function you would see no change in the memory usage. The good news is that with virtual memory, all of that has almost zero cost. What matters is how much ram you are actively using.
>>>
>> I want to load and send a file via network...when I load the entire file into memory it's very stupid that I can't release that memory again...
>>
>> OK I'll try it with the C API but thanks for your help!
>>
> 
> C's runtime (malloc/free) doesn't return memory to the OS either.

Um .. yes it does. :)

#include <stdio.h>
#include <stdlib.h>

int main() {
  void *p = calloc(1024*1024*256, 1);
  system("sleep 10");
  free(p);
  system("sleep 10");
  return 0;
}

Run that, wait 10s, and watch the memory usage go down.

@downs:

That's what I even had before....just allocate and release memory is not difficult - just if you want to access the data the GC copies all the memory into its heap...

I found a new way which is really good and easy:
Just use the Win32 Memory API for allocating and releasing your data
with functions like GlobalAlloc or GlobalFree

http://msdn.microsoft.com/en-us/library/ms810603.aspx

downs wrote:
> BCS wrote:
>> Hello AxelS,
>>
>>> BCS Wrote:
>>>
>>>> You can't. The D runtime (and most other runtimes) don't ever reduce the amount of memory they keep in the heap. If you where to allocate another 25MB right after that function you would see no change in the memory usage. The good news is that with virtual memory, all of that has almost zero cost. What matters is how much ram you are actively using.
>>>>
>>> I want to load and send a file via network...when I load the entire file into memory it's very stupid that I can't release that memory again...
>>>
>>> OK I'll try it with the C API but thanks for your help!
>>>
>> C's runtime (malloc/free) doesn't return memory to the OS either.
> 
> Um .. yes it does. :)
> 
	No it doesn't (not always). Depending on your C runtime and the
size of the allocations, memory may or may not be returned to the OS.

	For example, on Unix systems, there are two system calls to get
memory from the OS: (s)brk and mmap. AFAIK, there is no way to
return memory allocated through (s)brk. Those calls are only able to
allocate blocs of a given size (the OS page size) and the runtime
manages smaller and larger sizes to balance speed, and efficiency.
Some C runtimes will allocate small blocs through (s)brk and large
blocs through mmap, some will allocate everything with (s)brk, some
will use mmap when the requested size is a multiple of the OS page
size and everything else through (s)brk. Depending on the scheme
used, the runtime may not be able to return the memory to the OS.

	Like BCS said, the only way to make sure that the memory will be
returned is to use mmap/munmap directly (or their equivalent on your
platform).

		Jerome
-- 
mailto:jeberger@free.fr
http://jeberger.free.fr
Jabber: jeberger@jabber.fr

AxelS wrote:
> @downs:
> 
> That's what I even had before....just allocate and release memory is not difficult - just if you want to access the data the GC copies all the memory into its heap...

Can you recommend a good crack dealer? ;)

Seriously, there are no GC calls associated with memory accesses. Perhaps malloc causes the OS to assign some virtual memory to the process, but only accessing it causes a page fault which the OS handles by committing physical memory.


-- 
Tomasz Stachowiak
http://h3.team0xf.com/
h3/h3r3tic on #D freenode

Hello Jérôme,

> Like BCS said, the only way to make sure that the memory will be
> returned is to use mmap/munmap directly (or their equivalent on your
> platform).

That maybe true, but it wasn't my point. What I was trying to get at was that if you want to load a file into a buffer then asking mmap to do it removes the need to allocate a buffer from the language runtime.

On Sat, 04 Jul 2009 05:11:39 -0400, Daniel Keep <daniel.keep.lists@gmail.com> wrote:

>
> BCS wrote:
>> ... The good news is that with virtual memory, all of that has almost
>> zero cost. What matters is how much ram you are actively using.
>
> You've obviously never used a netbook with no swap file.  :)

Swap files are needed for hibernation, probably a very highly used feature for netbooks...

-Steve

Reply to Steven,

> On Sat, 04 Jul 2009 05:11:39 -0400, Daniel Keep
> <daniel.keep.lists@gmail.com> wrote:
> 
>> BCS wrote:
>> 
>>> ... The good news is that with virtual memory, all of that has
>>> almost zero cost. What matters is how much ram you are actively
>>> using.
>>> 
>> You've obviously never used a netbook with no swap file.  :)
>> 
> Swap files are needed for hibernation, probably a very highly used
> feature  for netbooks...
> 
> -Steve
> 

Last I checked, windows doesn't use the swap file to hibernate (it seems to have to create a new file when you turn on the feature). I haven't a clue why it doesn't but that seems to be the case.