Having fleshed out an SMTP server, I've now started testing it against a good load level.

Apart from the various thread issues in phobos (getHostbyName is not thread safe etc./ using udns solved alot of this...) - my bigger worry is memory management.

With GC enabled/or even disabled, the memory pool will rise to ~ 200Mb from the baseline of ~20K and run out of memory on the box it's running on. The test box we have peaks at about 30 concurrent connections/threads. - compiled using GDC (as it's FreeBSD)

From what i've seen, there is no real way to disable the Garbage collection, in terms of stopping it from managing the allocation/deallocation of memory? as it is too intertwined in the management of things like char[] etc..

Has anyone found good ways to  analyze where the application may be leaking memory/allocating to much memory - hooks in the in/out properties of methods ?

Is there any way to compact the memory, as the pool appears to jump quite dramatically from ~ 64k->100Mb->250Mb, even though it looks like the used pool is significantly smaller..

I'm basically fishing for ideas here, as I've got a whole day tomorrow to start sticking in debugging code to find some answers...

Regards
Alan

Alan Knowles wrote:
> Having fleshed out an SMTP server, I've now started testing it against a good load level.
> 
> Apart from the various thread issues in phobos (getHostbyName is not thread safe etc./ using udns solved alot of this...) - my bigger worry is memory management.
> 
> With GC enabled/or even disabled, the memory pool will rise to ~ 200Mb from the baseline of ~20K and run out of memory on the box it's running on. The test box we have peaks at about 30 concurrent connections/threads. - compiled using GDC (as it's FreeBSD)
> 
>  From what i've seen, there is no real way to disable the Garbage collection, in terms of stopping it from managing the allocation/deallocation of memory? as it is too intertwined in the management of things like char[] etc..
> 
> Has anyone found good ways to  analyze where the application may be leaking memory/allocating to much memory - hooks in the in/out properties of methods ?

With Tango you can attach a cleanupHandler and use it to dump info on objects being collected.  I'm planning to add more debugging features as well, but they aren't in yet.  However, on Tango I'd add some logging to lib/gc/basic/gc.d and on Phobos I'd add it to internal/gc/gc.d.

> Is there any way to compact the memory, as the pool appears to jump quite dramatically from ~ 64k->100Mb->250Mb, even though it looks like the used pool is significantly smaller..

Tango offers a GC.minimize() routine that may help.  I believe the same routine is in Phobos as well but it's not actually implemented.


Sean

Unfortunatly Tango doesnt seem to install on freeBSD -
I just get:
make: don't know how to make aaA.o. Stop

I'm having a go adding the minimize code from tango to phobos - any thoughts on if it should work? - it takes forever to build on this freebsd box...

Regards
Alan



Sean Kelly wrote:
> Alan Knowles wrote:
>> Having fleshed out an SMTP server, I've now started testing it against a good load level.
>>
>> Apart from the various thread issues in phobos (getHostbyName is not thread safe etc./ using udns solved alot of this...) - my bigger worry is memory management.
>>
>> With GC enabled/or even disabled, the memory pool will rise to ~ 200Mb from the baseline of ~20K and run out of memory on the box it's running on. The test box we have peaks at about 30 concurrent connections/threads. - compiled using GDC (as it's FreeBSD)
>>
>>  From what i've seen, there is no real way to disable the Garbage collection, in terms of stopping it from managing the allocation/deallocation of memory? as it is too intertwined in the management of things like char[] etc..
>>
>> Has anyone found good ways to  analyze where the application may be leaking memory/allocating to much memory - hooks in the in/out properties of methods ?
> 
> With Tango you can attach a cleanupHandler and use it to dump info on objects being collected.  I'm planning to add more debugging features as well, but they aren't in yet.  However, on Tango I'd add some logging to lib/gc/basic/gc.d and on Phobos I'd add it to internal/gc/gc.d.
> 
>> Is there any way to compact the memory, as the pool appears to jump quite dramatically from ~ 64k->100Mb->250Mb, even though it looks like the used pool is significantly smaller..
> 
> Tango offers a GC.minimize() routine that may help.  I believe the same routine is in Phobos as well but it's not actually implemented.
> 
> 
> Sean

Alan Knowles wrote:

> Unfortunatly Tango doesnt seem to install on freeBSD -
> I just get:
> make: don't know how to make aaA.o. Stop

This sounds more like something else is missing, like a D compiler.

For Tango itself, look to ticket 896 for patches related to get Tango up and running on FreeBSD.

-- 
Lars Ivar Igesund
blog at http://larsivi.net
DSource, #d.tango & #D: larsivi
Dancing the Tango

Ok  - adding the minimize code from tango into phobos seems ok - it does reduce the pools...

However - Its not really a fix, but a bandaid to a worse problem.. (leaky code...)

It took me quite a while to track down the major causes of my memory leaks, here are some of the tricks (which would be really useful if they where in phobos/tango core..)

setMaxAlloc() - ability to set the maximum memory that alloc can be called with. - and just return null if it's more than that..
- Why? - well, quite a common error is
a ~= ..... (where a is an ever growing string) -- this can quickly eat up pages of memory, and is quite difficult to track down. - forcing a segfault by not allowing memory allocation, makes finding the cause with backtrace in gdb very simple.

allocLogOn() ...  this would help alot - seeing that you are allocating alot of small buffers eg. this code

char[] recv() {
   ubyte[] buf  = new ubyte[1024];
   numb = stream.read(buf);
   ...
}
which eats up memory very quickly - is better done using a class property.... eg.
ubyte[] buf  = new ubyte[1024];
char[] recv() {
   numb = stream.read(buf);
   ...
}


I'm going to look at thread registering of memory (eg. having an associative array of memusage[threadid] = size) so it's easier to work out where a thread might be leaking memory, even after a fullCollect().





Regards
Alan




Alan Knowles wrote:
> Having fleshed out an SMTP server, I've now started testing it against a good load level.
> 
> Apart from the various thread issues in phobos (getHostbyName is not thread safe etc./ using udns solved alot of this...) - my bigger worry is memory management.
> 
> With GC enabled/or even disabled, the memory pool will rise to ~ 200Mb from the baseline of ~20K and run out of memory on the box it's running on. The test box we have peaks at about 30 concurrent connections/threads. - compiled using GDC (as it's FreeBSD)
> 
>  From what i've seen, there is no real way to disable the Garbage collection, in terms of stopping it from managing the allocation/deallocation of memory? as it is too intertwined in the management of things like char[] etc..
> 
> Has anyone found good ways to  analyze where the application may be leaking memory/allocating to much memory - hooks in the in/out properties of methods ?
> 
> Is there any way to compact the memory, as the pool appears to jump quite dramatically from ~ 64k->100Mb->250Mb, even though it looks like the used pool is significantly smaller..
> 
> I'm basically fishing for ideas here, as I've got a whole day tomorrow to start sticking in debugging code to find some answers...
> 
> Regards
> Alan