D programs are a vital part of my home computer infrastructure. I run some 60 D processes at almost any time.... and have recently been running out of memory.

Each individual process eats ~30-100 MB, but that times 60 = trouble. They start off small, like 5 MB, and grow over weeks or months, so it isn't something I can easily isolate in a debugger after recompiling.

I'm pretty sure this is the result of wasteful code somewhere in my underlying libraries, but nothing is obviously jumping out at me in the code. So I want to look at some of my existing processes and see just what is responsible for this.

I tried attaching to one and `call gc_stats()` in gdb... and it segfaulted. Whoops.




I am willing to recompile and run again, though I need to actually use the programs, so if instrumenting makes them unusable it won't really help. Is there a magic --DRT- argument perhaps? Or some trick with gdb attaching to a running process I don't know?

What I'm hoping to do is get an idea of which line of code allocates the most that isn't subsequently freed.

On Wednesday, 17 April 2019 at 16:27:02 UTC, Adam D. Ruppe wrote:
> D programs are a vital part of my home computer infrastructure. I run some 60 D processes at almost any time.... and have recently been running out of memory.
>
> Each individual process eats ~30-100 MB, but that times 60 = trouble. They start off small, like 5 MB, and grow over weeks or months, so it isn't something I can easily isolate in a debugger after recompiling.
>
> I'm pretty sure this is the result of wasteful code somewhere in my underlying libraries, but nothing is obviously jumping out at me in the code. So I want to look at some of my existing processes and see just what is responsible for this.
>
> I tried attaching to one and `call gc_stats()` in gdb... and it segfaulted. Whoops.
>
>
>
>
> I am willing to recompile and run again, though I need to actually use the programs, so if instrumenting makes them unusable it won't really help. Is there a magic --DRT- argument perhaps? Or some trick with gdb attaching to a running process I don't know?
>
> What I'm hoping to do is get an idea of which line of code allocates the most that isn't subsequently freed.

One thing you can try, without recompiling, is using pmap -x
on one of the bloated processes, and then dumping a large
memory region to file, and then just looking at the binary.

It might be something obvious on visual inspection.

You can dump memory with

  gdb -p $pid --eval-command 'dump binary memory file.bin 0xfromLL 0xtoLL' -batch

On Wednesday, 17 April 2019 at 16:27:02 UTC, Adam D. Ruppe wrote:
> Each individual process eats ~30-100 MB, but that times 60 = trouble. They start off small, like 5 MB, and grow over weeks or months, so it isn't something I can easily isolate in a

Do you run GC.minimize ?

On 2019-04-17 18:27, Adam D. Ruppe wrote:

> I am willing to recompile and run again, though I need to actually use the programs, so if instrumenting makes them unusable it won't really help. Is there a magic --DRT- argument perhaps? Or some trick with gdb attaching to a running process I don't know?

Perhaps try some of these flags [1] and [2]. I tried to look for other `--DRT-` flags but unfortunately it's spread across the druntime code base and not handled in a single place. There's no documentation and there's no generic `--DRT-help` flag. It's a mess.

[1] https://dlang.org/changelog/2.067.0.html#gc-options
[2] https://dlang.org/changelog/2.068.0.html#gc-api-profile

-- 
/Jacob Carlborg

On Wednesday, 17 April 2019 at 16:57:51 UTC, Julian wrote:
> It might be something obvious on visual inspection.

Huh, indeed this got me the biggest obvious-in-retrospect-but-i-didnt-think-to-look-there win of the day - the memory dump showed a super-bloated scrollback buffer in my terminal emulator. I removed 24 bit color support and slashed that in half, then instituted some limits to bring the peak down a bit more.

Still have more to go, but this little thing actually added up to a whopping gigabyte across my whole system.

>
> You can dump memory with

thanks for the tip!

On Wednesday, 17 April 2019 at 17:03:20 UTC, Martin Krejcirik wrote:
> Do you run GC.minimize ?

Not explicitly, but I did try `call gc_minimize()` from the debugger when attached to processes and it made no difference.

Maybe I'll add a hook to the program to call that on a hotkey press for the future though, I can see some situations where it might make a difference.

(though I'd be kinda surprised if it didn't at least sometimes run automatically...)

On Wednesday, 17 April 2019 at 19:07:46 UTC, Jacob Carlborg wrote:
> Perhaps try some of these flags [1] and [2].

oooh, those are very interesting too.

What I was kinda hoping is it would have stats for which file and line of code was responsible for most allocations; a detailed profile. But even so, this is an interesting gem.

> There's no documentation and there's no generic `--DRT-help` flag. It's a mess.

Indeed, we need to fix that. But I'm too lazy to do it myself :(

On Wednesday, 17 April 2019 at 22:37:38 UTC, Adam D. Ruppe wrote:
> On Wednesday, 17 April 2019 at 19:07:46 UTC, Jacob Carlborg wrote:
>> Perhaps try some of these flags [1] and [2].
>
> oooh, those are very interesting too.
>
> What I was kinda hoping is it would have stats for which file and line of code was responsible for most allocations; a detailed profile. But even so, this is an interesting gem.

Not at all what you want, but it may be useful for figuring out where the leaks are. Have you tried compiling with -vgc? https://dlang.org/dmd-windows.html#switch-vgc

On Wednesday, 17 April 2019 at 23:57:41 UTC, Meta wrote:
> Not at all what you want, but it may be useful for figuring out where the leaks are. Have you tried compiling with -vgc?

That wouldn't help me much here because I know parts are GC allocating, and I'm ok with that. What I want to know are the parts the GC is not collecting for whatever reason. These parts may be malloc'd too; it isn't necessary the GC's fault.

If you have a slow memory leak, you can speed it up by a stress test. Also the debug built application can run in a separate environment.