Hi, is there any way to get from the GC all allocated objects, so I can see their size and find where I'm leaking memory? Or perhaps a good tool to help with this issue...

I tried building my program with "profile-gc" but I got an invalid MemoryOperationError with no stack trace... so no luck using that. And the leak is only obvious when running in production, where after 6 hours I've seen 1.5Gb of memory usage.

Right now, I'm able to extend the life of my program by injecting calls to:
GC.collect
GCminimize

Within an event that happens every 5 seconds, which I estimate will give me about a week before the program's memory is again at +1.5Gb.

On Sunday, 21 January 2018 at 17:28:13 UTC, Andres Clari wrote:
> Hi, is there any way to get from the GC all allocated objects, so I can see their size and find where I'm leaking memory? Or perhaps a good tool to help with this issue...
>
> I tried building my program with "profile-gc" but I got an invalid MemoryOperationError with no stack trace... so no luck using that. And the leak is only obvious when running in production, where after 6 hours I've seen 1.5Gb of memory usage.


Are you by chance using vibe.d?
In versions 0.8.x I observe a huge memory leak. Might be your problem.

>
> Right now, I'm able to extend the life of my program by injecting calls to:
> GC.collect
> GCminimize
>
> Within an event that happens every 5 seconds, which I estimate will give me about a week before the program's memory is again at +1.5Gb.

On Monday, 22 January 2018 at 06:15:24 UTC, Dmitry Olshansky wrote:
> On Sunday, 21 January 2018 at 17:28:13 UTC, Andres Clari wrote:
>> Hi, is there any way to get from the GC all allocated objects, so I can see their size and find where I'm leaking memory? Or perhaps a good tool to help with this issue...
>>
>> I tried building my program with "profile-gc" but I got an invalid MemoryOperationError with no stack trace... so no luck using that. And the leak is only obvious when running in production, where after 6 hours I've seen 1.5Gb of memory usage.
>
>
> Are you by chance using vibe.d?
> In versions 0.8.x I observe a huge memory leak. Might be your problem.
>
>>
>> Right now, I'm able to extend the life of my program by injecting calls to:
>> GC.collect
>> GCminimize
>>
>> Within an event that happens every 5 seconds, which I estimate will give me about a week before the program's memory is again at +1.5Gb.

Yeah.

Although after doing some manual tracking and isolation of the suspect routines, I found that using "std.concurrency.spawn" to run a http post request with the "requests" library was the leaking suspect.

I'm triggering some 15 requests every 5 seconds in production, so it's obvious there, but querying "GC.usedSize" I was able to observe the same on my dev environment.

After removing the "spawn" call, and wrapping up the post code inside a "runTask", it seems to work properly now, and so far I have a consistently small memory size in production.

Not sure why "spawn" would leak like that tho. I would assume that once the thread exits, it would get destroyed and it's resources reclaimed, specially when I have calls to "GC.collect and GC.minimize".

On Monday, 22 January 2018 at 06:48:00 UTC, Andres Clari wrote:
> Not sure why "spawn" would leak like that tho. I would assume that once the thread exits, it would get destroyed and it's resources reclaimed, specially when I have calls to "GC.collect and GC.minimize".

All threads withing a process share the same heap, so whatever one thread allocated in that heap is not freed or reclaimed automatically when thread dies, it stays in the heap.

On Monday, 22 January 2018 at 15:56:53 UTC, thedeemon wrote:
> On Monday, 22 January 2018 at 06:48:00 UTC, Andres Clari wrote:
>> Not sure why "spawn" would leak like that tho. I would assume that once the thread exits, it would get destroyed and it's resources reclaimed, specially when I have calls to "GC.collect and GC.minimize".
>
> All threads withing a process share the same heap, so whatever one thread allocated in that heap is not freed or reclaimed automatically when thread dies, it stays in the heap.

Well the destructor of some Json objects and strings should be called when the thread exits tho. Why wouldn’t the GC take care of this?

On Monday, 22 January 2018 at 16:37:19 UTC, Andres Clari wrote:
>> All threads withing a process share the same heap, so whatever one thread allocated in that heap is not freed or reclaimed automatically when thread dies, it stays in the heap.
>
> Well the destructor of some Json objects and strings should be called when the thread exits tho.

If some objects were on the stack of the thread proc, their destructors run, yes, as usual for leaving a scope. Strings don't have destructors, afaik.
Thread-local storage gets cleaned up too. But everything that lives in heap, stays there, until GC finds them. Objects in heap don't have any information on them  which thread they belong to.