Thread overview | ||||||||
---|---|---|---|---|---|---|---|---|
|
November 03, 2005 memory leakage in D | ||||
---|---|---|---|---|
| ||||
Hi all, The following code produces very strange behaviour in D. Try out the routine 1, routine 2, routine 3 or routine 4! Apparently there is a bug in the garbage collector and a memory leakage is occouring. A test-case file goes in attachment. Is this a programmer bug or a bug in D, i.e. did I miss something? Thanks! Best Regards, Tiago - -- Tiago Gasiba (M.Sc.) - http://www.gasiba.de Everything should be made as simple as possible, but not simpler. |
November 04, 2005 Re: memory leakage in D | ||||
---|---|---|---|---|
| ||||
Posted in reply to Tiago Gasiba | Tiago Gasiba wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Hi all,
>
> The following code produces very strange behaviour in D.
> Try out the routine 1, routine 2, routine 3 or routine 4!
> Apparently there is a bug in the garbage collector and a memory leakage is occouring.
> A test-case file goes in attachment.
> Is this a programmer bug or a bug in D, i.e. did I miss something? Thanks!
>
> Best Regards,
> Tiago
> [snip]
I think that memory usage keeps increasing, because garbage collection doesn't kick in until the app runs out of system memory.. Try calling std.gc.fullCollect and see if it helps..
xs0
|
November 04, 2005 Re: memory leakage in D | ||||
---|---|---|---|---|
| ||||
Posted in reply to xs0 | xs0 schrieb: > > I think that memory usage keeps increasing, because garbage collection doesn't kick in until the app runs out of system memory.. Try calling std.gc.fullCollect and see if it helps.. Yes, it helps, but only in the function bar(), before the return's Putting the fullCollect in the foo() routine, it does not do anything!!! This must be a GC bug IMHO, because the "user" is not supposed to manually call fullCollect()... I have also posted another message entitled "Garbage Collector Bug?" whereby I have reported a series of errors reported by Valgrind. In this very same program, the same errors occour, i.e. I get some errors like: (This sort of errors should definitly not occour...) <snip> ==7748== Conditional jump or move depends on uninitialised value(s) ==7748== at 0x8053406: _D3gcx3Gcx4markFPvPvZv (in /home/gasiba/PhD/D/bugs/bug1/bug) ==7748== by 0x80535F7: _D3gcx3Gcx11fullcollectFPvZk (in /home/gasiba/PhD/D/bugs/bug1/bug) ==7748== by 0x80534B9: _D3gcx3Gcx16fullcollectshellFZk (in /home/gasiba/PhD/D/bugs/bug1/bug) ==7748== by 0x8053167: _D3gcx3Gcx8bigAllocFkZPv (in /home/gasiba/PhD/D/bugs/bug1/bug) ==7748== by 0x8052693: _D3gcx2GC12mallocNoSyncFkZPv (in /home/gasiba/PhD/D/bugs/bug1/bug) ==7748== by 0x8052536: _D3gcx2GC6mallocFkZPv (in /home/gasiba/PhD/D/bugs/bug1/bug) ==7748== by 0x80526C4: _D3gcx2GC6callocFkkZPv (in /home/gasiba/PhD/D/bugs/bug1/bug) ==7748== by 0x804ED04: _d_arraysetlength (in /home/gasiba/PhD/D/bugs/bug1/bug) ==7748== by 0x804B720: _D3bug3fooFdZd (in /home/gasiba/PhD/D/bugs/bug1/bug) ==7748== by 0x804B773: _Dmain (in /home/gasiba/PhD/D/bugs/bug1/bug) ==7748== by 0x804B7FE: main (in /home/gasiba/PhD/D/bugs/bug1/bug) ==7748== Conditional jump or move depends on uninitialised value(s) ==7748== at 0x8052FDD: _D3gcx3Gcx8findPoolFPvZPS3gcx4Pool (in /home/gasiba/PhD/D/bugs/bug1/bug) ==7748== by 0x805340F: _D3gcx3Gcx4markFPvPvZv (in /home/gasiba/PhD/D/bugs/bug1/bug) ==7748== by 0x80535F7: _D3gcx3Gcx11fullcollectFPvZk (in /home/gasiba/PhD/D/bugs/bug1/bug) ==7748== by 0x80534B9: _D3gcx3Gcx16fullcollectshellFZk (in /home/gasiba/PhD/D/bugs/bug1/bug) ==7748== by 0x8053167: _D3gcx3Gcx8bigAllocFkZPv (in /home/gasiba/PhD/D/bugs/bug1/bug) ==7748== by 0x8052693: _D3gcx2GC12mallocNoSyncFkZPv (in /home/gasiba/PhD/D/bugs/bug1/bug) ==7748== by 0x8052536: _D3gcx2GC6mallocFkZPv (in /home/gasiba/PhD/D/bugs/bug1/bug) ==7748== by 0x80526C4: _D3gcx2GC6callocFkkZPv (in /home/gasiba/PhD/D/bugs/bug1/bug) ==7748== by 0x804ED04: _d_arraysetlength (in /home/gasiba/PhD/D/bugs/bug1/bug) ==7748== by 0x804B720: _D3bug3fooFdZd (in /home/gasiba/PhD/D/bugs/bug1/bug) ==7748== by 0x804B773: _Dmain (in /home/gasiba/PhD/D/bugs/bug1/bug) ==7748== by 0x804B7FE: main (in /home/gasiba/PhD/D/bugs/bug1/bug) ==7748== Use of uninitialised value of size 4 ==7748== at 0x8053D98: _D6gcbits6GCBits4testFkZk (in /home/gasiba/PhD/D/bugs/bug1/bug) ==7748== by 0x8053457: _D3gcx3Gcx4markFPvPvZv (in /home/gasiba/PhD/D/bugs/bug1/bug) ==7748== by 0x80535F7: _D3gcx3Gcx11fullcollectFPvZk (in /home/gasiba/PhD/D/bugs/bug1/bug) ==7748== by 0x80534B9: _D3gcx3Gcx16fullcollectshellFZk (in /home/gasiba/PhD/D/bugs/bug1/bug) ==7748== by 0x8052AC8: _D3gcx2GC11fullCollectFZv (in /home/gasiba/PhD/D/bugs/bug1/bug) ==7748== by 0x804E7E0: _D3std2gc11fullCollectFZv (in /home/gasiba/PhD/D/bugs/bug1/bug) ==7748== by 0x804B6B9: _D3bug3barFKlZd (in /home/gasiba/PhD/D/bugs/bug1/bug) ==7748== by 0x804B74C: _D3bug3fooFdZd (in /home/gasiba/PhD/D/bugs/bug1/bug) ==7748== by 0x804B773: _Dmain (in /home/gasiba/PhD/D/bugs/bug1/bug) ==7748== by 0x804B7FE: main (in /home/gasiba/PhD/D/bugs/bug1/bug) <snip> etc... Thanks for the reply, Tiago -- Tiago Gasiba (M.Sc.) - http://www.gasiba.de Everything should be made as simple as possible, but not simpler. |
November 04, 2005 Re: memory leakage in D | ||||
---|---|---|---|---|
| ||||
Posted in reply to Tiago Gasiba | > > xs0 schrieb: >> >> I think that memory usage keeps increasing, because garbage collection doesn't kick in until the app runs out of system memory.. Try calling std.gc.fullCollect and see if it helps.. > > std.gc.fullCollect() does NOT help in the following example: <snip> import std.gc; double bar(inout long pvar){ int jj; long kk; double temp; static long myvar2=123456789; static long iy=0; static long iv[32]; kk = (pvar/53668); pvar = 40014*(pvar-kk*53668)-kk*12211; if( pvar<0 ) pvar += 2147483563; if( myvar2<0 ) myvar2 += 2147483399; jj = cast(int)(iy/(1+2147483562/32)); jj = jj&31; iy = iv[jj]-myvar2; iv[jj] = pvar; // std.gc.fullCollect(); // here it helps.... why? // program becomes much slower, though if( (temp=(1.0/2147483563)*iy) > (1.0-1.2e-7)) return (1.0-1.2e-7); else return temp; } /* This routine has a memory leakage??? Memory consumption keeps up increasing... even though I specifically call std.gc.fullCollect() */ double foo( double varx ){ int N = 1000000; double[] X; int ii; long pvar = 0x7623627; X.length = N; for( ii=0; ii<X.length; ii++ ) X[ii] = bar(pvar); std.gc.fullCollect(); return 0.0; } int main( ){ while( 1==1 ) foo(0); return 0; } <snip> Tiago -- Tiago Gasiba (M.Sc.) - http://www.gasiba.de Everything should be made as simple as possible, but not simpler. |
November 06, 2005 Re: memory leakage in D | ||||
---|---|---|---|---|
| ||||
Posted in reply to Tiago Gasiba | Tiago Gasiba wrote: >>xs0 schrieb: >> >>>I think that memory usage keeps increasing, because garbage collection >>>doesn't kick in until the app runs out of system memory.. Try calling >>>std.gc.fullCollect and see if it helps.. >> >> > > std.gc.fullCollect() does NOT help in the following example: I made a simple mp3 player in D a while ago. First, it read 4000 files and stored the key-value-pairs of the song tags in a linked list. Then I decided to have multiple playlists. When I "deleted" a playlist and tried the std.gc.fullCollect(), no memory was freed. I also tried something like #Playlist a = null; #std.gc.fullCollect(); and #Playlist a = null #for (int i=0; i<1000; i++) { doSomethingElse(); std.gc.fullCollect(); } but neither of those helped. I think the GC doesn't want to free memory until it has waited for a while and we are actually allocating more memory. But this is kind of stupid behaviour. Of course there are other programs that may need some memory or at least use it for buffering, but this GC seems to be a bit greedy, what do you think? |
November 11, 2005 Re: memory leakage in D | ||||
---|---|---|---|---|
| ||||
Posted in reply to Jari-Matti Mäkelä | Jari-Matti Mäkelä wrote:
> Tiago Gasiba wrote:
>
>>> xs0 schrieb:
>>>
>>>> I think that memory usage keeps increasing, because garbage
>>>> collection doesn't kick in until the app runs out of system
>>>> memory.. Try calling std.gc.fullCollect and see if it helps..
>>>
>>>
>>>
>>
>> std.gc.fullCollect() does NOT help in the following example:
>
>
> I made a simple mp3 player in D a while ago. First, it read 4000
> files and stored the key-value-pairs of the song tags in a linked
> list. Then I decided to have multiple playlists. When I "deleted" a
> playlist and tried the std.gc.fullCollect(), no memory was freed. I
> also tried something like
>
> #Playlist a = null; #std.gc.fullCollect();
>
> and
>
> #Playlist a = null #for (int i=0; i<1000; i++) { doSomethingElse();
> std.gc.fullCollect(); }
>
> but neither of those helped. I think the GC doesn't want to free
> memory until it has waited for a while and we are actually allocating
> more memory. But this is kind of stupid behaviour. Of course there
> are other programs that may need some memory or at least use it for
> buffering, but this GC seems to be a bit greedy, what do you think?
I just had a look into the sources, and it looks like garbage collecting never does reduce the memory footprint of a D program. Seems it isn't even supposed to. GC just collects memory for _itself_ from dead objects.
For the situations where the footprint should be reduced (like right after a memory hungry operation is finished, before continuing), the programmer should first run a full collect, then run
std.gc.minimize();
That would be the first time the OS reports our D program as using less memory, i.e. that it actually has released some memory back.
---
Except that minimize() isn't implemented yet. (Don't tell anybody.)
---
OK, so what can one do when one needs something done that not only takes a long time but also needs tons of memory at the start, after which it needs very little but runs very long?
|
Copyright © 1999-2021 by the D Language Foundation