Hi,

I'm trying to clear big associative arrays, but I always get an object error - what I currently doing is:

private string[uint] myAA;

void main() {

   fill(myAA);
   myAA.clear();
   fill(myAA); // object.Error: Access Violation
}

void fill(string[uint] aa) {
   for (uint i = 0; i < 10_000_000_000; i++) {
      myAA[i] = std.conv.to!(string)(i);
   }
}

I already saw the bug report at http://www.digitalmars.com/d/archives/
digitalmars/D/bugs/
Issue_5683_New_Calling_.clear_on_a_fresh_associative_array_causes_subsequent_segfault_28632.html
and I'm also using 2.052 - as I saw at the change-log of 2.053 it's
also unfixed. So... is there any solution to clear associative arrays
without memory leaking?

useo:

> So... is there any solution to clear associative arrays without memory leaking?

What about remove an item at a time inside a loop (iterations on the keys array), and then doing a rehash?

Bye,
bearophile

I realize that this might sound strange, but try setting myAA to null after clear(), this should fix the crash.

David


On 5/31/11 4:00 PM, useo wrote:
> Hi,
>
> I'm trying to clear big associative arrays, but I always get an
> object error - what I currently doing is:
>
> private string[uint] myAA;
>
> void main() {
>
>     fill(myAA);
>     myAA.clear();
>     fill(myAA); // object.Error: Access Violation
> }
>
> void fill(string[uint] aa) {
>     for (uint i = 0; i<  10_000_000_000; i++) {
>        myAA[i] = std.conv.to!(string)(i);
>     }
> }
>
> I already saw the bug report at http://www.digitalmars.com/d/archives/
> digitalmars/D/bugs/
> Issue_5683_New_Calling_.clear_on_a_fresh_associative_array_causes_subsequent_segfault_28632.html
> and I'm also using 2.052 - as I saw at the change-log of 2.053 it's
> also unfixed. So... is there any solution to clear associative arrays
> without memory leaking?

== Auszug aus David Nadlinger (see@klickverbot.at)'s Artikel
> I realize that this might sound strange, but try setting myAA to
null
> after clear(), this should fix the crash.
> David
> On 5/31/11 4:00 PM, useo wrote:
> > Hi,
> >
> > I'm trying to clear big associative arrays, but I always get an object error - what I currently doing is:
> >
> > private string[uint] myAA;
> >
> > void main() {
> >
> >     fill(myAA);
> >     myAA.clear();
> >     fill(myAA); // object.Error: Access Violation
> > }
> >
> > void fill(string[uint] aa) {
> >     for (uint i = 0; i<  10_000_000_000; i++) {
> >        myAA[i] = std.conv.to!(string)(i);
> >     }
> > }
> >
> > I already saw the bug report at http://www.digitalmars.com/d/
archives/
> > digitalmars/D/bugs/
> >
Issue_5683_New_Calling_.clear_on_a_fresh_associative_array_causes_subsequent_segfault_28632.html
> > and I'm also using 2.052 - as I saw at the change-log of 2.053
it's
> > also unfixed. So... is there any solution to clear associative
arrays
> > without memory leaking?

Thanks a lot,

I solved it by using

pGraphics.clear();
pGraphics = null;
// pGraphics.rehash();

I thought this'll solve my problem, but it doesn't... I've to fill my AA every 500 ms with around 250 values (in principle much less than 10_000_000_000). When I start my application (in this context, my game), it works as needed (around 2000 fps/s) but after some seconds, the fps-rate drops down to 1/s and back to normal (and again, and again...). When I remove the command to clear the array, it works without dropping down to 1 fps. I this case I can't update my screen as needed. I also tried to delete the class instance which contains the array and create an new but it's the same as before (it drops down to 1 fps). Is there anything I forgot to consider?

useo:

> Is there anything I forgot to consider?

If the key and values are primitive values or structs of primitive values then you may try another AA implementation that doesn't use the GC.

Bye,
bearophile

> useo:
> > Is there anything I forgot to consider?
> If the key and values are primitive values or structs of primitive
values then you may try another AA implementation that doesn't use the GC.
> Bye,
> bearophile

I tried some different implementations and I also reduced the size of
my arrays without any success :(. It always drops down to 1 FPS for 1
second and after that it runs for some seconds normal until next 1
FPS-second.
I also tried to do the work within a separate thread, also without
success. My current code looks like:

toRender.clear();
toRender = null;

for (int y = pFromY; y < pToY && y < pTiles.length; y++) {
	for (int x = pFromX; x < pToX && x < pTiles[y].length; x++) {
		toRender[pTiles[y][x].texture] ~= pTiles[y][x];
	}
}

toRender is my AA which contains the textures as key (instances of my
class "Texture") and the tiles (position of them) as values. When I
remove the first two lines (clear and set null) it doesn't drops down
to 1 FPS, it runs normal.

I hope anyone know a solution :)

useo:

> toRender is my AA which contains the textures as key (instances of my
> class "Texture") and the tiles (position of them) as values. When I
> remove the first two lines (clear and set null) it doesn't drops down
> to 1 FPS, it runs normal.
> 
> I hope anyone know a solution :)

Try to disable the GC before that double loop and enable after it. Generally those appends create lot of garbage, so it's not something to do in a performance critical part of the program (try to use fixed-sized arrays, or structs with a fixed-sized array plus a length). You can also try an AA of appender. Also try to profile your code, and take a look at GC stats.

Bye,
bearophile