Hi

I'm running Windows Vista 64 with dmd 2.062.

I have a simple program:

import std.stdio, core.memory, std.cstream;
void main()
{
	string[] temp_array;

	for(int i=0;i<5000000;i++)
	{
		++temp_array.length;
		temp_array[temp_array.length - 1] = "aaaaaaaaaaaaaaaaaaaaa";
	}
	
	temp_array = null;

	GC.collect();
        writeln("end");	
	din.getc();
}

When the program waits at "din.getc();", memory usage in the Task Manager is 150MB.

Why isn't the memory deallocating?

P.S.;
I tried temp_array.clear() and destroy(temp_array), but nothing changed.

On 05/07/2013 04:09 PM, Matic Kukovec wrote:> Hi

> When the program waits at "din.getc();", memory usage in the Task
> Manager is 150MB.
>
> Why isn't the memory deallocating?
>
> P.S.;
> I tried temp_array.clear() and destroy(temp_array), but nothing changed.

GC.minimize() may work.

Ali

On Tuesday, 7 May 2013 at 23:14:20 UTC, Ali Çehreli wrote:
> On 05/07/2013 04:09 PM, Matic Kukovec wrote:> Hi
>
> > When the program waits at "din.getc();", memory usage in the
> Task
> > Manager is 150MB.
> >
> > Why isn't the memory deallocating?
> >
> > P.S.;
> > I tried temp_array.clear() and destroy(temp_array), but
> nothing changed.
>
> GC.minimize() may work.
>
> Ali

Thanks for the quick reply, Ali.

Tried it, no changes.

On 05/07/2013 04:18 PM, Matic Kukovec wrote:

> On Tuesday, 7 May 2013 at 23:14:20 UTC, Ali Çehreli wrote:
>> GC.minimize() may work.

> Tried it, no changes.

Works for your test program under Linux but as the documentation says, it is not guaranteed to have any effect at all.

Ali

On Tuesday, 7 May 2013 at 23:31:41 UTC, Ali Çehreli wrote:
> On 05/07/2013 04:18 PM, Matic Kukovec wrote:
>
> > On Tuesday, 7 May 2013 at 23:14:20 UTC, Ali Çehreli wrote:
> >> GC.minimize() may work.
>
> > Tried it, no changes.
>
> Works for your test program under Linux but as the documentation says, it is not guaranteed to have any effect at all.
>
> Ali

I found this problem with a program that reads a large xml file (250000+ lines), then stores the lines in a string[], does a comparison with another array and finally clears the original array.
On the second or third file I always get an OutOfMemoryError, when the Task Manager shows about 1.3GB memory usage.

Is this a Windows specific thing or am I doing something wrong?

On 05/07/2013 04:42 PM, Matic Kukovec wrote:

> On Tuesday, 7 May 2013 at 23:31:41 UTC, Ali Çehreli wrote:
>> On 05/07/2013 04:18 PM, Matic Kukovec wrote:
>>
>> > On Tuesday, 7 May 2013 at 23:14:20 UTC, Ali Çehreli wrote:
>> >> GC.minimize() may work.
>>
>> > Tried it, no changes.
>>
>> Works for your test program under Linux but as the documentation says,
>> it is not guaranteed to have any effect at all.
>>
>> Ali
>
> I found this problem with a program that reads a large xml file (250000+
> lines), then stores the lines in a string[], does a comparison with
> another array and finally clears the original array.

You don't need to clear the original array but it should be harmless.

> On the second or third file I always get an OutOfMemoryError, when the
> Task Manager shows about 1.3GB memory usage.
>
> Is this a Windows specific thing or am I doing something wrong?

Is this a 32-bit platform? If so, the reason may be the conservative GC that dmd uses. What happens is, unrelated 32-bit values in other parts of the program may look like pointers into the allocated space and the GC thinks that they are still in use.

Ali

On Tuesday, 7 May 2013 at 23:58:53 UTC, Ali Çehreli wrote:
> On 05/07/2013 04:42 PM, Matic Kukovec wrote:
>
> > On Tuesday, 7 May 2013 at 23:31:41 UTC, Ali Çehreli wrote:
> >> On 05/07/2013 04:18 PM, Matic Kukovec wrote:
> >>
> >> > On Tuesday, 7 May 2013 at 23:14:20 UTC, Ali Çehreli wrote:
> >> >> GC.minimize() may work.
> >>
> >> > Tried it, no changes.
> >>
> >> Works for your test program under Linux but as the
> documentation says,
> >> it is not guaranteed to have any effect at all.
> >>
> >> Ali
> >
> > I found this problem with a program that reads a large xml
> file (250000+
> > lines), then stores the lines in a string[], does a
> comparison with
> > another array and finally clears the original array.
>
> You don't need to clear the original array but it should be harmless.
>
> > On the second or third file I always get an OutOfMemoryError,
> when the
> > Task Manager shows about 1.3GB memory usage.
> >
> > Is this a Windows specific thing or am I doing something
> wrong?
>
> Is this a 32-bit platform? If so, the reason may be the conservative GC that dmd uses. What happens is, unrelated 32-bit values in other parts of the program may look like pointers into the allocated space and the GC thinks that they are still in use.
>
> Ali

The system is Windows Vista 64bit. DMD is 2.062.

Matic Kukovec:

> The system is Windows Vista 64bit. DMD is 2.062.

DMD doesn't yet produce 64 bit binaries on Windows.

I have tried to solve your problem using GC.free, but I am not seeing good results...

Bye,
bearophile

>
> Why isn't the memory deallocating?

The memory might be free, but still not released to the OS.

Especially in windows, when you free memory it still isn't freed from
your process.
But you can reuse it in your program if the GC has collected it.

The correct test would be to copy and paste the same code below, and see if it
stays at 150Mb or it jumps to 300Mb.

If it jumps to 300Mb, then the GC hasn't collected it.

If it stays at 150Mb, the GC had collected it but windows didn't claim it.

--jm

> I found this problem with a program that reads a large xml file (250000+ lines), then stores the lines in a string[], does a comparison with another array and finally clears the original array.
> On the second or third file I always get an OutOfMemoryError, when the Task Manager shows about 1.3GB memory usage.
>
> Is this a Windows specific thing or am I doing something wrong?

Try the following before getting rid of your arrays:

      myarray[] = null; //if it is an array of classes or strings

      myIntArray[] = 0; //if it is an array of ints

      myBigString[] = '\0';

      etc.

This clears the contents of the arrays, and helps the garbage collector, so that it doesn't confuse data with pointers.

Also, avoid growing arrays little by little. This is VERY bad for garbage accumulation. Use instead an appender!(string[])() for string arrays for instance.

--jm