Jump to page: 1 2
Thread overview
Using closure causes GC allocation
Sep 02, 2017
Vino.B
Sep 02, 2017
Moritz Maxeiner
Sep 02, 2017
vino.b
Sep 02, 2017
Moritz Maxeiner
Sep 02, 2017
Vino.B
Sep 02, 2017
Moritz Maxeiner
Sep 02, 2017
Vino.B
Sep 04, 2017
Vino.B
Sep 04, 2017
Nicholas Wilson
Sep 04, 2017
Azi Hassan
Sep 06, 2017
Vino.B
September 02, 2017
Hi All,

   Request your help on how to solve the issue in the below code as when i execute the program with -vgc it state as below:

NewTD.d(21): vgc: using closure causes GC allocation
NewTD.d(25): vgc: array literal may cause GC allocation

void logClean (string[] Lglst, int LogAge) {   //Line 21
	if (!Lglst[0].exists) { mkdir(Lglst[0]); }
	auto ct1 = Clock.currTime();
	auto st1 = ct1 + days(-LogAge);
	auto dFiles = dirEntries(Lglst[0], SpanMode.shallow).filter!(a => a.exists && a.isFile && a.timeCreated < st1).map!(a =>[a.name]).array;  // Line 25
	  dFiles.each!(f => f[0].remove);
}

From,
Vino.B
September 02, 2017
On Saturday, 2 September 2017 at 17:43:08 UTC, Vino.B wrote:
> Hi All,
>
>    Request your help on how to solve the issue in the below code as when i execute the program with -vgc it state as below:
>
> NewTD.d(21): vgc: using closure causes GC allocation
> NewTD.d(25): vgc: array literal may cause GC allocation
>
> void logClean (string[] Lglst, int LogAge) {   //Line 21
> 	if (!Lglst[0].exists) { mkdir(Lglst[0]); }
> 	auto ct1 = Clock.currTime();
> 	auto st1 = ct1 + days(-LogAge);
> 	auto dFiles = dirEntries(Lglst[0], SpanMode.shallow).filter!(a => a.exists && a.isFile && a.timeCreated < st1).map!(a =>[a.name]).array;  // Line 25
> 	  dFiles.each!(f => f[0].remove);
> }

Line 25 happens because of `[a.name]`. You request a new array: the memory for this has to be allocated (the reason why the compiler says "may" is because sometimes, e.g. if the array literal itself contains only literals, the allocations needn't happen at runtime and no GC call is necessary). Since you don't actually use the array, get rid of it:

---
void logClean (string[] Lglst, int LogAge) {   //Line 21
	if (!Lglst[0].exists) { mkdir(Lglst[0]); }
	auto ct1 = Clock.currTime();
	auto st1 = ct1 + days(-LogAge);
	auto dFiles = dirEntries(Lglst[0], SpanMode.shallow).filter!(a => a.exists && a.isFile && a.timeCreated < st1).array;  // Line 25
	  dFiles.each!(f => f.remove);
}
---

I cannot reproduce the line 21 report, though.
Since you use `timeCreated` I assume you're on Windows, but what's your D compiler, which D frontend version are you using, etc. (all the things needed to attempt to reproduce the error).
September 02, 2017
On Saturday, 2 September 2017 at 18:02:06 UTC, Moritz Maxeiner wrote:
> On Saturday, 2 September 2017 at 17:43:08 UTC, Vino.B wrote:
>> [...]
>
> Line 25 happens because of `[a.name]`. You request a new array: the memory for this has to be allocated (the reason why the compiler says "may" is because sometimes, e.g. if the array literal itself contains only literals, the allocations needn't happen at runtime and no GC call is necessary). Since you don't actually use the array, get rid of it:
>
> [...]

Hi,

  Thank you for your help and the DMD version that i am using is DMD 2.076.0 and yes I am on windows.

From,
Vino.B
September 02, 2017
On Saturday, 2 September 2017 at 18:08:19 UTC, vino.b wrote:
> On Saturday, 2 September 2017 at 18:02:06 UTC, Moritz Maxeiner wrote:
>> On Saturday, 2 September 2017 at 17:43:08 UTC, Vino.B wrote:
>>> [...]
>>
>> Line 25 happens because of `[a.name]`. You request a new array: the memory for this has to be allocated (the reason why the compiler says "may" is because sometimes, e.g. if the array literal itself contains only literals, the allocations needn't happen at runtime and no GC call is necessary). Since you don't actually use the array, get rid of it:
>>
>> [...]
>
> Hi,
>
>   Thank you for your help and the DMD version that i am using is DMD 2.076.0 and yes I am on windows.

Please post a compilable, minimal example including how that function gets called that yields you that compiler output.
September 02, 2017
On Saturday, 2 September 2017 at 18:32:55 UTC, Moritz Maxeiner wrote:
> On Saturday, 2 September 2017 at 18:08:19 UTC, vino.b wrote:
>> On Saturday, 2 September 2017 at 18:02:06 UTC, Moritz Maxeiner wrote:
>>> On Saturday, 2 September 2017 at 17:43:08 UTC, Vino.B wrote:
>>>> [...]
>>>
>>> Line 25 happens because of `[a.name]`. You request a new array: the memory for this has to be allocated (the reason why the compiler says "may" is because sometimes, e.g. if the array literal itself contains only literals, the allocations needn't happen at runtime and no GC call is necessary). Since you don't actually use the array, get rid of it:
>>>
>>> [...]
>>
>> Hi,
>>
>>   Thank you for your help and the DMD version that i am using is DMD 2.076.0 and yes I am on windows.
>
> Please post a compilable, minimal example including how that function gets called that yields you that compiler output.

Hi,

 Please find the example code below,

import std.stdio: File,writeln;
import std.datetime.systime: Clock, days, SysTime;
import std.file: SpanMode, dirEntries, exists, isFile, mkdir, remove;
import std.typecons: tuple;
import std.algorithm:  filter, map, each;
import std.array: array;

void logClean (string[] Lglst, int LogAge) {
	if (!Lglst[0].exists) { mkdir(Lglst[0]); }
	auto ct1 = Clock.currTime();
	auto st1 = ct1 + days(-LogAge);
	auto dFiles = dirEntries(Lglst[0], SpanMode.shallow).filter!(a => a.exists && a.isFile && a.timeCreated < st1).map!(a => tuple(a.name)).array;
	  dFiles.each!(a => a[0].remove);
}

void main () {
string[] LogDir = ["C:\\Users\\bheev1\\Desktop\\Current\\Script\\D\\Logs"];
int  LogAge = 1;
logClean(LogDir,LogAge);
}

Another similar issue :
 I removed the [a.name] and the issue in line 25 has resolved, but for another function i am getting the same error

string[][] cleanFiles(string FFs, string Step) {
	auto dFiles = dirEntries(FFs, SpanMode.shallow).filter!(a => a.isFile).map!(a =>[a.name , a.timeCreated.toSimpleString[0 .. 20]]).array;  -> Issue in this line
    if (Step == "run")
        dFiles.each!(a => a[0].remove);
		return dFiles;
}

if the replace the line in error as below then i am getting the error "Error: cannot implicitly convert expression dFiles of type Tuple!(string, string)[] to string[][]"

auto dFiles = dirEntries(FFs, SpanMode.shallow).filter!(a => a.isFile).map!(a => tuple(a.name, a.timeCreated.toSimpleString[0 .. 20])).array;

September 02, 2017
On Saturday, 2 September 2017 at 18:59:30 UTC, Vino.B wrote:
> On Saturday, 2 September 2017 at 18:32:55 UTC, Moritz Maxeiner wrote:
>> On Saturday, 2 September 2017 at 18:08:19 UTC, vino.b wrote:
>>> On Saturday, 2 September 2017 at 18:02:06 UTC, Moritz Maxeiner wrote:
>>>> On Saturday, 2 September 2017 at 17:43:08 UTC, Vino.B wrote:
>>>>> [...]
>>>>
>>>> Line 25 happens because of `[a.name]`. You request a new array: the memory for this has to be allocated (the reason why the compiler says "may" is because sometimes, e.g. if the array literal itself contains only literals, the allocations needn't happen at runtime and no GC call is necessary). Since you don't actually use the array, get rid of it:
>>>>
>>>> [...]
>>>
>>> Hi,
>>>
>>>   Thank you for your help and the DMD version that i am using is DMD 2.076.0 and yes I am on windows.
>>
>> Please post a compilable, minimal example including how that function gets called that yields you that compiler output.
>
> Hi,
>
>  Please find the example code below,
>
> [...]

Cannot reproduce under Linux with dmd 2.076.0 (with commented out Windows-only check). I'll try to see what happens on Windows once I have a VM setup.

>
> Another similar issue :
>  I removed the [a.name] and the issue in line 25 has resolved, but for another function i am getting the same error
>
> string[][] cleanFiles(string FFs, string Step) {
> 	auto dFiles = dirEntries(FFs, SpanMode.shallow).filter!(a => a.isFile).map!(a =>[a.name , a.timeCreated.toSimpleString[0 .. 20]]).array;  -> Issue in this line
>     if (Step == "run")
>         dFiles.each!(a => a[0].remove);
> 		return dFiles;
> }
>
> if the replace the line in error as below then i am getting the error "Error: cannot implicitly convert expression dFiles of type Tuple!(string, string)[] to string[][]"
>
> auto dFiles = dirEntries(FFs, SpanMode.shallow).filter!(a => a.isFile).map!(a => tuple(a.name, a.timeCreated.toSimpleString[0 .. 20])).array;

You changed the type of dFiles, which you return from cleanFiles, without changing the return type of cleanFiles. Change the return type of cleanFiles to the type the compiler error above tells you it should be (`Tuple!(string, string)[]` instead of `string[][]`), or let the compiler infer it via auto (`auto cleanFiles(...`).
September 02, 2017
On Saturday, 2 September 2017 at 20:10:58 UTC, Moritz Maxeiner wrote:
> On Saturday, 2 September 2017 at 18:59:30 UTC, Vino.B wrote:
>> [...]
>
> Cannot reproduce under Linux with dmd 2.076.0 (with commented out Windows-only check). I'll try to see what happens on Windows once I have a VM setup.
>
>> [...]
>
> You changed the type of dFiles, which you return from cleanFiles, without changing the return type of cleanFiles. Change the return type of cleanFiles to the type the compiler error above tells you it should be (`Tuple!(string, string)[]` instead of `string[][]`), or let the compiler infer it via auto (`auto cleanFiles(...`).

Hi,

 Thank you very much, was able to resolve the second code issue by changing the return type of the function.
September 04, 2017
On Saturday, 2 September 2017 at 20:54:03 UTC, Vino.B wrote:
> On Saturday, 2 September 2017 at 20:10:58 UTC, Moritz Maxeiner wrote:
>> On Saturday, 2 September 2017 at 18:59:30 UTC, Vino.B wrote:
>>> [...]
>>
>> Cannot reproduce under Linux with dmd 2.076.0 (with commented out Windows-only check). I'll try to see what happens on Windows once I have a VM setup.
>>
>>> [...]
>>
>> You changed the type of dFiles, which you return from cleanFiles, without changing the return type of cleanFiles. Change the return type of cleanFiles to the type the compiler error above tells you it should be (`Tuple!(string, string)[]` instead of `string[][]`), or let the compiler infer it via auto (`auto cleanFiles(...`).
>
> Hi,
>
>  Thank you very much, was able to resolve the second code issue by changing the return type of the function.

Hi,

  In order to resolve the issue "Using closure causes GC allocation" it was stated that we need to use delegates, can you please help me on how to as i have not gone that much far in D programming.

import std.stdio: File,writeln;
import std.datetime.systime: Clock, days, SysTime;
import std.file: SpanMode, dirEntries, exists, isFile, mkdir, remove;
import std.typecons: tuple, Tuple;
import std.algorithm:  filter, map, each;
import std.array: array;

Tuple!(string)[] logClean (string[] Lglst, int LogAge) {
	if (!Lglst[0].exists) { mkdir(Lglst[0]); }
	auto ct1 = Clock.currTime();
	auto st1 = ct1 + days(-LogAge);
	auto dFiles = dirEntries(Lglst[0], SpanMode.shallow).filter!(a => a.exists && a.isFile && a.timeCreated < st1).map!(a => tuple(a.name)).array;
	dFiles.each!(a => a[0].remove);
	return dFiles;
}

void main () {
string[] LogDir = ["C:\\Users\\bheev1\\Desktop\\Current\\Script\\D\\Logs"];
int  LogAge = 1;
logClean(LogDir,LogAge);
}

From,
Vino.B
September 04, 2017
On Monday, 4 September 2017 at 05:45:18 UTC, Vino.B wrote:
> On Saturday, 2 September 2017 at 20:54:03 UTC, Vino.B wrote:
>> On Saturday, 2 September 2017 at 20:10:58 UTC, Moritz Maxeiner wrote:
>>> On Saturday, 2 September 2017 at 18:59:30 UTC, Vino.B wrote:
>>>> [...]
>>>
>>> Cannot reproduce under Linux with dmd 2.076.0 (with commented out Windows-only check). I'll try to see what happens on Windows once I have a VM setup.
>>>
>>>> [...]
>>>
>>> You changed the type of dFiles, which you return from cleanFiles, without changing the return type of cleanFiles. Change the return type of cleanFiles to the type the compiler error above tells you it should be (`Tuple!(string, string)[]` instead of `string[][]`), or let the compiler infer it via auto (`auto cleanFiles(...`).
>>
>> Hi,
>>
>>  Thank you very much, was able to resolve the second code issue by changing the return type of the function.
>
> Hi,
>
>   In order to resolve the issue "Using closure causes GC allocation" it was stated that we need to use delegates, can you please help me on how to as i have not gone that much far in D programming.
>
> import std.stdio: File,writeln;
> import std.datetime.systime: Clock, days, SysTime;
> import std.file: SpanMode, dirEntries, exists, isFile, mkdir, remove;
> import std.typecons: tuple, Tuple;
> import std.algorithm:  filter, map, each;
> import std.array: array;
>
> Tuple!(string)[] logClean (string[] Lglst, int LogAge) {
> 	if (!Lglst[0].exists) { mkdir(Lglst[0]); }
> 	auto ct1 = Clock.currTime();
> 	auto st1 = ct1 + days(-LogAge);
> 	auto dFiles = dirEntries(Lglst[0], SpanMode.shallow).filter!(a => a.exists && a.isFile && a.timeCreated < st1).map!(a => tuple(a.name)).array;
> 	dFiles.each!(a => a[0].remove);
> 	return dFiles;
> }
>
> void main () {
> string[] LogDir = ["C:\\Users\\bheev1\\Desktop\\Current\\Script\\D\\Logs"];
> int  LogAge = 1;
> logClean(LogDir,LogAge);
> }
>
> From,
> Vino.B

If you are getting Using closure causes GC allocation from the above, it will be because you reference a local in the template delegate argument to filter:

filter!(a => a.exists &&
        a.isFile &&
        a.timeCreated < st1)
                       ^^^
because of the local (st1) it needs a closure. I can't remember how to transform delegates to take parameters and then call with the parameter set to the local, someone else will have to fill you in on that one.
September 04, 2017
On Monday, 4 September 2017 at 05:45:18 UTC, Vino.B wrote:
>   In order to resolve the issue "Using closure causes GC allocation" it was stated that we need to use delegates

Alternatively you can drop the functional style and use a foreach loop that doesn't require delegates, but you'd still need the GC to store the result in an array. And even then you could still use Array (from std.container).
« First   ‹ Prev
1 2