Hi All,

 Is there any better ways to get the size of folders , The below code perfectly works , but i need return type as Array!(Tuple!(string, string)) rather then using the "Result.insertBack(d); Result.insertBack(to!string(SdFiles[].sum))" as per the below example.

E.g:
Array!(Tuple!(string, string)) Result;
Result = (d, to!string(SdFiles[].sum));

Program:
import std.algorithm: filter, map, sum, uniq;
import std.container.array;
import std.file: dirEntries, SpanMode, isDir, isFile;
import std.stdio: writeln;
import std.typecons: tuple, Tuple;
import std.conv: to;
/******************************************/
/* Sub Function : Size of Dir List        */
/******************************************/
auto mSize () {
	string FFs = "C:\\Temp\\BACKUP";
	Array!string Result;
	auto dFiles = Array!string ((dirEntries(FFs, SpanMode.shallow).filter!(a => a.isDir)).map!(a => a.name));
	foreach (d; dFiles[]) 	{
		auto SdFiles = Array!ulong((dirEntries(d, SpanMode.depth).filter!(a => a.isFile)).map!(a => a.size));
		if (SdFiles[].sum / 1024 / 1024  > 30) { Result.insertBack(d); Result.insertBack(to!string(SdFiles[].sum)); }
								}
		return Result;
}

void main() {
writeln(mSize[]);
}

From,
Vino.B

On Tuesday, 5 December 2017 at 17:21:29 UTC, Vino wrote:
> Hi All,
>
>  Is there any better ways to get the size of folders , The below code perfectly works , but i need return type as Array!(Tuple!(string, string)) rather then using the "Result.insertBack(d); Result.insertBack(to!string(SdFiles[].sum))" as per the below example.
>
> E.g:
> Array!(Tuple!(string, string)) Result;
> Result = (d, to!string(SdFiles[].sum));
>
> Program:
> import std.algorithm: filter, map, sum, uniq;
> import std.container.array;
> import std.file: dirEntries, SpanMode, isDir, isFile;
> import std.stdio: writeln;
> import std.typecons: tuple, Tuple;
> import std.conv: to;
> /******************************************/
> /* Sub Function : Size of Dir List        */
> /******************************************/
> auto mSize () {
> 	string FFs = "C:\\Temp\\BACKUP";
> 	Array!string Result;
> 	auto dFiles = Array!string ((dirEntries(FFs, SpanMode.shallow).filter!(a => a.isDir)).map!(a => a.name));
> 	foreach (d; dFiles[]) 	{
> 		auto SdFiles = Array!ulong((dirEntries(d, SpanMode.depth).filter!(a => a.isFile)).map!(a => a.size));
> 		if (SdFiles[].sum / 1024 / 1024  > 30) { Result.insertBack(d); Result.insertBack(to!string(SdFiles[].sum)); }
> 								}
> 		return Result;
> }
>
> void main() {
> writeln(mSize[]);
> }
>
> From,
> Vino.B

Something like:

auto mSize () {
	string FFs = "C:\\Temp\\BACKUP";
	
   return dirEntries(FFs, SpanMode.shallow)
   .filter!(a => a.isDir)
   .map!(a => tuple(a.name, a.dirEntries(SpanMode.depth).filter!(a=>a.isFile).map!(a => a.size).sum))
   .filter!(a => a[1] > 1024*1024*30)
   .map!(a => tuple(a[0], a[1].to!string))
   .array;
}

?

On Wednesday, 6 December 2017 at 09:16:56 UTC, Andrea Fontana wrote:
> On Tuesday, 5 December 2017 at 17:21:29 UTC, Vino wrote:
>> Hi All,
>>
>>  Is there any better ways to get the size of folders , The below code perfectly works , but i need return type as Array!(Tuple!(string, string)) rather then using the "Result.insertBack(d); Result.insertBack(to!string(SdFiles[].sum))" as per the below example.
>>
>> E.g:
>> Array!(Tuple!(string, string)) Result;
>> Result = (d, to!string(SdFiles[].sum));
>>
>> Program:
>> import std.algorithm: filter, map, sum, uniq;
>> import std.container.array;
>> import std.file: dirEntries, SpanMode, isDir, isFile;
>> import std.stdio: writeln;
>> import std.typecons: tuple, Tuple;
>> import std.conv: to;
>> /******************************************/
>> /* Sub Function : Size of Dir List        */
>> /******************************************/
>> auto mSize () {
>> 	string FFs = "C:\\Temp\\BACKUP";
>> 	Array!string Result;
>> 	auto dFiles = Array!string ((dirEntries(FFs, SpanMode.shallow).filter!(a => a.isDir)).map!(a => a.name));
>> 	foreach (d; dFiles[]) 	{
>> 		auto SdFiles = Array!ulong((dirEntries(d, SpanMode.depth).filter!(a => a.isFile)).map!(a => a.size));
>> 		if (SdFiles[].sum / 1024 / 1024  > 30) { Result.insertBack(d); Result.insertBack(to!string(SdFiles[].sum)); }
>> 								}
>> 		return Result;
>> }
>>
>> void main() {
>> writeln(mSize[]);
>> }
>>
>> From,
>> Vino.B
>
> Something like:
>
> auto mSize () {
> 	string FFs = "C:\\Temp\\BACKUP";
> 	
>    return dirEntries(FFs, SpanMode.shallow)
>    .filter!(a => a.isDir)
>    .map!(a => tuple(a.name, a.dirEntries(SpanMode.depth).filter!(a=>a.isFile).map!(a => a.size).sum))
>    .filter!(a => a[1] > 1024*1024*30)
>    .map!(a => tuple(a[0], a[1].to!string))
>    .array;
> }
>
> ?

Hi Andrea,

  Thank you very much, as your code is pretty good for our scenario, just one request, the above is a part of our main code where we have many such sub code and all of our sub code use the container array(std.container.array) rather than standard array(std.array), so can you please guide me on how to implement the same code using the container array.

From,
Vino.B.

On Wednesday, 6 December 2017 at 14:49:48 UTC, Vino wrote:

> Hi Andrea,
>
>   Thank you very much, as your code is pretty good for our scenario, just one request, the above is a part of our main code where we have many such sub code and all of our sub code use the container array(std.container.array) rather than standard array(std.array), so can you please guide me on how to implement the same code using the container array.
>
> From,
> Vino.B.

Just use Array! constructor.

auto mSize () {
	string FFs = "/home/andrea/Scaricati";
	
   return
   Array!(Tuple!(string,string))(
   dirEntries(FFs, SpanMode.shallow)
   .filter!(a => a.isDir)
   .map!(a => tuple(a.name, a.dirEntries(SpanMode.depth).filter!(a=>a.isFile).map!(a => a.size).sum))
   .filter!(a => a[1] > 1024*1024*30)
   .map!(a => tuple(a[0], a[1].to!string))
   );
}

On Wednesday, 6 December 2017 at 15:04:55 UTC, Andrea Fontana wrote:
> On Wednesday, 6 December 2017 at 14:49:48 UTC, Vino wrote:
>
>> Hi Andrea,
>>
>>   Thank you very much, as your code is pretty good for our scenario, just one request, the above is a part of our main code where we have many such sub code and all of our sub code use the container array(std.container.array) rather than standard array(std.array), so can you please guide me on how to implement the same code using the container array.
>>
>> From,
>> Vino.B.
>
> Just use Array! constructor.
>
> auto mSize () {
> 	string FFs = "/home/andrea/Scaricati";
> 	
>    return
>    Array!(Tuple!(string,string))(
>    dirEntries(FFs, SpanMode.shallow)
>    .filter!(a => a.isDir)
>    .map!(a => tuple(a.name, a.dirEntries(SpanMode.depth).filter!(a=>a.isFile).map!(a => a.size).sum))
>    .filter!(a => a[1] > 1024*1024*30)
>    .map!(a => tuple(a[0], a[1].to!string))
>    );
> }

Hi Andrea,

  I test your code, initially it error ed out stating "patch does not exist", the reason for this error is that the length of the path is more than 256 , so added the UNC path to the code as below.

auto mSize () {
string FFs = "C:\Temp\BACKUP";
ulong SGb = 1024 * 1024;
int SizeDir = 10;

return Array!(Tuple!(string,real))(
 dirEntries(join(["\\\\?\\", FFs]), SpanMode.shallow).filter!(a => a.isDir)
 .map!(a => tuple(a.name, a.dirEntries(SpanMode.depth).filter!(a=>a.isFile)
 .map!(a => a.size).sum)).filter!(a => a[1] >  (SGb * SizeDir))
 .map!(a => tuple(a[0], ((a[1].to!real)/ SGb ))));
}

The output of the code is as below
\\?\C:\Temp\BACKUP\dir1                                 34.90
\\?\C:\Temp\BACKUP\dir2                                 36.18

So how do we print the output without UNC path
C:\Temp\BACKUP\dir1                                 34.90
C:\Temp\BACKUP\dir2                                 36.18

From,
Vino.B

On Thursday, 7 December 2017 at 09:04:19 UTC, Vino wrote:
> On Wednesday, 6 December 2017 at 15:04:55 UTC, Andrea Fontana wrote:
>> On Wednesday, 6 December 2017 at 14:49:48 UTC, Vino wrote:
>>
>>> [...]
>>
>> Just use Array! constructor.
>>
>> auto mSize () {
>> 	string FFs = "/home/andrea/Scaricati";
>> 	
>>    return
>>    Array!(Tuple!(string,string))(
>>    dirEntries(FFs, SpanMode.shallow)
>>    .filter!(a => a.isDir)
>>    .map!(a => tuple(a.name, a.dirEntries(SpanMode.depth).filter!(a=>a.isFile).map!(a => a.size).sum))
>>    .filter!(a => a[1] > 1024*1024*30)
>>    .map!(a => tuple(a[0], a[1].to!string))
>>    );
>> }
>
> Hi Andrea,
>
>   I test your code, initially it error ed out stating "patch does not exist", the reason for this error is that the length of the path is more than 256 , so added the UNC path to the code as below.
>
> auto mSize () {
> string FFs = "C:\Temp\BACKUP";
> ulong SGb = 1024 * 1024;
> int SizeDir = 10;
>
> return Array!(Tuple!(string,real))(
>  dirEntries(join(["\\\\?\\", FFs]), SpanMode.shallow).filter!(a => a.isDir)
>  .map!(a => tuple(a.name, a.dirEntries(SpanMode.depth).filter!(a=>a.isFile)
>  .map!(a => a.size).sum)).filter!(a => a[1] >  (SGb * SizeDir))
>  .map!(a => tuple(a[0], ((a[1].to!real)/ SGb ))));
> }
>
> The output of the code is as below
> \\?\C:\Temp\BACKUP\dir1                                 34.90
> \\?\C:\Temp\BACKUP\dir2                                 36.18
>
> So how do we print the output without UNC path
> C:\Temp\BACKUP\dir1                                 34.90
> C:\Temp\BACKUP\dir2                                 36.18
>
> From,
> Vino.B

Hi Andrea,

 Was able to find a solution to the above issue by adding the replace function as below, the the code is working as expected, is there any chance of using parallel function as the file system contains about 500+ folders and the size of each folder ranges from 5GB - 500 GB so the run time of the above code is about an 2 hours.

.map!(a => tuple(a[0].replace(",\\?\", ""), ((a[1].to!real)/ SGb ))));

From,
Vino.B

On Thursday, 7 December 2017 at 12:19:00 UTC, Vino wrote:
> On Thursday, 7 December 2017 at 09:04:19 UTC, Vino wrote:
>> [...]
>
> Hi Andrea,
>
>  Was able to find a solution to the above issue by adding the replace function as below, the the code is working as expected, is there any chance of using parallel function as the file system contains about 500+ folders and the size of each folder ranges from 5GB - 500 GB so the run time of the above code is about an 2 hours.
>
> .map!(a => tuple(a[0].replace(",\\?\", ""), ((a[1].to!real)/ SGb ))));
>
> From,
> Vino.B

Hi All,

 Any update on the above request.

From,
Vino.B

On Friday, 8 December 2017 at 19:10:09 UTC, vino wrote:
> On Thursday, 7 December 2017 at 12:19:00 UTC, Vino wrote:
>> On Thursday, 7 December 2017 at 09:04:19 UTC, Vino wrote:
>>> [...]
>>
>> Hi Andrea,
>>
>>  Was able to find a solution to the above issue by adding the replace function as below, the the code is working as expected, is there any chance of using parallel function as the file system contains about 500+ folders and the size of each folder ranges from 5GB - 500 GB so the run time of the above code is about an 2 hours.
>>
>> .map!(a => tuple(a[0].replace(",\\?\", ""), ((a[1].to!real)/ SGb ))));
>>
>> From,
>> Vino.B
>
> Hi All,
>
>  Any update on the above request.
>
> From,
> Vino.B

Hi Andrea,

  Your code is good, when compared to the performance between your code and my earlier code , my code seem to be much faster, my code execution takes 40 mins where as your code takes 2+ hours(no change in the environment) as in my code i have used parallel and tired to add the parallelism to your code but was not able to succeed. Hence i need to switch back to my code, in case if you find any method to add parallelism you your code please do let me know. Thank you very much for you help.

From,
Vino.B