September 08, 2017 Re: Container Array | ||||
---|---|---|---|---|
| ||||
Posted in reply to Vino.B | On Friday, 8 September 2017 at 12:14:46 UTC, Vino.B wrote:
> On Friday, 8 September 2017 at 09:51:38 UTC, Ali Çehreli wrote:
>> On 09/07/2017 11:21 PM, Vino.B wrote:
>>
>> > At last was able to print the output, but i am getting some
>> > "Deprecation" warnings like below and also can you help me in
>> formating
>> > the output to display ulong.
>> >
>> > Output:
>> > Size.d(9): Deprecation: std.container.array.RangeT(A) is not
>> visible
>> > from module Size
>>
>> That's due to std.container.array.RangeT being private. The deprecation warning is about a bug that leaked such private symbols when they were imported selectively (I think). Now the bug is fixed, you won't be able to access the symbol at the end of the deprecation period.
>>
>> > import std.algorithm: filter, map, fold;
>> > import std.container;
>> > import std.file: SpanMode, dirEntries, isDir;
>> > import std.stdio: File, writefln, writeln;
>> > import std.typecons: tuple, Tuple;
>> > import std.parallelism: parallel;
>> > import std.conv;
>> > import std.range;
>> > Tuple!(RangeT!(Array!(Tuple!string)), RangeT!(Array!ulong))
>> > coSizeDirList () {
>> > string FFs = "C:\\Temp\\sapnas2\\BACKUP";
>> > int SizeDir = 1;
>> > ulong subdirTotal;
>> > ulong subdirTotalGB;
>> > Array!(ulong) Subdata;
>> > auto dFiles = Array!(Tuple!(string)) (dirEntries(FFs,
>> > SpanMode.shallow).filter!(a => a.isDir).map!(a =>
>> tuple(a.name)));
>> > foreach (d; dFiles[]) {
>> > auto SdFiles = Array!(Tuple!(ulong))
>> (dirEntries(d[0],
>> > SpanMode.depth).map!(a => tuple(a.size)));
>> > foreach(f; SdFiles[]) { subdirTotal +=
>> f.fold!((a, b) =>
>> > a + b); }
>> > subdirTotalGB = (subdirTotal/1024/1024);
>> > if (subdirTotalGB > SizeDir) { Subdata ~=
>> > subdirTotalGB; }
>> > subdirTotal = 0;
>> > }
>> > return tuple (dFiles[], Subdata[]);
>> > }
>> >
>> > void main () {
>> > writeln(coSizeDirList[]);
>> > //writefln("%(%-(%-63s %)\n%)", coSizeDirList[]);
>> > }
>>
>> I apologize for not really having time to look at what you're trying to achieve. I gave you advice which ended up trying to solve compilation errors.
>>
>> I think the main problem here is to determine directories above a certain size. So, I think Array should enter the picture only if built-in arrays are not usable for some reason. Even better, one should stay with lazy range algorithms as long as it's possible.
>>
>> How about the following approach, which you can either use directly or populate an Array with:
>>
>> import std.algorithm: filter, map, sum;
>> import std.file: SpanMode, dirEntries, isDir, DirEntry;
>> import std.stdio: writeln;
>>
>> auto dFiles(string dirName) {
>> return dirEntries(dirName, SpanMode.shallow).filter!(a => a.isDir);
>> }
>>
>> auto totalSize(DirEntry dir) {
>> return dirEntries(dir, SpanMode.depth).map!(a => a.size).sum;
>> }
>>
>> struct DirInfo {
>> string name;
>> ulong size;
>> }
>>
>> auto coSizeDirList (string dirName, ulong sizeLimit) {
>> return dFiles(dirName).map!(dir => DirInfo(dir.name, dir.totalSize)).filter!(info => info.size > sizeLimit);
>> }
>>
>> void main () {
>> writeln(coSizeDirList("./deleteme", 1));
>>
>> // Only if Array is really needed:
>> import std.container : Array;
>> auto arr = Array!DirInfo(coSizeDirList("./deleteme", 42));
>> writeln(arr[]);
>> }
>>
>> Ali
>
> Hi Ali,
>
> As stated earlier my release 1 code are still using std.array, so now in release 2 i am converting all my standard array to container array. My program has 5 function and I was able to adopt 4 function to container array, and facing issue with this 1 function. I would like to have all my function to be neither in standard array nor in container array, more over I am facing gc issue in standard array and this is the main reason I would like to convert my function to container array, as this function would find the size of folder which are greater then a specified size in a 5 file system each with 10 TB, so i have added the thread(with Local storage) and parallelism to my function as container array give's me the option of reserving memory, so that i would bump it up as and when it is required with any gc issues. All help to resolve this issue would be appreciated.
>
> Updated Code:
> import std.algorithm: filter, map, fold;
> import std.container;
> import std.file: SpanMode, dirEntries, isDir;
> import std.stdio: File, writefln, writeln;
> import std.typecons: tuple, Tuple;
> import std.parallelism: parallel;
> import std.conv;
> import std.range;
>
> Tuple!(RangeT!(Array!string), RangeT!(Array!ulong)) coSizeDirList () {
> string FFs = "C:\\Temp\\sapnas2\\BACKUP";
> int SizeDir = 1;
> ulong subdirTotal;
> ulong subdirTotalGB;
> Array!(string) Subdir;
> Array!(ulong) Subsize;
> Tuple!((Array!string), (Array!string)) Result;
> auto dFiles = Array!string ((dirEntries(FFs, SpanMode.shallow).filter!(a => a.isDir))[].map!(a => a.name));
> foreach (d; dFiles[]) {
> auto SdFiles = Array!ulong ((dirEntries(d, SpanMode.depth)).map!(a => a.size));
> foreach(f; SdFiles[]) { subdirTotal += f; }
> subdirTotalGB = (subdirTotal/1024/1024); { Subdir ~= d; Subsize ~= subdirTotalGB; }
> if (subdirTotalGB > SizeDir)
> subdirTotal = 0;
> }
> return tuple (Subdir[], Subsize[]);
> }
>
> void main () {
> writeln(coSizeDirList[]);
> }
>
> From,
> Vino.B
Hi Ali,
Was able to resolve the above issue but not sure whether it is correct and now i am getting the output as below, request your help.
Output:
C:\Temp\sapnas2\BACKUP\dir1, 34, C:\Temp\sapnas2\BACKUP\DND3, 1, C:\Temp\sapnas2\BACKUP\DND5, 5
Required Output:
C:\Temp\sapnas2\BACKUP\dir1 34
C:\Temp\sapnas2\BACKUP\DND3 1
C:\Temp\sapnas2\BACKUP\DND5 5
Program:
import std.algorithm: filter, map, fold;
import std.container;
import std.file: SpanMode, dirEntries, isDir, isFile;
import std.stdio: File, writefln, writeln;
import std.typecons: tuple, Tuple;
import std.parallelism: parallel;
import std.conv;
import std.range;
Array!string coSizeDirList () {
string FFs = "C:\\Temp\\sapnas2\\BACKUP";
int SizeDir = 1;
ulong subdirTotal;
ulong subdirTotalGB;
Array!(string) Subsize;
Array!string Result;
auto dFiles = Array!string ((dirEntries(FFs, SpanMode.shallow).filter!(a => a.isDir))[].map!(a => a.name));
foreach (d; dFiles[]) {
auto SdFiles = Array!ulong(dirEntries(d, SpanMode.depth).map!(a => a.size));
foreach(f; SdFiles[]) { subdirTotal += f; }
subdirTotalGB = (subdirTotal/1024/1024); { Result ~= d; Result ~= to!string(subdirTotalGB); }
if (subdirTotalGB > SizeDir)
subdirTotal = 0;
}
return Result;
}
void main () {
writefln("%-(%s, %)", coSizeDirList[]);
}
From,
Vino.B
|
September 08, 2017 Re: Container Array | ||||
---|---|---|---|---|
| ||||
Posted in reply to Vino.B | On Friday, 8 September 2017 at 15:48:47 UTC, Vino.B wrote:
> On Friday, 8 September 2017 at 12:14:46 UTC, Vino.B wrote:
>> On Friday, 8 September 2017 at 09:51:38 UTC, Ali Çehreli wrote:
>>> [...]
>>
>> Hi Ali,
>>
>> As stated earlier my release 1 code are still using std.array, so now in release 2 i am converting all my standard array to container array. My program has 5 function and I was able to adopt 4 function to container array, and facing issue with this 1 function. I would like to have all my function to be neither in standard array nor in container array, more over I am facing gc issue in standard array and this is the main reason I would like to convert my function to container array, as this function would find the size of folder which are greater then a specified size in a 5 file system each with 10 TB, so i have added the thread(with Local storage) and parallelism to my function as container array give's me the option of reserving memory, so that i would bump it up as and when it is required with any gc issues. All help to resolve this issue would be appreciated.
>>
>> Updated Code:
>> import std.algorithm: filter, map, fold;
>> import std.container;
>> import std.file: SpanMode, dirEntries, isDir;
>> import std.stdio: File, writefln, writeln;
>> import std.typecons: tuple, Tuple;
>> import std.parallelism: parallel;
>> import std.conv;
>> import std.range;
>>
>> Tuple!(RangeT!(Array!string), RangeT!(Array!ulong)) coSizeDirList () {
>> string FFs = "C:\\Temp\\sapnas2\\BACKUP";
>> int SizeDir = 1;
>> ulong subdirTotal;
>> ulong subdirTotalGB;
>> Array!(string) Subdir;
>> Array!(ulong) Subsize;
>> Tuple!((Array!string), (Array!string)) Result;
>> auto dFiles = Array!string ((dirEntries(FFs, SpanMode.shallow).filter!(a => a.isDir))[].map!(a => a.name));
>> foreach (d; dFiles[]) {
>> auto SdFiles = Array!ulong ((dirEntries(d, SpanMode.depth)).map!(a => a.size));
>> foreach(f; SdFiles[]) { subdirTotal += f; }
>> subdirTotalGB = (subdirTotal/1024/1024); { Subdir ~= d; Subsize ~= subdirTotalGB; }
>> if (subdirTotalGB > SizeDir)
>> subdirTotal = 0;
>> }
>> return tuple (Subdir[], Subsize[]);
>> }
>>
>> void main () {
>> writeln(coSizeDirList[]);
>> }
>>
>> From,
>> Vino.B
>
> Hi Ali,
>
> Was able to resolve the above issue but not sure whether it is correct and now i am getting the output as below, request your help.
>
> Output:
> C:\Temp\sapnas2\BACKUP\dir1, 34, C:\Temp\sapnas2\BACKUP\DND3, 1, C:\Temp\sapnas2\BACKUP\DND5, 5
>
> Required Output:
> C:\Temp\sapnas2\BACKUP\dir1 34
> C:\Temp\sapnas2\BACKUP\DND3 1
> C:\Temp\sapnas2\BACKUP\DND5 5
>
> Program:
> import std.algorithm: filter, map, fold;
> import std.container;
> import std.file: SpanMode, dirEntries, isDir, isFile;
> import std.stdio: File, writefln, writeln;
> import std.typecons: tuple, Tuple;
> import std.parallelism: parallel;
> import std.conv;
> import std.range;
>
> Array!string coSizeDirList () {
> string FFs = "C:\\Temp\\sapnas2\\BACKUP";
> int SizeDir = 1;
> ulong subdirTotal;
> ulong subdirTotalGB;
> Array!(string) Subsize;
> Array!string Result;
> auto dFiles = Array!string ((dirEntries(FFs, SpanMode.shallow).filter!(a => a.isDir))[].map!(a => a.name));
> foreach (d; dFiles[]) {
> auto SdFiles = Array!ulong(dirEntries(d, SpanMode.depth).map!(a => a.size));
> foreach(f; SdFiles[]) { subdirTotal += f; }
> subdirTotalGB = (subdirTotal/1024/1024); { Result ~= d; Result ~= to!string(subdirTotalGB); }
> if (subdirTotalGB > SizeDir)
> subdirTotal = 0;
> }
> return Result;
> }
>
> void main () {
> writefln("%-(%s, %)", coSizeDirList[]);
> }
>
> From,
> Vino.B
Hi Ali,
At last was able to resolve the issue including the output too, thank you very much for your help, please let me know in case if you find any issue with the below code.
import std.algorithm: filter, map, fold;
import std.container;
import std.file: SpanMode, dirEntries, isDir, isFile;
import std.stdio: File, writefln, writeln;
import std.typecons: tuple, Tuple;
import std.parallelism: parallel;
import std.conv;
import std.range;
string[][] coSizeDirList () {
string FFs = "C:\\Temp\\sapnas2\\BACKUP";
int SizeDir = 1;
ulong subdirTotal;
ulong subdirTotalGB;
Array!(string) Subsize;
string[][] Result;
auto dFiles = Array!string ((dirEntries(FFs, SpanMode.shallow).filter!(a => a.isDir))[].map!(a => a.name));
foreach (d; dFiles[]) {
auto SdFiles = Array!ulong(dirEntries(d, SpanMode.depth).map!(a => a.size));
foreach(f; SdFiles[]) { subdirTotal += f; }
subdirTotalGB = (subdirTotal/1024/1024);
if (subdirTotalGB > SizeDir) { Result ~= [[d] ~ [to!string(subdirTotalGB)]]; }
subdirTotal = 0;
}
return Result;
}
void main () {
writefln("%(%-(%-63s %)\n%)", coSizeDirList[]);
}
From,
Vino.B
|
Copyright © 1999-2021 by the D Language Foundation