Thread overview | |||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
December 17, 2017 Fold in Parallelism | ||||
---|---|---|---|---|
| ||||
HI All, As per the document form std.parallelism it states that we can use taskPool.reduce so can we use the same for fold (taskPool.fold) as basically both are same with slight variation on seed values, if possible can can we define the same in the below lines Tried the below but getting an error auto SdFiles = Array!ulong(dirEntries("C:\\TEMP\\BACKUP", SpanMode.depth).map!(a => a.size).taskPool.fold!((a,b) => a + b) (x))[]; Error : Srvnscleaner.d(89): Error: function std.parallelism.taskPool () is not callable using argument types (MapResult!(__lambda2, DirIterator)) auto SdFiles = Array!ulong(dirEntries("C:\\TEMP\\BACKUP", SpanMode.depth).map!(a => a.size).fold!((a,b) => a + b) (x)).taskPool[]; Error : Size.d(89): Error: function std.parallelism.taskPool () is not callable using argument types (Array!ulong) From, Vino.B |
December 17, 2017 Re: Fold in Parallelism | ||||
---|---|---|---|---|
| ||||
Posted in reply to Vino | On 12/17/2017 08:11 AM, Vino wrote: > As per the document form std.parallelism it states that we can use > taskPool.reduce so can we use the same for fold (taskPool.fold) as > basically both are same with slight variation on seed values, if > possible can can we define the same in the below lines fold has only been added to std.algorithm. Opened an enhancement request: https://issues.dlang.org/show_bug.cgi?id=18096 Ali |
December 18, 2017 Re: Fold in Parallelism | ||||
---|---|---|---|---|
| ||||
Posted in reply to Ali Çehreli | On Sunday, 17 December 2017 at 20:00:53 UTC, Ali Çehreli wrote:
> On 12/17/2017 08:11 AM, Vino wrote:
>
> > As per the document form std.parallelism it states that we
> can use
> > taskPool.reduce so can we use the same for fold
> (taskPool.fold) as
> > basically both are same with slight variation on seed values,
> if
> > possible can can we define the same in the below lines
>
> fold has only been added to std.algorithm. Opened an enhancement request:
>
> https://issues.dlang.org/show_bug.cgi?id=18096
>
> Ali
Hi Ali,
Thank you very much, may we know if possible as to when this would be added.
From,
Vino.B
|
December 18, 2017 Re: Fold in Parallelism | ||||
---|---|---|---|---|
| ||||
Posted in reply to Vino | On 12/18/2017 02:18 AM, Vino wrote:
> On Sunday, 17 December 2017 at 20:00:53 UTC, Ali Çehreli wrote:
>> On 12/17/2017 08:11 AM, Vino wrote:
>>
>> > As per the document form std.parallelism it states that we
>> can use
>> > taskPool.reduce so can we use the same for fold
>> (taskPool.fold) as
>> > basically both are same with slight variation on seed values,
>> if
>> > possible can can we define the same in the below lines
>>
>> fold has only been added to std.algorithm. Opened an enhancement request:
>>
>> https://issues.dlang.org/show_bug.cgi?id=18096
>>
>> Ali
>
> Hi Ali,
>
> Thank you very much, may we know if possible as to when this would be added.
>
> From,
> Vino.B
The implementation is almost a copy+paste from std.algorithm.fold. You can simply copy the following fold template to your project and start using it:
import std.parallelism;
import std.stdio;
int adder(int result, int element) {
return result + element;
}
template fold(fun...)
if (fun.length >= 1)
{
import std.parallelism : TaskPool;
auto fold(R, S...)(TaskPool t, R r, S seed)
{
static if (S.length < 2)
{
return t.reduce!fun(seed, r);
}
else
{
import std.typecons : tuple;
return t.reduce!fun(tuple(seed), r);
}
}
}
unittest {
import std.range;
const N = 10_000;
const expected = N * (N - 1) / 2;
foreach (numThreads; 1 .. 100) {
auto t = new TaskPool(numThreads);
const result = t.fold!adder(N.iota);
assert(result == expected);
t.finish();
}
}
void main() {
}
If you want to provide an explicit seed value, good luck making sense of how it affects the final result. :) (The documentation of TaskPool.reduce explains it but I find it too involved to understand.)
Ali
|
December 18, 2017 Re: Fold in Parallelism | ||||
---|---|---|---|---|
| ||||
Posted in reply to Ali Çehreli Attachments:
| Ali, Shouldn't this be a pull request for std.parallelism to be extended? If the function is in std.algorithm, then people should not have to write it for themselves in std.parallelism. On Mon, 2017-12-18 at 11:01 -0800, Ali Çehreli via Digitalmars-d-learn wrote: > […] > > Hi Ali, > > > > Thank you very much, may we know if possible as to when this > > would be > > added. > > > > From, > > Vino.B > > The implementation is almost a copy+paste from std.algorithm.fold. > You > can simply copy the following fold template to your project and > start > using it: > > import std.parallelism; > import std.stdio; > > int adder(int result, int element) { > return result + element; > } > > template fold(fun...) > if (fun.length >= 1) > { > import std.parallelism : TaskPool; > auto fold(R, S...)(TaskPool t, R r, S seed) > { > static if (S.length < 2) > { > return t.reduce!fun(seed, r); > } > else > { > import std.typecons : tuple; > return t.reduce!fun(tuple(seed), r); > } > } > } > > unittest { > import std.range; > > const N = 10_000; > const expected = N * (N - 1) / 2; > > foreach (numThreads; 1 .. 100) { > auto t = new TaskPool(numThreads); > const result = t.fold!adder(N.iota); > assert(result == expected); > t.finish(); > } > } > > void main() { > } > > If you want to provide an explicit seed value, good luck making sense > of > how it affects the final result. :) (The documentation of > TaskPool.reduce explains it but I find it too involved to > understand.) > > Ali -- Russel. =========================================== Dr Russel Winder t: +44 20 7585 2200 41 Buckmaster Road m: +44 7770 465 077 London SW11 1EN, UK w: www.russel.org.uk |
December 19, 2017 Re: Fold in Parallelism | ||||
---|---|---|---|---|
| ||||
Posted in reply to Russel Winder | On Monday, 18 December 2017 at 20:53:28 UTC, Russel Winder wrote:
> Ali,
>
> Shouldn't this be a pull request for std.parallelism to be extended?
>
> If the function is in std.algorithm, then people should not have to write it for themselves in std.parallelism.
>
>
> On Mon, 2017-12-18 at 11:01 -0800, Ali Çehreli via Digitalmars-d-learn wrote:
>> [...]
> […]
>> [...]
Hi Ali,
Sorry, I would like to echo the statement from Russel, as an user(newbie) I personally do not want to fiddle around the libraries even though it is a simple code copy+paste, as this might cause some serious issue in case if anything went wrong, so i would like to leave it to expert's like you to help us. If possible can we added this to the next release (78).
From,
Vino.B.
|
December 20, 2017 Re: Fold in Parallelism | ||||
---|---|---|---|---|
| ||||
Posted in reply to Vino | On 12/19/2017 02:32 AM, Vino wrote: > even though it is a simple code copy+paste The change was a little more complicated than my naive adaptation from std.algorithm.fold. Here is the pull request: https://github.com/dlang/phobos/pull/5951 Ali |
December 21, 2017 Re: Fold in Parallelism | ||||
---|---|---|---|---|
| ||||
Posted in reply to Ali Çehreli Attachments:
| On Wed, 2017-12-20 at 22:31 -0800, Ali Çehreli via Digitalmars-d-learn wrote: > On 12/19/2017 02:32 AM, Vino wrote: > > > even though it is a simple code copy+paste > > The change was a little more complicated than my naive adaptation > from > std.algorithm.fold. Here is the pull request: > > https://github.com/dlang/phobos/pull/5951 > > Ali Thanks for doing this. Having consistency is a good thing. -- Russel. =========================================== Dr Russel Winder t: +44 20 7585 2200 41 Buckmaster Road m: +44 7770 465 077 London SW11 1EN, UK w: www.russel.org.uk |
December 22, 2017 Re: Fold in Parallelism | ||||
---|---|---|---|---|
| ||||
Posted in reply to Ali Çehreli | On Thursday, 21 December 2017 at 06:31:52 UTC, Ali Çehreli wrote:
> On 12/19/2017 02:32 AM, Vino wrote:
>
> > even though it is a simple code copy+paste
>
> The change was a little more complicated than my naive adaptation from std.algorithm.fold. Here is the pull request:
>
> https://github.com/dlang/phobos/pull/5951
>
> Ali
Hi Ali,
Thank you very much, the pull request is in open state, so can you please let me know when can we can test the same.
From,
Vino.B
|
December 22, 2017 Re: Fold in Parallelism | ||||
---|---|---|---|---|
| ||||
Posted in reply to Vino | On Friday, 22 December 2017 at 00:12:45 UTC, Vino wrote:
> On Thursday, 21 December 2017 at 06:31:52 UTC, Ali Çehreli wrote:
>> On 12/19/2017 02:32 AM, Vino wrote:
>>
>> > even though it is a simple code copy+paste
>>
>> The change was a little more complicated than my naive adaptation from std.algorithm.fold. Here is the pull request:
>>
>> https://github.com/dlang/phobos/pull/5951
>>
>> Ali
>
> Hi Ali,
>
> Thank you very much, the pull request is in open state, so can you please let me know when can we can test the same.
>
> From,
> Vino.B
It will take a couple of days for this pull request to reach a ready form and to be approved. The best way to help it to move forward is to review it yourself or at least vote for it on GitHub ;-)
Once merged it will appear on dmd-nightly on the next day, but I'm not sure why you depend on his pull request?
Can't you just use the code directly?
|
Copyright © 1999-2021 by the D Language Foundation