Thread overview | |||||||
---|---|---|---|---|---|---|---|
|
May 18, 2012 Distributed work load | ||||
---|---|---|---|---|
| ||||
I'm working on a tool which reads a an unknown number of files, does some processing on the content and writes out the result to new files on disk. The processing of the content is completely independent of any other processing, therefore I thought it might be a good idea to do this in parallel. So what I basically want is something like this:
foreach (file ; files)
{
executInParallel((file) {
auto content = read(file);
process(content);
write(newFile);
});
}
"executInParallel" would then distribute this on an appropriate amount of threads and cores.
Is this what std.parallelism does ?
--
/Jacob Carlborg
|
May 18, 2012 Re: Distributed work load | ||||
---|---|---|---|---|
| ||||
Posted in reply to Jacob Carlborg | On 18.05.2012 17:16, Jacob Carlborg wrote: > I'm working on a tool which reads a an unknown number of files, does > some processing on the content and writes out the result to new files on > disk. The processing of the content is completely independent of any > other processing, therefore I thought it might be a good idea to do this > in parallel. So what I basically want is something like this: > > foreach (file ; files) > { > executInParallel((file) { > auto content = read(file); > process(content); > write(newFile); > }); > } > > "executInParallel" would then distribute this on an appropriate amount > of threads and cores. > > Is this what std.parallelism does ? > Yes. Start with: foreach (file; parallel(files)) { ... } -- Dmitry Olshansky |
May 18, 2012 Re: Distributed work load | ||||
---|---|---|---|---|
| ||||
Posted in reply to Dmitry Olshansky | On 2012-05-18 16:56, Dmitry Olshansky wrote: > Yes. Start with: > foreach (file; parallel(files)) > { > ... > } > Aha, I was looking at "task". Thanks. -- /Jacob Carlborg |
May 18, 2012 Re: Distributed work load | ||||
---|---|---|---|---|
| ||||
Posted in reply to Jacob Carlborg | On 05/18/2012 06:16 AM, Jacob Carlborg wrote: > I'm working on a tool which reads a an unknown number of files, does > some processing on the content and writes out the result to new files on > disk. The processing of the content is completely independent of any > other processing, therefore I thought it might be a good idea to do this > in parallel. So what I basically want is something like this: > > foreach (file ; files) > { > executInParallel((file) { > auto content = read(file); > process(content); > write(newFile); > }); > } > > "executInParallel" would then distribute this on an appropriate amount > of threads and cores. > > Is this what std.parallelism does ? > I have tried to summarize what parallel() and its sisters do here: http://ddili.org/ders/d.en/parallelism.html I hope at least the Summary section at the end is a useful cheat sheet. Ali -- D Programming Language Tutorial: http://ddili.org/ders/d.en/index.html |
May 18, 2012 Re: Distributed work load | ||||
---|---|---|---|---|
| ||||
Posted in reply to Ali Çehreli | On 2012-05-18 20:44, Ali Çehreli wrote: > I have tried to summarize what parallel() and its sisters do here: > > http://ddili.org/ders/d.en/parallelism.html > > I hope at least the Summary section at the end is a useful cheat sheet. > > Ali > I'lL have to take a look at that. -- /Jacob Carlborg |
Copyright © 1999-2021 by the D Language Foundation