Thread overview | |||||
---|---|---|---|---|---|
|
December 07, 2020 how to print progress of a long running parallel() loop? | ||||
---|---|---|---|---|
| ||||
https://dlang.org/phobos/std_parallelism.html#.parallel auto logs = new double[1_000_000]; foreach (i, ref elem; parallel(logs)) { elem = log(i + 1.0); } In Python, using joblib, I can use `verbose` level like this: r = Parallel(n_jobs=2, verbose=10)(delayed(sleep)(.2) for _ in range(10)) to print out the progress. How to do this in D's parallel loop? thanks. |
December 07, 2020 Re: how to print progress of a long running parallel() loop? | ||||
---|---|---|---|---|
| ||||
Posted in reply to mw | On Monday, 7 December 2020 at 08:16:50 UTC, mw wrote:
> r = Parallel(n_jobs=2, verbose=10)(delayed(sleep)(.2) for _ in range(10))
>
> to print out the progress.
>
> How to do this in D's parallel loop?
>
> thanks.
Allocate a `shared int` before the foreach loop. In the loop when, let's say `!(i & 0xFFF)`, atomically increment the shared variable and print it's number.
Disclaimer: no experience about using `shared`, but this is what I'd try.
|
December 07, 2020 Re: how to print progress of a long running parallel() loop? | ||||
---|---|---|---|---|
| ||||
Posted in reply to Dukc | On 12/7/20 4:48 AM, Dukc wrote: > On Monday, 7 December 2020 at 08:16:50 UTC, mw wrote: >> r = Parallel(n_jobs=2, verbose=10)(delayed(sleep)(.2) for _ in range(10)) >> >> to print out the progress. >> >> How to do this in D's parallel loop? >> >> thanks. > > Allocate a `shared int` before the foreach loop. In the loop when, let's say `!(i & 0xFFF)`, atomically increment the shared variable and print it's number. > > Disclaimer: no experience about using `shared`, but this is what I'd try. Yes, that would be the way but I don't know the effect of using shared or not. But because the progress needs to be somehow reported, an atomic variable may not be sufficient (unless a separate thread is reading it, etc.) This issue came up during one of my recent presentations at this point: https://www.youtube.com/watch?v=dRORNQIB2wA&feature=youtu.be&t=1527 I used a synchronized block in that code because it did not affect performance in my use case. Ali |
Copyright © 1999-2021 by the D Language Foundation