Thread overview | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|
|
January 10, 2016 ndslice, using a slice in place of T[] in template parameters | ||||
---|---|---|---|---|
| ||||
I cut this median template from Jack Stouffer's article and was attempting to use it in a parallel function. As shown, it builds and execute correctly, but it failed to compile if I attempting to use medians[i] = median(vec,slb[task]); in place of the medians[i] = median(vec,dbuf[j .. k]); Is there a cast needed? ================ import std.array : array; import std.algorithm; import std.datetime; import std.conv : to; import std.stdio; import std.experimental.ndslice; shared double[] medians; double[] data; shared double[] dbuf; int numTasks; const int smalld = 1000; const int bigd = 10_000; const int fulld = bigd*smalld; /** Params: r = input range buf = buffer with length no less than the number of elements in `r` Returns: median value over the range `r` */ T median(Range, T)(Range r, T[] buf) { import std.algorithm.sorting: sort; size_t n; foreach (e; r) { buf[n++] = e; } buf[0 .. n].sort(); immutable m = n >> 1; return n & 1 ? buf[m] : cast(T)((buf[m - 1] + buf[m]) / 2); } void f3() { import std.parallelism; auto sl = data.sliced(smalld,bigd); auto slb = dbuf.sliced(numTasks,bigd); foreach(i,vec; parallel(sl)){ int task = taskPool.workerIndex; int j = task*bigd; int k = j+bigd; medians[i] = median(vec,dbuf[j .. k]); } } void main() { import std.parallelism; numTasks = taskPool.size+1; data = new double[fulld]; dbuf = new double[bigd*numTasks]; medians = new double[smalld]; for(int i=0;i<fulld;i++){ data[i] = i/(fulld*1.0);} StopWatch sw3; sw3.start(); f3() ; auto r3 = sw3.peek().msecs; writeln("medians parallel:",medians); writeln("parallel time medians msec:",r3); } |
January 10, 2016 Re: ndslice, using a slice in place of T[] in template parameters | ||||
---|---|---|---|---|
| ||||
Posted in reply to Jay Norwood | On Sunday, 10 January 2016 at 22:00:20 UTC, Jay Norwood wrote:
> I cut this median template from Jack Stouffer's article and was attempting to use it in a parallel function. As shown, it builds and execute correctly, but it failed to compile if I attempting to use
> medians[i] = median(vec,slb[task]);
>
> [...]
Could you please provide full code and error (git gists)? -- Ilya
|
January 10, 2016 Re: ndslice, using a slice in place of T[] in template parameters | ||||
---|---|---|---|---|
| ||||
Posted in reply to Ilya Yaroshenko | On Sunday, 10 January 2016 at 22:23:18 UTC, Ilya Yaroshenko wrote: > Could you please provide full code and error (git gists)? -- Ilya ok, thanks. I'm building with DMD32 D Compiler v2.069.2 on Win32. The dub.json is included. https://gist.github.com/jnorwood/affd05b69795c20989a3 |
January 10, 2016 Re: ndslice, using a slice in place of T[] in template parameters | ||||
---|---|---|---|---|
| ||||
Posted in reply to Jay Norwood | On Sunday, 10 January 2016 at 23:24:24 UTC, Jay Norwood wrote: > On Sunday, 10 January 2016 at 22:23:18 UTC, Ilya Yaroshenko wrote: > >> Could you please provide full code and error (git gists)? -- Ilya > > ok, thanks. > I'm building with DMD32 D Compiler v2.069.2 on Win32. The dub.json is included. > > https://gist.github.com/jnorwood/affd05b69795c20989a3 Just use normal arrays for buffer (median accepts array on second argument for optimisation reasons). BTW, dip80-ndslice moved to http://code.dlang.org/packages/mir -- Ilya |
January 11, 2016 Re: ndslice, using a slice in place of T[] in template parameters | ||||
---|---|---|---|---|
| ||||
Posted in reply to Jay Norwood | On Sunday, 10 January 2016 at 23:24:24 UTC, Jay Norwood wrote: > On Sunday, 10 January 2016 at 22:23:18 UTC, Ilya Yaroshenko wrote: > >> Could you please provide full code and error (git gists)? -- Ilya > > ok, thanks. > I'm building with DMD32 D Compiler v2.069.2 on Win32. The dub.json is included. > > https://gist.github.com/jnorwood/affd05b69795c20989a3 I have create parallel test to (it requires mir v0.10.0-beta ) https://github.com/9il/examples/blob/parallel/image_processing/median-filter/source/app.d Could you please create a benchmark with default values of nc & nc for single thread app, your parallel version, and my. My version has some additional overhead and I am interesting if it is significant. -- Ilya |
January 11, 2016 Re: ndslice, using a slice in place of T[] in template parameters | ||||
---|---|---|---|---|
| ||||
Posted in reply to Ilya Yaroshenko | On Sunday, 10 January 2016 at 23:31:47 UTC, Ilya Yaroshenko wrote:
> Just use normal arrays for buffer (median accepts array on second argument for optimisation reasons).
ok, I think I see. I created a slice(numTasks, bigd) over an allocated double[] dbuf, but slb[task] will be returning some struct instead of the double[] that i need in this case.
If I add .array to the Slice, it does compile, and executes, but slower than using the buffer directly.
medians[i] = median(vec, slb[task].array);
parallel time medians msec:113
original version using the computed slice of the original allocated dbuf.
medians[i] = median(vec,dbuf[j .. k]);
parallel time medians msec:85
The .array appears to make a copy. Is there some other call in ndslice to return the double[] slice of the original array?
|
January 11, 2016 Re: ndslice, using a slice in place of T[] in template parameters | ||||
---|---|---|---|---|
| ||||
Posted in reply to Jay Norwood | On Monday, 11 January 2016 at 00:39:04 UTC, Jay Norwood wrote: > On Sunday, 10 January 2016 at 23:31:47 UTC, Ilya Yaroshenko wrote: >> Just use normal arrays for buffer (median accepts array on second argument for optimisation reasons). > > ok, I think I see. I created a slice(numTasks, bigd) over an allocated double[] dbuf, but slb[task] will be returning some struct instead of the double[] that i need in this case. > > If I add .array to the Slice, it does compile, and executes, but slower than using the buffer directly. > > medians[i] = median(vec, slb[task].array); > parallel time medians msec:113 > > original version using the computed slice of the original allocated dbuf. > medians[i] = median(vec,dbuf[j .. k]); > parallel time medians msec:85 > > The .array appears to make a copy. Is there some other call in ndslice to return the double[] slice of the original array? I will add such function. But it is not safe to do so (Slice can have strides not equal to 1). So it is like a hack (&ret[0, 0, 0])[0 .. ret.elementsCount]). Have you made comparison between my and yours parallel versions? https://github.com/9il/examples/blob/parallel/image_processing/median-filter/source/app.d -- Ilya |
January 11, 2016 Re: ndslice, using a slice in place of T[] in template parameters | ||||
---|---|---|---|---|
| ||||
Posted in reply to Ilya Yaroshenko | On Monday, 11 January 2016 at 00:50:37 UTC, Ilya Yaroshenko wrote: > I will add such function. But it is not safe to do so (Slice can have strides not equal to 1). So it is like a hack (&ret[0, 0, 0])[0 .. ret.elementsCount]). > > Have you made comparison between my and yours parallel versions? > https://github.com/9il/examples/blob/parallel/image_processing/median-filter/source/app.d > -- Ilya Thanks. No, I haven't studied it previously, but I see how you used the 'hack' in your code, and it works out to the statement below in my case. medians[i] = median(vec, (&slb[task,0])[0 .. bigd]); which compiled. It ran in the faster time without the .array copying. parallel time medians msec:87 That 'hack' seems to be related to the third from below. https://dlang.org/spec/arrays.html b = a; b = a[]; b = a[0 .. a.length]; |
Copyright © 1999-2021 by the D Language Foundation