Thread overview
ndslice, using a slice in place of T[] in template parameters
Jan 10, 2016
Jay Norwood
Jan 10, 2016
Ilya Yaroshenko
Jan 10, 2016
Jay Norwood
Jan 10, 2016
Ilya Yaroshenko
Jan 11, 2016
Jay Norwood
Jan 11, 2016
Ilya Yaroshenko
Jan 11, 2016
Jay Norwood
Jan 11, 2016
Ilya Yaroshenko
January 10, 2016
I cut this median template from Jack Stouffer's article and was attempting to use it in  a parallel function.  As shown, it builds and execute correctly, but it failed to compile if I attempting to use
medians[i] = median(vec,slb[task]);

in place of the
medians[i] = median(vec,dbuf[j .. k]);

Is there a cast needed?

================
import std.array : array;
import std.algorithm;
import std.datetime;
import std.conv : to;
import std.stdio;
import std.experimental.ndslice;

shared double[] medians;
double[] data;
shared double[] dbuf;
int numTasks;
const int smalld = 1000;
const int bigd = 10_000;
const int fulld = bigd*smalld;

/**
Params:
r = input range
buf = buffer with length no less than the number of elements in `r`
Returns:
median value over the range `r`
*/
T median(Range, T)(Range r, T[] buf)
{
    import std.algorithm.sorting: sort;

    size_t n;

    foreach (e; r) {
        buf[n++] = e;
    }

    buf[0 .. n].sort();
    immutable m = n >> 1;
    return n & 1 ? buf[m] : cast(T)((buf[m - 1] + buf[m]) / 2);
}


void f3() {
    import std.parallelism;
    auto sl = data.sliced(smalld,bigd);
    auto slb = dbuf.sliced(numTasks,bigd);
    foreach(i,vec; parallel(sl)){
        int task = taskPool.workerIndex;
        int j = task*bigd;
        int k = j+bigd;
        medians[i] = median(vec,dbuf[j .. k]);
    }
}


void main() {
    import std.parallelism;
    numTasks = taskPool.size+1;
    data = new double[fulld];
    dbuf = new double[bigd*numTasks];
    medians = new double[smalld];

    for(int i=0;i<fulld;i++){ data[i] = i/(fulld*1.0);}
    StopWatch sw3;

    sw3.start();
    f3() ;
    auto r3 = sw3.peek().msecs;
    writeln("medians parallel:",medians);

    writeln("parallel time medians msec:",r3);
}
January 10, 2016
On Sunday, 10 January 2016 at 22:00:20 UTC, Jay Norwood wrote:
> I cut this median template from Jack Stouffer's article and was attempting to use it in  a parallel function.  As shown, it builds and execute correctly, but it failed to compile if I attempting to use
> medians[i] = median(vec,slb[task]);
>
> [...]

Could you please provide full code and error (git gists)? -- Ilya
January 10, 2016
On Sunday, 10 January 2016 at 22:23:18 UTC, Ilya Yaroshenko wrote:

> Could you please provide full code and error (git gists)? -- Ilya

ok, thanks.
I'm building with DMD32 D Compiler v2.069.2 on Win32.  The dub.json is included.

https://gist.github.com/jnorwood/affd05b69795c20989a3

January 10, 2016
On Sunday, 10 January 2016 at 23:24:24 UTC, Jay Norwood wrote:
> On Sunday, 10 January 2016 at 22:23:18 UTC, Ilya Yaroshenko wrote:
>
>> Could you please provide full code and error (git gists)? -- Ilya
>
> ok, thanks.
> I'm building with DMD32 D Compiler v2.069.2 on Win32.  The dub.json is included.
>
> https://gist.github.com/jnorwood/affd05b69795c20989a3

Just use normal arrays for buffer (median accepts array on second argument for optimisation reasons).

BTW, dip80-ndslice moved to http://code.dlang.org/packages/mir
-- Ilya
January 11, 2016
On Sunday, 10 January 2016 at 23:24:24 UTC, Jay Norwood wrote:
> On Sunday, 10 January 2016 at 22:23:18 UTC, Ilya Yaroshenko wrote:
>
>> Could you please provide full code and error (git gists)? -- Ilya
>
> ok, thanks.
> I'm building with DMD32 D Compiler v2.069.2 on Win32.  The dub.json is included.
>
> https://gist.github.com/jnorwood/affd05b69795c20989a3

I have create parallel test to (it requires mir v0.10.0-beta )
https://github.com/9il/examples/blob/parallel/image_processing/median-filter/source/app.d
Could you please create a benchmark with default values of nc & nc for single thread app, your parallel version, and my.
My version has some additional overhead and I am interesting if it is significant.
-- Ilya
January 11, 2016
On Sunday, 10 January 2016 at 23:31:47 UTC, Ilya Yaroshenko wrote:
> Just use normal arrays for buffer (median accepts array on second argument for optimisation reasons).

ok, I think I see. I created a slice(numTasks, bigd) over an allocated double[] dbuf, but slb[task] will be returning some struct instead of the double[] that i need in this case.

If I add .array to the Slice, it does compile, and executes, but slower than using the buffer directly.

medians[i] = median(vec, slb[task].array);
parallel time medians msec:113

original version using the computed slice of the original allocated dbuf.
medians[i] = median(vec,dbuf[j .. k]);
parallel time medians msec:85

The .array appears to make a copy. Is there some other call in ndslice to return the double[] slice of the original array?

January 11, 2016
On Monday, 11 January 2016 at 00:39:04 UTC, Jay Norwood wrote:
> On Sunday, 10 January 2016 at 23:31:47 UTC, Ilya Yaroshenko wrote:
>> Just use normal arrays for buffer (median accepts array on second argument for optimisation reasons).
>
> ok, I think I see. I created a slice(numTasks, bigd) over an allocated double[] dbuf, but slb[task] will be returning some struct instead of the double[] that i need in this case.
>
> If I add .array to the Slice, it does compile, and executes, but slower than using the buffer directly.
>
> medians[i] = median(vec, slb[task].array);
> parallel time medians msec:113
>
> original version using the computed slice of the original allocated dbuf.
> medians[i] = median(vec,dbuf[j .. k]);
> parallel time medians msec:85
>
> The .array appears to make a copy. Is there some other call in ndslice to return the double[] slice of the original array?

I will add such function. But it is not safe to do so (Slice can have strides not equal to 1). So it is like a hack (&ret[0, 0, 0])[0 .. ret.elementsCount]).

Have you made comparison between my and yours parallel versions?
https://github.com/9il/examples/blob/parallel/image_processing/median-filter/source/app.d
-- Ilya

January 11, 2016
On Monday, 11 January 2016 at 00:50:37 UTC, Ilya Yaroshenko wrote:
> I will add such function. But it is not safe to do so (Slice can have strides not equal to 1). So it is like a hack (&ret[0, 0, 0])[0 .. ret.elementsCount]).
>
> Have you made comparison between my and yours parallel versions?
> https://github.com/9il/examples/blob/parallel/image_processing/median-filter/source/app.d
> -- Ilya

Thanks.  No, I haven't studied it previously, but I see how you used the 'hack' in your code, and it works out to the statement below in my case.

medians[i] = median(vec, (&slb[task,0])[0 .. bigd]);

which compiled.  It ran in the faster time without the .array copying.
parallel time medians msec:87

That 'hack' seems to be related to the third from below.
https://dlang.org/spec/arrays.html
b = a;
b = a[];
b = a[0 .. a.length];