Thread overview
[dmd-concurrency] How to use parallel foreach to process a part of array?
Oct 29, 2013
Ilya Sazonov
Oct 29, 2013
Brad Roberts
Oct 30, 2013
Ilya Sazonov
October 29, 2013
Suppose I have the following piece of code

import std.parallelism;

void main() {
    int[] data = new int[1000_000_00];

    foreach(i, ref el; parallel(data)) {
        if (i > 10 && i < 1000_000_00 - 10)
            data[i] = data[i-10] + data[i+10];
    }
}

That's right, I want to process all the elements of the array, except for
first 10 and last 10.
Is there a way to do that without if statement?

I mean something like foreach starting from element with index 10 and ending with lendth - 10


October 29, 2013
Slice it.

foreach(i, ref el; parallel(data[10..$-10]))

-Steve

On Oct 29, 2013, at 3:49 AM, Ilya Sazonov <poxvuibr@gmail.com> wrote:

> Suppose I have the following piece of code
> 
> import std.parallelism;
> 
> void main() {
>     int[] data = new int[1000_000_00];
> 
>     foreach(i, ref el; parallel(data)) {
>         if (i > 10 && i < 1000_000_00 - 10)
>             data[i] = data[i-10] + data[i+10];
>     }
> }
> 
> That's right, I want to process all the elements of the array, except for first 10 and last 10. Is there a way to do that without if statement?
> 
> I mean something like foreach starting from element with index 10 and ending with lendth - 10
> 
> _______________________________________________
> dmd-concurrency mailing list
> dmd-concurrency@puremagic.com
> http://lists.puremagic.com/mailman/listinfo/dmd-concurrency

_______________________________________________
dmd-concurrency mailing list
dmd-concurrency@puremagic.com
http://lists.puremagic.com/mailman/listinfo/dmd-concurrency

October 29, 2013
That will eliminate the if, but over looks a larger issue.. parallel assumes that there's no interdependencies and that the order of execution is irrelevant.  That's very much not the case in the code below.  Each value in that code depends on other elements.  Unless you notice that all values are 0 and can thus just be replaced with the foreach entirely removed.

On 10/29/13 6:41 AM, Steven Schveighoffer wrote:
> Slice it.
>
> foreach(i, ref el; parallel(data[10..$-10]))
>
> -Steve
>
> On Oct 29, 2013, at 3:49 AM, Ilya Sazonov <poxvuibr@gmail.com> wrote:
>
>> Suppose I have the following piece of code
>>
>> import std.parallelism;
>>
>> void main() {
>>      int[] data = new int[1000_000_00];
>>
>>      foreach(i, ref el; parallel(data)) {
>>          if (i > 10 && i < 1000_000_00 - 10)
>>              data[i] = data[i-10] + data[i+10];
>>      }
>> }
>>
>> That's right, I want to process all the elements of the array, except for first 10 and last 10.
>> Is there a way to do that without if statement?
>>
>> I mean something like foreach starting from element with index 10 and ending with lendth - 10
>>

_______________________________________________
dmd-concurrency mailing list
dmd-concurrency@puremagic.com
http://lists.puremagic.com/mailman/listinfo/dmd-concurrency

October 29, 2013
Right, I didn't check over the full code.

The code is broken with or without my change, but with my change it's shorter broken code :)

-Steve

On Oct 29, 2013, at 4:33 PM, Brad Roberts <braddr@puremagic.com> wrote:

> That will eliminate the if, but over looks a larger issue.. parallel assumes that there's no interdependencies and that the order of execution is irrelevant.  That's very much not the case in the code below.  Each value in that code depends on other elements.  Unless you notice that all values are 0 and can thus just be replaced with the foreach entirely removed.
> 
> On 10/29/13 6:41 AM, Steven Schveighoffer wrote:
>> Slice it.
>> 
>> foreach(i, ref el; parallel(data[10..$-10]))
>> 
>> -Steve
>> 
>> On Oct 29, 2013, at 3:49 AM, Ilya Sazonov <poxvuibr@gmail.com> wrote:
>> 
>>> Suppose I have the following piece of code
>>> 
>>> import std.parallelism;
>>> 
>>> void main() {
>>>     int[] data = new int[1000_000_00];
>>> 
>>>     foreach(i, ref el; parallel(data)) {
>>>         if (i > 10 && i < 1000_000_00 - 10)
>>>             data[i] = data[i-10] + data[i+10];
>>>     }
>>> }
>>> 
>>> That's right, I want to process all the elements of the array, except for first 10 and last 10. Is there a way to do that without if statement?
>>> 
>>> I mean something like foreach starting from element with index 10 and ending with lendth - 10
>>> 
> 
> _______________________________________________
> dmd-concurrency mailing list
> dmd-concurrency@puremagic.com
> http://lists.puremagic.com/mailman/listinfo/dmd-concurrency

_______________________________________________
dmd-concurrency mailing list
dmd-concurrency@puremagic.com
http://lists.puremagic.com/mailman/listinfo/dmd-concurrency

October 30, 2013
Sorry, i made the code as short as I could to ask the question and it looks
broken now :(.
It should have been something like:

import std.parallelism;

void main() {
    int[] data = new int[1000_000_00];
    int[] new_data = new int[1000_000_00];

    foreach(i, ref el; parallel(new_data)) {
        if (i > 10 && i < 1000_000_00 - 10)
            el = data[i-10] + data[i+10];
    }
}

I read from one array and write results to another.


Thank you for your answer, it solved my issue, I didn't know about slices.
Am I right, than I'll have to add an offset to index, because foreach
always starts with zero?
Like that?
    foreach(i, ref el; parallel(new_data[10..$-10)) {
         el = data[i + 10 -10] + data[i + 10 +10];
    }

Or there is some way to start foreach with specified index?



On Wed, Oct 30, 2013 at 12:56 AM, Steven Schveighoffer <schveiguy@yahoo.com>wrote:

> Right, I didn't check over the full code.
>
> The code is broken with or without my change, but with my change it's shorter broken code :)
>
> -Steve
>
> On Oct 29, 2013, at 4:33 PM, Brad Roberts <braddr@puremagic.com> wrote:
>
> > That will eliminate the if, but over looks a larger issue.. parallel
> assumes that there's no interdependencies and that the order of execution is irrelevant.  That's very much not the case in the code below.  Each value in that code depends on other elements.  Unless you notice that all values are 0 and can thus just be replaced with the foreach entirely removed.
> >
> > On 10/29/13 6:41 AM, Steven Schveighoffer wrote:
> >> Slice it.
> >>
> >> foreach(i, ref el; parallel(data[10..$-10]))
> >>
> >> -Steve
> >>
> >> On Oct 29, 2013, at 3:49 AM, Ilya Sazonov <poxvuibr@gmail.com> wrote:
> >>
> >>> Suppose I have the following piece of code
> >>>
> >>> import std.parallelism;
> >>>
> >>> void main() {
> >>>     int[] data = new int[1000_000_00];
> >>>
> >>>     foreach(i, ref el; parallel(data)) {
> >>>         if (i > 10 && i < 1000_000_00 - 10)
> >>>             data[i] = data[i-10] + data[i+10];
> >>>     }
> >>> }
> >>>
> >>> That's right, I want to process all the elements of the array, except
> for first 10 and last 10.
> >>> Is there a way to do that without if statement?
> >>>
> >>> I mean something like foreach starting from element with index 10 and
> ending with lendth - 10
> >>>
> >
> > _______________________________________________
> > dmd-concurrency mailing list
> > dmd-concurrency@puremagic.com
> > http://lists.puremagic.com/mailman/listinfo/dmd-concurrency
>
> _______________________________________________
> dmd-concurrency mailing list
> dmd-concurrency@puremagic.com
> http://lists.puremagic.com/mailman/listinfo/dmd-concurrency
>


October 30, 2013
Correct

-Steve

> On Oct 30, 2013, at 3:18 AM, Ilya Sazonov <poxvuibr@gmail.com> wrote:
> 
> Sorry, i made the code as short as I could to ask the question and it looks broken now :(. It should have been something like:
> 
> import std.parallelism;
> 
> void main() {
>     int[] data = new int[1000_000_00];
>     int[] new_data = new int[1000_000_00];
> 
>     foreach(i, ref el; parallel(new_data)) {
>         if (i > 10 && i < 1000_000_00 - 10)
>             el = data[i-10] + data[i+10];
>     }
> }
> 
> I read from one array and write results to another.
> 
> 
> Thank you for your answer, it solved my issue, I didn't know about slices.
> Am I right, than I'll have to add an offset to index, because foreach always starts with zero?
> Like that?
>     foreach(i, ref el; parallel(new_data[10..$-10)) {
>          el = data[i + 10 -10] + data[i + 10 +10];
>     }
> 
> Or there is some way to start foreach with specified index?
> 
> 
> 
>> On Wed, Oct 30, 2013 at 12:56 AM, Steven Schveighoffer <schveiguy@yahoo.com> wrote: Right, I didn't check over the full code.
>> 
>> The code is broken with or without my change, but with my change it's shorter broken code :)
>> 
>> -Steve
>> 
>> On Oct 29, 2013, at 4:33 PM, Brad Roberts <braddr@puremagic.com> wrote:
>> 
>> > That will eliminate the if, but over looks a larger issue.. parallel assumes that there's no interdependencies and that the order of execution is irrelevant.  That's very much not the case in the code below.  Each value in that code depends on other elements.  Unless you notice that all values are 0 and can thus just be replaced with the foreach entirely removed.
>> >
>> > On 10/29/13 6:41 AM, Steven Schveighoffer wrote:
>> >> Slice it.
>> >>
>> >> foreach(i, ref el; parallel(data[10..$-10]))
>> >>
>> >> -Steve
>> >>
>> >> On Oct 29, 2013, at 3:49 AM, Ilya Sazonov <poxvuibr@gmail.com> wrote:
>> >>
>> >>> Suppose I have the following piece of code
>> >>>
>> >>> import std.parallelism;
>> >>>
>> >>> void main() {
>> >>>     int[] data = new int[1000_000_00];
>> >>>
>> >>>     foreach(i, ref el; parallel(data)) {
>> >>>         if (i > 10 && i < 1000_000_00 - 10)
>> >>>             data[i] = data[i-10] + data[i+10];
>> >>>     }
>> >>> }
>> >>>
>> >>> That's right, I want to process all the elements of the array, except for first 10 and last 10. Is there a way to do that without if statement?
>> >>>
>> >>> I mean something like foreach starting from element with index 10 and ending with lendth - 10
>> >>>
>> >
>> > _______________________________________________
>> > dmd-concurrency mailing list
>> > dmd-concurrency@puremagic.com
>> > http://lists.puremagic.com/mailman/listinfo/dmd-concurrency
>> 
>> _______________________________________________
>> dmd-concurrency mailing list
>> dmd-concurrency@puremagic.com
>> http://lists.puremagic.com/mailman/listinfo/dmd-concurrency
> 
> _______________________________________________
> dmd-concurrency mailing list
> dmd-concurrency@puremagic.com
> http://lists.puremagic.com/mailman/listinfo/dmd-concurrency