I ran my calculation on an 8-core at work, by the way, and it was very enjoyable to see a manyfold speed-up just by changing a few lines of code.

We should definitely get this into Phobos.

-Lars

----- Reply message -----
From: "David Simcha" <dsimcha at gmail.com>
Date: Tue, Aug 31, 2010 14:13
Subject: [phobos] std.parallelism:  Request for review/comment
To: "Discuss the phobos library for D" <phobos at puremagic.com>

  On 8/31/2010 6:22 AM, Lars Tandle Kyllingstad wrote:
> Point (3) is pretty cool.  I just used your module for my current project at work, and the ability to get the index made the code a lot nicer.
>
> Another question:  Why have you chosen the default number of work units to be just two units per thread?  In my experience, it's not uncommon that calculations are harder on some parts of the range than others, and then there is a risk of some cores running out of work to do.  I'd think that having more work units, 3-4 per thread, say, would allow for better distribution of work between cores.
>
> -Lars

Good point.  I should probably change this, as the more I think about it
the more I realize that I never use the default for the reason you
mention.  It seemed like a good idea in iteration 1, and then I just
never reconsidered.
_______________________________________________
phobos mailing list
phobos at puremagic.com
http://lists.puremagic.com/mailman/listinfo/phobos
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.puremagic.com/pipermail/phobos/attachments/20100831/66eaafd6/attachment.html>

I really want good reviews from Sean (threading guru) and Andrei (general design guru) before this gets into Phobos.  Otherwise I feel like scientific computing people (like us) might be the only people that find this module to be any good.  Also, no matter what, I'm probably going to wait until after the next release to check it in, because I want to test it thoroughly on 64, and I can't do that w/o a 64 compiler.

On Tue, Aug 31, 2010 at 9:34 AM, Lars Tandle Kyllingstad <lars at kyllingen.net
> wrote:

> I ran my calculation on an 8-core at work, by the way, and it was very enjoyable to see a manyfold speed-up just by changing a few lines of code.
>
> We should definitely get this into Phobos.
>
> -Lars
>
> ----- Reply message -----
> From: "David Simcha" <dsimcha at gmail.com>
> Date: Tue, Aug 31, 2010 14:13
> Subject: [phobos] std.parallelism:  Request for review/comment
> To: "Discuss the phobos library for D" <phobos at puremagic.com>
>
>
>  On 8/31/2010 6:22 AM, Lars Tandle Kyllingstad wrote:
> > Point (3) is pretty cool.  I just used your module for my current project at work, and the ability to get the index made the code a lot nicer.
> >
> > Another question:  Why have you chosen the default number of work units to be just two units per thread?  In my experience, it's not uncommon that calculations are harder on some parts of the range than others, and then there is a risk of some cores running out of work to do.  I'd think that having more work units, 3-4 per thread, say, would allow for better distribution of work between cores.
> >
> > -Lars
>
> Good point.  I should probably change this, as the more I think about it
> the more I realize that I never use the default for the reason you
> mention.  It seemed like a good idea in iteration 1, and then I just
> never reconsidered.
> _______________________________________________
> phobos mailing list
> phobos at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/phobos
>
>
>
> _______________________________________________
> phobos mailing list
> phobos at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/phobos
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.puremagic.com/pipermail/phobos/attachments/20100831/0d828a57/attachment.html>

Just a comment on 64-bit support, you could say that about any module in phobos :)

My recommendation is to just include it when you feel it's good enough, and we'll deal with 64-bit when we get there.  Or are there specific 64-bit problems that you envision that will affect the design?

BTW, I haven't looked at the module, but it seems like a nifty idea.

-Steve


>
>From: David Simcha <dsimcha at gmail.com>
>To: Discuss the phobos library for D <phobos at puremagic.com>
>Sent: Tue, August 31, 2010 9:42:59 AM
>Subject: Re: [phobos] std.parallelism: Request for review/comment
>
>I really want good reviews from Sean (threading guru) and Andrei (general design guru) before this gets into Phobos.  Otherwise I feel like scientific computing people (like us) might be the only people that find this module to be any good. Also, no matter what, I'm probably going to wait until after the next release to check it in, because I want to test it thoroughly on 64, and I can't do that w/o a 64 compiler.
>
>
>On Tue, Aug 31, 2010 at 9:34 AM, Lars Tandle Kyllingstad <lars at kyllingen.net> wrote:
>
>I ran my calculation on an 8-core at work, by the way, and it was very enjoyable to see a manyfold speed-up just by changing a few lines of code.
>>
>>We should definitely get this into Phobos.
>>
>>-Lars
>>
>>----- Reply message -----
>>From: "David Simcha" <dsimcha at gmail.com>
>>Date: Tue, Aug 31, 2010 14:13
>>Subject: [phobos] std.parallelism:  Request for review/comment
>>To: "Discuss the phobos library for D" <phobos at puremagic.com>
>>
>>
>> On 8/31/2010 6:22 AM, Lars Tandle Kyllingstad wrote:
>>> Point (3) is pretty cool.  I just used your module for my current project at work, and the ability to get the index made the code a lot nicer.
>>>
>>> Another question:  Why have you chosen the default number of work units to be just two units per thread?  In my experience, it's not uncommon that calculations are harder on some parts of the range than others, and then there is a risk of some cores running out of work to do.  I'd think that having more work units, 3-4 per thread, say, would allow for better distribution of work between cores.
>>>
>>> -Lars
>>
>>Good point.  I should probably change this, as the more I think about it the more I realize that I never use the default for the reason you mention.  It seemed like a good idea in iteration 1, and then I just never reconsidered.
>>
>>_______________________________________________
>>phobos mailing list
>>phobos at puremagic.com
>>http://lists.puremagic.com/mailman/listinfo/phobos
>>
>>
>>
>>_______________________________________________
>>phobos mailing list
>>phobos at puremagic.com
>>http://lists.puremagic.com/mailman/listinfo/phobos
>>
>



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.puremagic.com/pipermail/phobos/attachments/20100831/9b502f9c/attachment-0001.html>

Just the fact that it's a systems-ish module and therefore is more susceptible to changes in CPU architecture than other stuff.  Also, it depends on core.atomic, which is even more systems-ish and hasn't been tested on 64 AFAIK.

On Tue, Aug 31, 2010 at 10:01 AM, Steve Schveighoffer <schveiguy at yahoo.com>wrote:

> Just a comment on 64-bit support, you could say that about any module in phobos :)
>
> My recommendation is to just include it when you feel it's good enough, and we'll deal with 64-bit when we get there.  Or are there specific 64-bit problems that you envision that will affect the design?
>
> BTW, I haven't looked at the module, but it seems like a nifty idea.
>
> -Steve
>
>
> *From:* David Simcha <dsimcha at gmail.com>
> *To:* Discuss the phobos library for D <phobos at puremagic.com>
> *Sent:* Tue, August 31, 2010 9:42:59 AM
> *Subject:* Re: [phobos] std.parallelism: Request for review/comment
>
> I really want good reviews from Sean (threading guru) and Andrei (general design guru) before this gets into Phobos.  Otherwise I feel like scientific computing people (like us) might be the only people that find this module to be any good.  Also, no matter what, I'm probably going to wait until after the next release to check it in, because I want to test it thoroughly on 64, and I can't do that w/o a 64 compiler.
>
> On Tue, Aug 31, 2010 at 9:34 AM, Lars Tandle Kyllingstad < lars at kyllingen.net> wrote:
>
>> I ran my calculation on an 8-core at work, by the way, and it was very enjoyable to see a manyfold speed-up just by changing a few lines of code.
>>
>> We should definitely get this into Phobos.
>>
>> -Lars
>>
>> ----- Reply message -----
>> From: "David Simcha" <dsimcha at gmail.com>
>> Date: Tue, Aug 31, 2010 14:13
>> Subject: [phobos] std.parallelism:  Request for review/comment
>> To: "Discuss the phobos library for D" <phobos at puremagic.com>
>>
>>
>>  On 8/31/2010 6:22 AM, Lars Tandle Kyllingstad wrote:
>> > Point (3) is pretty cool.  I just used your module for my current project at work, and the ability to get the index made the code a lot nicer.
>> >
>> > Another question:  Why have you chosen the default number of work units to be just two units per thread?  In my experience, it's not uncommon that calculations are harder on some parts of the range than others, and then there is a risk of some cores running out of work to do.  I'd think that having more work units, 3-4 per thread, say, would allow for better distribution of work between cores.
>> >
>> > -Lars
>>
>> Good point.  I should probably change this, as the more I think about it
>> the more I realize that I never use the default for the reason you
>> mention.  It seemed like a good idea in iteration 1, and then I just
>> never reconsidered.
>> _______________________________________________
>> phobos mailing list
>> phobos at puremagic.com
>> http://lists.puremagic.com/mailman/listinfo/phobos
>>
>>
>>
>> _______________________________________________
>> phobos mailing list
>> phobos at puremagic.com
>> http://lists.puremagic.com/mailman/listinfo/phobos
>>
>
>
>
> _______________________________________________
> phobos mailing list
> phobos at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/phobos
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.puremagic.com/pipermail/phobos/attachments/20100831/2d524b93/attachment.html>

Oh, that and I'm looking forward to the next release enough that I don't want to hold it up.

On Tue, Aug 31, 2010 at 10:08 AM, David Simcha <dsimcha at gmail.com> wrote:

> Just the fact that it's a systems-ish module and therefore is more susceptible to changes in CPU architecture than other stuff.  Also, it depends on core.atomic, which is even more systems-ish and hasn't been tested on 64 AFAIK.
>
>
> On Tue, Aug 31, 2010 at 10:01 AM, Steve Schveighoffer <schveiguy at yahoo.com
> > wrote:
>
>> Just a comment on 64-bit support, you could say that about any module in phobos :)
>>
>> My recommendation is to just include it when you feel it's good enough, and we'll deal with 64-bit when we get there.  Or are there specific 64-bit problems that you envision that will affect the design?
>>
>> BTW, I haven't looked at the module, but it seems like a nifty idea.
>>
>> -Steve
>>
>>
>> *From:* David Simcha <dsimcha at gmail.com>
>> *To:* Discuss the phobos library for D <phobos at puremagic.com>
>> *Sent:* Tue, August 31, 2010 9:42:59 AM
>> *Subject:* Re: [phobos] std.parallelism: Request for review/comment
>>
>> I really want good reviews from Sean (threading guru) and Andrei (general design guru) before this gets into Phobos.  Otherwise I feel like scientific computing people (like us) might be the only people that find this module to be any good.  Also, no matter what, I'm probably going to wait until after the next release to check it in, because I want to test it thoroughly on 64, and I can't do that w/o a 64 compiler.
>>
>> On Tue, Aug 31, 2010 at 9:34 AM, Lars Tandle Kyllingstad < lars at kyllingen.net> wrote:
>>
>>> I ran my calculation on an 8-core at work, by the way, and it was very enjoyable to see a manyfold speed-up just by changing a few lines of code.
>>>
>>> We should definitely get this into Phobos.
>>>
>>> -Lars
>>>
>>> ----- Reply message -----
>>> From: "David Simcha" <dsimcha at gmail.com>
>>> Date: Tue, Aug 31, 2010 14:13
>>> Subject: [phobos] std.parallelism:  Request for review/comment
>>> To: "Discuss the phobos library for D" <phobos at puremagic.com>
>>>
>>>
>>>  On 8/31/2010 6:22 AM, Lars Tandle Kyllingstad wrote:
>>> > Point (3) is pretty cool.  I just used your module for my current project at work, and the ability to get the index made the code a lot nicer.
>>> >
>>> > Another question:  Why have you chosen the default number of work units to be just two units per thread?  In my experience, it's not uncommon that calculations are harder on some parts of the range than others,
>>> and
>>> > then there is a risk of some cores running out of work to do.  I'd
>>> think
>>> > that having more work units, 3-4 per thread, say, would allow for
>>> better
>>> > distribution of work between cores.
>>> >
>>> > -Lars
>>>
>>> Good point.  I should probably change this, as the more I think about it
>>> the more I realize that I never use the default for the reason you
>>> mention.  It seemed like a good idea in iteration 1, and then I just
>>> never reconsidered.
>>> _______________________________________________
>>> phobos mailing list
>>> phobos at puremagic.com
>>>  http://lists.puremagic.com/mailman/listinfo/phobos
>>>
>>>
>>>
>>> _______________________________________________
>>> phobos mailing list
>>> phobos at puremagic.com
>>> http://lists.puremagic.com/mailman/listinfo/phobos
>>>
>>
>>
>>
>> _______________________________________________
>> phobos mailing list
>> phobos at puremagic.com
>> http://lists.puremagic.com/mailman/listinfo/phobos
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.puremagic.com/pipermail/phobos/attachments/20100831/f268b688/attachment.html>

On Tue, Aug 31, 2010 at 10:01 AM, Steve Schveighoffer <schveiguy at yahoo.com>wrote:

> Just a comment on 64-bit support, you could say that about any module in phobos :)
>
> My recommendation is to just include it when you feel it's good enough, and we'll deal with 64-bit when we get there.  Or are there specific 64-bit problems that you envision that will affect the design?
>
> BTW, I haven't looked at the module, but it seems like a nifty idea.
>
> -Steve
>

When I initially said I was going to hold off, I thought 64 support and the next release was imminent anyhow.  Now that it looks like it's gonna be a while (since Walter went into bug fixing mode and put off finishing 64 support) I think I would like to try to get this in for next release.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.puremagic.com/pipermail/phobos/attachments/20100902/6a6292fc/attachment.html>