Or what I really want to ask: why can't I call amap from std.parallelism with a lambda? I assume it's because it's a member function but I'm not 100% sure.

I hardly ever call map with a named function (named local functions don't work with TaskPool.amap either), it's always a closure. Not just in D, pretty much in any language I write code in.

I had to roll my own parallel map today, but at least I did get a nice 3x speedup.

Atila

On Thursday, 18 September 2014 at 19:49:00 UTC, Atila Neves wrote:
> Or what I really want to ask: why can't I call amap from std.parallelism with a lambda? I assume it's because it's a member function but I'm not 100% sure.
>
> Atila

You have to tell DMD that the lambda is not in fact a delegate.

import std.stdio;
import std.range;
import std.parallelism;

void main()
{
    auto w = iota(0,1_000_000);
    int[] foo;

    // Not OK, dmd can't infer lambda isn't a delegate
    // foo = taskPool().amap!(a => a + 1)(w);

    // OK:
    foo = taskPool().amap!`a+1`(w); // string lambdas, yeah!
    foo = taskPool().amap!(function int(int a) => a + 1)(w);
    static int func(int a) { return a + 1; }
    foo = taskPool().amap!func(w);
}

The point is I _want_ a delegate.

Atila

On Thursday, 18 September 2014 at 20:51:30 UTC, Jared wrote:
> On Thursday, 18 September 2014 at 19:49:00 UTC, Atila Neves wrote:
>> Or what I really want to ask: why can't I call amap from std.parallelism with a lambda? I assume it's because it's a member function but I'm not 100% sure.
>>
>> Atila
>
> You have to tell DMD that the lambda is not in fact a delegate.
>
> import std.stdio;
> import std.range;
> import std.parallelism;
>
> void main()
> {
>     auto w = iota(0,1_000_000);
>     int[] foo;
>
>     // Not OK, dmd can't infer lambda isn't a delegate
>     // foo = taskPool().amap!(a => a + 1)(w);
>
>     // OK:
>     foo = taskPool().amap!`a+1`(w); // string lambdas, yeah!
>     foo = taskPool().amap!(function int(int a) => a + 1)(w);
>     static int func(int a) { return a + 1; }
>     foo = taskPool().amap!func(w);
> }

On Friday, 19 September 2014 at 07:17:50 UTC, Atila Neves wrote:
> The point is I _want_ a delegate.
>
> Atila
>
> On Thursday, 18 September 2014 at 20:51:30 UTC, Jared wrote:
>> On Thursday, 18 September 2014 at 19:49:00 UTC, Atila Neves wrote:
>>> Or what I really want to ask: why can't I call amap from std.parallelism with a lambda? I assume it's because it's a member function but I'm not 100% sure.

Check out this: https://issues.dlang.org/show_bug.cgi?id=5710

It seems that the compiler doesn't know what to do with non-global templates and delegates since the delegate would end up with two context pointers. Looks like there are efforts to fix it.

On Thursday, 18 September 2014 at 19:49:00 UTC, Atila Neves wrote:
> I had to roll my own parallel map today, but at least I did get a nice 3x speedup.

Is your own parallel map public somewhere? It would be interesting to see it.

On Sat, 2014-09-20 at 06:46 +0000, "Nordlöw" via Digitalmars-d-learn wrote:
> On Thursday, 18 September 2014 at 19:49:00 UTC, Atila Neves wrote:
> > I had to roll my own parallel map today, but at least I did get a nice 3x speedup.

How many cores? Is the problem a data parallel one and hence should show linear speedup?

> Is your own parallel map public somewhere? It would be interesting to see it.

Particularly if it can be used to improved the code in std.parallelism.
-- 
Russel. ============================================================================= Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder@ekiga.net 41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel@winder.org.uk London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder

On Saturday, 20 September 2014 at 06:46:43 UTC, Nordlöw wrote:
> On Thursday, 18 September 2014 at 19:49:00 UTC, Atila Neves wrote:
>> I had to roll my own parallel map today, but at least I did get a nice 3x speedup.
>
> Is your own parallel map public somewhere? It would be interesting to see it.

I just did the simplest, stupidest thing that would work, so it's probably buggy. It works where I used it (and was faster) so that's all I needed to know. To even think of releasing this I'd use atomics instead of the mutex and try to break it in all sorts of ways. But here it is anyway:

private auto pmap(alias fun, R)(R range) if(isInputRange!R) {
    import std.parallelism;
    import core.sync.mutex;

    static __gshared Mutex mutex;
    if(mutex is null) mutex = new Mutex;
    typeof(fun(range.front))[] values;
    foreach(i, value; range.parallel) {
        auto newValue = fun(value);
        synchronized(mutex) {
            if(values.length < i + 1) values.length = i + 1;
            values[i] = newValue;
        }
    }

    return values;
}

Oh, and the reason I don't just append to `values` is that I need to preserve the original order.

Atila

On Saturday, 20 September 2014 at 07:25:45 UTC, Russel Winder via Digitalmars-d-learn wrote:
> On Sat, 2014-09-20 at 06:46 +0000, "Nordlöw" via Digitalmars-d-learn
> wrote:
>> On Thursday, 18 September 2014 at 19:49:00 UTC, Atila Neves wrote:
>> > I had to roll my own parallel map today, but at least I did get a nice 3x speedup.
>
> How many cores? Is the problem a data parallel one and hence should show
> linear speedup?

It depends on the data. I was running analysis on a build dependency graph. It'd depend on the dependency tree topology. I have 4 cores with hyperthreading. 3x speedup seems good to me, especially since the implementation is recursive.

>
>> Is your own parallel map public somewhere? It would be interesting to see it.
>
> Particularly if it can be used to improved the code in std.parallelism.

Now that I looked at the bug id referenced above, I don't think it'd help. I can see all sorts of problems with guaranteeing no mutable references are captured by the delegate.

Atila