Sorry for the double-post; I have asked the same question on D.learn earlier but I think this is more of a question to this forum.

Tested on Ubuntu 11.10 64-bit dmd.

The following program gets stuck during the writeln() call.

- Note that the foo() call alone works fine.

- Also note that the program works fine when there is no writeln() call nor foo() call. All elements get processed in that case and the results are ignored.

Am I using taskPool.map incorrectly or is this a bug? Can you help identify where the problem may be? How is writeln() using the range differently than foo() to cause this behavior?

import std.stdio;
import std.parallelism;
import core.thread;

int func(int i)
{
    writeln("processing ", i);
    return i;
}

void main()
{
    auto results = taskPool.map!func([1,2,3,4,5,6,7,8], 2);

    writeln(results);  // <-- Gets stuck HERE

    foo(results); // this works fine
}

void foo(R)(R range)
{
    for ( ; !range.empty; range.popFront()) {
        writeln(range.front);
    }
}

Thank you,
Ali

On Saturday, 11 February 2012 at 01:31:29 UTC, Ali Çehreli wrote:
> Sorry for the double-post; I have asked the same question on D.learn earlier but I think this is more of a question to this forum.
>
> Tested on Ubuntu 11.10 64-bit dmd.
>
> The following program gets stuck during the writeln() call.
>
> - Note that the foo() call alone works fine.
>
> - Also note that the program works fine when there is no writeln() call nor foo() call. All elements get processed in that case and the results are ignored.
>
> Am I using taskPool.map incorrectly or is this a bug? Can you help identify where the problem may be? How is writeln() using the range differently than foo() to cause this behavior?
>
> import std.stdio;
> import std.parallelism;
> import core.thread;
>
> int func(int i)
> {
>    writeln("processing ", i);
>    return i;
> }
>
> void main()
> {
>    auto results = taskPool.map!func([1,2,3,4,5,6,7,8], 2);
>
>    writeln(results);  // <-- Gets stuck HERE
>
>    foo(results); // this works fine
> }
>
> void foo(R)(R range)
> {
>    for ( ; !range.empty; range.popFront()) {
>        writeln(range.front);
>    }
> }
>
> Thank you,
> Ali

For what I see, maybe that's because it's a "non-random access ranges". So to access "results", you need access from it's aggregate...

try this inside your main:


  foreach(int r; results)
      writeln(r);

//   foo(results); // this works fin

On Sat, 11 Feb 2012 02:31:29 +0100, Ali Çehreli <acehreli@yahoo.com> wrote:

> Sorry for the double-post; I have asked the same question on D.learn earlier but I think this is more of a question to this forum.
>
> Tested on Ubuntu 11.10 64-bit dmd.
>
> The following program gets stuck during the writeln() call.
>
> - Note that the foo() call alone works fine.
>
> - Also note that the program works fine when there is no writeln() call nor foo() call. All elements get processed in that case and the results are ignored.
>
> Am I using taskPool.map incorrectly or is this a bug? Can you help identify where the problem may be? How is writeln() using the range differently than foo() to cause this behavior?
>
> import std.stdio;
> import std.parallelism;
> import core.thread;
>
> int func(int i)
> {
>      writeln("processing ", i);
>      return i;
> }
>
> void main()
> {
>      auto results = taskPool.map!func([1,2,3,4,5,6,7,8], 2);
>
>      writeln(results);  // <-- Gets stuck HERE
>
>      foo(results); // this works fine
> }
>
> void foo(R)(R range)
> {
>      for ( ; !range.empty; range.popFront()) {
>          writeln(range.front);
>      }
> }
>
> Thank you,
> Ali

Yeah, you have a deadlock in there, it's somewhat hidden though.
The issue is that writeln will take lock on stdout once. This will deadlock
with the lazy processing of the map range.

On 02/11/2012 12:56 AM, Martin Nowak wrote:
> On Sat, 11 Feb 2012 02:31:29 +0100, Ali Çehreli <acehreli@yahoo.com> wrote:
>
>> Sorry for the double-post; I have asked the same question on D.learn
>> earlier but I think this is more of a question to this forum.
>>
>> Tested on Ubuntu 11.10 64-bit dmd.
>>
>> The following program gets stuck during the writeln() call.
>>
>> - Note that the foo() call alone works fine.
>>
>> - Also note that the program works fine when there is no writeln()
>> call nor foo() call. All elements get processed in that case and the
>> results are ignored.
>>
>> Am I using taskPool.map incorrectly or is this a bug? Can you help
>> identify where the problem may be? How is writeln() using the range
>> differently than foo() to cause this behavior?
>>
>> import std.stdio;
>> import std.parallelism;
>> import core.thread;
>>
>> int func(int i)
>> {
>> writeln("processing ", i);
>> return i;
>> }
>>
>> void main()
>> {
>> auto results = taskPool.map!func([1,2,3,4,5,6,7,8], 2);
>>
>> writeln(results); // <-- Gets stuck HERE
>>
>> foo(results); // this works fine
>> }
>>
>> void foo(R)(R range)
>> {
>> for ( ; !range.empty; range.popFront()) {
>> writeln(range.front);
>> }
>> }
>>
>> Thank you,
>> Ali
>
> Yeah, you have a deadlock in there, it's somewhat hidden though.
> The issue is that writeln will take lock on stdout once. This will deadlock
> with the lazy processing of the map range.

Thank you.

I was trying to visualize the semi-lazy nature of taskPool.map. Now I get what I want when the writeln() call in main() is changed to be on stderr:

import std.stdio;
import std.parallelism;
import core.thread;

int func(int i)
{
    writeln("processing ", i);
    Thread.sleep(dur!"seconds"(1));
    return i;
}

void main()
{
    auto results = taskPool.map!func([1,2,3,4,5,6,7,8], 2);

    stderr.writeln(results);    // <-- now on stderr
}

Now the output hints at how taskPool.map is semi-lazy:

processing 1
processing 2
[processing 3
1, 2processing 4
, 3processing 5
, 4processing 6
, 5, 6processing 7
processing 8
, 7, 8]

Good. :)

Ali

On Sat, 11 Feb 2012 17:18:21 +0100, Ali Çehreli <acehreli@yahoo.com> wrote:

> On 02/11/2012 12:56 AM, Martin Nowak wrote:
>  > On Sat, 11 Feb 2012 02:31:29 +0100, Ali Çehreli <acehreli@yahoo.com> wrote:
>  >
>  >> Sorry for the double-post; I have asked the same question on D.learn
>  >> earlier but I think this is more of a question to this forum.
>  >>
>  >> Tested on Ubuntu 11.10 64-bit dmd.
>  >>
>  >> The following program gets stuck during the writeln() call.
>  >>
>  >> - Note that the foo() call alone works fine.
>  >>
>  >> - Also note that the program works fine when there is no writeln()
>  >> call nor foo() call. All elements get processed in that case and the
>  >> results are ignored.
>  >>
>  >> Am I using taskPool.map incorrectly or is this a bug? Can you help
>  >> identify where the problem may be? How is writeln() using the range
>  >> differently than foo() to cause this behavior?
>  >>
>  >> import std.stdio;
>  >> import std.parallelism;
>  >> import core.thread;
>  >>
>  >> int func(int i)
>  >> {
>  >> writeln("processing ", i);
>  >> return i;
>  >> }
>  >>
>  >> void main()
>  >> {
>  >> auto results = taskPool.map!func([1,2,3,4,5,6,7,8], 2);
>  >>
>  >> writeln(results); // <-- Gets stuck HERE
>  >>
>  >> foo(results); // this works fine
>  >> }
>  >>
>  >> void foo(R)(R range)
>  >> {
>  >> for ( ; !range.empty; range.popFront()) {
>  >> writeln(range.front);
>  >> }
>  >> }
>  >>
>  >> Thank you,
>  >> Ali
>  >
>  > Yeah, you have a deadlock in there, it's somewhat hidden though.
>  > The issue is that writeln will take lock on stdout once. This will deadlock
>  > with the lazy processing of the map range.
>
> Thank you.
>
> I was trying to visualize the semi-lazy nature of taskPool.map. Now I get what I want when the writeln() call in main() is changed to be on stderr:
>
> import std.stdio;
> import std.parallelism;
> import core.thread;
>
> int func(int i)
> {
>      writeln("processing ", i);
>      Thread.sleep(dur!"seconds"(1));
>      return i;
> }
>
> void main()
> {
>      auto results = taskPool.map!func([1,2,3,4,5,6,7,8], 2);
>
>      stderr.writeln(results);    // <-- now on stderr
> }
>
> Now the output hints at how taskPool.map is semi-lazy:
>
> processing 1
> processing 2
> [processing 3
> 1, 2processing 4
> , 3processing 5
> , 4processing 6
> , 5, 6processing 7
> processing 8
> , 7, 8]
>
> Good. :)
>
> Ali
>
Going with foreach(e; results) will work too and you can write to the right output.