Thread overview | |||||||
---|---|---|---|---|---|---|---|
|
February 11, 2012 Bug? taskPool.map() with bufSize and writeln() gets stuck | ||||
---|---|---|---|---|
| ||||
Sorry for the double-post; I have asked the same question on D.learn earlier but I think this is more of a question to this forum. Tested on Ubuntu 11.10 64-bit dmd. The following program gets stuck during the writeln() call. - Note that the foo() call alone works fine. - Also note that the program works fine when there is no writeln() call nor foo() call. All elements get processed in that case and the results are ignored. Am I using taskPool.map incorrectly or is this a bug? Can you help identify where the problem may be? How is writeln() using the range differently than foo() to cause this behavior? import std.stdio; import std.parallelism; import core.thread; int func(int i) { writeln("processing ", i); return i; } void main() { auto results = taskPool.map!func([1,2,3,4,5,6,7,8], 2); writeln(results); // <-- Gets stuck HERE foo(results); // this works fine } void foo(R)(R range) { for ( ; !range.empty; range.popFront()) { writeln(range.front); } } Thank you, Ali |
February 11, 2012 Re: Bug? taskPool.map() with bufSize and writeln() gets stuck | ||||
---|---|---|---|---|
| ||||
Posted in reply to Ali Çehreli | On Saturday, 11 February 2012 at 01:31:29 UTC, Ali Çehreli wrote:
> Sorry for the double-post; I have asked the same question on D.learn earlier but I think this is more of a question to this forum.
>
> Tested on Ubuntu 11.10 64-bit dmd.
>
> The following program gets stuck during the writeln() call.
>
> - Note that the foo() call alone works fine.
>
> - Also note that the program works fine when there is no writeln() call nor foo() call. All elements get processed in that case and the results are ignored.
>
> Am I using taskPool.map incorrectly or is this a bug? Can you help identify where the problem may be? How is writeln() using the range differently than foo() to cause this behavior?
>
> import std.stdio;
> import std.parallelism;
> import core.thread;
>
> int func(int i)
> {
> writeln("processing ", i);
> return i;
> }
>
> void main()
> {
> auto results = taskPool.map!func([1,2,3,4,5,6,7,8], 2);
>
> writeln(results); // <-- Gets stuck HERE
>
> foo(results); // this works fine
> }
>
> void foo(R)(R range)
> {
> for ( ; !range.empty; range.popFront()) {
> writeln(range.front);
> }
> }
>
> Thank you,
> Ali
For what I see, maybe that's because it's a "non-random access ranges". So to access "results", you need access from it's aggregate...
try this inside your main:
foreach(int r; results)
writeln(r);
// foo(results); // this works fin
|
February 11, 2012 Re: Bug? taskPool.map() with bufSize and writeln() gets stuck | ||||
---|---|---|---|---|
| ||||
Posted in reply to Ali Çehreli | On Sat, 11 Feb 2012 02:31:29 +0100, Ali Çehreli <acehreli@yahoo.com> wrote:
> Sorry for the double-post; I have asked the same question on D.learn earlier but I think this is more of a question to this forum.
>
> Tested on Ubuntu 11.10 64-bit dmd.
>
> The following program gets stuck during the writeln() call.
>
> - Note that the foo() call alone works fine.
>
> - Also note that the program works fine when there is no writeln() call nor foo() call. All elements get processed in that case and the results are ignored.
>
> Am I using taskPool.map incorrectly or is this a bug? Can you help identify where the problem may be? How is writeln() using the range differently than foo() to cause this behavior?
>
> import std.stdio;
> import std.parallelism;
> import core.thread;
>
> int func(int i)
> {
> writeln("processing ", i);
> return i;
> }
>
> void main()
> {
> auto results = taskPool.map!func([1,2,3,4,5,6,7,8], 2);
>
> writeln(results); // <-- Gets stuck HERE
>
> foo(results); // this works fine
> }
>
> void foo(R)(R range)
> {
> for ( ; !range.empty; range.popFront()) {
> writeln(range.front);
> }
> }
>
> Thank you,
> Ali
Yeah, you have a deadlock in there, it's somewhat hidden though.
The issue is that writeln will take lock on stdout once. This will deadlock
with the lazy processing of the map range.
|
February 11, 2012 Re: Bug? taskPool.map() with bufSize and writeln() gets stuck | ||||
---|---|---|---|---|
| ||||
Posted in reply to Martin Nowak | On 02/11/2012 12:56 AM, Martin Nowak wrote: > On Sat, 11 Feb 2012 02:31:29 +0100, Ali Çehreli <acehreli@yahoo.com> wrote: > >> Sorry for the double-post; I have asked the same question on D.learn >> earlier but I think this is more of a question to this forum. >> >> Tested on Ubuntu 11.10 64-bit dmd. >> >> The following program gets stuck during the writeln() call. >> >> - Note that the foo() call alone works fine. >> >> - Also note that the program works fine when there is no writeln() >> call nor foo() call. All elements get processed in that case and the >> results are ignored. >> >> Am I using taskPool.map incorrectly or is this a bug? Can you help >> identify where the problem may be? How is writeln() using the range >> differently than foo() to cause this behavior? >> >> import std.stdio; >> import std.parallelism; >> import core.thread; >> >> int func(int i) >> { >> writeln("processing ", i); >> return i; >> } >> >> void main() >> { >> auto results = taskPool.map!func([1,2,3,4,5,6,7,8], 2); >> >> writeln(results); // <-- Gets stuck HERE >> >> foo(results); // this works fine >> } >> >> void foo(R)(R range) >> { >> for ( ; !range.empty; range.popFront()) { >> writeln(range.front); >> } >> } >> >> Thank you, >> Ali > > Yeah, you have a deadlock in there, it's somewhat hidden though. > The issue is that writeln will take lock on stdout once. This will deadlock > with the lazy processing of the map range. Thank you. I was trying to visualize the semi-lazy nature of taskPool.map. Now I get what I want when the writeln() call in main() is changed to be on stderr: import std.stdio; import std.parallelism; import core.thread; int func(int i) { writeln("processing ", i); Thread.sleep(dur!"seconds"(1)); return i; } void main() { auto results = taskPool.map!func([1,2,3,4,5,6,7,8], 2); stderr.writeln(results); // <-- now on stderr } Now the output hints at how taskPool.map is semi-lazy: processing 1 processing 2 [processing 3 1, 2processing 4 , 3processing 5 , 4processing 6 , 5, 6processing 7 processing 8 , 7, 8] Good. :) Ali |
February 11, 2012 Re: Bug? taskPool.map() with bufSize and writeln() gets stuck | ||||
---|---|---|---|---|
| ||||
Posted in reply to Ali Çehreli | On Sat, 11 Feb 2012 17:18:21 +0100, Ali Çehreli <acehreli@yahoo.com> wrote:
> On 02/11/2012 12:56 AM, Martin Nowak wrote:
> > On Sat, 11 Feb 2012 02:31:29 +0100, Ali Çehreli <acehreli@yahoo.com> wrote:
> >
> >> Sorry for the double-post; I have asked the same question on D.learn
> >> earlier but I think this is more of a question to this forum.
> >>
> >> Tested on Ubuntu 11.10 64-bit dmd.
> >>
> >> The following program gets stuck during the writeln() call.
> >>
> >> - Note that the foo() call alone works fine.
> >>
> >> - Also note that the program works fine when there is no writeln()
> >> call nor foo() call. All elements get processed in that case and the
> >> results are ignored.
> >>
> >> Am I using taskPool.map incorrectly or is this a bug? Can you help
> >> identify where the problem may be? How is writeln() using the range
> >> differently than foo() to cause this behavior?
> >>
> >> import std.stdio;
> >> import std.parallelism;
> >> import core.thread;
> >>
> >> int func(int i)
> >> {
> >> writeln("processing ", i);
> >> return i;
> >> }
> >>
> >> void main()
> >> {
> >> auto results = taskPool.map!func([1,2,3,4,5,6,7,8], 2);
> >>
> >> writeln(results); // <-- Gets stuck HERE
> >>
> >> foo(results); // this works fine
> >> }
> >>
> >> void foo(R)(R range)
> >> {
> >> for ( ; !range.empty; range.popFront()) {
> >> writeln(range.front);
> >> }
> >> }
> >>
> >> Thank you,
> >> Ali
> >
> > Yeah, you have a deadlock in there, it's somewhat hidden though.
> > The issue is that writeln will take lock on stdout once. This will deadlock
> > with the lazy processing of the map range.
>
> Thank you.
>
> I was trying to visualize the semi-lazy nature of taskPool.map. Now I get what I want when the writeln() call in main() is changed to be on stderr:
>
> import std.stdio;
> import std.parallelism;
> import core.thread;
>
> int func(int i)
> {
> writeln("processing ", i);
> Thread.sleep(dur!"seconds"(1));
> return i;
> }
>
> void main()
> {
> auto results = taskPool.map!func([1,2,3,4,5,6,7,8], 2);
>
> stderr.writeln(results); // <-- now on stderr
> }
>
> Now the output hints at how taskPool.map is semi-lazy:
>
> processing 1
> processing 2
> [processing 3
> 1, 2processing 4
> , 3processing 5
> , 4processing 6
> , 5, 6processing 7
> processing 8
> , 7, 8]
>
> Good. :)
>
> Ali
>
Going with foreach(e; results) will work too and you can write to the right output.
|
Copyright © 1999-2021 by the D Language Foundation