http://d.puremagic.com/issues/show_bug.cgi?id=8214

           Summary: blocking option for TaskPool.finish()
           Product: D
           Version: D2
          Platform: All
        OS/Version: All
            Status: NEW
          Severity: enhancement
          Priority: P2
         Component: Phobos
        AssignedTo: nobody@puremagic.com
        ReportedBy: john@neggie.net


--- Comment #0 from John Belmonte <john@neggie.net> 2012-06-09 19:13:35 PDT ---
In std.parallelism, I have a simple need to block on all tasks in a pool being completed.  (In my use case tasks don't return a value, and errors within a task are discarded.)  Currently the only way to block would be to manually maintain a collection of all tasks added to a pool.

I propose that TaskPool.finish() take a backwards-compatible option to block:

   finish(bool block=false)

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=8214


John Belmonte <john@neggie.net> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |dsimcha@yahoo.com


--- Comment #1 from John Belmonte <john@neggie.net> 2012-06-09 20:01:48 PDT ---
Looking at source I see the original version includes TaskPool.join() which has the desired behavior.  However the method definition is disabled:


https://github.com/D-Programming-Language/phobos/blob/master/std/parallelism.d#L3097

In 2011 dsimcha mentioned the join function [1]:

    "One thing Andrei mentioned that I'm really not sure about is what to do
with TaskPool.join().  My example for it is still terrible, because I
think it's an evolutionary artifact.  It was useful in earlier designs
that were never released and didn't have high-level data parallelism
primitives.  I never use it, don't have any good use cases for it and
would be inclined to remove it entirely.  Andrei seems to have some good
use cases in mind, but he has not detailed any that I believe are
reasonably implementable and I'm not sure whether they could be solved
better using the higher-level data parallelism primitives."

and

    "I put TaskPool.join() in a version(none) statement and removed all
references to it from the documentation.  If anyone has a compelling
reason why this is useful (preferably with a good, short example), I'll
add it back.  Otherwise, it's gone, since IMHO it's an evolutionary
artifact that makes sense in theory but that I can't think of a use for
anymore in practice."

Well, I have a use case :)  I'm writing a D implementation of luaproc, the Lua share-nothing message passing framework described in "Exploring Lua for Concurrent Programming" [2].  The std.parallelism module nicely replaces several hundred lines of low level multithreading code in the original C implementation.    The design requires a single queue of tasks serviced by multiple threads, where the tasks do not yield a return value (they only optionally schedule more tasks), and any errors within a task are discarded (i.e. exceptions don't need to be propagated to parent tasks).  The design specifies a function which blocks on the completion of all tasks, and this is the only thing std.parallelism is missing.


[1] http://forum.dlang.org/thread/imehn5$h78$1@digitalmars.com [2] http://www.inf.puc-rio.br/~roberto/docs/ry08-05.pdf

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=8214



--- Comment #2 from John Belmonte <john@neggie.net> 2012-06-09 20:39:07 PDT ---
I do prefer this being an option to finish() rather than a separate join()
function.  We all understand what finish() does and it's not a stretch at all
to accept that there's blocking and non-blocking invocations (as evidenced by
current doc needing to clearly state that it's non-blocking).

Also I don't think the docs need a big example for the blocking case.  It should suffice to say that blocking invocation only makes sense when task results aren't needed (otherwise the corresponding Task.*Force() calls would provide the blocking).

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=8214


John Belmonte <john@neggie.net> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |ASSIGNED
         AssignedTo|nobody@puremagic.com        |john@neggie.net


--- Comment #3 from John Belmonte <john@neggie.net> 2012-06-17 06:14:56 PDT ---
https://github.com/D-Programming-Language/phobos/pull/634

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=8214



--- Comment #4 from github-bugzilla@puremagic.com 2012-06-17 07:01:14 PDT ---
Commits pushed to master at https://github.com/D-Programming-Language/phobos

https://github.com/D-Programming-Language/phobos/commit/01cd4f3eed7809080a5ef0f51387b588f1f55b2e Issue 8214 - blocking option for TaskPool.finish()

Add backwards compatible blocking option to finish().  Remove disabled
join() method.  Add unit tests for finish().

https://github.com/D-Programming-Language/phobos/commit/a1e79c76335aff5cacd2c259265ae3ca4921500a Merge pull request #634 from belm0/taskpool_finish

Issue 8214 - blocking option for TaskPool.finish()

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=8214


John Belmonte <john@neggie.net> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|ASSIGNED                    |RESOLVED
         Resolution|                            |FIXED


-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=8214


Brad Roberts <braddr@puremagic.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |REOPENED
                 CC|                            |braddr@puremagic.com
         Resolution|FIXED                       |


--- Comment #5 from Brad Roberts <braddr@puremagic.com> 2012-06-17 11:38:05 PDT ---
The std.parallelism tests aren't passing reliably (though also not failing consistently either) any more, on most platforms after this was committed.

freebsd/32, linux/*, osx/64/32:
core.exception.AssertError@std.parallelism(4061): unittest failure

So far, no win/32 or freebsd/64 failures, but I suspect it's only a matter of time.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=8214



--- Comment #6 from John Belmonte <john@neggie.net> 2012-06-17 11:55:52 PDT ---
Addressing flaky test:

https://github.com/D-Programming-Language/phobos/pull/636

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=8214



--- Comment #7 from John Belmonte <john@neggie.net> 2012-06-17 12:00:57 PDT ---
@braddr:  would you kindly add me to while list for pull auto-tester?

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=8214



--- Comment #8 from Brad Roberts <braddr@puremagic.com> 2012-06-17 12:03:29 PDT ---
You were already.  Both this pull and the previous one ran through the auto tester, and show(ed) failures.  You're not done fixing things yet based on the current test results for this pull request.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------