Thread overview | |||||||
---|---|---|---|---|---|---|---|
|
November 27, 2011 How do I redirect stderr of a spawned process into an internal buffer? | ||||
---|---|---|---|---|
| ||||
I've talked about this before, there's a problem with spawning multiple processes and letting them write to stderr asynchronously It seems like this might be a Windows-only problem, I couldn't recreate on Ubuntu but maybe that's because it was virtualized. Take buggy.d: import std.concurrency; import std.process; void test(int i) { system("dmd buggy.d buggy.d"); } void main() { foreach (i; 0 .. 100) { spawn(&test, i); } } Run it via rdmd: rdmd buggy.d The output is messy (ran via cmd.exe): http://paste.pocoo.org/show/513763/ So it just gets worse the more processes are spawned. One workaround is to redirect each process' stderr to a unique file. Example: import core.thread; import std.concurrency; import std.process; import std.string; import std.stdio; import std.file : readText; void test(int i) { // redirect each process stderr to its own file system(format("dmd workaround.d workaround.d 2> error_%s.txt", i)); } void main() { foreach (i; 0 .. 100) { spawn(&test, i); } thread_joinAll(); foreach (i; 0 .. 100) { writeln(format("Output %s = %s", i, readText(format("error_%s.txt", i)))); } } So now the output is perfect: http://paste.pocoo.org/show/513771/ But writing to files is rather slow. Is there anything in Phobos I can use to spawn a process and redirect stderr to an internal buffer? Tango had this sort of thing for D1, but I'm not seeing anything in Phobos. Since I can only recreate this on Windows I'm ok with using a WinAPI function if it provides this feature. |
November 28, 2011 Re: How do I redirect stderr of a spawned process into an internal buffer? | ||||
---|---|---|---|---|
| ||||
Posted in reply to Andrej Mitrovic | On Sun, 27 Nov 2011 17:37:24 -0500, Andrej Mitrovic <andrej.mitrovich@gmail.com> wrote:
> I've talked about this before, there's a problem with spawning
> multiple processes and letting them write to stderr asynchronously It
> seems like this might be a Windows-only problem, I couldn't recreate
> on Ubuntu but maybe that's because it was virtualized.
>
> Take buggy.d:
>
> import std.concurrency;
> import std.process;
>
> void test(int i)
> {
> system("dmd buggy.d buggy.d");
> }
>
> void main()
> {
> foreach (i; 0 .. 100)
> {
> spawn(&test, i);
> }
> }
>
> Run it via rdmd: rdmd buggy.d
>
> The output is messy (ran via cmd.exe): http://paste.pocoo.org/show/513763/
>
> So it just gets worse the more processes are spawned. One workaround
> is to redirect each process' stderr to a unique file. Example:
>
> import core.thread;
> import std.concurrency;
> import std.process;
> import std.string;
> import std.stdio;
> import std.file : readText;
>
> void test(int i)
> {
> // redirect each process stderr to its own file
> system(format("dmd workaround.d workaround.d 2> error_%s.txt", i));
> }
>
> void main()
> {
> foreach (i; 0 .. 100)
> {
> spawn(&test, i);
> }
>
> thread_joinAll();
>
> foreach (i; 0 .. 100)
> {
> writeln(format("Output %s = %s", i,
> readText(format("error_%s.txt", i))));
> }
> }
>
> So now the output is perfect: http://paste.pocoo.org/show/513771/
>
> But writing to files is rather slow. Is there anything in Phobos I can
> use to spawn a process and redirect stderr to an internal buffer?
>
> Tango had this sort of thing for D1, but I'm not seeing anything in
> Phobos. Since I can only recreate this on Windows I'm ok with using a
> WinAPI function if it provides this feature.
the yet-to-be-reviewed std.process upgrade should fix all your problems. However, I need to get Walter to include a fix in dmc so reading past the end of a pipe produces EOF instead of an invalid handle error.
I'll work on getting that pull request to Walter, this keeps coming up...
-Steve
|
November 28, 2011 Re: How do I redirect stderr of a spawned process into an internal buffer? | ||||
---|---|---|---|---|
| ||||
Posted in reply to Steven Schveighoffer | Cool stuff Steve, I'll be eager to review it when the time comes. |
November 28, 2011 Re: How do I redirect stderr of a spawned process into an internal buffer? | ||||
---|---|---|---|---|
| ||||
Btw, is the new std.process hosted anywhere? My solution is windows-specific and probably doesn't handle all the edge-cases. Even if the new std.process isn't ready yet I'd love to look at your implementation. |
November 29, 2011 Re: How do I redirect stderr of a spawned process into an internal buffer? | ||||
---|---|---|---|---|
| ||||
Posted in reply to Andrej Mitrovic | On Mon, 28 Nov 2011 17:42:54 -0500, Andrej Mitrovic <andrej.mitrovich@gmail.com> wrote:
> Btw, is the new std.process hosted anywhere? My solution is
> windows-specific and probably doesn't handle all the edge-cases. Even
> if the new std.process isn't ready yet I'd love to look at your
> implementation.
Yes,but in order for it to work you need a new dmc runtime.
it's on my github account. I'd have to send you the dmc runtime privately, send me a private email.
-Steve
|
Copyright © 1999-2021 by the D Language Foundation