This post https://dlang.org/library/std/process/pipe.html mentions:

>Pipes can, for example, be used for interprocess communication by spawning a new process and passing one end of the pipe to the child, while the parent uses the other end. (See also pipeProcess and pipeShell for an easier way of doing this.)

```
auto p = pipe();
auto outFile = File("D downloads.txt", "w");
auto cpid = spawnProcess(["curl", "http://dlang.org/download.html"],
                         stdin, p.writeEnd);
scope(exit) wait(cpid);
auto gpid = spawnProcess(["grep", "-o", `http://\S*\.zip`],
                         p.readEnd, outFile);
scope(exit) wait(gpid);
```

How am I able to compose the pipes from pipeProcess if I can't pass in an existing pipe into the function. Eg.

```
auto p = pipeProcess("ls");
auto q = pipeProcess("cat", stdin = p.stdout); //it would be good to do this or something like it
```

Do I need to manually extract the output from pipes.stdin.readln and place it in pipes.stdout.writeln?
Or is there an easier way to do this?

Thanks

On 1/28/21 2:16 AM, Anthony wrote:

> auto p = pipeProcess("ls");
> auto q = pipeProcess("cat", stdin = p.stdout); //it would be good to do

That would work if `cat` received the *contents* of the files (and with a "-" command line switch). Since `ls` produces file names, you would have to make the complete `cat` command line from `ls`'s output.

> Do I need to manually extract the output from pipes.stdin.readln

Seems to be so for the `ls | cat` case. But the following `find | grep` example shows how two ends of pipes can be connected:

import std.stdio;
import std.process;
import std.range;

// BONUS: Enable one of the following lines to enjoy an issue.
// version = bonus_bug;
// version = bonus_bug_but_this_works;

void main() {
  // Writes to 'a':
  auto a = pipe();
  auto lsPid = spawnProcess([ "find", "."], stdin, a.writeEnd);
  scope (exit) wait(lsPid);

  // Reads from 'a', writes to 'b':
  auto b = pipe();
  auto catPid = spawnProcess([ "grep", "-e", `\.d$` ], a.readEnd, b.writeEnd);
  scope (exit) wait(catPid);

  version (bonus_bug) {
    // Fails with the following error.
    //
    // "/usr/include/dmd/phobos/std/typecons.d(6540): Error:
    // `"Attempted to access an uninitialized payload."`"
    writefln!"Some of the D source files under the current directory:\n%-(  %s\n%)"(
      b.readEnd.byLine);

  } else version (bonus_bug_but_this_works) {
    // Note .take at the end:
    writefln!"Some of the D source files under the current directory:\n%-(  %s\n%)"(
      b.readEnd.byLine.take(1000));

  } else {
    // The results are read from 'b':
    writeln(b.readEnd.byLine);
  }
}

I've discovered a strange issue, which can be observed by uncommenting the 'version = bonus_bug;' line above. But comment that one back in and uncomment the next line, now it works. (?)

Ali

On Thursday, 28 January 2021 at 17:18:46 UTC, Ali Çehreli wrote:
> On 1/28/21 2:16 AM, Anthony wrote:
>
> > auto p = pipeProcess("ls");
> > auto q = pipeProcess("cat", stdin = p.stdout); //it would be
> good to do
>
> That would work if `cat` received the *contents* of the files (and with a "-" command line switch). Since `ls` produces file names, you would have to make the complete `cat` command line from `ls`'s output.
>
> > Do I need to manually extract the output from
> pipes.stdin.readln
>
> Seems to be so for the `ls | cat` case. But the following `find | grep` example shows how two ends of pipes can be connected:
>
> import std.stdio;
> import std.process;
> import std.range;
>
> // BONUS: Enable one of the following lines to enjoy an issue.
> // version = bonus_bug;
> // version = bonus_bug_but_this_works;
>
> void main() {
>   // Writes to 'a':
>   auto a = pipe();
>   auto lsPid = spawnProcess([ "find", "."], stdin, a.writeEnd);
>   scope (exit) wait(lsPid);
>
>   // Reads from 'a', writes to 'b':
>   auto b = pipe();
>   auto catPid = spawnProcess([ "grep", "-e", `\.d$` ], a.readEnd, b.writeEnd);
>   scope (exit) wait(catPid);
>
>   version (bonus_bug) {
>     // Fails with the following error.
>     //
>     // "/usr/include/dmd/phobos/std/typecons.d(6540): Error:
>     // `"Attempted to access an uninitialized payload."`"
>     writefln!"Some of the D source files under the current directory:\n%-(  %s\n%)"(
>       b.readEnd.byLine);
>
>   } else version (bonus_bug_but_this_works) {
>     // Note .take at the end:
>     writefln!"Some of the D source files under the current directory:\n%-(  %s\n%)"(
>       b.readEnd.byLine.take(1000));
>
>   } else {
>     // The results are read from 'b':
>     writeln(b.readEnd.byLine);
>   }
> }
>
> I've discovered a strange issue, which can be observed by uncommenting the 'version = bonus_bug;' line above. But comment that one back in and uncomment the next line, now it works. (?)
>
> Ali


Thanks Ali.
I was messing around and below seems to work well enough for me.

```

struct AccumulatorPipe {
    Pid[] pids;

    File stdin;
    File stdout;
}

AccumulatorPipe run(string cmd) {
    AccumulatorPipe acc;

    auto p = P.pipeShell(cmd, P.Redirect.stdout);

    acc.pids ~= p.pid;
    acc.stdout = p.stdout;

    return acc;
}

AccumulatorPipe pipe(AccumulatorPipe acc0, string cmd) {
    AccumulatorPipe acc;
    Pipe p = P.pipe();

    acc.stdin = p.writeEnd;
    acc.stdout = p.readEnd;

    auto pid = P.spawnShell(cmd, acc0.stdout, acc.stdin);
    acc.pids = acc0.pids ~ pid;

    return acc;
}

void end(AccumulatorPipe acc) {
    auto pids = acc.pids ~ P.spawnShell("cat", acc.stdout);

    foreach (pid; pids) {
        P.wait(pid);
    }
}
```


So now I can do something like:
```
run("find source -name '*.d'")
        .pipe("entr ./make.d tests")
        .end(),
```


> That would work if `cat` received the *contents* of the files (and with a "-" command line switch)

I was actually trying to use cat to just spit out the filenames of the directory as a test.
But I see what you mean.

On 1/28/21 3:45 PM, Anthony wrote:

> void end(AccumulatorPipe acc) {
>      auto pids = acc.pids ~ P.spawnShell("cat", acc.stdout);
>
>      foreach (pid; pids) {
>          P.wait(pid);
>      }
> }
> ```
>
>
> So now I can do something like:
> ```
> run("find source -name '*.d'")
>          .pipe("entr ./make.d tests")
>          .end(),

Cool but there should be one improvement because I don't think end() is guaranteed to be executed in that code, which may leave zombie processes around. From 'man waitpid':

  "A child that terminates, but has not been waited for becomes a "zombie".

Which is relayed to std.process documentation as "to avoid child processes becoming "zombies"".

Ali

On Friday, 29 January 2021 at 03:49:38 UTC, Ali Çehreli wrote:
> On 1/28/21 3:45 PM, Anthony wrote:
>
> > void end(AccumulatorPipe acc) {
> >      auto pids = acc.pids ~ P.spawnShell("cat", acc.stdout);
> >
> >      foreach (pid; pids) {
> >          P.wait(pid);
> >      }
> > }
> > ```
> >
> >
> > So now I can do something like:
> > ```
> > run("find source -name '*.d'")
> >          .pipe("entr ./make.d tests")
> >          .end(),
>
> Cool but there should be one improvement because I don't think end() is guaranteed to be executed in that code, which may leave zombie processes around. From 'man waitpid':
>
>   "A child that terminates, but has not been waited for becomes a "zombie".
>
> Which is relayed to std.process documentation as "to avoid child processes becoming "zombies"".
>
> Ali

I take it you're referring to missing scope guards like in your code
`scope (exit) wait(lsPid);`

Yeah, that is a tricky one. I can't think of a way to have a nice interface that also closes the pids on exit since scope guards are handled on function exit in this case.

Perhaps thats just the nature of the problem though.

I'll take a look at what scriptlike does https://github.com/Abscissa/scriptlike#script-style-shell-commands