I'm trying to interact with a process using std.process and
redirected stdin/stdout/stderr.
What would be the recommended way?

For example:
----
auto pipes=pipeShell("myprocess",Redirect.all);
while(true){
  pipes.stdin.rawWrite(some_command);
  foreach (line; pipes.stdout.byLine) {
    //do something with line
  }
}
----

This doesn't work because it might block inside pipes.stdout.byLine, as the process is requesting more inputs to be written to its stdin before outputting more bytes to its stdout.

What's the right approach?
* fcntl(fd, F_SETFL, O_NONBLOCK); didn't seem to work
* reading pipes.stdout inside a separate thread?
In that second case, how to cleanly dispose of a blocked thread when we no
longer need it?

Any detailed example would help.
Thanks!

On Monday, 15 July 2013 at 03:49:10 UTC, Timothee Cour wrote:
> I'm trying to interact with a process using std.process and
> redirected stdin/stdout/stderr.
> What would be the recommended way?
>
> For example:
> ----
> auto pipes=pipeShell("myprocess",Redirect.all);
> while(true){
>   pipes.stdin.rawWrite(some_command);
>   foreach (line; pipes.stdout.byLine) {
>     //do something with line
>   }
> }
> ----
>
> This doesn't work because it might block inside pipes.stdout.byLine, as the
> process is requesting more inputs to be written to its stdin before
> outputting more bytes to its stdout.
>
> What's the right approach?
> * fcntl(fd, F_SETFL, O_NONBLOCK); didn't seem to work
> * reading pipes.stdout inside a separate thread?
> In that second case, how to cleanly dispose of a blocked thread when we no
> longer need it?
>
> Any detailed example would help.
> Thanks!



I tried using a separate thread for reading the process' stdout. It works, except that sometimes the output is shuffled out of order.

Is there anything buggy in this:

----
__gshared string output;

void readBlocking(){
while ((c = fgetc(filepointer)) >= 0)
      output~=cast(char) c;
//NOTE: i can use something more efficient here but that's beside the question
}

thread = new Thread(& readBlocking);
output=null;
while(true){
        Thread.sleep(...);
        if(condition) break;
}
//now output is shuffled out of order sometimes
----

Furthermore, is there a standard way to tell when a process is waiting for stdin input ? (cf condition above). Currently I'm checking whether 'output' was modified within a timeout period T, but that's fragile and incurs of penalty of T at least.

On Monday, 15 July 2013 at 06:46:52 UTC, timotheecour wrote:
> On Monday, 15 July 2013 at 03:49:10 UTC, Timothee Cour wrote:
>> I'm trying to interact with a process using std.process and
>> redirected stdin/stdout/stderr.
>> What would be the recommended way?
>>
>> For example:
>> ----
>> auto pipes=pipeShell("myprocess",Redirect.all);
>> while(true){
>>  pipes.stdin.rawWrite(some_command);
>>  foreach (line; pipes.stdout.byLine) {
>>    //do something with line
>>  }
>> }
>> ----
>>
>> This doesn't work because it might block inside pipes.stdout.byLine, as the
>> process is requesting more inputs to be written to its stdin before
>> outputting more bytes to its stdout.
>>
>> What's the right approach?
>> * fcntl(fd, F_SETFL, O_NONBLOCK); didn't seem to work
>> * reading pipes.stdout inside a separate thread?
>> In that second case, how to cleanly dispose of a blocked thread when we no
>> longer need it?
>>
>> Any detailed example would help.
>> Thanks!
>
>
>
> I tried using a separate thread for reading the process' stdout. It works, except that sometimes the output is shuffled out of order.
>
> Is there anything buggy in this:
>
> ----
> __gshared string output;
>
> void readBlocking(){
> while ((c = fgetc(filepointer)) >= 0)
>       output~=cast(char) c;
> //NOTE: i can use something more efficient here but that's beside the question
> }
>
> thread = new Thread(& readBlocking);
> output=null;
> while(true){
>         Thread.sleep(...);
>         if(condition) break;
> }
> //now output is shuffled out of order sometimes
> ----
>
> Furthermore, is there a standard way to tell when a process is waiting for stdin input ? (cf condition above). Currently I'm checking whether 'output' was modified within a timeout period T, but that's fragile and incurs of penalty of T at least.

Are you looking for select() or poll()
Poll I believe is posix only but I think select is more widely available.
Not much of an answer but I hope it helps

On Tue, Jul 16, 2013 at 10:01 AM, Anthony Goins <neontotem@gmail.com> wrote:

> On Monday, 15 July 2013 at 06:46:52 UTC, timotheecour wrote:
>
>> On Monday, 15 July 2013 at 03:49:10 UTC, Timothee Cour wrote:
>>
>>> I'm trying to interact with a process using std.process and
>>> redirected stdin/stdout/stderr.
>>> What would be the recommended way?
>>>
>>> For example:
>>> ----
>>> auto pipes=pipeShell("myprocess",**Redirect.all);
>>> while(true){
>>>  pipes.stdin.rawWrite(some_**command);
>>>  foreach (line; pipes.stdout.byLine) {
>>>    //do something with line
>>>  }
>>> }
>>> ----
>>>
>>> This doesn't work because it might block inside pipes.stdout.byLine, as
>>> the
>>> process is requesting more inputs to be written to its stdin before
>>> outputting more bytes to its stdout.
>>>
>>> What's the right approach?
>>> * fcntl(fd, F_SETFL, O_NONBLOCK); didn't seem to work
>>> * reading pipes.stdout inside a separate thread?
>>> In that second case, how to cleanly dispose of a blocked thread when we
>>> no
>>> longer need it?
>>>
>>> Any detailed example would help.
>>> Thanks!
>>>
>>
>>
>>
>> I tried using a separate thread for reading the process' stdout. It works, except that sometimes the output is shuffled out of order.
>>
>> Is there anything buggy in this:
>>
>> ----
>> __gshared string output;
>>
>> void readBlocking(){
>> while ((c = fgetc(filepointer)) >= 0)
>>       output~=cast(char) c;
>> //NOTE: i can use something more efficient here but that's beside the
>> question
>> }
>>
>> thread = new Thread(& readBlocking);
>> output=null;
>> while(true){
>>         Thread.sleep(...);
>>         if(condition) break;
>> }
>> //now output is shuffled out of order sometimes
>> ----
>>
>> Furthermore, is there a standard way to tell when a process is waiting for stdin input ? (cf condition above). Currently I'm checking whether 'output' was modified within a timeout period T, but that's fragile and incurs of penalty of T at least.
>>
>
> Are you looking for select() or poll()
> Poll I believe is posix only but I think select is more widely available.
> Not much of an answer but I hope it helps
>
>

Thanks, I had actually used select to solve another problem I had:
[std.process: how to process stdout chunk by chunk without waiting for
process termination]
That should work for here as well.

I think std.process is a bit limited currently, I keep having to implement basic stuff it doesn't support.