My friend using a mac has had severe issues getting my project (recently ported to Tango) to run on his machine.

Looking at debug output and doing some tests, it appears that Stdout is not flushing.

Typical offending code:
auto outStream = new Print!(char)(new Layout!(char),Stdout);
outStream.formatln("= {}\n", valueToReturn); // adds extra \n and flushes


Sometimes I get a non-response too, but restarting the program usually fixes it.  It appears that on the mac this is not the case.  Any ideas what could be wrong?  I do not know which version of tango my friend is using.  I think I'm using Tango 0.99 rc1 with dmd 1.020.

Jason House wrote:
> My friend using a mac has had severe issues getting my project (recently ported to Tango) to run on his machine.
> 
> Looking at debug output and doing some tests, it appears that Stdout is not flushing.  
> 
> Typical offending code:
> auto outStream = new Print!(char)(new Layout!(char),Stdout);

Is this really the line used?  The second parameter is supposed to be an OutputStream, and Stdout is not.

> outStream.formatln("= {}\n", valueToReturn); // adds extra \n and flushes

Seems okay, though it's difficult to say from just that line.  Is output redirected?  Are objects being re-used?  Is .close() perhaps not being called if output is to a file?


Sean

Sean Kelly Wrote:

> Jason House wrote:
> > Typical offending code:
> > auto outStream = new Print!(char)(new Layout!(char),Stdout);
> 
> Is this really the line used?  The second parameter is supposed to be an OutputStream, and Stdout is not.


No, I did a quick hack to make it short.  The code was more generic with the last input parameter being passed in from elsewhere.  I took a quick swag at it without verifying with other code.  The real code compiles ;)

Here's the key lines from the class:

Print!(char) outStream;
this(InputStream _inStream=Cin.stream, OutputStream _outStream=Cout.stream, double _pollPeriod=0.1){
  outStream = new Print!(char)(new Layout!(char),_outStream);
  ...


> > outStream.formatln("= {}\n", valueToReturn); // adds extra \n and flushes
> 
> Seems okay, though it's difficult to say from just that line.  Is output redirected?  Are objects being re-used?  Is .close() perhaps not being called if output is to a file?


Output is going to Cout.stream and is never closed.  The code relies on formatln to flush the output.  It does real time communication in plain text with a 3rd party controlling application.  This line really was from the program.  In this particular example, valueToReturn is of type char[] and is literally "HouseBot"

Jason House wrote:
> Sean Kelly Wrote:
> 
>> Jason House wrote:
>>> outStream.formatln("= {}\n", valueToReturn); // adds extra \n and flushes
>> Seems okay, though it's difficult to say from just that line.  Is output redirected?  Are objects being re-used?  Is .close() perhaps not being called if output is to a file?
> 
> 
> Output is going to Cout.stream and is never closed.  The code relies on formatln to flush the output.  It does real time communication in plain text with a 3rd party controlling application.  This line really was from the program.  In this particular example, valueToReturn is of type char[] and is literally "HouseBot"

This part is likely your problem: "It does real time communication in plain text with a 3rd party controlling application"
That probably means the output is redirected and not sent to an actual console, right?
The default behavior for Stdout is to auto-flush only if the output is sent to an actual console (i.e. isn't redirected).

To make sure flushing occurs, execute "Stdout.flush = true;" somewhere at the start of your program (main(), a static this(), or just right before the first output).

Frits van Bommel wrote:
> Jason House wrote:
>> Sean Kelly Wrote:
>>
>>> Jason House wrote:
>>>> outStream.formatln("= {}\n", valueToReturn); // adds extra \n and flushes
>>> Seems okay, though it's difficult to say from just that line.  Is output redirected?  Are objects being re-used?  Is .close() perhaps not being called if output is to a file?
>>
>>
>> Output is going to Cout.stream and is never closed.  The code relies on formatln to flush the output.  It does real time communication in plain text with a 3rd party controlling application.  This line really was from the program.  In this particular example, valueToReturn is of type char[] and is literally "HouseBot"
> 
> This part is likely your problem: "It does real time communication in plain text with a 3rd party controlling application"
> That probably means the output is redirected and not sent to an actual console, right?
> The default behavior for Stdout is to auto-flush only if the output is sent to an actual console (i.e. isn't redirected).
> 
> To make sure flushing occurs, execute "Stdout.flush = true;" somewhere at the start of your program (main(), a static this(), or just right before the first output).


Yeah, Stdout and Cout inhibit *automatic* flush when the console is redirected. If you think about it, the whole automatic flush thing is a 'shortcut' so that output to the real console doesn't need an explicit call to flush() each time. It's a royal PITA to support that :)

Anyway, the most effective route here is simply to invoke flush() explicitly, whenever you actually need to. For example:

# outStream.format("blah blah").flush;

kris wrote:
> Anyway, the most effective route here is simply to invoke flush() explicitly, whenever you actually need to. For example:
> 
> # outStream.format("blah blah").flush;


I thought I checked to verify that formatln calls newline which calls flush.  I therefore concluded that an explicit flush call was useless. My friend had the issue, so I couldn't really test.  When I hear more about testing fixes, I'll post back on the list.

Jason House wrote:
> 
> kris wrote:
>> Anyway, the most effective route here is simply to invoke flush() explicitly, whenever you actually need to. For example:
>>
>> # outStream.format("blah blah").flush;
> 
> 
> I thought I checked to verify that formatln calls newline which calls flush.  I therefore concluded that an explicit flush call was useless. My friend had the issue, so I couldn't really test.  When I hear more about testing fixes, I'll post back on the list.


Well, newline *does* call flush, but only if the console has not been redirected. This was not the case in some earlier Tango releases, but the change was made to accommodate requests for faster throughput (flush is very expensive on some platforms). In short, we may have caused your app to break. Please accept my apologies, and I hope it didn't cause too much trouble for you

Sadly, this simple fix isn't the solution.  Here's what I got back (via e-mail)...

> This part is likely your problem: "It does real time communication in
> plain text with a 3rd party controlling application"
> That probably means the output is redirected and not sent to an actual
> console, right?
> The default behavior for Stdout is to auto-flush only if the output is
> sent to an actual console (i.e. isn't redirected).

Well, it is sent to an actual console. I mean I just start housebot
by hand
and interact with it.

> To make sure flushing occurs, execute "Stdout.flush = true;" somewhere
> at the start of your program (main(), a static this(), or just right
> before the first output).

I tried that. I added the import for tango.io.Stdout to housebot.d
and added
"Stdout.flush = true;" at the beginning of main(), just after the
variable
declarations. But nothing changed.

Given that enabling a flush doesn't solve the problem, what is the path forward for figuring out how to fix this?  It works on all other systems tested with dmd, but doesn't work on mac (with gdc).  (I've had big problems getting gdc and tango to play nicely with each other in a cygwin environment, but that's probably another thread)

Jason House Wrote:

> Sadly, this simple fix isn't the solution.  Here's what I got back (via e-mail)...
> 
> > This part is likely your problem: "It does real time communication in
> > plain text with a 3rd party controlling application"
> > That probably means the output is redirected and not sent to an actual
> > console, right?
> > The default behavior for Stdout is to auto-flush only if the output is
> > sent to an actual console (i.e. isn't redirected).
> 
> Well, it is sent to an actual console. I mean I just start housebot
> by hand
> and interact with it.
> 
> > To make sure flushing occurs, execute "Stdout.flush = true;" somewhere
> > at the start of your program (main(), a static this(), or just right
> > before the first output).
> 
> I tried that. I added the import for tango.io.Stdout to housebot.d
> and added
> "Stdout.flush = true;" at the beginning of main(), just after the
> variable
> declarations. But nothing changed.

For what it's worth, flush() merely writes all data buffered by Tango to the underlying device.  On non-Win32 platforms, there is no actual kernel call performed to force an immediate flush of the data to wherever.  Is it possible that OSX does buffering within the kernel and the data is lingering in there?  And if so, do you know of a routine that could be called to perform a hard flush?  Typically, the commit() method is used for this purpose, but it's empty on Posix systems right now.

Jason House wrote:
> Given that enabling a flush doesn't solve the problem, what is the path forward for figuring out how to fix this?  It works on all other systems tested with dmd, but doesn't work on mac (with gdc).  (I've had big problems getting gdc and tango to play nicely with each other in a cygwin environment, but that's probably another thread)
> 
> Jason House Wrote:
> 
>> Sadly, this simple fix isn't the solution.  Here's what I got back (via e-mail)...
>>
>>> This part is likely your problem: "It does real time communication in
>>> plain text with a 3rd party controlling application"
>>> That probably means the output is redirected and not sent to an actual
>>> console, right?
>>> The default behavior for Stdout is to auto-flush only if the output is
>>> sent to an actual console (i.e. isn't redirected).
>> Well, it is sent to an actual console. I mean I just start housebot
>> by hand
>> and interact with it.
>>
>>> To make sure flushing occurs, execute "Stdout.flush = true;" somewhere
>>> at the start of your program (main(), a static this(), or just right
>>> before the first output).
>> I tried that. I added the import for tango.io.Stdout to housebot.d
>> and added
>> "Stdout.flush = true;" at the beginning of main(), just after the
>> variable
>> declarations. But nothing changed.
>