I need some help understanding where extra '\r' come from when output is redirected to file on Windows.

First, this works correctly:
 rdmd --eval="(\"hello\" ~ newline).toFile(\"out.txt\");"
As expected, out.txt contains "hello\r\n".

I would expect the following to do the same, but it doesn't:
 rdmd --eval="write(\"hello\" ~ newline);" > out.txt
Now out.txt contains "hello\r\r\n".

Who is doing the extra conversion here, and how do I stop it?

Thanks!
Bastiaan.

On 04/06/2018 3:24 AM, Bastiaan Veelo wrote:
> I need some help understanding where extra '\r' come from when output is redirected to file on Windows.
> 
> First, this works correctly:
>   rdmd --eval="(\"hello\" ~ newline).toFile(\"out.txt\");"
> As expected, out.txt contains "hello\r\n".
> 
> I would expect the following to do the same, but it doesn't:
>   rdmd --eval="write(\"hello\" ~ newline);" > out.txt
> Now out.txt contains "hello\r\r\n".
> 
> Who is doing the extra conversion here, and how do I stop it?
> 
> Thanks!
> Bastiaan.

That would be cmd. Not sure you can stop it without piping it after rdmd to remove the \r.

On Sunday, 3 June 2018 at 15:42:48 UTC, rikki cattermole wrote:
> On 04/06/2018 3:24 AM, Bastiaan Veelo wrote:
>> I need some help understanding where extra '\r' come from when output is redirected to file on Windows.
>> 
>> First, this works correctly:
>>   rdmd --eval="(\"hello\" ~ newline).toFile(\"out.txt\");"
>> As expected, out.txt contains "hello\r\n".
>> 
>> I would expect the following to do the same, but it doesn't:
>>   rdmd --eval="write(\"hello\" ~ newline);" > out.txt
>> Now out.txt contains "hello\r\r\n".
>> 
>> Who is doing the extra conversion here, and how do I stop it?
>> 
>> Thanks!
>> Bastiaan.
>
> That would be cmd. Not sure you can stop it without piping it after rdmd to remove the \r.

Thanks. It is starting to dawn on me that I shouldn't use `newline` and `toFile` to write text files, but rather always use "\n" as line ending and use `write` for both writing to stdout and file.
 rdmd --eval="File(\"out.txt\", \"w\").write(\"hello\n\");"
and
 rdmd --eval="write(\"hello\n\");" > out.txt
both produce "hello\r\n" on Windows.

Am I correct, or is there a more idiomatic way of writing strings to text files?

On 6/3/18 12:24 PM, Bastiaan Veelo wrote:
> On Sunday, 3 June 2018 at 15:42:48 UTC, rikki cattermole wrote:
>> On 04/06/2018 3:24 AM, Bastiaan Veelo wrote:
>>> I need some help understanding where extra '\r' come from when output is redirected to file on Windows.
>>>
>>> First, this works correctly:
>>>   rdmd --eval="(\"hello\" ~ newline).toFile(\"out.txt\");"
>>> As expected, out.txt contains "hello\r\n".
>>>
>>> I would expect the following to do the same, but it doesn't:
>>>   rdmd --eval="write(\"hello\" ~ newline);" > out.txt
>>> Now out.txt contains "hello\r\r\n".
>>>
>>> Who is doing the extra conversion here, and how do I stop it?
>>>
>>> Thanks!
>>> Bastiaan.
>>
>> That would be cmd. Not sure you can stop it without piping it after rdmd to remove the \r.

No, it's not cmd. It's File, or more specifically, FILE * from C.

> 
> Thanks. It is starting to dawn on me that I shouldn't use `newline` and `toFile` to write text files, but rather always use "\n" as line ending and use `write` for both writing to stdout and file.
>   rdmd --eval="File(\"out.txt\", \"w\").write(\"hello\n\");"
> and
>   rdmd --eval="write(\"hello\n\");" > out.txt
> both produce "hello\r\n" on Windows.
> 
> Am I correct, or is there a more idiomatic way of writing strings to text files?
> 

Windows C library has this bizarro mode for FILE * called "text" mode, which is the default. In this mode, it scans all output, and anywhere it sees a '\n', it replaces it with "\r\n".

the `newline` variable contains "\r\n". So what you have done is, output "hello\r\n", and File helpfully replaces that "\n" to "\r\n", giving you "\r\r\n".

The correct answer, as you have guessed, is don't use newline :) Just use \n. It's portable, and line endings on Windows are less important these days.

If you want to turn off text mode, set the mode to binary, as this should work (note the "wb" mode):

rdmd --eval="File(\"out.txt\", \"wb\").write(\"hello\" ~ newline);"

Note there is also a rawWrite method, which temporarily turns it into binary mode, and will work as well:

rdmd --eval="File(\"out.txt\", \"w\").rawWrite(\"hello\" ~ newline);"

-Steve

On Monday, 4 June 2018 at 15:31:04 UTC, Steven Schveighoffer wrote:
> Windows C library has this bizarro mode for FILE * called "text" mode, which is the default. In this mode, it scans all output, and anywhere it sees a '\n', it replaces it with "\r\n".

Thanks, Steven.