How to use readText to read utf16 file? Or other encoding file.

On Tuesday, 17 November 2015 at 02:40:14 UTC, Domain wrote:
> How to use readText to read utf16 file?

readText!wstring("filename")

should do it for utf16. It will return a wstring, which is utf-16.

You can do utf32 with readText!dstring. The default, of course, is string, which is utf8.

It doesn't support conversions or any other encoding.

On Tuesday, 17 November 2015 at 02:42:29 UTC, Adam D. Ruppe wrote:
> On Tuesday, 17 November 2015 at 02:40:14 UTC, Domain wrote:
>> How to use readText to read utf16 file?
>
> readText!wstring("filename")
>
> should do it for utf16. It will return a wstring, which is utf-16.
>
> You can do utf32 with readText!dstring. The default, of course, is string, which is utf8.
>
> It doesn't support conversions or any other encoding.

Thanks! But how to remove BOM? Slice the result myself?

On Tuesday, 17 November 2015 at 02:50:44 UTC, Domain wrote:
> Thanks! But how to remove BOM? Slice the result myself?

Yeah. Do something like if(result.length &&result[0] == bom) { result = result[1..$]; } and you'll have it.

On 11/16/15 10:00 PM, Adam D. Ruppe wrote:
> On Tuesday, 17 November 2015 at 02:50:44 UTC, Domain wrote:
>> Thanks! But how to remove BOM? Slice the result myself?
>
> Yeah. Do something like if(result.length &&result[0] == bom) { result =
> result[1..$]; } and you'll have it.

To be technically correct, you can do:

if(!result.empty && result.front == bom) result.popFront();

This should work for all 3 types of strings.

-Steve

On Tuesday, 17 November 2015 at 03:12:47 UTC, Steven Schveighoffer wrote:
> On 11/16/15 10:00 PM, Adam D. Ruppe wrote:
>> On Tuesday, 17 November 2015 at 02:50:44 UTC, Domain wrote:
>>> Thanks! But how to remove BOM? Slice the result myself?
>>
>> Yeah. Do something like if(result.length &&result[0] == bom) { result =
>> result[1..$]; } and you'll have it.
>
> To be technically correct, you can do:
>
> if(!result.empty && result.front == bom) result.popFront();
>
> This should work for all 3 types of strings.
>
> -Steve

Thank you! Now another question: how to handle endianness?

On Tuesday, 17 November 2015 at 05:37:55 UTC, Domain wrote:
>
> Thank you! Now another question: how to handle endianness?

If your file follow a file format:
    the endianness should be defined there.
else it's a random text file:
    either it will have a BOM with endianness,
    or you will have to guess.

On Tuesday, 17 November 2015 at 02:40:14 UTC, Domain wrote:
> How to use readText to read utf16 file? Or other encoding file.

Here's a helpful resource when working with text files in D.

http://nomad.so/2015/09/working-with-files-in-the-d-programming-language/

On Tuesday, 17 November 2015 at 13:00:11 UTC, Gary Willoughby wrote:
> On Tuesday, 17 November 2015 at 02:40:14 UTC, Domain wrote:
>> How to use readText to read utf16 file? Or other encoding file.
>
> Here's a helpful resource when working with text files in D.
>
> http://nomad.so/2015/09/working-with-files-in-the-d-programming-language/

the nomad.so site is now about minecraft, the old content here: https://web.archive.org/web/20171109072958/http://nomad.so/2015/09/working-with-files-in-the-d-programming-language/