August 06, 2011
On 8/5/2011 11:12 AM, Andrej Mitrovic wrote:
> On 8/4/11, Walter Bright<newshound2@digitalmars.com>  wrote:
>> I've used this for 25 years now.
>>
>> http://www.digitalmars.com/rtl/disp.html
>>
>
> That's freaking great. And it's already in snn.lib. However this
> doesn't work on Linux, right?

Right, though porting it wouldn't be hard.

>
> Works great in D on win32. Thanks W.

August 07, 2011
A readText() function that would read a text file (**and** autodetect its encoding from its BOM) would be of great help.

August 07, 2011
On 07.08.2011 12:09, Mehrdad wrote:
> A readText() function that would read a text file (**and** autodetect its encoding from its BOM) would be of great help.
>
Well the name is here, dunno if it meets your expectations:
http://d-programming-language.org/phobos/std_file.html#readText

-- 
Dmitry Olshansky

August 07, 2011
On Sunday 07 August 2011 14:08:06 Dmitry Olshansky wrote:
> On 07.08.2011 12:09, Mehrdad wrote:
> > A readText() function that would read a text file (**and** autodetect
> > its encoding from its BOM) would be of great help.
> 
> Well the name is here, dunno if it meets your expectations: http://d-programming-language.org/phobos/std_file.html#readText

D (and Phobos in general) assumes that char in UTF-8, wchar is UTF-16, and dchar is UTF-32. You're going to get an exception thrown pretty quickly if you're trying to use those types with values that don't match those encodings. As such, readText assumes that the file is in whatever encoding the character type is that it's instantiated with. So, if you try and read in a file which doesn't match the character encoding of the character type that you're using (which is char by default), you're going to get a UtfException.

What Mehrdad wants is a way to read in a file with an encoding other than UTF-8, UTF-16, or UTF-32, have it autodetect the encoding by reading the file's BOM, and then convert it it to whatever encoding is that the character type that readText is using uses. readText doesn't currently do anything of the sort.

At this point, dealing with anything which has an encoding other than UTF-8, UTF-16, or UTF-32 is problematic in D. std.encoding helps, but it's not necessarily all that good (Andrei considers it a failed experiment which either needs to be redesigned or removed). So, one of the things that still needs to be figured out for Phobos is how to better handle encodings other than UTF-8, UTF-16, and UTF-32. For the most part, other encodings are likely to be dealt with only when reading or writing I/O while UTF-8, UTF-16, and UTF-32 are dealt with inside of D programs, but we still need to fix things so that we can readily deal with I/O that isn't UTF-8, UTF-16, or UTF-32.

- Jonathan M Davis
August 07, 2011
On 8/7/11 5:08 AM, Dmitry Olshansky wrote:
> On 07.08.2011 12:09, Mehrdad wrote:
>> A readText() function that would read a text file (**and** autodetect
>> its encoding from its BOM) would be of great help.
>>
> Well the name is here, dunno if it meets your expectations:
> http://d-programming-language.org/phobos/std_file.html#readText

I think we could and should change readText to do the BOM trick. It's been on my mind forever.

Andrei
August 07, 2011
On 8/7/2011 3:21 AM, Jonathan M Davis wrote:
> On Sunday 07 August 2011 14:08:06 Dmitry Olshansky wrote:
>> On 07.08.2011 12:09, Mehrdad wrote:
>>> A readText() function that would read a text file (**and** autodetect
>>> its encoding from its BOM) would be of great help.
>> Well the name is here, dunno if it meets your expectations:
>> http://d-programming-language.org/phobos/std_file.html#readText
> ...
> What Mehrdad wants is a way to read in a file with an encoding other than
> UTF-8, UTF-16, or UTF-32, have it autodetect the encoding by reading the file's
> BOM, and then convert it it to whatever encoding is that the character type
> that readText is using uses.
Yeah, although I don't mean anything /other/ than those -- I only care about Unicode, but I think it should be auto-detected, not based on the template parameter.


On 8/7/2011 6:21 AM, Andrei Alexandrescu wrote:
> I think we could and should change readText to do the BOM trick. It's been on my mind forever.
I /do/ have an implementation, but it's (1) only for Windows, (2) hastily written (no error checking or whatever), and (3) doesn't work for UTF-16 BE (although it works for LE), and (4) only returns the result in UTF-8.
It's a starting point, though. An added bonus is the fact that it actually looks at the file data as well, so the heuristic is rather nice.

    pragma(lib, "advapi32.lib");
    extern(Windows) BOOL IsTextUnicode(in void* pBuffer, int cb, int* lpi);
    string readText(const(char)[] name)
    {
        auto data = cast(char[])file.read(name);
        int test = 0xFFFF;
        if (IsTextUnicode(data.ptr, data.length, &test))
        { return (cast(wchar[])(test & 0x00088 ? data[2 .. $] : data)).toUTF8(); }
        else
        { return (data.startsWith([0xEF, 0xBB, 0xBF]) ? data[3 .. $] : data).toUTF8(); }
    }
August 08, 2011
"Mehrdad" <wfunction@hotmail.com> wrote in message news:j1lh84$2n2j$1@digitalmars.com...
>A readText() function that would read a text file (**and** autodetect its encoding from its BOM) would be of great help.
>

http://www.dsource.org/projects/semitwist/browser/trunk/src/semitwist/util/io.d#L24

I agree a function like that needs to be in Phobos, though.


August 10, 2011
On Sun, 31 Jul 2011 07:27:19 +0200, Jonathan M Davis <jmdavisProg@gmx.com> wrote:

> So, what major functionality which we don't currently have would you like to
> see in either Phobos or in a 3rd party library so that you could use it in
> your D programs?

Database connectivity, XML, OpenGL bindings, full OS bindings,
vector math (think games).

-- 
  Simen
August 10, 2011
On Sun, 31 Jul 2011 15:49:42 +0200, bearophile <bearophileHUGS@lycos.com> wrote:

> - std.operator: standard operators as functions, to be used with higher order functions;

Nice idea. So simple yet so useful.

-- 
  Simen
August 10, 2011
On Tue, 02 Aug 2011 20:04:40 +0200, Paul D. Anderson <paul.d.removethis.anderson@comcast.andthis.net> wrote:

> I'm nearing completion on an arbitrary-precision floating point library, along with implementations of decimal32, decimal64 and decimal128 structs.

Awesome.

-- 
  Simen
5 6 7 8 9 10 11 12 13 14 15 16
Top | Discussion index | About this forum | D home