View mode: basic / threaded / horizontal-split · Log in · Help
August 06, 2011
Re: What library functionality would you most like to see in D?
On 8/5/2011 11:12 AM, Andrej Mitrovic wrote:
> On 8/4/11, Walter Bright<newshound2@digitalmars.com>  wrote:
>> I've used this for 25 years now.
>>
>> http://www.digitalmars.com/rtl/disp.html
>>
>
> That's freaking great. And it's already in snn.lib. However this
> doesn't work on Linux, right?

Right, though porting it wouldn't be hard.

>
> Works great in D on win32. Thanks W.
August 07, 2011
Re: What library functionality would you most like to see in D?
A readText() function that would read a text file (**and** autodetect 
its encoding from its BOM) would be of great help.
August 07, 2011
Re: What library functionality would you most like to see in D?
On 07.08.2011 12:09, Mehrdad wrote:
> A readText() function that would read a text file (**and** autodetect 
> its encoding from its BOM) would be of great help.
>
Well the name is here, dunno if it meets your expectations:
http://d-programming-language.org/phobos/std_file.html#readText

-- 
Dmitry Olshansky
August 07, 2011
Re: What library functionality would you most like to see in D?
On Sunday 07 August 2011 14:08:06 Dmitry Olshansky wrote:
> On 07.08.2011 12:09, Mehrdad wrote:
> > A readText() function that would read a text file (**and** autodetect
> > its encoding from its BOM) would be of great help.
> 
> Well the name is here, dunno if it meets your expectations:
> http://d-programming-language.org/phobos/std_file.html#readText

D (and Phobos in general) assumes that char in UTF-8, wchar is UTF-16, and 
dchar is UTF-32. You're going to get an exception thrown pretty quickly if 
you're trying to use those types with values that don't match those encodings. 
As such, readText assumes that the file is in whatever encoding the character 
type is that it's instantiated with. So, if you try and read in a file which 
doesn't match the character encoding of the character type that you're using 
(which is char by default), you're going to get a UtfException.

What Mehrdad wants is a way to read in a file with an encoding other than 
UTF-8, UTF-16, or UTF-32, have it autodetect the encoding by reading the file's 
BOM, and then convert it it to whatever encoding is that the character type 
that readText is using uses. readText doesn't currently do anything of the 
sort. 

At this point, dealing with anything which has an encoding other than UTF-8, 
UTF-16, or UTF-32 is problematic in D. std.encoding helps, but it's not 
necessarily all that good (Andrei considers it a failed experiment which 
either needs to be redesigned or removed). So, one of the things that still 
needs to be figured out for Phobos is how to better handle encodings other than 
UTF-8, UTF-16, and UTF-32. For the most part, other encodings are likely to be 
dealt with only when reading or writing I/O while UTF-8, UTF-16, and UTF-32 
are dealt with inside of D programs, but we still need to fix things so that we 
can readily deal with I/O that isn't UTF-8, UTF-16, or UTF-32.

- Jonathan M Davis
August 07, 2011
Re: What library functionality would you most like to see in D?
On 8/7/11 5:08 AM, Dmitry Olshansky wrote:
> On 07.08.2011 12:09, Mehrdad wrote:
>> A readText() function that would read a text file (**and** autodetect
>> its encoding from its BOM) would be of great help.
>>
> Well the name is here, dunno if it meets your expectations:
> http://d-programming-language.org/phobos/std_file.html#readText

I think we could and should change readText to do the BOM trick. It's 
been on my mind forever.

Andrei
August 07, 2011
Re: What library functionality would you most like to see in D?
On 8/7/2011 3:21 AM, Jonathan M Davis wrote:
> On Sunday 07 August 2011 14:08:06 Dmitry Olshansky wrote:
>> On 07.08.2011 12:09, Mehrdad wrote:
>>> A readText() function that would read a text file (**and** autodetect
>>> its encoding from its BOM) would be of great help.
>> Well the name is here, dunno if it meets your expectations:
>> http://d-programming-language.org/phobos/std_file.html#readText
> ...
> What Mehrdad wants is a way to read in a file with an encoding other than
> UTF-8, UTF-16, or UTF-32, have it autodetect the encoding by reading the file's
> BOM, and then convert it it to whatever encoding is that the character type
> that readText is using uses.
Yeah, although I don't mean anything /other/ than those -- I only care 
about Unicode, but I think it should be auto-detected, not based on the 
template parameter.


On 8/7/2011 6:21 AM, Andrei Alexandrescu wrote:
> I think we could and should change readText to do the BOM trick. It's 
> been on my mind forever.
I /do/ have an implementation, but it's (1) only for Windows, (2) 
hastily written (no error checking or whatever), and (3) doesn't work 
for UTF-16 BE (although it works for LE), and (4) only returns the 
result in UTF-8.
It's a starting point, though. An added bonus is the fact that it 
actually looks at the file data as well, so the heuristic is rather nice.

    pragma(lib, "advapi32.lib");
    extern(Windows) BOOL IsTextUnicode(in void* pBuffer, int cb, int* lpi);
    string readText(const(char)[] name)
    {
        auto data = cast(char[])file.read(name);
        int test = 0xFFFF;
        if (IsTextUnicode(data.ptr, data.length, &test))
        { return (cast(wchar[])(test & 0x00088 ? data[2 .. $] : 
data)).toUTF8(); }
        else
        { return (data.startsWith([0xEF, 0xBB, 0xBF]) ? data[3 .. $] : 
data).toUTF8(); }
    }
August 08, 2011
Re: What library functionality would you most like to see in D?
"Mehrdad" <wfunction@hotmail.com> wrote in message 
news:j1lh84$2n2j$1@digitalmars.com...
>A readText() function that would read a text file (**and** autodetect its 
>encoding from its BOM) would be of great help.
>

http://www.dsource.org/projects/semitwist/browser/trunk/src/semitwist/util/io.d#L24

I agree a function like that needs to be in Phobos, though.
August 10, 2011
Re: What library functionality would you most like to see in D?
On Sun, 31 Jul 2011 07:27:19 +0200, Jonathan M Davis <jmdavisProg@gmx.com>  
wrote:

> So, what major functionality which we don't currently have would you  
> like to
> see in either Phobos or in a 3rd party library so that you could use it  
> in
> your D programs?

Database connectivity, XML, OpenGL bindings, full OS bindings,
vector math (think games).

-- 
  Simen
August 10, 2011
Re: What library functionality would you most like to see in D?
On Sun, 31 Jul 2011 15:49:42 +0200, bearophile <bearophileHUGS@lycos.com>  
wrote:

> - std.operator: standard operators as functions, to be used with higher  
> order functions;

Nice idea. So simple yet so useful.

-- 
  Simen
August 10, 2011
Re: What library functionality would you most like to see in D?
On Tue, 02 Aug 2011 20:04:40 +0200, Paul D. Anderson  
<paul.d.removethis.anderson@comcast.andthis.net> wrote:

> I'm nearing completion on an arbitrary-precision floating point library,  
> along with implementations of decimal32, decimal64 and decimal128  
> structs.

Awesome.

-- 
  Simen
11 12 13 14 15 16
Top | Discussion index | About this forum | D home