This is the signature of a function of std.ascii:

http://dlang.org/phobos/std_ascii.html#toLower

pure nothrow @safe dchar toLower(dchar c);

If this function is supposed to be used on ASCII strings, what's the point of returning a dchar? When I use it I have usually to cast its result back to char, and I prefer to avoid casts in my code in D.

Bye,
bearophile

Sorry, the thread title was "About std.ascii.toLower"...

On Thursday, 20 September 2012 at 16:00:18 UTC, bearophile wrote:
> This is the signature of a function of std.ascii:
>
> http://dlang.org/phobos/std_ascii.html#toLower
>
> pure nothrow @safe dchar toLower(dchar c);
>
> If this function is supposed to be used on ASCII strings, what's the point of returning a dchar? When I use it I have usually to cast its result back to char, and I prefer to avoid casts in my code in D.
>
> Bye,
> bearophile

It's not, it only *operates* on ASCII, but non ascii is still a legal arg:

----
import std.stdio;
import std.ascii;

void main(){
    string s = "héllö";
    write("\"");
    foreach(c; s)
        write(c.toUpper);
    write("\"");
}
----
HéLLö
----

monarch_dodra:

> It's not, it only *operates* on ASCII, but non ascii is still a legal arg:

Then maybe std.ascii.toLower needs a pre-condition that constraints it to just ASCII inputs, so it's free to return a char.

Bye,
bearophile

On Thursday, 20 September 2012 at 16:34:22 UTC, bearophile wrote:
> monarch_dodra:
>
>> It's not, it only *operates* on ASCII, but non ascii is still a legal arg:
>
> Then maybe std.ascii.toLower needs a pre-condition that constraints it to just ASCII inputs, so it's free to return a char.
>
> Bye,
> bearophile

I was thinking the exact same thing right after replying actually.

Would that actually change anything though? I mean what with alignment and everything, wouldn't returning a char be just as expansive? I'm not 100% sure. What is your use case that would require this?

monarch_dodra:

> Would that actually change anything though? I mean what with alignment and everything, wouldn't returning a char be just as expansive? I'm not 100% sure.

If you are thinking about the number of operations, then it's the same, as both a char and dchar value go in a register. The run time is the same, especially after inlining.


> What is your use case that would require this?

I have a char[] like:

['a','x','b','a','c','x','f']

Every char encodes something. Putting it to upper case means that that data was already used:

['a','X','b','a','C','x','f']

In this case to use toUpper I have to use:

cast(char)toUpper(foo[1])

What's I am trying to minimize is the number of cast(). On the other hand even in C toupper returns a type larger than char:

http://www.acm.uiuc.edu/webmonkeys/book/c_guide/2.2.html

It's just D has contract programming, and this module is written for ASCII, so it's able to be smarter than C functions, and return a char.

Bye,
bearophile

On Thursday, September 20, 2012 18:35:21 bearophile wrote:
> monarch_dodra:
> > It's not, it only *operates* on ASCII, but non ascii is still a
> 
> > legal arg:
> Then maybe std.ascii.toLower needs a pre-condition that constraints it to just ASCII inputs, so it's free to return a char.

Goodness no.

1. Operating on a char is almost always the wrong thing to do. If you really want to do that, then cast. It should _not_ be encouraged.

2. It would be disastrous if std.ascii's funtions didn't work on unicode. Right now, you can use them with ranges on strings which are unicode, which can be very useful. I grant you that that's more obvious with something like isDigit than toLower, but regardless, std.ascii is designed such that its functions will all operate on unicode strings. It just doesn't alter unicode characters and returns false for them with any of the query functions.

- Jonathan M Davis

On Thursday, 20 September 2012 at 17:05:18 UTC, bearophile wrote:
> monarch_dodra:
>
>> Would that actually change anything though? I mean what with alignment and everything, wouldn't returning a char be just as expansive? I'm not 100% sure.
>
> If you are thinking about the number of operations, then it's the same, as both a char and dchar value go in a register. The run time is the same, especially after inlining.
>
>
>> What is your use case that would require this?
>
> I have a char[] like:
>
> ['a','x','b','a','c','x','f']
>
> Every char encodes something. Putting it to upper case means that that data was already used:
>
> ['a','X','b','a','C','x','f']
>
> In this case to use toUpper I have to use:
>
> cast(char)toUpper(foo[1])
>
> What's I am trying to minimize is the number of cast(). On the other hand even in C toupper returns a type larger than char:
>
> http://www.acm.uiuc.edu/webmonkeys/book/c_guide/2.2.html
>
> It's just D has contract programming, and this module is written for ASCII, so it's able to be smarter than C functions, and return a char.
>
> Bye,
> bearophile

That's what I thought. You have a valid point (IMO) but at the same time, using the ASCII methods on non-ascii characters is also legit operation.

I guess we'd need the extra "std.strictascii" module (!) for operations that would accept ASCII char, and return a ASCII char.

I'd support such an ER, I think it would. Allow users (such as you) to have tighter constraints if needed, while still keeping std.ascii for "safer" ASCII operations.

Jonathan M Davis:

> Goodness no.

:-)


> 1. Operating on a char is almost always the wrong thing to do.

A single char is often not so useful but I have to keep many mutable chars, keeping them as char[] instead of dchar[] saves both memory and reduces cache misses. The same is true for types like short or float, single ones are not so useful, but they sometimes become useful when you have many of them in arrays.

If I have to modify such char[], using toUpper() requires me a cast. And in my opinion it's not a good idea to return a dchar if you know the both the input and output of the function are a char.


> If you really want to do that, then cast.

On the other hand casts in D have a certain risk, so reducing their number as much as possible is a good idea.


> It should _not_ be encouraged.

This is silly, see the above explanation.


> 2. It would be disastrous if std.ascii's funtions didn't work on unicode.
> Right now, you can use them with ranges on strings which are unicode, which
> can be very useful. [...] but regardless, std.ascii is designed such that its
> functions will all operate on unicode strings. It just doesn't alter unicode
> characters and returns false for them with any of the query functions.

I see, and I didn't know this, I have misunderstood. I have thought of std.ascii functions as functions meant to work on just ASCII characters/text. But they are better defined as Unicode-passing functions. And yeah, it's written at the top of the module:

>Functions which operate on ASCII characters. All of the functions in std.ascii accept unicode characters but effectively ignore them. All isX functions return false for unicode characters, and all toX functions do nothing to unicode characters.<

So now I'd like a new set of functions designed for ASCII text, with contracts to refuse not-ASCII things ;-)

Thank you for the answers Jonathan.

Bye,
bearophile

On Thursday, 20 September 2012 at 17:32:52 UTC, bearophile wrote:
> Jonathan M Davis:
>>Functions which operate on ASCII characters. All of the functions in std.ascii accept unicode characters but effectively ignore them. All isX functions return false for unicode characters, and all toX functions do nothing to unicode characters.<
>
> So now I'd like a new set of functions designed for ASCII text, with contracts to refuse not-ASCII things ;-)
>
> Thank you for the answers Jonathan.
>
> Bye,
> bearophile

What do you (you two) think of my proposition for a "std.strictascii" module?

The signatures would be:
char toLower(dchar c);

And the implementations be like:

----
char toLower(dchar c)
in
{
    assert(c.std.ascii.isAscii());
}
body
{
    cast(char) c.std.ascii.toLower();
}
----

The rational for taking a dchar as input is so that it's own input can be correctly validated, and so that it can easilly operate with foreach etc, doing the cast internally. The returned value would be pre-cast to char.

Usage:

----
import std.stdio;
import std.strictascii;

void main(){
    string s1 = "axbacxf";
    string s2 = "àxbécxf";
    char[] cs = new char[](7);

    //bearophile use case: no casts
    foreach(i, c; s1)
        cs[i] = c.toUpper();

    //illegal use case: correct input validation
    foreach(i, c; s1)
        cs[i] = c.toUpper(); //in assert
}
----

It doesn't add *much* functionality, and arguably, it is a specialized functionality, but there are usecases where you want to operate ONLY on ascii, as pointed out by bearophile.

Just curious if I should even consider investing some effort in this.