http://d.puremagic.com/issues/show_bug.cgi?id=10717

           Summary: std.ascii.toLower and toUpper should return char
                    instead of dchar and avoid me to use a bad cast(char)
           Product: D
           Version: D2
          Platform: All
        OS/Version: All
            Status: NEW
          Severity: normal
          Priority: P2
         Component: Phobos
        AssignedTo: nobody@puremagic.com
        ReportedBy: bearophile_hugs@eml.cc


--- Comment #0 from bearophile_hugs@eml.cc 2013-07-26 05:06:14 PDT ---
If I use functions from std.ascii I am stating that I am working with ASCII chars.

Also std.ascii docs say: "All of the functions in std.ascii accept unicode characters but effectively ignore them."

So if I use std.ascii.toLower on an ASCII char, in most times (all times so
far) I want the result to be a char. If std.ascii.toLower returns a dchar it
forces me to use a cast(char) in most (or all) cases. Such cast doesn't
increase safety at all, it actually decreases it.

So I'd like std.ascii.toLower and std.ascii.toUpper to return char.

(This has a severity 'normal' instead of 'enhancement' because I regard that as a bug in the design of those two functions.)

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=10717


Jonathan M Davis <jmdavisProg@gmx.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |jmdavisProg@gmx.com
           Severity|normal                      |enhancement


--- Comment #1 from Jonathan M Davis <jmdavisProg@gmx.com> 2013-07-26 05:17:49 PDT ---
I really don't think that this is a bug. By taking dchar, they work with ranges of dchar and allow you to operate on strings that contain Unicode in cases where all you care about are particular ASCII characters (like when you only care about ASCII whitespace in a string - not Unicode whitespace - but the string contains Unicode characters). And as soon the functions accept dchar, they must return dchar, or they'll destroy any Unicode characters that get passed in.

A reasonable enhancement request would be to overload these functions with overloads which take char and return char, but there is actual value in having them accept and return dchar, and it would break existing code if they stopped. So, the functions that are there are there to stay, but you may get overloads which operate specifically on char.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=10717



--- Comment #2 from bearophile_hugs@eml.cc 2013-07-26 07:33:37 PDT ---
(In reply to comment #1)

> So, the functions that are there are there to stay, but you may get overloads which operate specifically on char.

OK. Thank for your comments and corrections Jonathan.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=10717


Jonathan M Davis <jmdavisProg@gmx.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|                            |pull


--- Comment #3 from Jonathan M Davis <jmdavisProg@gmx.com> 2013-07-27 02:41:04 PDT ---
https://github.com/D-Programming-Language/phobos/pull/1436

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=10717



--- Comment #4 from bearophile_hugs@eml.cc 2013-07-27 05:46:05 PDT ---
(In reply to comment #3)
> https://github.com/D-Programming-Language/phobos/pull/1436

Thank you again Jonathan :-)

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=10717


monarchdodra@gmail.com changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |monarchdodra@gmail.com


--- Comment #5 from monarchdodra@gmail.com 2013-07-31 02:43:39 PDT ---
(In reply to comment #4)
> (In reply to comment #3)
> > https://github.com/D-Programming-Language/phobos/pull/1436
> 
> Thank you again Jonathan :-)

The new semantics are:

//----
If $(D c) is an uppercase ASCII character, then its corresponding lowercase
letter is returned. Otherwise, $(D c) is returned.

$(D C) can be any type which implicitly converts to $(D dchar). In the case
where it's a built-in type, $(D Unqual!C) is returned, whereas if it's a
user-defined type, $(D dchar) is returned.
//----

Does this fit the bill, or do you see anything else that needs to be addressed?

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=10717



--- Comment #6 from bearophile_hugs@eml.cc 2013-07-31 05:15:58 PDT ---
(In reply to comment #5)

> Does this fit the bill, or do you see anything else that needs to be addressed?

I have tried it, and the semantics seems OK.


But the implementation is:

auto toLower(C)(C c) if(is(C : dchar)) {
    static if(isScalarType!C)
        return isUpper(c) ? cast(Unqual!C)(c + 'a' - 'A') : cast(Unqual!C)c;
    else
        return toLower!dchar(c);
}


So a call to toLower can cause two function calls. toLower() is a tiny function that could be called many many times, so perhaps in debug mode (without inlining) doubling the number of function calls slows down the user code a bit.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=10717



--- Comment #7 from monarchdodra@gmail.com 2013-07-31 12:26:58 PDT ---
(In reply to comment #6)
> So a call to toLower can cause two function calls. toLower() is a tiny function that could be called many many times, so perhaps in debug mode (without inlining) doubling the number of function calls slows down the user code a bit.

Well, it can cause two function calls only for user defined types, so the case should not be that common. Doing it this way means you'll only end up instanciating `toLower!dchar` for all user types. Which is also a plus.

One of the implementation I had suggested was:

////--------
//Public template that only filters and changes return type
auto toUpper(C)(C c) @safe pure nothrow
    if (is(C : dchar))
out(result)
{
    assert(!isLower(result));
}
body
{
    alias Ret = Select!(isScalarType!C, C, dchar);
    return cast(Ret) toUpperImpl(c);
}

//Non template that does actual job.
private dchar toUpperImpl(dchar c) @safe pure nothrow
{
    return isLower(c) ? cast(dchar)(c - ('a' - 'A')) : c;
}
////--------

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=10717



--- Comment #8 from github-bugzilla@puremagic.com 2013-08-02 01:49:55 PDT ---
Commits pushed to master at https://github.com/D-Programming-Language/phobos

https://github.com/D-Programming-Language/phobos/commit/28c1a8480a61054e3ea64b3e55b8c90a3c1754a0 Fix issue# 10717.

This makes it so that if you're operating on chars or wchars with std.ascii functions, you don't have to cast the return values back to char or wchar, but it still allows you to operate on dchars without mangling any Unicode values.

https://github.com/D-Programming-Language/phobos/commit/f7a593fe9cb0567bbb59a79653ae3a8538ffc7b0 Merge pull request #1436 from jmdavis/10717

Fix issue# 10717.

Merged.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=10717


Jonathan M Davis <jmdavisProg@gmx.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|                            |FIXED


-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------