Thread overview | |||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
July 30, 2012 UTF8 + SIMD = win | ||||
---|---|---|---|---|
| ||||
http://woboq.com/blog/utf-8-processing-using-simd.html All in the article. As D include Unicode as a language feature, I think it is interesting to mention here. |
July 30, 2012 Re: UTF8 + SIMD = win | ||||
---|---|---|---|---|
| ||||
Posted in reply to deadalnix | deadalnix:
> http://woboq.com/blog/utf-8-processing-using-simd.html
So many things to do, so little time to do them :-)
Bye,
bearophile
|
July 31, 2012 Re: UTF8 + SIMD = win | ||||
---|---|---|---|---|
| ||||
Posted in reply to deadalnix | On 07/30/12 21:13, deadalnix wrote:
> http://woboq.com/blog/utf-8-processing-using-simd.html
>
> All in the article. As D include Unicode as a language feature, I think it is interesting to mention here.
Very interesting, thx for sharing. This NG definitely is a horn of plenty :)
|
July 31, 2012 Re: UTF8 + SIMD = win | ||||
---|---|---|---|---|
| ||||
Posted in reply to deadalnix | On 7/30/2012 12:13 PM, deadalnix wrote: > http://woboq.com/blog/utf-8-processing-using-simd.html > > All in the article. As D include Unicode as a language feature, I think it is > interesting to mention here. If someone wants to fix std.utf http://dlang.org/phobos/std_utf.html to use SIMD instructions, that would be cool! |
July 31, 2012 Re: UTF8 + SIMD = win | ||||
---|---|---|---|---|
| ||||
Posted in reply to Walter Bright | Walter Bright:
> to use SIMD instructions, that would be cool!
I think in D the most needed UTF operation is UTF8 -> UTF32.
Bye,
bearophile
|
July 31, 2012 Re: UTF8 + SIMD = win | ||||
---|---|---|---|---|
| ||||
Posted in reply to bearophile | On Tuesday, 31 July 2012 at 10:57:23 UTC, bearophile wrote:
> Walter Bright:
>
>> to use SIMD instructions, that would be cool!
>
> I think in D the most needed UTF operation is UTF8 -> UTF32.
>
> Bye,
> bearophile
Where is UTF-32 actually used?
|
July 31, 2012 Re: UTF8 + SIMD = win | ||||
---|---|---|---|---|
| ||||
Posted in reply to Bernard Helyer | Bernard Helyer:
> Where is UTF-32 actually used?
I think all std.algorithm and std.range yield UTF-32 dchars, when you give them a string in input.
Bye,
bearophile
|
July 31, 2012 Re: UTF8 + SIMD = win | ||||
---|---|---|---|---|
| ||||
Posted in reply to bearophile | On Tuesday, 31 July 2012 at 12:11:25 UTC, bearophile wrote:
> Bernard Helyer:
>
>> Where is UTF-32 actually used?
>
> I think all std.algorithm and std.range yield UTF-32 dchars, when you give them a string in input.
>
> Bye,
> bearophile
In addition, foreach over a string with a dchar loop variable does implicit UTF-8 decoding.
|
July 31, 2012 Re: UTF8 + SIMD = win | ||||
---|---|---|---|---|
| ||||
Posted in reply to Jakob Ovrum | On 7/31/2012 5:24 AM, Jakob Ovrum wrote:
> On Tuesday, 31 July 2012 at 12:11:25 UTC, bearophile wrote:
>> Bernard Helyer:
>>
>>> Where is UTF-32 actually used?
>>
>> I think all std.algorithm and std.range yield UTF-32 dchars, when you give
>> them a string in input.
>>
>> Bye,
>> bearophile
>
> In addition, foreach over a string with a dchar loop variable does implicit
> UTF-8 decoding.
>
SIMD isn't going to speed things up at all for decoding one character. It is for transcoding a large array.
|
July 31, 2012 Re: UTF8 + SIMD = win | ||||
---|---|---|---|---|
| ||||
Posted in reply to Walter Bright | On Tuesday, 31 July 2012 at 19:28:03 UTC, Walter Bright wrote:
> On 7/31/2012 5:24 AM, Jakob Ovrum wrote:
>> On Tuesday, 31 July 2012 at 12:11:25 UTC, bearophile wrote:
>>> Bernard Helyer:
>>>
>>>> Where is UTF-32 actually used?
>>>
>>> I think all std.algorithm and std.range yield UTF-32 dchars, when you give
>>> them a string in input.
>>>
>>> Bye,
>>> bearophile
>>
>> In addition, foreach over a string with a dchar loop variable does implicit
>> UTF-8 decoding.
>>
>
> SIMD isn't going to speed things up at all for decoding one character. It is for transcoding a large array.
Duh, good point, I totally forgot the context.
|
Copyright © 1999-2021 by the D Language Foundation