View mode: basic / threaded / horizontal-split · Log in · Help
July 30, 2012
UTF8 + SIMD = win
http://woboq.com/blog/utf-8-processing-using-simd.html

All in the article. As D include Unicode as a language feature, I think 
it is interesting to mention here.
July 30, 2012
Re: UTF8 + SIMD = win
deadalnix:
> http://woboq.com/blog/utf-8-processing-using-simd.html

So many things to do, so little time to do them :-)

Bye,
bearophile
July 31, 2012
Re: UTF8 + SIMD = win
On 07/30/12 21:13, deadalnix wrote:
> http://woboq.com/blog/utf-8-processing-using-simd.html
> 
> All in the article. As D include Unicode as a language feature, I think
> it is interesting to mention here.

Very interesting, thx for sharing. This NG definitely is a horn of plenty :)
July 31, 2012
Re: UTF8 + SIMD = win
On 7/30/2012 12:13 PM, deadalnix wrote:
> http://woboq.com/blog/utf-8-processing-using-simd.html
>
> All in the article. As D include Unicode as a language feature, I think it is
> interesting to mention here.

If someone wants to fix std.utf

   http://dlang.org/phobos/std_utf.html

to use SIMD instructions, that would be cool!
July 31, 2012
Re: UTF8 + SIMD = win
Walter Bright:

> to use SIMD instructions, that would be cool!

I think in D the most needed UTF operation is UTF8 -> UTF32.

Bye,
bearophile
July 31, 2012
Re: UTF8 + SIMD = win
On Tuesday, 31 July 2012 at 10:57:23 UTC, bearophile wrote:
> Walter Bright:
>
>> to use SIMD instructions, that would be cool!
>
> I think in D the most needed UTF operation is UTF8 -> UTF32.
>
> Bye,
> bearophile

Where is UTF-32 actually used?
July 31, 2012
Re: UTF8 + SIMD = win
Bernard Helyer:

> Where is UTF-32 actually used?

I think all std.algorithm and std.range yield UTF-32 dchars, when 
you give them a string in input.

Bye,
bearophile
July 31, 2012
Re: UTF8 + SIMD = win
On Tuesday, 31 July 2012 at 12:11:25 UTC, bearophile wrote:
> Bernard Helyer:
>
>> Where is UTF-32 actually used?
>
> I think all std.algorithm and std.range yield UTF-32 dchars, 
> when you give them a string in input.
>
> Bye,
> bearophile

In addition, foreach over a string with a dchar loop variable 
does implicit UTF-8 decoding.
July 31, 2012
Re: UTF8 + SIMD = win
On 7/31/2012 5:24 AM, Jakob Ovrum wrote:
> On Tuesday, 31 July 2012 at 12:11:25 UTC, bearophile wrote:
>> Bernard Helyer:
>>
>>> Where is UTF-32 actually used?
>>
>> I think all std.algorithm and std.range yield UTF-32 dchars, when you give
>> them a string in input.
>>
>> Bye,
>> bearophile
>
> In addition, foreach over a string with a dchar loop variable does implicit
> UTF-8 decoding.
>

SIMD isn't going to speed things up at all for decoding one character. It is for 
transcoding a large array.
July 31, 2012
Re: UTF8 + SIMD = win
On Tuesday, 31 July 2012 at 19:28:03 UTC, Walter Bright wrote:
> On 7/31/2012 5:24 AM, Jakob Ovrum wrote:
>> On Tuesday, 31 July 2012 at 12:11:25 UTC, bearophile wrote:
>>> Bernard Helyer:
>>>
>>>> Where is UTF-32 actually used?
>>>
>>> I think all std.algorithm and std.range yield UTF-32 dchars, 
>>> when you give
>>> them a string in input.
>>>
>>> Bye,
>>> bearophile
>>
>> In addition, foreach over a string with a dchar loop variable 
>> does implicit
>> UTF-8 decoding.
>>
>
> SIMD isn't going to speed things up at all for decoding one 
> character. It is for transcoding a large array.

Duh, good point, I totally forgot the context.
« First   ‹ Prev
1 2
Top | Discussion index | About this forum | D home