Thread overview
[Issue 14641] Use SIMD to accelerate comment lexing
Nov 20, 2016
Walter Bright
Dec 17, 2022
Iain Buclaw
June 01, 2015
https://issues.dlang.org/show_bug.cgi?id=14641

briancschott@gmail.com changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |briancschott@gmail.com

--- Comment #1 from briancschott@gmail.com ---
The best way to do this that I've found is to skip everything other than a set of bytes that varies based on the comment being lexed:

For /* */ comments:
0x0c (\n)
0x0d (\r)
0x2a (*)
0x2f (/)
x0e2 (Beginning of multi-byte UTF-8 newline)

For /+ +/ comments:
0x0c (\n)
0x0d (\r)
0x2b (+)
0x2f (/)
x0e2 (Beginning of multi-byte UTF-8 newline)

For // comments:
0x0c (\n)
0x0d (\r)
x0e2 (Beginning of multi-byte UTF-8 newline)

The instruction used in libdparse to do this is "pcmpestri", which requires SSE4.2 (First released in 2008 according to wikipedia). My advice is to leave most of the logic intact and implement the assembly code such that it may advance the lexer 0 or more bytes, so that the rest of the algorithm is not disrupted on machines that don't support SSE4.2.

--
November 20, 2016
https://issues.dlang.org/show_bug.cgi?id=14641

Walter Bright <bugzilla@digitalmars.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|                            |SIMD

--
December 17, 2022
https://issues.dlang.org/show_bug.cgi?id=14641

Iain Buclaw <ibuclaw@gdcproject.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Priority|P1                          |P4

--
4 days ago
https://issues.dlang.org/show_bug.cgi?id=14641

--- Comment #2 from dlangBugzillaToGithub <robert.schadek@posteo.de> ---
THIS ISSUE HAS BEEN MOVED TO GITHUB

https://github.com/dlang/dmd/issues/18997

DO NOT COMMENT HERE ANYMORE, NOBODY WILL SEE IT, THIS ISSUE HAS BEEN MOVED TO GITHUB

--