On Thursday, 3 October 2013 at 11:04:26 UTC, Dicebot wrote:
> Yes.
>
> ( I have not found any rules that prohibit review manager from voting :) )

I'd love to say yes, since I've been dreaming of the day when we finally have a lexer... but I decided to put my yes under the condition that it can lex itself using ctfe.

My first attempt with adding a "import(__FILE__)" unittest failed with v2.063.2:

Error: memcpy cannot be interpreted at compile time, because it has no available source code
lexer.d(1966):       called from here: move(lex)
lexer.d(454):        called from here: r.this(lexerSource(range), config)

Maybe this is this fixed in HEAD though?

On Wednesday, 2 October 2013 at 14:41:56 UTC, Dicebot wrote:
> After brief discussion with Brian and gathering data from the review thread, I have decided to start voting for `std.d.lexer` inclusion into Phobos.
>
> -----------------------------------------------------
>[...]
>
> Thanks for your attention.

Yes.

(Btw, someone got benchmarks of std.d.lexer?
I remember that Brain was benchmarking his module quite a lot in order to catch up with DMD's lexer but I can't find links in IRC logs. I wonder if he achieved his goal in this regard)

On Thursday, 3 October 2013 at 19:07:03 UTC, nazriel wrote:
> (Btw, someone got benchmarks of std.d.lexer?
> I remember that Brain was benchmarking his module quite a lot in order to catch up with DMD's lexer but I can't find links in IRC logs. I wonder if he achieved his goal in this regard)

The most recent set of timings that I have can be found here: https://raw.github.com/Hackerpilot/hackerpilot.github.com/master/experimental/std_lexer/images/times4.png

They're a bit old at this point, but not much has changed in the lexer internals. I can try running another set of benchmarks soon. (The hardest part is hacking DMD to just do the lexing)

The times on the X-axis are milliseconds.

On Thursday, 3 October 2013 at 19:07:03 UTC, nazriel wrote:
> On Wednesday, 2 October 2013 at 14:41:56 UTC, Dicebot wrote:
> ...

Please keep "btw"s in separate thread :)

On 10/3/13 12:47 PM, Brian Schott wrote:
> On Thursday, 3 October 2013 at 19:07:03 UTC, nazriel wrote:
>> (Btw, someone got benchmarks of std.d.lexer?
>> I remember that Brain was benchmarking his module quite a lot in order
>> to catch up with DMD's lexer but I can't find links in IRC logs. I
>> wonder if he achieved his goal in this regard)
>
> The most recent set of timings that I have can be found here:
> https://raw.github.com/Hackerpilot/hackerpilot.github.com/master/experimental/std_lexer/images/times4.png
>
>
> They're a bit old at this point, but not much has changed in the lexer
> internals. I can try running another set of benchmarks soon. (The
> hardest part is hacking DMD to just do the lexing)
>
> The times on the X-axis are milliseconds.

I see we're considerably behind dmd. If improving performance would come at the price of changing the API, it may be sensible to hold off adoption for a bit.

Andrei

On Wednesday, 2 October 2013 at 14:41:56 UTC, Dicebot wrote:
> If you need to ask any last moment questions before making your decision, please do it in last review thread (linked in beginning of this post).
>
> Voting will last until the next weekend (Oct 12 23:59 GMT +0)
>
> Thanks for your attention.

I sadly have to vote no in the current state.

It is really needed to be able to reuse the same pool of identifier across several lexing (otherwize tooling around this lexer won't be able to manage mixins properly unless rolling its own identifier pool on top of the lexer's). This require the interface to change, so can't be introduced in a latter version without major breakage.

I'd vote yes if above condition is met or to integrate current module as experimental (not in std).

On Thursday, 3 October 2013 at 20:11:02 UTC, Andrei Alexandrescu wrote:
> On 10/3/13 12:47 PM, Brian Schott wrote:
>> On Thursday, 3 October 2013 at 19:07:03 UTC, nazriel wrote:
>>> (Btw, someone got benchmarks of std.d.lexer?
>>> I remember that Brain was benchmarking his module quite a lot in order
>>> to catch up with DMD's lexer but I can't find links in IRC logs. I
>>> wonder if he achieved his goal in this regard)
>>
>> The most recent set of timings that I have can be found here:
>> https://raw.github.com/Hackerpilot/hackerpilot.github.com/master/experimental/std_lexer/images/times4.png
>>
>>
>> They're a bit old at this point, but not much has changed in the lexer
>> internals. I can try running another set of benchmarks soon. (The
>> hardest part is hacking DMD to just do the lexing)
>>
>> The times on the X-axis are milliseconds.
>
> I see we're considerably behind dmd. If improving performance would come at the price of changing the API, it may be sensible to hold off adoption for a bit.
>
> Andrei

Considerably?  They look very similar to me.  dmd is just slightly winning.

On 10/3/13 1:15 PM, Brad Anderson wrote:
> On Thursday, 3 October 2013 at 20:11:02 UTC, Andrei Alexandrescu wrote:
>> On 10/3/13 12:47 PM, Brian Schott wrote:
>>> On Thursday, 3 October 2013 at 19:07:03 UTC, nazriel wrote:
>>>> (Btw, someone got benchmarks of std.d.lexer?
>>>> I remember that Brain was benchmarking his module quite a lot in order
>>>> to catch up with DMD's lexer but I can't find links in IRC logs. I
>>>> wonder if he achieved his goal in this regard)
>>>
>>> The most recent set of timings that I have can be found here:
>>> https://raw.github.com/Hackerpilot/hackerpilot.github.com/master/experimental/std_lexer/images/times4.png
>>>
>>>
>>>
>>> They're a bit old at this point, but not much has changed in the lexer
>>> internals. I can try running another set of benchmarks soon. (The
>>> hardest part is hacking DMD to just do the lexing)
>>>
>>> The times on the X-axis are milliseconds.
>>
>> I see we're considerably behind dmd. If improving performance would
>> come at the price of changing the API, it may be sensible to hold off
>> adoption for a bit.
>>
>> Andrei
>
> Considerably?  They look very similar to me.  dmd is just slightly winning.

To me 10% is considerable.

Andrei

Please express your opinion in a clear "Yes", "No" or "Yes, if" form. I can't really interpret discussions into voting results.

Of course, you and Walter also have "veto" votes in addition but it needs to be said explicitly.

On 10/3/13 3:03 PM, Dicebot wrote:
> Please express your opinion in a clear "Yes", "No" or "Yes, if" form. I
> can't really interpret discussions into voting results.
>
> Of course, you and Walter also have "veto" votes in addition but it
> needs to be said explicitly.

That's why I renamed the thread! I didn't vote.

Andrei