There are more and more projects requiring parsing D code (IDE plugins, DCT and dfmt now).

We definitely need a _standard_ fast D parser (suitable as tokenizer). We already have a good one at least in Visual D. Maybe dmd's parser is faster. If so, it can be (easily?) rewritten in D. We also already have some other parsers (in C# from Mono-D etc.).

What about to get one and call it standard?

-- 
Денис В. Шеломовский
Denis V. Shelomovskij

On Thursday, 5 July 2012 at 12:11:33 UTC, Denis Shelomovskij wrote:
> There are more and more projects requiring parsing D code (IDE plugins, DCT and dfmt now).
>
> We definitely need a _standard_ fast D parser (suitable as tokenizer). We already have a good one at least in Visual D. Maybe dmd's parser is faster. If so, it can be (easily?) rewritten in D. We also already have some other parsers (in C# from Mono-D etc.).
>
> What about to get one and call it standard?

Visual-D is not Boost-licensed (I think this would be possible to change)
Mono-D is written in C#, as you mentioned
Pegged may eventually become standard, if it will be performance optimized and a bit more customizable
Dscanner(https://github.com/Hackerpilot/Dscanner) from Brian Schott is pretty good, too.
SDC is another nice option
DIL (http://code.google.com/p/dil/) is very nice but GPL

I plan to try using Pegged inside my DCT project. Probably that will require huge modifications though...

Some more links from Pegged readme:

> Hisayuki Mima's CTPG(https://github.com/youkei/ctpg), very similar, also done in D. Have a look!
> Nick Sabalausky's Goldie (http://www.dsource.org/projects/goldie).
> Benjamin Shropshire's dparser (http://dsource.org/projects/scrapple/browser/trunk/dparser).

> Martin Nowak put these gists on the D newsgroup:
> https://gist.github.com/1255439 - lexer generator
> https://gist.github.com/1262321 - complete and fast D lexer

Forgot to add DDMD, which also has been forked and redesigned recently by someone (thus two more different compiler frontends).

But overall I doubt that any project could become standard very soon.

On 05-Jul-12 16:11, Denis Shelomovskij wrote:
> There are more and more projects requiring parsing D code (IDE plugins,
> DCT and dfmt now).
>
> We definitely need a _standard_ fast D parser (suitable as tokenizer).

Then do it. It's all about having something so obviously good, fast and flexible that other stuff can be considered only as toys.

I might do one, but I'd rather just help other folks make it faster ;)


-- 
Dmitry Olshansky

05.07.2012 16:30, Roman D. Boiko пишет:
> Forgot to add DDMD, which also has been forked and redesigned recently
> by someone (thus two more different compiler frontends).
>
> But overall I doubt that any project could become standard very soon.

Why? Even were they all almost equal we can select any one. The point is to select something (anything) to guide a coder who want to do something with this task to a good/up-to-date/supported parser.

-- 
Денис В. Шеломовский
Denis V. Shelomovskij

On Thursday, 5 July 2012 at 12:32:19 UTC, Dmitry Olshansky wrote:
> Then do it. It's all about having something so obviously good, fast and flexible that other stuff can be considered only as toys.
>
> I might do one, but I'd rather just help other folks make it faster ;)

Would you help to make Pegged faster?

On 05-Jul-12 17:04, Roman D. Boiko wrote:
> On Thursday, 5 July 2012 at 12:32:19 UTC, Dmitry Olshansky wrote:
>> Then do it. It's all about having something so obviously good, fast
>> and flexible that other stuff can be considered only as toys.
>>
>> I might do one, but I'd rather just help other folks make it faster ;)
>
> Would you help to make Pegged faster?

Well why not.
But first I'll need to deliver some stuff on my GSOC project.

-- 
Dmitry Olshansky

On Thursday, 5 July 2012 at 12:53:02 UTC, Denis Shelomovskij wrote:
> 05.07.2012 16:30, Roman D. Boiko пишет:
>> Forgot to add DDMD, which also has been forked and redesigned recently
>> by someone (thus two more different compiler frontends).
>>
>> But overall I doubt that any project could become standard very soon.
>
> Why? Even were they all almost equal we can select any one. The point is to select something (anything) to guide a coder who want to do something with this task to a good/up-to-date/supported parser.

Well, I guess we'd like to select one and not change it later, don't we?
I'm not sure we can decide which is the best currently and will stay the best in the future for all major use cases.

Anyway I propose to enumerate major use cases first.

On Thursday, 5 July 2012 at 13:05:41 UTC, Dmitry Olshansky wrote:
> On 05-Jul-12 17:04, Roman D. Boiko wrote:
> Well why not.
> But first I'll need to deliver some stuff on my GSOC project.

I bet that after you finish with GSOC optimizing Pegged will not be less relevant than it is now :) And as for DCT, it will take another half a year (at least) until it will become usable for any real needs (except the most trivial).

On 7/5/12 9:05 AM, Dmitry Olshansky wrote:
> On 05-Jul-12 17:04, Roman D. Boiko wrote:
>> On Thursday, 5 July 2012 at 12:32:19 UTC, Dmitry Olshansky wrote:
>>> Then do it. It's all about having something so obviously good, fast
>>> and flexible that other stuff can be considered only as toys.
>>>
>>> I might do one, but I'd rather just help other folks make it faster ;)
>>
>> Would you help to make Pegged faster?
>
> Well why not.
> But first I'll need to deliver some stuff on my GSOC project.

Good call :o).

Note that we can discuss tweaking the scope of the project after the midterm. Ping me if you have some ideas.


Andrei