View mode: basic / threaded / horizontal-split · Log in · Help
July 05, 2012
Re: Let's stop parser Hell
On Thursday, 5 July 2012 at 16:38:27 UTC, Jacob Carlborg wrote:
> On 2012-07-05 18:32, Roman D. Boiko wrote:
>
>> My vote would be for Pegged, I guess.
>
> Aren't you voting on your own project :)

Well, I'm going to base parsing on Pegged, after tweaking it to
my needs.
July 05, 2012
Re: Let's stop parser Hell
05.07.2012 20:14, Jacob Carlborg пишет:
> On 2012-07-05 18:04, Roman D. Boiko wrote:
>
>> Well, we did something like that for DCT... but I doubt that it would
>> fit general needs.
>
> Why wouldn't it.
>
>> If we had, why haven't they been analyzed, classified, discussed, etc.?
>> Or have they?
>
> I don't know. Here is what I wrote for DCT:
>
> * IDE integration
> * Refactoring tool
> * Static analysis
> * Compiler
> * Doc generating
> * Build tool
> * DI generating
>

Just want to mention I'm talking about parser only. Things like 
"Refactoring" have to work perfectly or are unusable so it require a 
full compiler at least as finished as current dmd.


-- 
Денис В. Шеломовский
Denis V. Shelomovskij
July 05, 2012
Re: Let's stop parser Hell
05.07.2012 20:28, Jonathan M Davis пишет:
> It's all too common for someone to suggest that we should
> do something or implement something without ever attempting to do it
> themselves, and in general, stuff around here gets done because someone really
> wants it done, takes the time to do it, and sees it through until its done and
> in Phobos.

I didn't want for someone to code anything at all! I on the contrary 
want them to stop writing parsers because it results in the only 
consequence: one have to spend more time to find a better parser.

-- 
Денис В. Шеломовский
Denis V. Shelomovskij
July 05, 2012
Re: Let's stop parser Hell
>> On 2012-07-05 18:32, Roman D. Boiko wrote:
>>
>>> My vote would be for Pegged, I guess.

As much as I'm flattered by that, my current impression is Pegged is very
far from being performant.

As a proof-of-concept that, in D,  it's possible to parse a string and
create a parse tree at compile-time and then generate code from this, it's
also successful. Go D!

As a parser proper, Pegged is awful :-) Nothing I'm ashamed of, as I learn
by coding. Hey, I just received the Dragon Book (International Edition),
I'm sure I'll learn many things in there.

So, if anyone is willing to change the code generated by Pegged, I'm game.
The results you showed me on keyword parsing are very interesting!

But, my impression is that the need for a 'D'-only parser and lexer is far
greater and much more imediate that the need for a parser generator. All
the reasons advanced upthread ask for a D parser, not a generic generator.
Parser generators are for those of us interested in having DSLs or macros
in D.
So Pegged or any other generator should *not* get the community focus right
now.

My plan would be as follow:

1- assemble a group of people knowing parsing. I don't think I'm exactly
knowledgeable, but I'm ready to be part of such a group.
2- create a github process.
3- translate an existing parser / adapt a D parser for Phobos. I'm ready to
be part of this (I'm sure I'll learn in the process)
4- spend 1-2 years fighting over LR parsing and such :) (Just kidding)
5- submit it to Phobos and have it adopted.

much later:
6- see the way the parser code is organized and tweak a code generator
(Pegged is a possibility if recursive parsing is OK) to produce an
equivalent code when fed the D grammar.

side-effect: maybe a std.tree or std.collection.tree to deal with trees in
a generic way.

Philippe
July 05, 2012
Re: Let's stop parser Hell
On 7/5/12 12:39 PM, Roman D. Boiko wrote:
> On Thursday, 5 July 2012 at 16:28:57 UTC, Jonathan M Davis wrote:
>> It's all too common for someone to suggest that we should
>> do something or implement something without ever attempting to do it
>> themselves, and in general, stuff around here gets done because
>> someone really wants it done, takes the time to do it, and sees it
>> through until its done and in Phobos.
>>
>> - Jonathan M Davis
>
> Resume: everybody is welcome to join effort of translating DMD front
> end, and improving Pegged.
>
>
> Also I would like to invite those interested in DCT project to help me
> with it. Right now I'm trying to understand whether it is possible to
> incorporate Pegged inside without losing anything critical (and I think
> it is very likely possible), and how exactly to do that.
>
> Dmitry proposed to help improve Pegged (or some other compiler's) speed.
>
> Anyone else?

I'd really want to create a task force on this, it is of strategic 
importance to D. In Walter's own words, no new feature is going to push 
us forward since we're not really using the great goodies we've got, and 
CTFE technology is the most important.

I also am actively opposed to a project of just translating D's 
front-end to D and dropping it into Phobos because it would smother (a) 
work on generic parser generators, and (b) strong, dependable 
formalization of D's syntax.


Andrei
July 05, 2012
Re: Let's stop parser Hell
On Thursday, July 05, 2012 22:23:00 Denis Shelomovskij wrote:
> 05.07.2012 20:28, Jonathan M Davis пишет:
> > It's all too common for someone to suggest that we should
> > do something or implement something without ever attempting to do it
> > themselves, and in general, stuff around here gets done because someone
> > really wants it done, takes the time to do it, and sees it through until
> > its done and in Phobos.
> 
> I didn't want for someone to code anything at all! I on the contrary
> want them to stop writing parsers because it results in the only
> consequence: one have to spend more time to find a better parser.

Well, until a lexer and parser for D make it into Phobos, more people are 
going to keep writing them, and even if/when they _do_ make it into Phobos, 
people will keep writing them, because some people like to write that kind of 
thing.

Honestly though, if your complaint is that there's too much choice, I don't 
have much sympathy for you. In general, we have too little D code out there, 
not too much. If there's a problem with more parsers being written, I think 
that it's almost purely an issue of some people's time probably being better 
spent on other projects, but it's their right to work on whatever they feel 
like working on.

- Jonathan M Davis
July 05, 2012
Re: Let's stop parser Hell
On 7/5/12 2:16 PM, Philippe Sigaud wrote:
> As much as I'm flattered by that, my current impression is Pegged is
> very far from being performant.
>
> As a proof-of-concept that, in D,  it's possible to parse a string and
> create a parse tree at compile-time and then generate code from this,
> it's also successful. Go D!
>
> As a parser proper, Pegged is awful :-) Nothing I'm ashamed of, as I
> learn by coding. Hey, I just received the Dragon Book (International
> Edition), I'm sure I'll learn many things in there.

I'll be glad to buy for you any book you might feel you need for this. 
Again, there are few things more important for D right now than 
exploiting its unmatched-by-competition features to great ends. I don't 
want the lack of educational material to hold you down. Please continue 
working on this and let me know of what you need.

> So, if anyone is willing to change the code generated by Pegged, I'm
> game. The results you showed me on keyword parsing are very interesting!
>
> But, my impression is that the need for a 'D'-only parser and lexer is
> far greater and much more imediate that the need for a parser generator.

I very strongly disagree. This is our chance to get things right instead 
of having a limited product that destroys competition (much like lex and 
yacc have been for years in the parser generator field).

> All the reasons advanced upthread ask for a D parser, not a generic
> generator. Parser generators are for those of us interested in having
> DSLs or macros in D.
> So Pegged or any other generator should *not* get the community focus
> right now.

Pegged should be the focus.

> My plan would be as follow:
>
> 1- assemble a group of people knowing parsing. I don't think I'm exactly
> knowledgeable, but I'm ready to be part of such a group.
> 2- create a github process.
> 3- translate an existing parser / adapt a D parser for Phobos. I'm ready
> to be part of this (I'm sure I'll learn in the process)
> 4- spend 1-2 years fighting over LR parsing and such :) (Just kidding)
> 5- submit it to Phobos and have it adopted.

Sounds good. Replace 1-2 years with 1-2 months.



Andrei
July 05, 2012
Re: Let's stop parser Hell
On Thursday, 5 July 2012 at 18:17:06 UTC, Philippe Sigaud wrote:
>>> On 2012-07-05 18:32, Roman D. Boiko wrote:
>>>
>>>> My vote would be for Pegged, I guess.
>
> As much as I'm flattered by that, my current impression is 
> Pegged is very
> far from being performant.
>
> As a proof-of-concept that, in D,  it's possible to parse a 
> string and
> create a parse tree at compile-time and then generate code from 
> this, it's
> also successful. Go D!
>
> As a parser proper, Pegged is awful :-) Nothing I'm ashamed of, 
> as I learn
> by coding. Hey, I just received the Dragon Book (International 
> Edition),
> I'm sure I'll learn many things in there.
>
> So, if anyone is willing to change the code generated by 
> Pegged, I'm game.
> The results you showed me on keyword parsing are very 
> interesting!
>
> But, my impression is that the need for a 'D'-only parser and 
> lexer is far
> greater and much more imediate that the need for a parser 
> generator. All
> the reasons advanced upthread ask for a D parser, not a generic 
> generator.
> Parser generators are for those of us interested in having DSLs 
> or macros
> in D.
> So Pegged or any other generator should *not* get the community 
> focus right
> now.

I'm sure it can generate **much** faster code. I'm going to focus 
on its part that generates D parser (i.e., to make it 
significantly faster and able to efficiently parse-as-you-type). 
Actually, I'm sure it will be able to beat any other parser with 
respect to performance. :)

1. So my plan is the following: invite whoever would want to help.
2. Prove my claims above in practice. :-)))))
July 05, 2012
Re: Let's stop parser Hell
On Thursday, 5 July 2012 at 18:28:21 UTC, Andrei Alexandrescu 
wrote:
> On 7/5/12 2:16 PM, Philippe Sigaud wrote:
>> So Pegged or any other generator should *not* get the 
>> community focus right now.
>
> Pegged should be the focus.

+10 (can I vote ten times?)

>> My plan would be as follow:
>>
>> 1- assemble a group of people knowing parsing. I don't think 
>> I'm exactly
>> knowledgeable, but I'm ready to be part of such a group.
>> 2- create a github process.
>> 3- translate an existing parser / adapt a D parser for Phobos. 
>> I'm ready
>> to be part of this (I'm sure I'll learn in the process)
>> 4- spend 1-2 years fighting over LR parsing and such :) (Just 
>> kidding)
>> 5- submit it to Phobos and have it adopted.
>
> Sounds good. Replace 1-2 years with 1-2 months.

Well, probably with 3-4 months... :)
July 05, 2012
Re: Let's stop parser Hell
On 05-Jul-12 22:22, Andrei Alexandrescu wrote:
> On 7/5/12 12:39 PM, Roman D. Boiko wrote:
>> On Thursday, 5 July 2012 at 16:28:57 UTC, Jonathan M Davis wrote:
>>> It's all too common for someone to suggest that we should
>>> do something or implement something without ever attempting to do it
>>> themselves, and in general, stuff around here gets done because
>>> someone really wants it done, takes the time to do it, and sees it
>>> through until its done and in Phobos.
>>>
>>> - Jonathan M Davis
>>
>> Resume: everybody is welcome to join effort of translating DMD front
>> end, and improving Pegged.
>>
>>
>> Also I would like to invite those interested in DCT project to help me
>> with it. Right now I'm trying to understand whether it is possible to
>> incorporate Pegged inside without losing anything critical (and I think
>> it is very likely possible), and how exactly to do that.
>>
>> Dmitry proposed to help improve Pegged (or some other compiler's) speed.
>>
>> Anyone else?
>
> I'd really want to create a task force on this, it is of strategic
> importance to D. In Walter's own words, no new feature is going to push
> us forward since we're not really using the great goodies we've got, and
> CTFE technology is the most important.
>

Count me as interested.
CTFE needs more correctness & speed though. So to put it blantly - no 
it's not possible right NOW.
BUT it doesn't prevent us from planing and doing a proof of concept. 
Pegged seems a good starting point even if we end up re-writing it from 
scratch.

> I also am actively opposed to a project of just translating D's
> front-end to D and dropping it into Phobos because it would smother (a)
> work on generic parser generators, and (b) strong, dependable
> formalization of D's syntax.
>

Well put. It shouldn't stop people from doing parsers, IMO the more the 
merrier.


-- 
Dmitry Olshansky
1 2 3 4 5 6 7
Top | Discussion index | About this forum | D home