On Thursday, 5 July 2012 at 16:38:27 UTC, Jacob Carlborg wrote:
> On 2012-07-05 18:32, Roman D. Boiko wrote:
>
>> My vote would be for Pegged, I guess.
>
> Aren't you voting on your own project :)

Well, I'm going to base parsing on Pegged, after tweaking it to
my needs.

05.07.2012 20:14, Jacob Carlborg пишет:
> On 2012-07-05 18:04, Roman D. Boiko wrote:
>
>> Well, we did something like that for DCT... but I doubt that it would
>> fit general needs.
>
> Why wouldn't it.
>
>> If we had, why haven't they been analyzed, classified, discussed, etc.?
>> Or have they?
>
> I don't know. Here is what I wrote for DCT:
>
> * IDE integration
> * Refactoring tool
> * Static analysis
> * Compiler
> * Doc generating
> * Build tool
> * DI generating
>

Just want to mention I'm talking about parser only. Things like "Refactoring" have to work perfectly or are unusable so it require a full compiler at least as finished as current dmd.


-- 
Денис В. Шеломовский
Denis V. Shelomovskij

05.07.2012 20:28, Jonathan M Davis пишет:
> It's all too common for someone to suggest that we should
> do something or implement something without ever attempting to do it
> themselves, and in general, stuff around here gets done because someone really
> wants it done, takes the time to do it, and sees it through until its done and
> in Phobos.

I didn't want for someone to code anything at all! I on the contrary want them to stop writing parsers because it results in the only consequence: one have to spend more time to find a better parser.

-- 
Денис В. Шеломовский
Denis V. Shelomovskij

>> On 2012-07-05 18:32, Roman D. Boiko wrote:
>>
>>> My vote would be for Pegged, I guess.

As much as I'm flattered by that, my current impression is Pegged is very far from being performant.

As a proof-of-concept that, in D,  it's possible to parse a string and create a parse tree at compile-time and then generate code from this, it's also successful. Go D!

As a parser proper, Pegged is awful :-) Nothing I'm ashamed of, as I learn by coding. Hey, I just received the Dragon Book (International Edition), I'm sure I'll learn many things in there.

So, if anyone is willing to change the code generated by Pegged, I'm game. The results you showed me on keyword parsing are very interesting!

But, my impression is that the need for a 'D'-only parser and lexer is far
greater and much more imediate that the need for a parser generator. All
the reasons advanced upthread ask for a D parser, not a generic generator.
Parser generators are for those of us interested in having DSLs or macros
in D.
So Pegged or any other generator should *not* get the community focus right
now.

My plan would be as follow:

1- assemble a group of people knowing parsing. I don't think I'm exactly
knowledgeable, but I'm ready to be part of such a group.
2- create a github process.
3- translate an existing parser / adapt a D parser for Phobos. I'm ready to
be part of this (I'm sure I'll learn in the process)
4- spend 1-2 years fighting over LR parsing and such :) (Just kidding)
5- submit it to Phobos and have it adopted.

much later:
6- see the way the parser code is organized and tweak a code generator
(Pegged is a possibility if recursive parsing is OK) to produce an
equivalent code when fed the D grammar.

side-effect: maybe a std.tree or std.collection.tree to deal with trees in a generic way.

Philippe

On 7/5/12 12:39 PM, Roman D. Boiko wrote:
> On Thursday, 5 July 2012 at 16:28:57 UTC, Jonathan M Davis wrote:
>> It's all too common for someone to suggest that we should
>> do something or implement something without ever attempting to do it
>> themselves, and in general, stuff around here gets done because
>> someone really wants it done, takes the time to do it, and sees it
>> through until its done and in Phobos.
>>
>> - Jonathan M Davis
>
> Resume: everybody is welcome to join effort of translating DMD front
> end, and improving Pegged.
>
>
> Also I would like to invite those interested in DCT project to help me
> with it. Right now I'm trying to understand whether it is possible to
> incorporate Pegged inside without losing anything critical (and I think
> it is very likely possible), and how exactly to do that.
>
> Dmitry proposed to help improve Pegged (or some other compiler's) speed.
>
> Anyone else?

I'd really want to create a task force on this, it is of strategic importance to D. In Walter's own words, no new feature is going to push us forward since we're not really using the great goodies we've got, and CTFE technology is the most important.

I also am actively opposed to a project of just translating D's front-end to D and dropping it into Phobos because it would smother (a) work on generic parser generators, and (b) strong, dependable formalization of D's syntax.


Andrei

On Thursday, July 05, 2012 22:23:00 Denis Shelomovskij wrote:
> 05.07.2012 20:28, Jonathan M Davis пишет:
> > It's all too common for someone to suggest that we should
> > do something or implement something without ever attempting to do it
> > themselves, and in general, stuff around here gets done because someone
> > really wants it done, takes the time to do it, and sees it through until
> > its done and in Phobos.
> 
> I didn't want for someone to code anything at all! I on the contrary want them to stop writing parsers because it results in the only consequence: one have to spend more time to find a better parser.

Well, until a lexer and parser for D make it into Phobos, more people are going to keep writing them, and even if/when they _do_ make it into Phobos, people will keep writing them, because some people like to write that kind of thing.

Honestly though, if your complaint is that there's too much choice, I don't have much sympathy for you. In general, we have too little D code out there, not too much. If there's a problem with more parsers being written, I think that it's almost purely an issue of some people's time probably being better spent on other projects, but it's their right to work on whatever they feel like working on.

- Jonathan M Davis

On 7/5/12 2:16 PM, Philippe Sigaud wrote:
> As much as I'm flattered by that, my current impression is Pegged is
> very far from being performant.
>
> As a proof-of-concept that, in D,  it's possible to parse a string and
> create a parse tree at compile-time and then generate code from this,
> it's also successful. Go D!
>
> As a parser proper, Pegged is awful :-) Nothing I'm ashamed of, as I
> learn by coding. Hey, I just received the Dragon Book (International
> Edition), I'm sure I'll learn many things in there.

I'll be glad to buy for you any book you might feel you need for this. Again, there are few things more important for D right now than exploiting its unmatched-by-competition features to great ends. I don't want the lack of educational material to hold you down. Please continue working on this and let me know of what you need.

> So, if anyone is willing to change the code generated by Pegged, I'm
> game. The results you showed me on keyword parsing are very interesting!
>
> But, my impression is that the need for a 'D'-only parser and lexer is
> far greater and much more imediate that the need for a parser generator.

I very strongly disagree. This is our chance to get things right instead of having a limited product that destroys competition (much like lex and yacc have been for years in the parser generator field).

> All the reasons advanced upthread ask for a D parser, not a generic
> generator. Parser generators are for those of us interested in having
> DSLs or macros in D.
> So Pegged or any other generator should *not* get the community focus
> right now.

Pegged should be the focus.

> My plan would be as follow:
>
> 1- assemble a group of people knowing parsing. I don't think I'm exactly
> knowledgeable, but I'm ready to be part of such a group.
> 2- create a github process.
> 3- translate an existing parser / adapt a D parser for Phobos. I'm ready
> to be part of this (I'm sure I'll learn in the process)
> 4- spend 1-2 years fighting over LR parsing and such :) (Just kidding)
> 5- submit it to Phobos and have it adopted.

Sounds good. Replace 1-2 years with 1-2 months.



Andrei

On Thursday, 5 July 2012 at 18:17:06 UTC, Philippe Sigaud wrote:
>>> On 2012-07-05 18:32, Roman D. Boiko wrote:
>>>
>>>> My vote would be for Pegged, I guess.
>
> As much as I'm flattered by that, my current impression is Pegged is very
> far from being performant.
>
> As a proof-of-concept that, in D,  it's possible to parse a string and
> create a parse tree at compile-time and then generate code from this, it's
> also successful. Go D!
>
> As a parser proper, Pegged is awful :-) Nothing I'm ashamed of, as I learn
> by coding. Hey, I just received the Dragon Book (International Edition),
> I'm sure I'll learn many things in there.
>
> So, if anyone is willing to change the code generated by Pegged, I'm game.
> The results you showed me on keyword parsing are very interesting!
>
> But, my impression is that the need for a 'D'-only parser and lexer is far
> greater and much more imediate that the need for a parser generator. All
> the reasons advanced upthread ask for a D parser, not a generic generator.
> Parser generators are for those of us interested in having DSLs or macros
> in D.
> So Pegged or any other generator should *not* get the community focus right
> now.

I'm sure it can generate **much** faster code. I'm going to focus on its part that generates D parser (i.e., to make it significantly faster and able to efficiently parse-as-you-type). Actually, I'm sure it will be able to beat any other parser with respect to performance. :)

1. So my plan is the following: invite whoever would want to help.
2. Prove my claims above in practice. :-)))))

On Thursday, 5 July 2012 at 18:28:21 UTC, Andrei Alexandrescu wrote:
> On 7/5/12 2:16 PM, Philippe Sigaud wrote:
>> So Pegged or any other generator should *not* get the community focus right now.
>
> Pegged should be the focus.

+10 (can I vote ten times?)

>> My plan would be as follow:
>>
>> 1- assemble a group of people knowing parsing. I don't think I'm exactly
>> knowledgeable, but I'm ready to be part of such a group.
>> 2- create a github process.
>> 3- translate an existing parser / adapt a D parser for Phobos. I'm ready
>> to be part of this (I'm sure I'll learn in the process)
>> 4- spend 1-2 years fighting over LR parsing and such :) (Just kidding)
>> 5- submit it to Phobos and have it adopted.
>
> Sounds good. Replace 1-2 years with 1-2 months.

Well, probably with 3-4 months... :)

On 05-Jul-12 22:22, Andrei Alexandrescu wrote:
> On 7/5/12 12:39 PM, Roman D. Boiko wrote:
>> On Thursday, 5 July 2012 at 16:28:57 UTC, Jonathan M Davis wrote:
>>> It's all too common for someone to suggest that we should
>>> do something or implement something without ever attempting to do it
>>> themselves, and in general, stuff around here gets done because
>>> someone really wants it done, takes the time to do it, and sees it
>>> through until its done and in Phobos.
>>>
>>> - Jonathan M Davis
>>
>> Resume: everybody is welcome to join effort of translating DMD front
>> end, and improving Pegged.
>>
>>
>> Also I would like to invite those interested in DCT project to help me
>> with it. Right now I'm trying to understand whether it is possible to
>> incorporate Pegged inside without losing anything critical (and I think
>> it is very likely possible), and how exactly to do that.
>>
>> Dmitry proposed to help improve Pegged (or some other compiler's) speed.
>>
>> Anyone else?
>
> I'd really want to create a task force on this, it is of strategic
> importance to D. In Walter's own words, no new feature is going to push
> us forward since we're not really using the great goodies we've got, and
> CTFE technology is the most important.
>

Count me as interested.
CTFE needs more correctness & speed though. So to put it blantly - no it's not possible right NOW.
BUT it doesn't prevent us from planing and doing a proof of concept. Pegged seems a good starting point even if we end up re-writing it from scratch.

> I also am actively opposed to a project of just translating D's
> front-end to D and dropping it into Phobos because it would smother (a)
> work on generic parser generators, and (b) strong, dependable
> formalization of D's syntax.
>

Well put. It shouldn't stop people from doing parsers, IMO the more the merrier.


-- 
Dmitry Olshansky