October 22, 2009
Matthias Fauconneau schrieb:
> Matthias Pleh Wrote:
>> Thanks for your code! The grammar was exactly what I was looking for. I will try to implement it in my IDE to support code-completion! (a plugin for SharpDevelop)
>>
>> Matthias
> 
> Are you going to use D_Parser ?
> If you are using a weaker parser, you may be interested in http://seatd.mainia.de/grammar.html
> 

I'm not so familiar with parser generator, so I was looking for a finished grammar for any generator.
I was glad to find this grammar!
SharpDevelop use Coco/R to generate the parser for the C# files, but I haven't found a grammar for it.
Best parser generator would be one, which can generate C#-files, since my plugin is written in C# :)

I will have a look at the grammar you mentioned!
October 22, 2009
Matthias Pleh wrote:
> Matthias Fauconneau schrieb:
>> Matthias Pleh Wrote:
>>> Thanks for your code! The grammar was exactly what I was looking for. I will try to implement it in my IDE to support code-completion! (a plugin for SharpDevelop)
>>>
>>> Matthias
>>
>> Are you going to use D_Parser ?
>> If you are using a weaker parser, you may be interested in
>> http://seatd.mainia.de/grammar.html
>>
> 
> I'm not so familiar with parser generator, so I was looking for a
> finished grammar for any generator.
> I was glad to find this grammar!
> SharpDevelop use Coco/R to generate the parser for the C# files, but I
> haven't found a grammar for it.
> Best parser generator would be one, which can generate C#-files, since
> my plugin is written in C# :)


ANTLR generates C# source code.

> 
> I will have a look at the grammar you mentioned!

That grammar looks okay, but I've counted at least 3 nitpicky problems, and I didn't read the whole thing (nor very closely)
October 22, 2009
> > I'm not so familiar with parser generator, so I was looking for a finished grammar for any generator.
This is the only other D grammar I know of.
It was made for ApageD (a discontinued parser project written in D) but it should be compatible with most parser generators.

> > Best parser generator would be one, which can generate C#-files, since my plugin is written in C# :)
If you want I could create a C library with the code analysis part of my IDE. It should be easy to bind in C#.
I'm currently providing a symbol table and links from a symbol use to definition.
The IDE use it to display an outliner and for a "Go to Definition" action.
Now, I'm currently coding the handling of expression (i.e finding the type of each subexpression) which will be used for code completion

> ANTLR generates C# source code.
I didn't try Coco/R or AntLR but I don't think they would let you specify the D syntax so concisely.

> That grammar looks okay, but I've counted at least 3 nitpicky problems, and I didn't read the whole thing (nor very closely)

I find this grammar is quite lengthy and hard to read, probably because it had to be accepted by a simple parser generator.
It doesn't benefit from the main advantages of using a parser generator which is simplicity and maintainability.

October 22, 2009
Matthias Fauconneau wrote:
> I didn't try Coco/R or AntLR but I don't think they would let you specify the D syntax so concisely.

Quite true. Your d_parser grammar is incredibly short (as opposed to my ANTLR grammar which is about 10 times larger). I bet d_parser is GLR?

And I don't mean to be nitpicky again, but it looks like it is missing even more than the other grammar.
> 
>> That grammar looks okay, but I've counted at least 3 nitpicky problems, and I didn't read the whole thing (nor very closely)
> 
> I find this grammar is quite lengthy and hard to read, probably because it had to be accepted by a simple parser generator.
> It doesn't benefit from the main advantages of using a parser generator which is simplicity and maintainability.
> 

Well, D is a lengthy language...
October 23, 2009
> Quite true. Your d_parser grammar is incredibly short (as opposed to my
ANTLR grammar which is about 10 times larger).

It's good I have chosen D_Parser then, 10 times is worth it. I hope it wont get much larger when I complete it.

>I bet d_parser is GLR?

it's a scannerless GLR parser

> And I don't mean to be nitpicky again,

No problem, I posted my project so I can get critics.

>but it looks like it is missing even more than the other grammar.

It did parse all my code but I had forgotten to disable automatic disambiguation (using greediness and height).
Now, I've been disambiguating my grammar.
I managed to get operator priorities and associativities working (the D_Parser manual is not very verbose on this)
but I'm running into some problems:

is D syntax supposed to be unambiguous ?

According to http://digitalmars.com/d/2.0/lex.html: "Each phase has no dependence on subsequent phases" and it lists syntax analysis and semantic analysis as separate phase.

"type* id;" is either a declaration or a multiplication and I don't see how to disambiguate this without a symbol table.

this case is ambiguous even with the symbol table:
"{ statements }  .ptr" could evaluate to the function literal context pointer or a scope + a call to ptr() defined at module scope

October 23, 2009
If you like, you can have an eyeball at my grammar:

http://www.dsource.org/projects/antlrd/browser/toys/v3d

note it's a port from a v2 grammar (which definitely works) and hasn't
been so extensively tested (or finished, for that matter..)

Matthias Fauconneau wrote:
> It did parse all my code but I had forgotten to disable automatic disambiguation (using greediness and height). Now, I've been disambiguating my grammar.

Now try parsing the standard libraries.

Here's a few things I noticed were missing:

* in/out/body blocks
* comma expressions
* c-style types
* anonymous classes
* parameters can be variadic and/or have no variable symbol
* unions should be able to take template arguments


> I managed to get operator priorities and associativities working (the D_Parser manual is not very verbose on this) but I'm running into some problems:
> 
> is D syntax supposed to be unambiguous ?

Ha! No.

> 
> According to http://digitalmars.com/d/2.0/lex.html: "Each phase has no dependence on subsequent phases" and it lists syntax analysis and semantic analysis as separate phase.
> 
> "type* id;" is either a declaration or a multiplication and I don't see how to disambiguate this without a symbol table.

convention: it's always a declaration (since as an expression it wouldn't have any effect, and we don't care about overloading. Or so I assume). There are other cases that do get resolved during semantic.

> 
> this case is ambiguous even with the symbol table:

they're aplenty

> "{ statements }  .ptr" could evaluate to the function literal context pointer or a scope + a call to ptr() defined at module scope
> 

Good one. I hadn't noticed that one before. Looks like the compiler will go with the second option.
October 29, 2009
Hello Matthias

Matthias Fauconneau schrieb:
>>> I'm not so familiar with parser generator, so I was looking for a finished grammar for any generator.
> This is the only other D grammar I know of.
> It was made for ApageD (a discontinued parser project written in D) but it should be compatible with most parser generators.
> 
>>> Best parser generator would be one, which can generate C#-files, since my plugin is written in C# :)
> If you want I could create a C library with the code analysis part of my IDE. It should be easy to bind in C#.
> I'm currently providing a symbol table and links from a symbol use to definition.
> The IDE use it to display an outliner and for a "Go to Definition" action.
> Now, I'm currently coding the handling of expression (i.e finding the type of each subexpression) which will be used for code completion

I've chosen the GOLD parsing system. There also exists a runtime for C# and the grammar is written in simple BNF.
I've tried to translate your grammar and the ANTLR-grammar from Ellery, but I was confronted with many reduce-reduce conflicts and I can't fix all of them.
So I try to create a grammar from scratch. But when the execution get to  slow, your option with the c-library will be a good option!
Maybe I come back to your offer! :)

>> ANTLR generates C# source code.
> I didn't try Coco/R or AntLR but I don't think they would let you specify the D syntax so concisely.
> 
>> That grammar looks okay, but I've counted at least 3 nitpicky problems,
>> and I didn't read the whole thing (nor very closely)
> 
> I find this grammar is quite lengthy and hard to read, probably because it had to be accepted by a simple parser generator.
> It doesn't benefit from the main advantages of using a parser generator which is simplicity and maintainability.
> 

October 29, 2009
Matthias Pleh wrote:
> Hello Matthias
> 
> Matthias Fauconneau schrieb:
>>>> I'm not so familiar with parser generator, so I was looking for a finished grammar for any generator.
>> This is the only other D grammar I know of.
>> It was made for ApageD (a discontinued parser project written in D)
>> but it should be compatible with most parser generators.
>>
>>>> Best parser generator would be one, which can generate C#-files, since my plugin is written in C# :)
>> If you want I could create a C library with the code analysis part of
>> my IDE. It should be easy to bind in C#.
>> I'm currently providing a symbol table and links from a symbol use to
>> definition.
>> The IDE use it to display an outliner and for a "Go to Definition"
>> action.
>> Now, I'm currently coding the handling of expression (i.e finding the
>> type of each subexpression) which will be used for code completion
> 
> I've chosen the GOLD parsing system. There also exists a runtime for C#
> and the grammar is written in simple BNF.
> I've tried to translate your grammar and the ANTLR-grammar from Ellery,
> but I was confronted with many reduce-reduce conflicts and I can't fix
> all of them.
> So I try to create a grammar from scratch. But when the execution get to
>  slow, your option with the c-library will be a good option!
> Maybe I come back to your offer! :)
> 

You are aware that those reduce-reduce conflicts are inherent to D, right? As in if you can't fix them in my grammar, you probably won't be able to fix them in yours.

Also, it took me maybe 6 months of reading through DMD's source to get something accurate together, and munging through syntactic and semantic ambiguities was more or less trivial (easier than in GOLD, from what I understand, though I haven't used GOLD), for what that's worth.

But never mind me. I'm just a big pessimist. Good luck with your grammar!
October 30, 2009
Hello Ellery
>>
> 
> You are aware that those reduce-reduce conflicts are inherent to D,
> right? As in if you can't fix them in my grammar, you probably won't be
> able to fix them in yours.
Yes of course, you are right!
But I couldn't fix them because it's not technically feasible, but I'm not so familair with your grammar and I can't overlook the whole grammar
at once.

> 
> Also, it took me maybe 6 months of reading through DMD's source to get
> something accurate together, and munging through syntactic and semantic
> ambiguities was more or less trivial (easier than in GOLD, from what I
> understand, though I haven't used GOLD), for what that's worth.
> 
I have taken the hole grammar from the Digital Mars Hompage (the grammar
for GOLD is nearly the same, as on the homepage) and I also used the same identifiers for the nonterminals, so when I get a conflict, I can easaly consult the online documentation, to understand the conflict!
And with your admission I take also some parts from your grammar to solve the conflicts.

> But never mind me. I'm just a big pessimist. Good luck with your grammar!
I am an optimist, and I want to do everything I can do to push the D programming language, because this is the language I looked so long for. :)
October 30, 2009
Matthias Pleh wrote:
>> Also, it took me maybe 6 months of reading through DMD's source to get something accurate together, and munging through syntactic and semantic ambiguities was more or less trivial (easier than in GOLD, from what I understand, though I haven't used GOLD), for what that's worth.
>>
> I have taken the hole grammar from the Digital Mars Hompage (the grammar for GOLD is nearly the same, as on the homepage) and I also used the same identifiers for the nonterminals, so when I get a conflict, I can easaly consult the online documentation, to understand the conflict! And with your admission I take also some parts from your grammar to solve the conflicts.

Note the reason I spent 4 months reading through the compiler is because the grammar in the spec is incorrect in many places. But yeah, feel free to use mine.