Having written a D2 parser a few month ago in the hope it will be helpful to Visual D (just finds it way into the plugin right now), I've noticed quite some inaccuracies in the "official" grammar on the website. Some of these are probably already in bugzilla, some of them might be personal taste and should be discussed, though.

I've set up a comparison with some notes here:

http://www.dsource.org/projects/visuald/wiki/GrammarComparison

The parser was written manually, so the proposed grammar was not directly used to generate the parser and might still contain errors.

I've extracted the website grammar using some combination of scripts from the ddoc sources, so if there is interest, I can dig them up...

Rainer

Walter Bright wrote:
> On 3/28/2011 9:02 AM, Luca Boasso wrote:
>> A complete D grammar is one of the objectives of my GSOC 2011 ANTLR
>> proposal, if I got accepted you will have one :)
> 
> The complete grammar should be part of the D spec. Please post any errors/fixes to the existing one to bugzilla!

On 3/28/2011 11:46 PM, Rainer Schuetze wrote:
>
> Having written a D2 parser a few month ago in the hope it will be helpful to
> Visual D (just finds it way into the plugin right now), I've noticed quite some
> inaccuracies in the "official" grammar on the website. Some of these are
> probably already in bugzilla, some of them might be personal taste and should be
> discussed, though.
>
> I've set up a comparison with some notes here:
>
> http://www.dsource.org/projects/visuald/wiki/GrammarComparison
>
> The parser was written manually, so the proposed grammar was not directly used
> to generate the parser and might still contain errors.
>
> I've extracted the website grammar using some combination of scripts from the
> ddoc sources, so if there is interest, I can dig them up...
>


I thought I fixed the grammar issues listed in Bugzilla.

Could you please add a bugzilla entry for your wiki notes?

Rainer Schuetze wrote:
> I've extracted the website grammar using some combination of scripts from the ddoc sources, so if there is interest, I can dig them up...

Yep, please make them publicly available so they don't get lost.
Though I think we should have an extra official page with the complete grammar in addition to the widespread snippets anyway.

On 03/29/2011 01:02 PM, Trass3r wrote:
> Rainer Schuetze wrote:
>> I've extracted the website grammar using some combination of scripts
>> from the ddoc sources, so if there is interest, I can dig them up...
>
> Yep, please make them publicly available so they don't get lost.
> Though I think we should have an extra official page with the complete grammar in addition to the widespread snippets anyway.

Sure, and this grammar should be the actual startpoint of actual parsers, so that we know it's correct ;-) (else, it's just some more blowing in the wind)

Denis
-- 
_________________
vita es estrany
spir.wikidot.com

Nick Sabalausky wrote:
...
> 
> Yea, I remember that too. Someone took all the BNF sections from D's docs on digitalmars.com, put them together, and filled in some missing/erroneous parts. Maybe it's on wiki4d?

Not sure if this is what you meant, but Jascha Wetzel had made a D grammar used by seatd / APaGeD: http://seatd.mainia.de/grammar.html

On 28/03/2011 18:19, Luca Boasso wrote:
> You can find an ANTLR grammar for D v1 at
> http://www.dsource.org/projects/antlrd/browser/toys/v3d/parsed.g (by
> Ellery Newcomer)
>
> The syntax is similar to EBNF, check the ANTLR documentation for details.
>
> I hope this might help you.
>
>
> Luca Boasso
>

Indeed, Ellery is (AFAIK) the only one who has worked on ANLTR-based D parser recently and possibly got any significant work done in that. It's definitely worth looking at his work and see if anything can be reused. However, according to his most recent message on the subject:

"
Been pretty busy this semester, so I haven't been doing much.

But the bottom line is, yes I have working antlr grammars for D1 and D2
if you don't mind
1) they're slow
2) they're tied to a hacked-out version of the netbeans fork of ANTLR2
3) they're tied to some custom java code
4) I haven't been keeping the tree grammars so up to date
"
http://www.digitalmars.com/webnews/newsgroups.php?art_group=digitalmars.D&article_id=122803

...it may not provide much of head-start.

-- 
Bruno Medeiros - Software Engineer

spir Wrote:
> Sure, and this grammar should be the actual startpoint of actual parsers, so that we know it's correct ;-) (else, it's just some more blowing in the wind)

Exactly!

Rainer Schuetze Wrote:
> I've set up a comparison with some notes here:
> 
> http://www.dsource.org/projects/visuald/wiki/GrammarComparison

Very nice comparison style.
What would be an appropriate way to discuss grammar changes?
I think many things should be simplified/clarified.

For example, isn't AutoDeclaration already covered by Decl -> StorageClasses Decl and so on?
Also there are 3 different instances of mixin: MixinExpression, MixinStatement and MixinDeclaration. Even though they all come down to the same thing.
Modifiers like shared are repeated, e.g. in BasicType, Attribute, TypeSpecialization. Also your proposed TypeWithModifier somewhat replicates parts of BasicType (just without the parentheses)

Walter Bright wrote:
> On 3/28/2011 11:46 PM, Rainer Schuetze wrote:
>>
>> I've set up a comparison with some notes here:
>>
>> http://www.dsource.org/projects/visuald/wiki/GrammarComparison
>>
> 
> 
> I thought I fixed the grammar issues listed in Bugzilla.

The comparison is from the time I wrote the parser, so it is a few month old. Checking the latest docs, there are already quite a number of issues fixed, but also some new have crept in.

I'll redo the comparison with the latest extracted grammar...

> 
> Could you please add a bugzilla entry for your wiki notes?

I hope you mean an entry with a link to the wiki. The comparison itself might a bit too large itself for bugzilla (html is larger than 800kB) ;-)

Trass3r wrote:
> Rainer Schuetze wrote:
>> I've extracted the website grammar using some combination of scripts from the ddoc sources, so if there is interest, I can dig them up...
> 
> Yep, please make them publicly available so they don't get lost.
> Though I think we should have an extra official page with the complete grammar in addition to the widespread snippets anyway.

I've uploaded the script and the grammar used for the comparison (a few month old) and the grammar from the current doc sources to

http://www.dsource.org/projects/visuald/browser/grammar

See the header of gen_grammar.sh for a very short description what to do to run the generation.