DScanner is a tool for analyzing D source code. It has the following features:

* Prints out a complete AST of a source file in XML format.
* Syntax checks code and prints warning/error messages
* Prints a listing of modules imported by a source file
* Syntax highlights code in HTML format
* Provides more meaningful "line of code" count than wc
* Counts tokens in a source file

The lexer/parser/AST are located in the "std/d" directory in the repository. These files should prove useful to anyone else working on D tooling.

https://github.com/Hackerpilot/Dscanner

Aside: the D grammar that I reverse-engineered can be located here: https://rawgithub.com/Hackerpilot/DGrammar/master/grammar.html

On Sun, 28 Jul 2013 00:27:34 +0200
"Brian Schott" <briancschott@gmail.com> wrote:

> DScanner is a tool for analyzing D source code. It has the following features:
> 
> * Prints out a complete AST of a source file in XML format.
> * Syntax checks code and prints warning/error messages
> * Prints a listing of modules imported by a source file
> * Syntax highlights code in HTML format
> * Provides more meaningful "line of code" count than wc
> * Counts tokens in a source file
> 
> The lexer/parser/AST are located in the "std/d" directory in the repository. These files should prove useful to anyone else working on D tooling.
> 
> https://github.com/Hackerpilot/Dscanner
> 
> Aside: the D grammar that I reverse-engineered can be located here: https://rawgithub.com/Hackerpilot/DGrammar/master/grammar.html

Sweet, I was *just* thinking about writing a D -> HTML syntax highlighter no more than about five minutes ago. Glad to see I won't have to :) I'll definitely be checking this out.

On Saturday, 27 July 2013 at 22:27:35 UTC, Brian Schott wrote:
> DScanner is a tool for analyzing D source code. It has the following features:
>
> * Prints out a complete AST of a source file in XML format.
> * Syntax checks code and prints warning/error messages
> * Prints a listing of modules imported by a source file
> * Syntax highlights code in HTML format
> * Provides more meaningful "line of code" count than wc
> * Counts tokens in a source file
>
> The lexer/parser/AST are located in the "std/d" directory in the repository. These files should prove useful to anyone else working on D tooling.
>
> https://github.com/Hackerpilot/Dscanner
>
> Aside: the D grammar that I reverse-engineered can be located here: https://rawgithub.com/Hackerpilot/DGrammar/master/grammar.html

Great! :)

I do not understand the LoC count, though. The description sounds like "Number of Statements"?

On Saturday, 27 July 2013 at 22:27:35 UTC, Brian Schott wrote:
> DScanner is a tool for analyzing D source code. It has the following features:
>
> * Prints out a complete AST of a source file in XML format.
> * Syntax checks code and prints warning/error messages
> * Prints a listing of modules imported by a source file
> * Syntax highlights code in HTML format
> * Provides more meaningful "line of code" count than wc
> * Counts tokens in a source file
>
> The lexer/parser/AST are located in the "std/d" directory in the repository. These files should prove useful to anyone else working on D tooling.
>
> https://github.com/Hackerpilot/Dscanner
>
> Aside: the D grammar that I reverse-engineered can be located here: https://rawgithub.com/Hackerpilot/DGrammar/master/grammar.html

Awesome! I hope it won't be forgotten by the time I need it :)

By the way, how far is that "std.d.*" stuff from ongoing Phobos inclusion review?

I suppose currently it does not do any semantical analysis? How hard it would be to implement dmd warnings on top of dscanner instead?

On Sunday, 28 July 2013 at 12:49:34 UTC, Dicebot wrote:

> Awesome! I hope it won't be forgotten by the time I need it :)
>
> By the way, how far is that "std.d.*" stuff from ongoing Phobos inclusion review?
>
> I suppose currently it does not do any semantical analysis? How hard it would be to implement dmd warnings on top of dscanner instead?

I don't think it's necessary for semantic analysis to be included in Phobos. It's enough to start with a lexer, then later add a parser and semantic analysis.

--
/Jacob Carlborg

On Saturday, 27 July 2013 at 22:27:35 UTC, Brian Schott wrote:
> Aside: the D grammar that I reverse-engineered can be located here: https://rawgithub.com/Hackerpilot/DGrammar/master/grammar.html

Does arrayLiteral support stray comma?

Also looks like enumBody supports multiple commas with nothing between them, but dmd doesn't support it. Sould be
'{' enumMember (',' enumMember)* ','? '}'

And arrayLiteral should be
'[' (assignExpression (',' assignExpression)* ','?)? ']'

The same for arrayInitializer:
'[' (arrayMemberInitialization (',' arrayMemberInitialization)* ','?)? ']'

Like typedef alias supports multiple declarators.
aliasDeclaration:
'alias' (aliasInitializer (',' aliasInitializer)* | type declarator (',' declarator)*) ';'

Also declarator supports initializer, but alias doesn't.
declarator:
    Identifier ('=' initializer)?
    ;

alias int a=1;
Error: alias cannot have initializer