On Thu, 1 Aug 2013 21:52:00 -0400
Nick Sabalausky <SeeWebsiteToContactMe@semitwist.com> wrote:

> On Sun, 28 Jul 2013 00:27:34 +0200
> "Brian Schott" <briancschott@gmail.com> wrote:
> 
> > DScanner is a tool for analyzing D source code.
> >[...]
> 
> When I try to compile it (DMD 2.063.2 Win32) I get this:
> 
> C:\Users\Nick\AppData\Roaming\dvm\compilers\dmd-2.063.2\bin\..\src\phobos\std\array.d(321):
> Error: function core.memory.GC.malloc (uint sz, uint ba = 0u) is not
> callable using argument types (ulong, BlkAttr)
> C:\Users\Nick\AppData\Roaming\dvm\compilers\dmd-2.063.2\bin\..\src\phobos\std\array.d(272):
> Error: template instance std.array.arrayAllocImpl!(false, ubyte[],
> ulong) error instantiating main.d(77):        instantiated from here:
> uninitializedArray!(ubyte[], ulong) main.d(77): Error: template
> instance std.array.uninitializedArray!(ubyte[], ulong) error
> instantiating

Actually, I dug into this more (it was problem on 32-bit) and made a
pull request:
https://github.com/Hackerpilot/Dscanner/pull/42

On Saturday, 27 July 2013 at 22:27:35 UTC, Brian Schott wrote:
> DScanner is a tool for analyzing D source code. It has the following features:
>
> * Prints out a complete AST of a source file in XML format.
> * Syntax checks code and prints warning/error messages
> * Prints a listing of modules imported by a source file
> * Syntax highlights code in HTML format
> * Provides more meaningful "line of code" count than wc
> * Counts tokens in a source file
>
> The lexer/parser/AST are located in the "std/d" directory in the repository. These files should prove useful to anyone else working on D tooling.
>
> https://github.com/Hackerpilot/Dscanner
>
> Aside: the D grammar that I reverse-engineered can be located here: https://rawgithub.com/Hackerpilot/DGrammar/master/grammar.html

Any idea on when we might see json output(i am not a fan of xml)? Other than that this is a fantastic project! I am planing some projects in the future and this will be of great help. Keep up the good work!

On Friday, 2 August 2013 at 13:52:14 UTC, Tofu Ninja wrote:
> Any idea on when we might see json output(i am not a fan of xml)?

Roughly the same time somebody submits a pull request.

I'm currently focusing my spare time on DCD, so the JSON output will happen after I'm able to get auto-completion working.

On Friday, 2 August 2013 at 18:01:01 UTC, Brian Schott wrote:
> On Friday, 2 August 2013 at 13:52:14 UTC, Tofu Ninja wrote:
>> Any idea on when we might see json output(i am not a fan of xml)?
>
> Roughly the same time somebody submits a pull request.
>
> I'm currently focusing my spare time on DCD, so the JSON output will happen after I'm able to get auto-completion working.

I will look into adding it my self if I get some time, but I don't think I will need to use this for a while. For what I want it for, there is a lot of legwork to be done before I get around to needing this.

Also roughly how difficult would it be to re-create source code from the xml? And does the xml preserve comments and if so does it do anything with ddoc?

On Friday, 2 August 2013 at 18:12:15 UTC, Tofu Ninja wrote:
> I will look into adding it my self if I get some time, but I don't think I will need to use this for a while. For what I want it for, there is a lot of legwork to be done before I get around to needing this.

The XML output is handled by this class:
https://github.com/Hackerpilot/Dscanner/blob/master/astprinter.d
It shouldn't be much more difficult than changing the print statements.

> Also roughly how difficult would it be to re-create source code from the xml? And does the xml preserve comments and if so does it do anything with ddoc?

Aside from the comments, it should be possible to recreate the source from the AST. If it's not, there's a bug in the AST output. Comments are skipped when syntax checking or generating the AST.

Brian Schott:

> DScanner is a tool for analyzing D source code. It has the following features:
> ...
> https://github.com/Hackerpilot/Dscanner

I have just compiled it on Windows32 and tried it.

The compilation using the given bat has failed to link:

OPTLINK (R) for Win32  Release 8.00.15
Copyright (C) Digital Mars 1989-2013  All rights reserved.
http://www.digitalmars.com/ctg/optlink.html
dscanner.obj(dscanner)
 Error 42: Symbol Undefined _D8analysis10ifelsesame15IfElseSameCheck6__ctorMFAyaZC8analysis10ifelsesame15IfElseSameCheck
dscanner.obj(dscanner)
 Error 42: Symbol Undefined _D8analysis12constructors16ConstructorCheck7__ClassZ
dscanner.obj(dscanner)
 Error 42: Symbol Undefined _D8analysis12constructors16ConstructorCheck6__ctorMFAyaZC8analysis12constructors16ConstructorCheck
dscanner.obj(dscanner)
 Error 42: Symbol Undefined _D8analysis10ifelsesame15IfElseSameCheck7__ClassZ
--- errorlevel 4


But I have built the DScanner successfully using the old "bud" tool.

I have seen that this basic usage is not supported:
dscanner -s *.d

I have seen it doesn't support source code with unicode identifiers or chars.

How do you enable/disable specific tests when you use -s?

This code gives four problems to Dscanner:

void main() {
    auto x = float(5);
    auto r = 1. + 2;
    int items[5];
    import std.stdio;
    int.max.writeln;
}


In about 25_000 CLOC lines of my code DScanner has found several usages of the old-style alias syntax, that I will fix. Plus it has found three usages of the implicit string concatenation, and two of them are (the same) bug! Andrei Alexandrescu was very wrong to think that implicit string concatenation is a speck of dust. I am not going to close down that enhancement request.

Bye,
bearophile

On 04/22/2014 07:08 PM, bearophile wrote:
> This code gives four problems to Dscanner:
> 
> void main() {
>      auto x = float(5);
This is not valid. DMD and the grammar spec both do not allow this.
>      auto r = 1. + 2;
Fixed. (https://issues.dlang.org/show_bug.cgi?id=12623)
>      int items[5];
I don't support C-style declarations. DMD doesn't really support them either. (https://issues.dlang.org/show_bug.cgi?id=953)
>      import std.stdio;
>      int.max.writeln;
Fixed.
> }

> Andrei Alexandrescu was very wrong to think that implicit string concatenation is a speck of dust. I am not going to close down that enhancement request.

It's a very bad feature. Every time I've ever used it has been a bug.

Brian Schott:

> This is not valid. DMD and the grammar spec both do not allow this.

This was changed weeks ago. Now D accepts that code.


>>     int items[5];
> I don't support C-style declarations. DMD doesn't really support them either. (https://issues.dlang.org/show_bug.cgi?id=953)

Then I suggest DScanner to support the half-C-declarations, because D compilers digests them just fine and they are very common in D code you will find in the wild. A code analyzer has to accept the real world code people write, otherwise it's much less useful :-)

Bye,
bearophile

On Thursday, 24 April 2014 at 21:36:24 UTC, bearophile wrote:
> This was changed weeks ago. Now D accepts that code.

You want tooling to support language features that aren't released?

> Then I suggest DScanner to support the half-C-declarations, because D compilers digests them just fine and they are very common in D code you will find in the wild. A code analyzer has to accept the real world code people write, otherwise it's much less useful :-)

Several people have indicated that they care about C-style declarations more than I do, and yet they still don't care enough to implement support for them and create a pull request.

On Thursday, 24 April 2014 at 22:19:29 UTC, Brian Schott wrote:
>> Then I suggest DScanner to support the half-C-declarations, because D compilers digests them just fine and they are very common in D code you will find in the wild. A code analyzer has to accept the real world code people write, otherwise it's much less useful :-)
>
> Several people have indicated that they care about C-style declarations more than I do, and yet they still don't care enough to implement support for them and create a pull request.

I think if such support is ever to be implemented it should result in immediate analysis message with demand to change it to D style declarations and never do it again. This is an unfortunate legacy that should not be encouraged.