Hello everyone. I was talking in the IRC channel today and I mentioned something small I wrote a couple of years ago which generates a Graphviz DOT file based on ModuleInfo, so you can see a visual class hierarchy with interfaces included. There was some interest in this in the channel at the time.

The code doesn't do much more than that at the moment, it could offer a few more options including outputing the hierarchy in other formats, and the graph it generates could perhaps look better. I'm just wondering if anyone here is interested in having a tool like this.

If enough people are interested and have a few suggestions for some features they like which aren't crazy massive feature lists, I'd be willing to expand it a bit, add some documentation, and offer it as a DUB package.

On Thursday, 26 March 2015 at 17:41:58 UTC, w0rp wrote:
> Hello everyone. I was talking in the IRC channel today and I mentioned something small I wrote a couple of years ago which generates a Graphviz DOT file based on ModuleInfo, so you can see a visual class hierarchy with interfaces included. There was some interest in this in the channel at the time.
>
> The code doesn't do much more than that at the moment, it could offer a few more options including outputing the hierarchy in other formats, and the graph it generates could perhaps look better. I'm just wondering if anyone here is interested in having a tool like this.
>
> If enough people are interested and have a few suggestions for some features they like which aren't crazy massive feature lists, I'd be willing to expand it a bit, add some documentation, and offer it as a DUB package.

I forgot to link the GitHub repository. Here it is.

https://github.com/w0rp/dhier

On Thursday, 26 March 2015 at 17:41:58 UTC, w0rp wrote:
> Hello everyone. I was talking in the IRC channel today and I mentioned something small I wrote a couple of years ago which generates a Graphviz DOT file based on ModuleInfo, so you can see a visual class hierarchy with interfaces included. There was some interest in this in the channel at the time.
>
> The code doesn't do much more than that at the moment, it could offer a few more options including outputing the hierarchy in other formats, and the graph it generates could perhaps look better. I'm just wondering if anyone here is interested in having a tool like this.
>
> If enough people are interested and have a few suggestions for some features they like which aren't crazy massive feature lists, I'd be willing to expand it a bit, add some documentation, and offer it as a DUB package.

+1

How much of a stretch would it be to extend the dependency graph to compositional types?

struct Foo {
  mixin Bar!(a,b,c);
}

alias Qux = Baz!Foo;

and so on?

On Thursday, 26 March 2015 at 17:43:46 UTC, Vlad Levenfeld wrote:
> +1
>
> How much of a stretch would it be to extend the dependency graph to compositional types?
>
> struct Foo {
>   mixin Bar!(a,b,c);
> }
>
> alias Qux = Baz!Foo;
>
> and so on?

I'm not sure about that. I'd have to think about it a bit. Templates are probably trickier.

Someone else has also suggested module dependency graphs. I could maybe offer something for that too.

On 3/26/15 10:41 AM, w0rp wrote:
> Hello everyone. I was talking in the IRC channel today and I mentioned
> something small I wrote a couple of years ago which generates a Graphviz
> DOT file based on ModuleInfo, so you can see a visual class hierarchy
> with interfaces included. There was some interest in this in the channel
> at the time.
>
> The code doesn't do much more than that at the moment, it could offer a
> few more options including outputing the hierarchy in other formats, and
> the graph it generates could perhaps look better. I'm just wondering if
> anyone here is interested in having a tool like this.
>
> If enough people are interested and have a few suggestions for some
> features they like which aren't crazy massive feature lists, I'd be
> willing to expand it a bit, add some documentation, and offer it as a
> DUB package.

That would definitely be of interest. Also, if some of that can be reused to show inter-module dependencies that would be even better. Thanks! -- Andrei

On Thursday, 26 March 2015 at 17:41:58 UTC, w0rp wrote:
> If enough people are interested and have a few suggestions for some features they like which aren't crazy massive feature lists, I'd be willing to expand it a bit, add some documentation, and offer it as a DUB package.

Very nice! PlantUML [1] output would be even nicer!

[1] http://plantuml.sourceforge.net/classes.html

On Friday, 27 March 2015 at 08:26:51 UTC, Dragos Carp wrote:
> On Thursday, 26 March 2015 at 17:41:58 UTC, w0rp wrote:
>> If enough people are interested and have a few suggestions for some features they like which aren't crazy massive feature lists, I'd be willing to expand it a bit, add some documentation, and offer it as a DUB package.
>
> Very nice! PlantUML [1] output would be even nicer!
>
> [1] http://plantuml.sourceforge.net/classes.html

I'll give that a go. I was thinking before of offerring a DOT file with just the class and interface names, and another one with the class and interface names and some methods. I might support that and also PlantUML. I've been considering some kind of web-friendly format, but I'm not sure what to do for that yet. Maybe it's enough to have Graphviz output an SVG.

Last night I simplified part of the code and re-arranged everything into a DUB package format, which wasn't too hard. I'll probably work on it over the weekend.

I just added a module dependency graph generator by following a similar process to the library which I used for class hierarchies. I'd be willing to bet that Andrei wants to apply it to Phobos, and that is indeed a thing that can be done.

There's an example on GitHub, and the readme explains how to run it. The example outputs... this. (Try not to scream.)

https://i.imgur.com/Af4RKuc.png

So like you might expect for Phobos. Everything importing everything else, just about. I have a few comments on this.

The library just gives you the DOT file as output, not the image. Graphviz probably lets you pick different strategies for how a graph should be drawn. Drawing a graph is really complicated, so it's best to leave that to Graphviz. I believe you can specify drawing co-ordinates for vertices in a DOT file. In any case, how the graph is drawn in the end is a probably a user decision.

Like the previous class example, you can filter out modules from the output with a regular expression. You will probably have to do this to get sensible output, as you might not care which standard library modules you're depending on, or you might care, but not for a few modules in particular, etc.

The filters and the methods can be combined to only look at particular sets of modules at a time, so you can generate graphs for only a subset of Phobos, etc. This might help in making the imports look more like a binary tree.

Back on the class hierarchies. I hit a bit of a snag with that. I have been using ModuleInfo and ClassInfo to generate the diagrams, but ClassInfo is a runtime feature which has no knowledge of method or attribute names. So I might have to use compile time reflection all the way for that. So in short, I haven't thought of a good way to get method and attribute names into the output yet.

As an aside, ModuleInfo has been undocumented forever, as far as I can tell. Would it be possible to get some documentation for ModuleInfo up on the site? I've been figuring out how to use it just by reading object_.d. If it was documented, someone else would have probably written this stuff I'm writing by now already, because it's really simple code.

In an attempt to get something, anything, easier to read, I wrote another function which produces a graph where the module nodes are ranked by their incoming edge counts. The only way I could figure out how to get Graphviz to do this was to write the numbers as edges too and so on. So it looks like this.

https://i.imgur.com/jPGdPRm.png

I don't know why Graphviz decides that the distance between the ranks and the proper graph must be so huge. There's probably a way to write the two as subgraphs, and then set the distance between them to be small. I used the othorgonal lines for this one. Maybe some node and edge colouring would make the lines easier to follow, but generally as more things import each other the graph will tend to look messier.

On 3/29/15 4:24 AM, w0rp wrote:
> I just added a module dependency graph generator by following a similar
> process to the library which I used for class hierarchies. I'd be
> willing to bet that Andrei wants to apply it to Phobos, and that is
> indeed a thing that can be done.
>
> There's an example on GitHub, and the readme explains how to run it. The
> example outputs... this. (Try not to scream.)
>
> https://i.imgur.com/Af4RKuc.png
>
> So like you might expect for Phobos. Everything importing everything
> else, just about. I have a few comments on this.

Not screaming, it's as expected :o). This tool will be of good help for weeding some dependencies out.

One other informative graph that should be much easier to generate is the direct imports graph (no transitivity).

I should add that removing dependencies for Phobos should be approached with due consideration. It shouldn't be a goal in and of itself; users don't care for it. It does have positive effects such as easier debugging of the compiler that need to be appraised.

[snip
> As an aside, ModuleInfo has been undocumented forever, as far as I can
> tell. Would it be possible to get some documentation for ModuleInfo up
> on the site? I've been figuring out how to use it just by reading
> object_.d. If it was documented, someone else would have probably
> written this stuff I'm writing by now already, because it's really
> simple code.

We should, and maybe we can take this opportunity to define it well. Once it's documented, it sets the stage for backward compatibility.


Andrei