The purpose of a static language compiler is to help the programmer spot the bugs in the code very well and to compile it. It's not the job of a compiler to create nice HTML/XML documentation files.

So I suggest to:
1) DMD can copy all documentation strings inside the JSON file generated with the -X option. DMD has just to recognize the /** */ and /// inside the D code and put only such comments inside the JSON (the JSON has to contain all the information necessary to create the HTML pages).
2) Move the code that understands DDoc syntax and generates the HTML docs inside an external utility written in D or Python, and keep such utility inside the zip of the standard DMD distribution. This program can ask DMD to generate the JSON files, can read them and generate the HTML pages.
3) Deprecate and then remove the -D option of DMD2.

Removing that HTML ddoc parsing & HTML generation from DMD allows such code to evolve faster and to be debugged more efficiently. Today some people are using

[Mostly unrelated: once xfbuild is well debugged it can be added inside the zip of the standard DMD distribution.]

Bye,
bearophile

bearophile wrote:
> Removing that HTML ddoc parsing & HTML generation from DMD allows such code
> to evolve faster and to be debugged more efficiently. Today some people are
> using

There are very good reasons why ddoc is part of dmd.

1. Being a defined part of D means it's ALWAYS there. That means there won't be D compilers without ddoc. That means that people can rely on it being there, and use it with confidence.

2. Please do not underestimate the power of something reliably being there. In my experience, it makes it FAR more likely that people will actually use it, and there's the big win. If it's a separate utility, I guarantee you that usage of ddoc will drop probably by 90%.

3. A separate ddoc utility means that, inevitably, dmd and ddoc will get out of sync with each other. People will update dmd and not ddoc, and vice versa. There is quite enough grief with people mixing one version of dmd with the wrong version of phobos and then being mystified by the results. I can't tell you how many times I've helped someone who INSISTED they were using the correct libphobos.a and there were no others on their machine, only to eventually sheepishly discover another old libphobos.a on their path that was being used.

4. Making it a separate utility means others will invent "improved" ddocs, with the resulting balkanization and non-portability of documentation comments. See C++ for the affect this has.

5. The documentation of Phobos took a HUGE leap forward with the integration of ddoc.

6. Similar advantages can be found with the integration of unittesting in D, and the integration of the preprocessor with C/C++ (I'm old enough to remember when that was a separate program).

7. Contrary to what you state, making ddoc a separate utility will make it far harder to maintain, not easier. Just outputting JSON files is not enough, as ddoc is also able to pretty-print the declarations and do color syntax highlighting of coding examples. If you look at the dmd source code changes, you'll find that ddoc tracks changes in D's grammar and syntax with rarely any changes at all in the ddoc code.

Walter Bright:
> There are very good reasons why ddoc is part of dmd.

Thank you for your many answers. I think you can even add them to the D2 FAQ :-)

Bye,
bearophile

On 8/1/2010 1:53 PM, bearophile wrote:
> 
> I think you can...
> 
> Bye,
> bearophile

I've been avoiding saying this for a while, but I suspect a lot of people have thought it at one time or another.  You probably ought to find a word other than 'can' to use.  Of course he's capable of... whatever.  Perhaps you meant to make it a suggestion?

Walter Bright Wrote:

> bearophile wrote:
> > Removing that HTML ddoc parsing & HTML generation from DMD allows such code to evolve faster and to be debugged more efficiently. Today some people are using
> 
> There are very good reasons why ddoc is part of dmd.
> 
> 1. Being a defined part of D means it's ALWAYS there. That means there won't be D compilers without ddoc. That means that people can rely on it being there, and use it with confidence.
> 
> 2. Please do not underestimate the power of something reliably being there. In my experience, it makes it FAR more likely that people will actually use it, and there's the big win. If it's a separate utility, I guarantee you that usage of ddoc will drop probably by 90%.

That's just silly. People using 15 different good document generators is much better than one s**y one with s**y templates you need to dig from old newsgroup posts.

> 
> 3. A separate ddoc utility means that, inevitably, dmd and ddoc will get out of sync with each other. People will update dmd and not ddoc, and vice versa. There is quite enough grief with people mixing one version of dmd with the wrong version of phobos and then being mystified by the results. I can't tell you how many times I've helped someone who INSISTED they were using the correct libphobos.a and there were no others on their machine, only to eventually sheepishly discover another old libphobos.a on their path that was being used.

His idea was probably to separate the tools and still provide them both as part of the standard distribution. It would be your duty to keep both up to date.

> 
> 4. Making it a separate utility means others will invent "improved" ddocs, with the resulting balkanization and non-portability of documentation comments. See C++ for the affect this has.

D already has several document generators because ddoc is way too primitive. Luckily the main competition (Tango) died because of an organized FUD attack.

> 
> 5. The documentation of Phobos took a HUGE leap forward with the integration of ddoc.

The output is still a huge joke compared to tools like doxygen and javadoc. No offense. And those are rather basic document generation tools -- better ones exist.

> 
> 6. Similar advantages can be found with the integration of unittesting in D, and the integration of the preprocessor with C/C++ (I'm old enough to remember when that was a separate program).

The integrated unit test is a joke. Spend 30 minutes with Java 6 someday and try JUnit. Or maybe few hours. The more you use, the more you learn. If you open your eyes, you might actually see how crippled your vision of unit testing is. The integrated unittest stuff brings almost zero value when doing serious software development.

> 
> 7. Contrary to what you state, making ddoc a separate utility will make it far harder to maintain, not easier.

One of the main reasons why 3rd party tool support sucks is that YOU have zero ability communicating technical details to the community. We can see it in documentation. Almost none exists. People make lots of questions here and instead of answering you always focus on bikeshedding.

> Just outputting JSON files is not enough, as ddoc is also able to pretty-print the declarations and do color syntax highlighting of coding examples. If you look at the dmd source code changes, you'll find that ddoc tracks changes in D's grammar and syntax with rarely any changes at all in the ddoc code.

That's not a surprise. If a docgen tool and a compiler frontend coming from a single vendor didn't share any code, I wouldn't call it software engineering. It would be a horrible toy project written by an amateur.

Walter Bright wrote:

> bearophile wrote:
>> Removing that HTML ddoc parsing & HTML generation from DMD allows such code to evolve faster and to be debugged more efficiently. Today some people are using
> 
> There are very good reasons why ddoc is part of dmd.
> 
> 1. Being a defined part of D means it's ALWAYS there. That means there won't be D compilers without ddoc. That means that people can rely on it being there, and use it with confidence.
...

I agree, though the standard output is not so pleasant. When the new website design is adopted, would you please consider shipping a .ddoc plus the css needed in a similar style for user documentation? No doubt people are willing to help with this, I certainly would.

Lutger wrote:
> I agree, though the standard output is not so pleasant. When the new website design is adopted, would you please consider shipping a .ddoc plus the css needed in a similar style for user documentation? No doubt people are willing to help with this, I certainly would.


Sure.

On Sun, 01 Aug 2010 13:22:53 -0700, Walter Bright wrote:
> 1. Being a defined part of D means it's ALWAYS there. That means there won't be D compilers without ddoc.

/me waves.

I'm writing a D compiler, and have zero plans to add DDoc support. I couldn't do it well enough for me to waste my time on it.

Bernard Helyer:
> I'm writing a D compiler, and have zero plans to add DDoc support. I couldn't do it well enough for me to waste my time on it.

Walter has said that putting ddoc inside the compiler helps keep ddoc uniform, so people don't invent/use something different. In theory this is a nice purpose, but in practice I have seen lot of people use other documentation means for D, Doxygen. Walter seems to ignore how much often such other systems are used instead of ddoc.

Bye,
bearophile

On Friday 06 August 2010 20:06:59 Bernard Helyer wrote:
> On Sun, 01 Aug 2010 13:22:53 -0700, Walter Bright wrote:
> > 1. Being a defined part of D means it's ALWAYS there. That means there won't be D compilers without ddoc.
> 
> /me waves.
> 
> I'm writing a D compiler, and have zero plans to add DDoc support. I couldn't do it well enough for me to waste my time on it.

Well, that's your choice, but then you haven't followed the D spec in that regard, so your compiler wouldn't be properly standard D. It is the sort of thing that I would expect to be somewhat lower in priority than other features though.

- Jonathan M Davis