The default macros built into DDoc leave something to be desired in terms of HTML structure, for example:

- presentational <font>, <big> and <small> tags, rather than CSS equivalents
- presentational details coded directly into the macros called by DDoc, rather than delegated to CSS classes
- <br><br> to separate paragraphs, rather than <p></p> to enclose them.

By redefining a lot of the macros in a .ddoc file, one can do better.  However, there remain a number of limitations:

(a) No way to make a paragraph of the DDoc source a paragraph in the generated HTML unless it's specified to be something else, such as a list or a table.  While one could wrap the actual paragraphs in $(P ... ) and define DDOC_BLANKLINE to be empty, this undermines the first two of DDoc's goals (looks good as embedded documentation, and is easy and natural to write).

(b) No macro to be seen that corresponds to the whole block of named sections.  So I can't (for example) wrap them all in a <dl>.  Meanwhile I've made do with a <dl> containing a single <dt> and <dd> for each named section.

(c) No way to user-define sections - you can change the markup generated for the standard sections, but for everything else there's just DDOC_SECTION and DDOC_SECTION_H, precluding any distinctive highlighting of certain sections.

There are probably others I thought of earlier, but I can't think what they are at the moment.  But as you see, DDoc falls short of what ought to be another of its goals: to support the generation of logically structured output.

Has anybody else got further with getting logical HTML out of DDoc?  Have you been using DDoc to generate some internal format and then some other program to turn it into HTML, or what?


But some improvements to DDoc could certainly make things better.

Dealing with (a) admittedly isn't as simple as it may seem at first.  While you could have a DDOC_PARAGRAPH macro for this, there are subtleties in determining what is and isn't a paragraph.

(b) is straightforward - just create and use a DDOC_NAMED_SECTIONS macro.

As for (c), well, maybe we could have it so that

Custom_Section:
    blah blah blah

first looks for a macro called DDOC_Custom_Section and, if it's present, uses it.  Or if no such macro is present, just use DDOC_SECTION.


I don't know how far we can go short of a doclet system like JavaDoc has.  Nor even how exactly a doclet system would work under a language that compiles to native code....

Stewart.

On Mar 10, 2012 8:10 PM, "Stewart Gordon" <smjg_1998@yahoo.com> wrote>
> I don't know how far we can go short of a doclet system like JavaDoc has.
 Nor even how exactly a doclet system would work under a language that
compiles to native code....
>
> Stewart.

I hate to say it but I think the ddoc system was way overthought.  It would be better to use a markup language such as Creole that is meant to look nice and you can convert that into anything you want.  You could leave the default html but provide an api for accessing a parsed tree.

On 11/03/2012 02:26, Kevin Cox wrote:
<snip>
> I hate to say it but I think the ddoc system was way overthought.  It would be better to
> use a markup language such as Creole that is meant to look nice and you can convert that
> into anything you want.  You could leave the default html but provide an api for accessing
> a parsed tree.

I think this is more or less how Java doclets work.  It works because Java's bytecode and dynamic class loading provide a means to call an arbitrary method of an arbitrary compiled class that need not be part of the same app.

But how are we going to do this with D?  It seems we would need to either:
(a) compile the doclet on the fly and pass the parse tree into it
(b) have in said API a parse tree constructor that translates it from a textual representation received through stdin.

Stewart.

Am 11.03.2012 03:26, schrieb Kevin Cox:
>
> On Mar 10, 2012 8:10 PM, "Stewart Gordon" <smjg_1998@yahoo.com
> <mailto:smjg_1998@yahoo.com>> wrote>
>  > I don't know how far we can go short of a doclet system like JavaDoc
> has.  Nor even how exactly a doclet system would work under a language
> that compiles to native code....
>  >
>  > Stewart.
>
> I hate to say it but I think the ddoc system was way overthought.  It
> would be better to use a markup language such as Creole that is meant to
> look nice and you can convert that into anything you want.  You could
> leave the default html but provide an api for accessing a parsed tree.
>

This is a bit what Go does with the go packages (ast, doc, etc)
http://tip.golang.org/pkg/go/

--
Paulo

On 03/11/2012 12:46 PM, Stewart Gordon wrote:
> On 11/03/2012 02:26, Kevin Cox wrote:
> <snip>
>> I hate to say it but I think the ddoc system was way overthought. It
>> would be better to
>> use a markup language such as Creole that is meant to look nice and
>> you can convert that
>> into anything you want. You could leave the default html but provide
>> an api for accessing
>> a parsed tree.
>
> I think this is more or less how Java doclets work. It works because
> Java's bytecode and dynamic class loading provide a means to call an
> arbitrary method of an arbitrary compiled class that need not be part of
> the same app.
>
> But how are we going to do this with D? It seems we would need to either:
> (a) compile the doclet on the fly and pass the parse tree into it
> (b) have in said API a parse tree constructor that translates it from a
> textual representation received through stdin.
>
> Stewart.

I don't understand why you related dynamic class' loading with a documentation system/syntax for D.

Some hours ago you closed a very old enhancement report I submited:

http://d.puremagic.com/issues/show_bug.cgi?id=3161

(in fact I had totally forgotten I implemented that :-P, so thanks for reminding it to me... thought I didn't like that you closed it as invalid)

Now, I implemented it purely based on dmd's code. I didn't add anything else to get that done. And D is not dynamic or doesn't have dynamic class loading.

So... can you explain a bit more?

And does somebody know why, being dmd able to compile files, which requires having exact type information for every method/template,whatever, generates such a poor documentation?

Another question: ddoc is used (or was used) to generate d's main site. Wut?? So it's much more than just documenting D classes. I think this is wrong. That's when you end up having macros and all of that, because when generating a page you want to reuse things. But when documentation a library, hmmm... I never needed such thing. And even though I do care about the format of the documentation (how it looks), I definitely don't put formatting information inside a method's documentation (except for code snippets and lists).

Some simple like markdown or rdoc (well, the idea, not rdoc itself for D :-P) should be more than enough.

If there's interesent, I can hack dmd into producing such documentation, and send a pull request.

On 11/03/2012 19:04, Ary Manzana wrote:
<snip>
> I don't understand why you related dynamic class' loading with a documentation
> system/syntax for D.

Because that's what Javadoc relies on as a means for a program distributed in binary form to call custom code.

OK, so there are native ways to do this, like DLLs and SOs.  I don't really know whether these are fit for the purpose.  It might get a bit complicated supporting the D object model with DMD being written in C++.

> Some hours ago you closed a very old enhancement report I submited:
>
> http://d.puremagic.com/issues/show_bug.cgi?id=3161
>
> (in fact I had totally forgotten I implemented that :-P, so thanks for reminding it to
> me... thought I didn't like that you closed it as invalid)
>
> Now, I implemented it purely based on dmd's code. I didn't add anything else to get that
> done. And D is not dynamic or doesn't have dynamic class loading.

OK, so writing a new documentation generator based on the DMD front end code is another approach.  Did you just write it in C++, or do the extra work so that you could write it in D?

Either way, it's good that you've written a D documentation generator that's better than standard Ddoc, and no doubt handles D much better than Doxygen does.  Have you released it anywhere?

> So... can you explain a bit more?
>
> And does somebody know why, being dmd able to compile files, which requires having exact
> type information for every method/template,whatever, generates such a poor documentation?

I guess because not that much work has gone into Ddoc so far.

> Another question: ddoc is used (or was used) to generate d's main site. Wut?? So it's much
> more than just documenting D classes. I think this is wrong. That's when you end up having
> macros and all of that, because when generating a page you want to reuse things. But when
> documentation a library, hmmm... I never needed such thing. And even though I do care
> about the format of the documentation (how it looks), I definitely don't put formatting
> information inside a method's documentation (except for code snippets and lists).

By "formatting information" are you talking about semantic/logical markup or presentational/physical markup?

http://webtips.dan.info/logical.html
talks about the differences between them - written from an HTML point of view but I suppose equally applicable to Ddoc macros.

> Some simple like markdown or rdoc (well, the idea, not rdoc itself for D :-P) should be
> more than enough.
>
> If there's interesent, I can hack dmd into producing such documentation, and send a pull
> request.

To replace the current Ddoc comment format, or to give the user the choice between that and whatever else?

Replacing the current format would be a breaking change.

Having two formats built into DMD would seem silly, aside from the question of how you would specify which is being used in a given project.  The idea of Ddoc is that it's the standard D way of doing documentation.  If a compiler has built-in documentation generation, it should use either a format that is part of the language standard or some established format such as Doxygen, rather than some non-portable ad-hoc format.  Which is what what you're thinking of doing would be, unless it's incorporated into the language standard.  But having two documentation comment formats in the language standard is itself something that doesn't seem to make sense.  It would be like the whole D language having two syntaxes.

Stewart.

On 03/11/2012 05:33 PM, Stewart Gordon wrote:
> On 11/03/2012 19:04, Ary Manzana wrote:
> <snip>
>> I don't understand why you related dynamic class' loading with a
>> documentation
>> system/syntax for D.
>
> Because that's what Javadoc relies on as a means for a program
> distributed in binary form to call custom code.

What do you mean by "custom code"?

>
> OK, so there are native ways to do this, like DLLs and SOs. I don't
> really know whether these are fit for the purpose. It might get a bit
> complicated supporting the D object model with DMD being written in C++.

Sorry, I still don't understand you :-(

>
>> Some hours ago you closed a very old enhancement report I submited:
>>
>> http://d.puremagic.com/issues/show_bug.cgi?id=3161
>>
>> (in fact I had totally forgotten I implemented that :-P, so thanks for
>> reminding it to
>> me... thought I didn't like that you closed it as invalid)
>>
>> Now, I implemented it purely based on dmd's code. I didn't add
>> anything else to get that
>> done. And D is not dynamic or doesn't have dynamic class loading.
>
> OK, so writing a new documentation generator based on the DMD front end
> code is another approach. Did you just write it in C++, or do the extra
> work so that you could write it in D?

I wrote it in Java, because back then I had dmd ported to Java and I found it easy to do it that way (I'm not very efficient/fast programming in C++).

>
> Either way, it's good that you've written a D documentation generator
> that's better than standard Ddoc, and no doubt handles D much better
> than Doxygen does. Have you released it anywhere?

I posted it back then, it's in Descent's source code. There wasn't much interest, though. And instead of working on a decent documentation system you end up with documentation like this one:

https://github.com/D-Programming-Language/phobos/blob/master/std/algorithm.d#L4

Nice, a javascript tag inside of the docs. And then an excerpt:

$(BOOKTABLE ,
$(TR $(TH Category) $(TH Functions)
)
$(TR $(TDNW Searching) $(TD $(MYREF balancedParens) $(MYREF

???

Ok, a table can't be done easily in markdown or rdoc either, but a list would have maybe been enough. Or just list the functions in alphabetical order, top down (because it's hard to do a binary search with the eyes from left to right, since it takes some time to spot the first letter of every word). Like finding join here:

http://dlang.org/phobos/std_array.html

instead of here:

http://pastebin.com/ib4dJt48

>
>> So... can you explain a bit more?
>>
>> And does somebody know why, being dmd able to compile files, which
>> requires having exact
>> type information for every method/template,whatever, generates such a
>> poor documentation?
>
> I guess because not that much work has gone into Ddoc so far.

Documentation is the primary interface to the users of a library, including the standard library. I believe it deserves a lot more attention than that.

>
>> Another question: ddoc is used (or was used) to generate d's main
>> site. Wut?? So it's much
>> more than just documenting D classes. I think this is wrong. That's
>> when you end up having
>> macros and all of that, because when generating a page you want to
>> reuse things. But when
>> documentation a library, hmmm... I never needed such thing. And even
>> though I do care
>> about the format of the documentation (how it looks), I definitely
>> don't put formatting
>> information inside a method's documentation (except for code snippets
>> and lists).
>
> By "formatting information" are you talking about semantic/logical
> markup or presentational/physical markup?
>
> http://webtips.dan.info/logical.html
> talks about the differences between them - written from an HTML point of
> view but I suppose equally applicable to Ddoc macros.

Interesting reading.

I talk about semantic/logical markup. But I'd like to use one which, when I read it, doesn't hurt my eyes (like std.algorithm docs).

>
>> Some simple like markdown or rdoc (well, the idea, not rdoc itself for
>> D :-P) should be
>> more than enough.
>>
>> If there's interesent, I can hack dmd into producing such
>> documentation, and send a pull
>> request.
>
> To replace the current Ddoc comment format, or to give the user the
> choice between that and whatever else?

Replace it. Macros to document code? I think that's a bit too much...

Write a list in DDoc:

$(UL
  $(LI Karatsuba multiplication)
  $(LI Squaring is optimized independently of multiplication)
  $(LI Divide-and-conquer division)
  $(LI Binary exponentiation)
)

Write a list in rdoc or markdown:

* Karatsuba multiplication
* Squaring is optimized independently of multiplication
* Divide-and-conquer division
* Binary exponentiation

Or links. With DDoc:

$(WEB sgi.com/tech/stl/, Alexander Stepanov's Standard Template Library)

With markdown

[Alexander Stepanov's Standard Template Library](sgi.com/tech/stl/)

(well, I admit there's not much difference in this last example. Still... it's shorter and cleaner :-P)

>
> Replacing the current format would be a breaking change.

Only for docs that heavily use macros. I hope there's not many!

And you can always make a tool to convert ddoc to the new format (I believe with ddoc macros?)

>
> Having two formats built into DMD would seem silly, aside from the
> question of how you would specify which is being used in a given
> project. The idea of Ddoc is that it's the standard D way of doing
> documentation. If a compiler has built-in documentation generation, it
> should use either a format that is part of the language standard or some
> established format such as Doxygen, rather than some non-portable ad-hoc
> format. Which is what what you're thinking of doing would be, unless
> it's incorporated into the language standard. But having two
> documentation comment formats in the language standard is itself
> something that doesn't seem to make sense. It would be like the whole D
> language having two syntaxes.

Just leave one syntax.

On 03/11/2012 03:26 AM, Kevin Cox wrote:
>
> On Mar 10, 2012 8:10 PM, "Stewart Gordon" <smjg_1998@yahoo.com
> <mailto:smjg_1998@yahoo.com>> wrote>
>  > I don't know how far we can go short of a doclet system like JavaDoc
> has.  Nor even how exactly a doclet system would work under a language
> that compiles to native code....
>  >
>  > Stewart.
>
> I hate to say it but I think the ddoc system was way overthought.  It
> would be better to use a markup language such as Creole that is meant to
> look nice and you can convert that into anything you want.  You could
> leave the default html but provide an api for accessing a parsed tree.
>

Afaik you can use ddoc together with any markup language you like, as long as it does not clash with the macro syntax.

On Sunday, 11 March 2012 at 20:59:23 UTC, Ary Manzana wrote:
> Write a list in DDoc:

Better:
  $(LIST
      Karatsuba multiplication,
      Squaring is optimized independently of multiplication,
      Divide-and-conquer division,
      Binary exponentiation)

Many of the default ddoc macros suck. They are hard
to use and generate bad html.

The ddoc system can do a LOT better.

On 11/03/2012 20:59, Ary Manzana wrote:
> On 03/11/2012 05:33 PM, Stewart Gordon wrote:
<snip>
>> Because that's what Javadoc relies on as a means for a program
>> distributed in binary form to call custom code.
>
> What do you mean by "custom code"?

Code written and compiled by a user of the tool, of course.  Or maybe written by a third party unconnected with the tool's creator.

<snip>
> Replace it. Macros to document code? I think that's a bit too much...

The advantage of a macro system is that it's easily extensible by the user.  Does your proposed system provide this?

<snip>
> And you can always make a tool to convert ddoc to the new format (I believe with ddoc
> macros?)
<snip>

You mean make a tool that leaves D code unchanged, but uses the ddoc macro system to convert documentation comments to the new format?

Stewart.