On 2013-12-31 23:27, Walter Bright wrote:

> I used this one in std.datetime:
>
>          LREF2=<a href="#$1">$(D $2)</a>

That would require specific knowledge about how Ddoc generate the HTML and file structure. Like replaces dots with an underscore in the module name for the file name. std.datetime becomes std_datetime.htm. Is that wise?

-- 
/Jacob Carlborg

On 2013-12-31 18:08, Andrei Alexandrescu wrote:

> That's an exaggeration. The top of std.algorithm produces complex output
> (two tables). The rest of the documentation is nice and legible.

std.algorithm contains 335 lines of ddoc comments. I would consider that "hundreds", or does "hundreds" mean something else. What I'm saying is that those two tables should be automatically generated, at least the first one.

> As one who's written both, I can testify this isn't true. Ddoc macros
> need to be mentioned once and then the closing paren completes them.
> Additionally, they can expand to arbitrarily complex output. In
> contrast, tags need to be mentioned twice (an often-mentioned issue with
> writing HTML by hand), are syntactically heavier, and therefore litter
> the text a lot more.

I'm considering it to be a sort of tag, just with a different syntax. Who said tags need to have an end tag. A less verbose a minimal reliance on tags would be Markdown or similar. Creating a list in Markdown is simple as:

* this
* is
* a
* list
* in Markdown

> No. There are two tables there. One is a categorization of algorithms,
> which is human-made. There are several solutions to materializing that
> table but they all would require the intervention of a human being. I
> figured std.algorithm adds new algorithms rarely enough to allow myself
> a solution that requires manual intervention.

If there's a specific macro that would indicate a category then it would be easy to generate a table like that.

/// $(CATEGORY searching)
void foo ()

> The second table is a "cheat sheet" with human-produced short
> descriptions for each function. Those short descriptions wouldn't fit
> well with the longer descriptions.

That can also be generated automatically. Ddoc can just insert the summary [1] for each symbol.

[1] The Ddoc documentation says:

"The first section is the Summary, and does not have a section name. It is first paragraph, up to a blank line or a section name. While the summary can be any length, try to keep it to one line."

http://dlang.org/ddoc.html

> Again no. You are misinformed. The PDF manual is generated via LaTeX
> from the same ddoc sources as the HTML pages.

Ok, fair enough.

-- 
/Jacob Carlborg

On 2013-12-31 22:08, Walter Bright wrote:

> Not at all. The same ddoc sources are used to generate an ebook and a
> Windows help file - which are based on html but require different html
> to be generated, a difference handled by ddoc. The ebook and chm both
> require post processing with tools supplied by Amazon and Microsoft,
> respectively.
>
> It works, it is not a failure because the source files do not need to be
> modified to get this working. A failure would be having to modify the
> source files each time a different output needs to be generated.

Ok, I would not consider it a failure if it needs to be converted to a different format. But adding cross referencing, generating an index or similar would still be a failure, if that requires post processing.

> We all of course want that - but there is certainly room for
> disagreement about what "best" is.

Yes, but who would not want cross referencing to work properly?

> Reinventing Amazon's "kindlegen.exe" seems pointless to me.

Yes, I agree.

-- 
/Jacob Carlborg

On Tue, Dec 31, 2013 at 06:12:02PM -0800, Andrei Alexandrescu wrote:
> On 12/31/13 11:51 AM, H. S. Teoh wrote:
> >On Tue, Dec 31, 2013 at 09:08:07AM -0800, Andrei Alexandrescu wrote:
> >>On 12/31/13 4:26 AM, Jacob Carlborg wrote:
> >[...]
> >>>>>4. It doesn't rely on embedded HTML, as such will impede extraction and formatting for other purposes.
> >>>
> >>>As far as I know this isn't very useful. For the other formats we use, like PDF, it uses the HTML output as a base.
> >>
> >>Again no. You are misinformed. The PDF manual is generated via LaTeX from the same ddoc sources as the HTML pages.
> >[...]
> >
> >This is true.
> >
> >I will note, however, that limitations in Ddoc make the LaTeX output less than ideal. Many fine points of LaTeX formatting are ignored or simply not possible (e.g., proper use of em- and en-dashes, proper use of '.\ ', non-breaking spaces, hyphenation hints, etc.). In some cases, postprocessing is needed to take full advantage of the LaTeX format (esp. for display of math formulas, which is one of LaTeX's biggest selling points).
> >
> >Even for HTML, the lack of semantic support for paragraphs means XHTML compliance is out of the question, and any proper tag nesting where paragraphs are involved is a fragile hack that is likely non-HTML compliant. But nobody notices this because most browsers are too permissive in what they accept.
> 
> These should be fixed in ddoc.
[...]

For HTML:

	https://d.puremagic.com/issues/show_bug.cgi?id=9731

For LaTeX, though, it would require some pretty radical changes to ddoc, that I'm not sure will be accepted.


T

-- 
Blunt statements really don't have a point.

On 1/1/2014 4:43 AM, Jacob Carlborg wrote:
> On 2013-12-31 23:27, Walter Bright wrote:
>
>> I used this one in std.datetime:
>>
>>          LREF2=<a href="#$1">$(D $2)</a>
>
> That would require specific knowledge about how Ddoc generate the HTML and file
> structure.

Ddoc doesn't generate HTML (the macros do), and the # means it's a local reference, no file structure required.

On Wednesday, 1 January 2014 at 20:38:43 UTC, Walter Bright wrote:
> Ddoc doesn't generate HTML (the macros do)

Ddoc doesn't generate HTML at all, not even with macros.

/// Returns true iff a > b
bool greaterThan(int a, int b);

Ddoc does /not/ do the right thing there. The best you can do is write $(GT), which is stupid and insufficient or > which is wrong.

On 1/1/14 4:56 AM, Jacob Carlborg wrote:
> On 2013-12-31 18:08, Andrei Alexandrescu wrote:
>
>> That's an exaggeration. The top of std.algorithm produces complex output
>> (two tables). The rest of the documentation is nice and legible.
>
> std.algorithm contains 335 lines of ddoc comments. I would consider that
> "hundreds", or does "hundreds" mean something else.

I replied to "Hundreds of lines of Ddoc madness" with "that's an exaggeration". Do you really think I was refuting the factual 335 lines?

> What I'm saying is
> that those two tables should be automatically generated, at least the
> first one.

Yah, that would be nice.

>> As one who's written both, I can testify this isn't true. Ddoc macros
>> need to be mentioned once and then the closing paren completes them.
>> Additionally, they can expand to arbitrarily complex output. In
>> contrast, tags need to be mentioned twice (an often-mentioned issue with
>> writing HTML by hand), are syntactically heavier, and therefore litter
>> the text a lot more.
>
> I'm considering it to be a sort of tag, just with a different syntax.
> Who said tags need to have an end tag. A less verbose a minimal reliance
> on tags would be Markdown or similar. Creating a list in Markdown is
> simple as:
>
> * this
> * is
> * a
> * list
> * in Markdown

I prefer ddoc to markdown for its flexibility. Ddoc is an almost pure macro expansion system with minimal syntax. Markdown defines a bunch of syntax but has (last time I looked) is weak on macro capability.

Case in point: markdown lists will be styled one way. With DDoc macros one can define any style of lists needed. (We've defined BOOKTABLE in addition to a couple of other TABLEs, which works beautifully in HTML and LaTeX; not sure how can be done in Markdown.

> If there's a specific macro that would indicate a category then it would
> be easy to generate a table like that.
>
> /// $(CATEGORY searching)
> void foo ()

The issue is with writing text out of order, something DDoc is not very apt at. Would be nice to improve on that (we do some of it with javascript).


Andrei

Am 01.01.2014 21:38, schrieb Walter Bright:
> the # means it's a local reference, no file structure required.

A declaration being inside the *same* file is *also* part of the file/directory structure. I could also be in different files. The point is that if this kind of knowledge is encoded in the documentation, or it is tied to that particular structure. This is the case for the Phobos documentation and is the reason why the DDOX based docs have to completely ignore the manual cross references.

On 2014-01-01 21:38, Walter Bright wrote:

> Ddoc doesn't generate HTML (the macros do), and the # means it's a local
> reference, no file structure required.

Right, I failed to see that. But then that macro won't work for symbols in other modules.

The file structure I was referring to was that for the Phobos documentation the HTML files are created with the same name as the fully qualified module name but with dots replaced with underscores. If I want to create a link to a symbol in an other module I need to know about that.

-- 
/Jacob Carlborg

On Wed, Jan 01, 2014 at 06:41:50PM -0800, Andrei Alexandrescu wrote:
> On 1/1/14 4:56 AM, Jacob Carlborg wrote:
[...]
> >I'm considering it to be a sort of tag, just with a different syntax. Who said tags need to have an end tag. A less verbose a minimal reliance on tags would be Markdown or similar. Creating a list in Markdown is simple as:
> >
> >* this
> >* is
> >* a
> >* list
> >* in Markdown
> 
> I prefer ddoc to markdown for its flexibility. Ddoc is an almost pure macro expansion system with minimal syntax. Markdown defines a bunch of syntax but has (last time I looked) is weak on macro capability.

Well, ddoc was designed to be a macro system, so from that POV, it's pretty good at what it was intended to do. Whether it's the best system for *generating documentation*, though, is a matter of opinion. Markdown has the advantage of having very close visual appearance in the input as compared to the output -- a stated goal of ddoc, but obviously the unified macro syntax was a higher priority in ddoc's case. For example, using `backticks` to write code snippets is definitely more readable than writing $(D backticks), among many other things. But that does introduce more syntax, which makes parsing more involved and also requires learning more syntax. So, it's a trade-off.


> Case in point: markdown lists will be styled one way. With DDoc macros one can define any style of lists needed. (We've defined BOOKTABLE in addition to a couple of other TABLEs, which works beautifully in HTML and LaTeX; not sure how can be done in Markdown.

Styling is an orthogonal issue. Markdown was designed to translate to HTML, so you'd just write CSS stylesheets to customize the output. Ddoc, OTOH, was designed to be a generic macro system, so obviously it's better at doing macro-based stuff.


> >If there's a specific macro that would indicate a category then it would be easy to generate a table like that.
> >
> >/// $(CATEGORY searching)
> >void foo ()
> 
> The issue is with writing text out of order, something DDoc is not very apt at. Would be nice to improve on that (we do some of it with javascript).
[...]

The limitation of macro systems is that, fundamentally speaking, no semantics are assigned to macros. A macro is just an arbitrary name that gets recursively substituted with some pre-specified pattern.  The macro engine doesn't know what any of the macros "mean", and therefore has no concept of things like cross-referencing, categorization, etc., which are all higher-level, logical concepts, that are required in order to move data around.

There are, of course, ways of simulating this with macros, such as with a global top-level macro that takes an arbitrary number of arguments and can permute them at will, but this is ultimately just a hack. Macro systems simply don't provide enough expressive power to build abstractions like "this block of text is a `declaration` with `identifier` as key, so add `identifier` to the index structure in the output". Postprocessing is the only way to reliably add this kind of semantics to it.

But postprocessing also removes some of the advantage of ddoc: if you translate everything down to HTML tags, the postprocessor will have to reparse the HTML and resynthesize information on where sections begin/end, what constitutes a declaration, etc., which may not be reliable. So now you have to use an intermediate format (like docbook) that still captures some of the original structure, so that the postprocessor can reliably recover the original document structure from it. But then, you might as well just write in the intermediate format to begin with, instead of using ddoc.

So while I do agree that ddoc is pretty good at what it does -- expanding macros -- I'm still on the fence as to whether that's the best approach to documentation generation.


T

-- 
Sometimes the best solution to morale problems is just to fire all of the unhappy people. -- despair.com