On Thursday, January 02, 2014 13:38:22 H. S. Teoh wrote:
> But postprocessing also removes some of the advantage of ddoc: if you translate everything down to HTML tags, the postprocessor will have to reparse the HTML and resynthesize information on where sections begin/end, what constitutes a declaration, etc., which may not be reliable. So now you have to use an intermediate format (like docbook) that still captures some of the original structure, so that the postprocessor can reliably recover the original document structure from it. But then, you might as well just write in the intermediate format to begin with, instead of using ddoc.

That depends on how you go about post-processing. If you design your ddoc macros with the idea that there's going to be post-processing, then they can convert to other ddoc macros which are designed for the post-processor to be able to handle, and then the post-processor can convert those into another set of macros that you run ddoc on again to get the final result. That's essentially what I've done when using ddoc for stuff like a table of contents or index.

Now, if the post-processing is really the last step rather than just post one of multiple runs of ddoc, then it's stuck dealing with whatever the target format is, but if you do multiple runs of ddoc with a separate program operating on the intermediate results of the macro expansion and then generating a new set of ddoc macros for another run of ddoc, then your post- processor doesn't have to understand the target format.

- Jonathan M Davis

Am 02.01.2014 22:38, schrieb H. S. Teoh:
> For example,
> using `backticks` to write code snippets is definitely more readable
> than writing $(D backticks), among many other things. But that does
> introduce more syntax, which makes parsing more involved and also
> requires learning more syntax. So, it's a trade-off.

I don't want to argue for as much syntax as Markdown has - even if I'd actually find that great, Markdown has become a kind of de-facto standard in the programming area - but, overall, at least the aspect of learning new syntax is not really worse than learning new macros. There may be cases where a form of syntax is more difficult to remember, but there are also many cases where the macros are actually harder to learn and remember.

Having said that, Markdown, if not implemented using a series of regexes, is a crazy format to parse. There is an existing parser in vibe.d, though.


>> Case in point: markdown lists will be styled one way. With DDoc
>> macros one can define any style of lists needed. (We've defined
>> BOOKTABLE in addition to a couple of other TABLEs, which works
>> beautifully in HTML and LaTeX; not sure how can be done in Markdown.
>
> Styling is an orthogonal issue. Markdown was designed to translate to
> HTML, so you'd just write CSS stylesheets to customize the output. Ddoc,
> OTOH, was designed to be a generic macro system, so obviously it's
> better at doing macro-based stuff.

Also, an important point is that markdown defines *semantic* primitives. They can be translated to whatever is required by the output format. In DDOC's case that could simply be some standard macros ($(UL), $(LI)).

BTW to Markdown HTML basically is what macros are to DDOC (just that it also has defined semantics). Inline HTML explicitly supported to handle advanced cases and, if restricted to semantic tags, provides a pretty good basis for translating into other output formats.

On 2014-01-03 01:56, Jonathan M Davis wrote:

> That depends on how you go about post-processing. If you design your ddoc
> macros with the idea that there's going to be post-processing, then they can
> convert to other ddoc macros which are designed for the post-processor to be
> able to handle, and then the post-processor can convert those into another set
> of macros that you run ddoc on again to get the final result. That's
> essentially what I've done when using ddoc for stuff like a table of contents
> or index.
>
> Now, if the post-processing is really the last step rather than just post one
> of multiple runs of ddoc, then it's stuck dealing with whatever the target
> format is, but if you do multiple runs of ddoc with a separate program
> operating on the intermediate results of the macro expansion and then
> generating a new set of ddoc macros for another run of ddoc, then your post-
> processor doesn't have to understand the target format.

Seems unnecessary complicated.

-- 
/Jacob Carlborg

On 2014-01-02 22:38, H. S. Teoh wrote:

> The limitation of macro systems is that, fundamentally speaking, no
> semantics are assigned to macros. A macro is just an arbitrary name that
> gets recursively substituted with some pre-specified pattern.  The macro
> engine doesn't know what any of the macros "mean", and therefore has no
> concept of things like cross-referencing, categorization, etc., which
> are all higher-level, logical concepts, that are required in order to
> move data around.

We could start adding semantic meaning to some macros. They would become built-in macros ddoc knows about.

-- 
/Jacob Carlborg

On Friday, January 03, 2014 12:12:44 Jacob Carlborg wrote:
> On 2014-01-03 01:56, Jonathan M Davis wrote:
> > That depends on how you go about post-processing. If you design your ddoc macros with the idea that there's going to be post-processing, then they can convert to other ddoc macros which are designed for the post-processor to be able to handle, and then the post-processor can convert those into another set of macros that you run ddoc on again to get the final result. That's essentially what I've done when using ddoc for stuff like a table of contents or index.
> > 
> > Now, if the post-processing is really the last step rather than just post one of multiple runs of ddoc, then it's stuck dealing with whatever the target format is, but if you do multiple runs of ddoc with a separate program operating on the intermediate results of the macro expansion and then generating a new set of ddoc macros for another run of ddoc, then your post- processor doesn't have to understand the target format.
> 
> Seems unnecessary complicated.

It depends on what you're trying to do. For generating something like a table of contents, it's pretty straightforward.

For instance,

$(CHAPTER Introduction,
    $(P Contents of chapter)
 )

could be processed by a short program to have that chapter's name put in the table of contents which it generates (which is then also ddoc). e.g.

$(TABLE_OF_CONTENTS
    $(CHAPTER_ENTRY Introduction, 1)
    $(CHAPTER_ENTRY Some Other Chapter, 2)
    $(CHAPTER_ENTRY Conclusion, 3)
)

and if any other operations need to be done on CHAPTER to make it suitable for your target formats, then it can be tweaked as necessary.

You then take the resulting .dd file and generate the documentation with the ddoc file that corresponds to the target format (e.g. html.ddoc or latex.ddoc or whatever). I've found that it works quite well.

But that's for generating a book, not documentation for code. You could do something similar to generate cross-references in code documentation, I'm sure, but I wasn't necessarily suggesting that we do. I was just pointing out that it's quite possible (and not necessarily difficult) to manipulate a .dd file (or a .d file with ddoc in it) to generate another .dd file which you then actually use to generate the html or latex or whatever you need to generate and that you don't necessarily need to manipulate the target format to get what you want.

- Jonathan M Davis

XSLT is pretty useful for format conversion. It is also built into most browsers, so you can just emit XML attach a XSLT stylesheet and send it to a browser for transformation and rendering.

Would be an advantage for anyone writing a book or article on D (transform XML over to docbook, and from there to any format).