Good afternoon, all,

I would still like to compile the D Lang Spec into EPUB (and possibly other formats) but, as we discussed in these threads:

http://forum.dlang.org/thread/bsbdpjyjubfxvmecwhjl@forum.dlang.org
http://forum.dlang.org/thread/uzdngvjzexukbgkxdzpi@forum.dlang.org

having the D Lang Specification written in DDoc macros is making it extremely difficult to work with.

I ask, therefore, what opposition would there be to me rewriting the DLang Spec files into another format that will be easier to parse and compile for the website, PDF, Latex, eBook and other formats? If the answer is 'minimal', 'go ahead' or 'it's your funeral', then my follow-up question is 'what format would be the easiest to write, debug and maintain?'

For greater clarity, I am NOT proposing to rewrite the DDoc-generated library documentation or any other pages outside of the spec. In the makefile, they are defined as the files covered in $(SPEC_ROOT).

With regards,

I hasten to add that I don't mean to criticise the original writers of the DLang Spec for writing it in DDoc macros. So far, I've found the documentation fairly easy to follow (as plain text) and so I don't want to lose any of that should the spec be rewritten.

It's also possible (although, in my opinion, less preferable) to keep the spec written in DDoc macros but reformatted to allow for easier conversion to other formats...

On Saturday, May 25, 2013 20:10:53 Borden wrote:
> Good afternoon, all,
> 
> I would still like to compile the D Lang Spec into EPUB (and possibly other formats) but, as we discussed in these threads:
> 
> http://forum.dlang.org/thread/bsbdpjyjubfxvmecwhjl@forum.dlang.org http://forum.dlang.org/thread/uzdngvjzexukbgkxdzpi@forum.dlang.org
> 
> having the D Lang Specification written in DDoc macros is making it extremely difficult to work with.
> 
> I ask, therefore, what opposition would there be to me rewriting the DLang Spec files into another format that will be easier to parse and compile for the website, PDF, Latex, eBook and other formats? If the answer is 'minimal', 'go ahead' or 'it's your funeral', then my follow-up question is 'what format would be the easiest to write, debug and maintain?'
> 
> For greater clarity, I am NOT proposing to rewrite the DDoc-generated library documentation or any other pages outside of the spec. In the makefile, they are defined as the files covered in $(SPEC_ROOT).
> 
> With regards,

Can you please give concrete examples of what doesn't work with ddoc? On the whole, I find ddoc to work extremely well. Depending on what you're problem is, it may be the case that the macros in question just need to be rearranged or redesigned. Or maybe we could add a fairly simple feature to ddoc to solve the problem. Certainly, my naturaly reaction is to be against rewriting any of dlang.org in something other than ddoc. It's all in ddoc right now, so it's quite consistent, and aside from you, I'm not aware of anyone complaining about it any time recently.

- Jonathan M Davis

On 5/25/13 2:16 PM, Borden wrote:
> I hasten to add that I don't mean to criticise the original writers of
> the DLang Spec for writing it in DDoc macros. So far, I've found the
> documentation fairly easy to follow (as plain text) and so I don't want
> to lose any of that should the spec be rewritten.
>
> It's also possible (although, in my opinion, less preferable) to keep
> the spec written in DDoc macros but reformatted to allow for easier
> conversion to other formats...

My attitude on DDoc has evolved in threes:

3 minutes: "wtf is this crap"
3 hours: "this sucks"
3 days: "grumble I'll make do with this although it totally sucks"
3 months: "this is pretty darn good"

To generate several formats from one source, a macro system is needed. One interesting thing I figured about macro systems is they're all dirty - they can't be really considered "languages" because they intermix the programming part with the very output generated. So, what macro system would you use? (Actual question.) Look at m4 - it won't win any beauty contests, either, and it's enormously complicated. DDoc is simple for what it does, it has somehow hit a sweet spot.


Andrei

On Saturday, 25 May 2013 at 23:28:46 UTC, Jonathan M Davis wrote:
> aside from you, I'm not aware of anyone complaining about it any time recently.

Good evening, Jonathan,

I'm not sure whether you mean that nobody's complained recently
about the spec being in DDoc lately, because, as in my first
link, I found that more people disliked the macros feature of
DDoc (2 + me) than liked it (0).

To answer your question, regarding concrete examples, these
issues are in the context of wanting to translate the DLangSpec
pages into HTML5 so that I can compile them into an ePUB document:

1) the DLangSpec files (pick any one) use SECTION# macros, where
# is a number. In the DDoc conversion files to HTML, these
SECTION# macros convert to <h#> tags and encase the contents
within them. However, I'm not aware of any facility within DDoc,
short of hand-writing a parser, to allow these SECTION# macros to
be nested in order to take advantage of HTML5's <section> tags.

2) The macros are not self-documenting. For example, consider
$(LNAME2 pointers, Pointers) in arrays.dd. The easiest way, I
know, to figure out what $(LNAME2) means is to read the posix.mak
to see that arrays.dd gets pumped through ddoc.dd. Now, a search
through doc.ddoc to find the declaration LINK2=<a
href="$1">$+</a> at last tells me that argument 1 is the path to
the link and everything that follows that is the text to appear
in the link. The point is that, as I struggle through modifying
the existing .ddoc templates to compile to HTML5, I need to keep
flipping back and forth between the source and the .ddoc to make
sure that anything I'm redefining I'm doing correctly.

3) Again using LINK2, if I were to delete the LINK2= line from
doc.ddoc and forget to readd it, my experience is that dmd -D
will quietly drop instances of $(LINK2) without telling me.

4) Again using the same example, if LINK2 gets defined in
multiple DDoc files, how do I know for certain which definition
it calls when dmd runs against the files?

5) I find that a lot of the DLangSpec is written from an HTML
point of view, so maybe it just needs rewriting to make the
macros descriptive. For example, consider $(B dmd) and $(B -gc)
on lines 881-882 of abi.dd. By default, these get converted into
<b>dmd</b> and <b>-gc</b> Say I want commands (like dmd) to be
bolded but I want command-line arguments not to be bolded.
There's no way to write B= to single out some $(B)s and not
others. Now, I know the knee-jerk response is "Yes, but HTML
works the exact same way." That's true, but CSS *does* give you a
bunch of selectors to cherry-pick, say, "only the <b> tags of
class X" or "the element with this id." Meaning that all I have
to do is find the Bs I want to change and add a class="" without
having to worry about updating any of the other Bs. Is there a
simple way to do this in DDoc?

These are just a few of the observations that I have. As I said
in the other threads, my ePUB effort has ground to a halt because
I find that I'm fighting to read the spec sources rather than
figuring out how to produce clean and pretty HTML5 code that can
get compiled into an ePUB.

On Sunday, 26 May 2013 at 00:28:05 UTC, Andrei Alexandrescu wrote:
> To generate several formats from one source, a macro system is needed. One interesting thing I figured about macro systems is they're all dirty - they can't be really considered "languages" because they intermix the programming part with the very output generated. So, what macro system would you use? (Actual question.) Look at m4 - it won't win any beauty contests, either, and it's enormously complicated. DDoc is simple for what it does, it has somehow hit a sweet spot.

Good evening, Professor,

I'm not arguing with the macro system or proposing a replacement. I think, for what it's designed to do, it works perfectly well and, you're right, is somewhat faster and cleaner than XML tags. (and this is from someone who's biased in favour of HTML tags)

My contention is that, for the purposes of writing lengthy, non-code documentation like the DLang spec (I'm not referring to any other documentation or pages on the site), enclosing the entire exposition in macros has made the source too inflexible for me to work with without awkward workarounds or having to write my own parser. Again, the idea is to use the features of HTML5 and compile the DLang spec into an ePub document that I can read on my brand-new Kobo.

Therefore, I'm not proposing a radical overhaul of DDoc or recommending that all known DDoc be recoded in HTML, as I mention in my other forum threads. Rather, I'm mentioning that, for my purposes, the DLang Spec source is too difficult for me to work with in its current state for my purposes and I'm offering to recode it in a language that will allow me to accomplish what I want to do; keep the DLang spec easy to read, write and maintain; and tread on as few toes as possible.

I want to keep this discussion focussed on the DLang spec source code. If we want to debate the features of DDoc, we should do it in another thread.

However, as not to appear full of cricism but short of ideas, I'm going to break my own rule and suggest, at least for the purposes of solving some of the issues I've run into with the DLang spec source, that integrating some wiki-markup into DDoc may help. For example:

1) Allowing sections to be defined using == Heading == or === Heading === instead of $(HEADING ) or variants. The advantage that Wiki syntax has over macro-syntax is that it automatically works out the section nesting (which is essential for building tables of contents in things such as, hint hint, eBooks) whereas macros can only do it if the subheadings are nested as arguments.
1a) Using ==Headings== and the existing /** */ code standards, DDoc could have a predefined $(TOC) macro which would auto-generate the TOC. /** */ would form the main headings and ==Heading== would be the subheadings, prettily nested when formatted.
2) Adopting Latex's rule that a double line break means a new paragraph. This will effectively make the $(P) macros rampant in the DLang spec documentation unnecessary.
3) Defining tables using the +---+ syntax. I know that this will be unpopular due to the existing /++ documentation code rules (and thus is open to alternatives). However, one must admire how simply Wiki markup has elegantly solved a problem that Latex and XML dosen't.
4) Using either * or - to indicate bullet points, similar to Wiki markup. Again, I know that it'll have to be coded as not to confuse the parser with /** */ and operands.
4a) Maybe #) to indicate ordered lists? (again, similar to Wiki markup)
5) Use the [[Link|Link name]] instead of $(Link) macros to cross-link. By default, Link would be a reference to some other DDoc and allow links to be handled automagically.

How will this all help the DLang spec? Well, if the spec could be rewritten entirely in a /** */ block, with reasonable macro use, then couldn't it be parsed more readily into the necessary formats? Wouldn't it also make the source more readable and editable without all of the nested parentheses? Wouldn't the syntax be self-documenting?

Anyway, just throwing stuff against the wall to see what sticks...

On Sunday, 26 May 2013 at 01:22:17 UTC, Borden wrote:
> 2) Adopting Latex's rule that a double line break means a new paragraph. This will effectively make the $(P) macros rampant in the DLang spec documentation unnecessary.

Oops. I realised that this has already been done. OK, so I guess the question is why does the DLang spec need $(P) macros? How could it be rewritten without them and let dmd worry about them?

On 5/25/13 8:56 PM, Borden wrote:
> My contention is that, for the purposes of writing lengthy, non-code
> documentation like the DLang spec (I'm not referring to any other
> documentation or pages on the site), enclosing the entire exposition in
> macros has made the source too inflexible for me to work with without
> awkward workarounds or having to write my own parser. Again, the idea is
> to use the features of HTML5 and compile the DLang spec into an ePub
> document that I can read on my brand-new Kobo.

This is a worthy goal. We manage to generate mobi files for the spec (and Phobos in a pull request), is the ebook format very different?

Andrei

On 5/25/13 9:22 PM, Borden wrote:
> 1) Allowing sections to be defined using == Heading == or === Heading
> === instead of $(HEADING ) or variants. The advantage that Wiki syntax
> has over macro-syntax is that it automatically works out the section
> nesting (which is essential for building tables of contents in things
> such as, hint hint, eBooks) whereas macros can only do it if the
> subheadings are nested as arguments.

Not getting this at all. You can define in DDoc things like H1, H2, etc. or whatever you want.

Besides, you are proposing a bunch of... just sugar aimed at reading the text as is. That's not part of DDoc's charter. Besides, it does not add power (you can do the same with macros) and it makes everything awfully complicated. Do you want to be the guy writing the parser for all that sugar?

If ddoc has anything going for it, it's simplicity of syntax. It has like 5 syntactic rules in total. Parsing ddoc is quite simple. What vexes me is that all the sugar you propose goes against what you opened with, which was:

> I ask, therefore, what opposition would there be to me rewriting the
> DLang Spec files into another format that will be easier to parse and
> compile for the website, PDF, Latex, eBook and other formats?

I really don't understand. Far as I can tell you are trying to accomplish a well-defined goal: compile the spec in ebook format. Then every step you're sketching on the way there takes you just away from your goal.

We can generate LaTeX from ddoc (there's a pull request for doing it even better) and I can hypothesize that the shortest path between where we are and what you're trying to accomplish is a few dozens of macro definitions. Did you try doing that and failed?


Andrei