On 5/25/2013 9:59 PM, Borden wrote:
> On Sunday, 26 May 2013 at 04:57:12 UTC, Borden wrote:
>> On Sunday, 26 May 2013 at 04:30:46 UTC, Walter Bright wrote:
>>> Again, this is deliberate. Macros are set up so that the last one overrides
>>> all the previous ones, enabling a hierarchy of them using ddoc files. It's a
>>> simple form of 'inheritance'.
>>
>> And perhaps this point could be clarified (and, when I next attack the source
>> I'll test it). I have one.ddoc two.ddoc and src.dd. In src.dd, I use
>> $(MY_MACRO x). one.ddoc has the line MY_MACRO=<p>Called one on $1</p>;
>> two.ddoc has the line MY_MACRO=<p>Called two on $1</p>.
>>
>> So, I now run dmd -o- -D one.ddoc two.ddoc src.dd. What does src.html say?
>
> and by $1 I mean, of course, $0.

The lexically last definition of MY_MACRO is used.

On Saturday, May 25, 2013 21:30:44 Walter Bright wrote:
> On 5/25/2013 8:55 PM, Jonathan M Davis wrote:
> >> 3) Again using LINK2, if I were to delete the LINK2= line from doc.ddoc and forget to readd it, my experience is that dmd -D will quietly drop instances of $(LINK2) without telling me.
> > 
> > Then perhaps dmd should be fixed so that it complains. That's a quality of implementation issue and probably easily fixed.
> 
> It's quite deliberate, is not a QoI issue, and doesn't need to be fixed.

Hmmmm. Because it's designed with idea that you can make multiple passes? Well, regardless of why, the fact that it doesn't give an error doesn't harm ddoc's expressiveness. So, in questions of whether ddoc is powerful enough or expressive enough to do something (which appears to be the thrust of Borden's complaints) aren't affected by it.

My main complaint about ddoc is actually not a complaint about ddoc but about html. I find it very annoying to have to put $(P ) around every paragraph. Stuff like LaTeX does that automatically based on blank lines, which is way better IMHO, but if you're targetting HTML, then unfortunately, you need to mark paragraphs. The only way to fix that with regards to ddoc would be to make it so that ddoc understood that blank lines meant new paragraphs and inserted <p></p> appropriately, when generating html, but that would make it so that ddoc was less general, and there might be other negatives to that I haven't thought of. So, we just get to deal with $(P ) I guess.

And it's easy enough to write a program to handle the stuff that ddoc _can't_ do (like generate a table of contents from all of your CHAPTER tags), that ddoc's limitations really aren't a big deal, and its flexibility is fantastic.

- Jonathan M Davis

On Sat, May 25, 2013 at 08:28:06PM -0400, Andrei Alexandrescu wrote: [...]
> My attitude on DDoc has evolved in threes:
> 
> 3 minutes: "wtf is this crap"
> 3 hours: "this sucks"
> 3 days: "grumble I'll make do with this although it totally sucks"
> 3 months: "this is pretty darn good"

LOL... Though for me, I think I stopped at the third step (or slightly
past that).


> To generate several formats from one source, a macro system is needed. One interesting thing I figured about macro systems is they're all dirty - they can't be really considered "languages" because they intermix the programming part with the very output generated. So, what macro system would you use? (Actual question.) Look at m4 - it won't win any beauty contests, either, and it's enormously complicated. DDoc is simple for what it does, it has somehow hit a sweet spot.
[...]

I don't know, to me DDoc is still lacking a major feature: a mechanism for per-character translation. The problem is that many output formats have a different scheme of metacharacters, and some (most notably LaTeX) require special transcription of certain characters. Right now, the only way to handle this correctly in DDoc is very painful: write macros for every special character and logical entity (like mdash, nbsp, and the like), which makes it very hard to write. Your text would look like:

	$(T)his is Mr$(DOT)$(NBSP)T$(APOS)s $(DOLLAR)0$(DOT)02
	recip$(EACUTE)$(MDASH)as seen on TV$(DOT)

This problem is mostly evaded when you're targeting a single output format. Once you start targeting more than a single output format, the number of required macros grow exponentially. Making the DDoc source targetable to *arbitrary* output formats requires practically wrapping every character inside a macro, which is impractical.

To work around this problem with the current version of DDoc, you'd need an external utility to do the transcriptions for you, which is a hassle.


T

-- 
Real men don't take backups. They put their source on a public FTP-server and let the world mirror it. -- Linus Torvalds

On 5/26/13 2:03 AM, H. S. Teoh wrote:
> I don't know, to me DDoc is still lacking a major feature: a mechanism
> for per-character translation. The problem is that many output formats
> have a different scheme of metacharacters, and some (most notably LaTeX)
> require special transcription of certain characters. Right now, the only
> way to handle this correctly in DDoc is very painful: write macros for
> every special character and logical entity (like mdash, nbsp, and the
> like), which makes it very hard to write. Your text would look like:
>
> 	$(T)his is Mr$(DOT)$(NBSP)T$(APOS)s $(DOLLAR)0$(DOT)02
> 	recip$(EACUTE)$(MDASH)as seen on TV$(DOT)
>
> This problem is mostly evaded when you're targeting a single output
> format. Once you start targeting more than a single output format, the
> number of required macros grow exponentially. Making the DDoc source
> targetable to *arbitrary* output formats requires practically wrapping
> every character inside a macro, which is impractical.
>
> To work around this problem with the current version of DDoc, you'd need
> an external utility to do the transcriptions for you, which is a hassle.

ESCAPES has been recently defined to partially fix that.

Also, LaTeX has about the same limitation. Someone defined an "ActiveTeX" derivative in which each character was active (and therefore potentially definable as a macro). As far as I know it didn't catch up, which may be a sign that people were okay without that capability.


Andrei

On 5/25/2013 10:34 PM, Jonathan M Davis wrote:
> My main complaint about ddoc is actually not a complaint about ddoc but about
> html. I find it very annoying to have to put $(P ) around every paragraph. Stuff
> like LaTeX does that automatically based on blank lines, which is way better
> IMHO, but if you're targetting HTML, then unfortunately, you need to mark
> paragraphs. The only way to fix that with regards to ddoc would be to make it
> so that ddoc understood that blank lines meant new paragraphs and inserted
> <p></p> appropriately, when generating html, but that would make it so that
> ddoc was less general, and there might be other negatives to that I haven't
> thought of. So, we just get to deal with $(P ) I guess.

The issue with implied paragraph breaks is that then ddoc would have to get a lot smarter to avoid putting $(P ) around everything with a blank lines, and then you are already down the path of creating a markup language, not a macro language.

On Sunday, May 26, 2013 00:32:01 Walter Bright wrote:
> On 5/25/2013 10:34 PM, Jonathan M Davis wrote:
> > My main complaint about ddoc is actually not a complaint about ddoc but about html. I find it very annoying to have to put $(P ) around every paragraph. Stuff like LaTeX does that automatically based on blank lines, which is way better IMHO, but if you're targetting HTML, then unfortunately, you need to mark paragraphs. The only way to fix that with regards to ddoc would be to make it so that ddoc understood that blank lines meant new paragraphs and inserted <p></p> appropriately, when generating html, but that would make it so that ddoc was less general, and there might be other negatives to that I haven't thought of. So, we just get to deal with $(P ) I guess.
> 
> The issue with implied paragraph breaks is that then ddoc would have to get a lot smarter to avoid putting $(P ) around everything with a blank lines, and then you are already down the path of creating a markup language, not a macro language.

Which is why I'm not pushing for any changes in that regard. For some of the stuff that I'm writing in ddoc right now, I considered having the program that does the build add the $(P) macros for me but decided that it was better to just suck it up and use $(P) rather than risk problems with code blocks with blank lines in them and whatnot (I'm using a D program to do the build because it's easier than writing makefile, and I needed a program to generate the table of contents and index anyway, since ddoc can't do that).

So, I'm not sure what the best solution with regards to $(P) is, and for the moment, it looks like it's just better to put up with it, but it does end up being my #1 annoyance when dealing with ddoc.

- Jonathan M Davis

On Sunday, 26 May 2013 at 06:43:46 UTC, Jonathan M Davis wrote:
> So, in questions of whether ddoc is powerful enough or
> expressive enough to do something (which appears to be the thrust of Borden's complaints) aren't affected by it.

How I'd rewrite DDoc from scratch as its own markup language is not quite what I'm trying to get at in this thread. From what I've gathered from Walter's responses, if I've understood correctly, is that the idea behind DDoc is to provide the simplest rules-based formatting scheme possible for the purposes of generating documentation at the same time one compiles code. I just want to make sure that I understand what I'm working with.

My 'complaint' - although I would prefer to have my observations about difficulties working with a markup system be called 'observations' - is that the current body of text files which comprise the DLang spec source cannot be easily compiled into clean, well-formed, XHTML5-compliant files from which I can build an ePUB file.

To solve this problem, and based on responses I got to previous related threads, I offered in my first post to translate the DLang spec files into a markup designed for documentation. This idea was promptly refuted as being unwelcome effort as, it was explained, the DDoc spec is written in a way which is both sufficient for its purposes and is independent of any particular markup language.

I am willing to keep working with the DDoc macros to try to get them to output the XHTML5 files that I want. However, before I can continue, I need guidance on:
a) How I can modify the DLang spec files to enable me to translate them into the HTML5 files that I need; and
b) Avoid breaking existing compilation into other formats (such as Latex, PDF, HTML4, etc.)

(I apologise if my message came across as hostile. It's rather late where I am and I wanted to get this into the aether before I went to bed. I don't mean any insult if anything I've writen could be interpreted that way)

On Sunday, 26 May 2013 at 08:09:16 UTC, Borden wrote:
> [...]
> My 'complaint' - although I would prefer to have my observations about difficulties working with a markup system be called 'observations' - is that the current body of text files which comprise the DLang spec source cannot be easily compiled into clean, well-formed, XHTML5-compliant files from which I can build an ePUB file.
> [...]

Maybe you can automatically convert HTML to XHTML, and then apply an XSLT transformation.

You mentioned somewhere that you needed something like a CSS transformation to target a <p> inside another element. You could do that with XSLT.

To convert from HTML to XHTML you could use the following:

http://www.codeproject.com/Articles/10792/Convert-HTML-to-XHTML-and-Clean-Unnecessary-Tags-a

It is made in C#, though if it works, I guess it could be ported to D.

ALso you could use Addam D. Ruppe XML DOM classes, which, though I'm not sure, seem to tolerate HTML4:

https://github.com/adamdruppe/misc-stuff-including-D-programming-language-web-stuff

(grab dom.d and characterencoding.d from there).

Or maybe the next generation xml library for D which will be revieed for inclusion, which supports XPATH queries:

      http://dsource.org/projects/xmlp

--jm

Thank you for the suggestions, Juan.

For the purposes of generating a single set of XHTML5 documents, your advice would work. What I'm trying to do, however, is update the makefiles for the website source so that ePUB files become a target.

I worry, therefore, that pumping the DLang spec through several conversions will give me less control over the resulting output. More importantly, though, it will make the makefiles less portable because anyone who wishes to use them will have to install all of the dependencies, so I'm trying to avoid that.

Again, what I'd ideally like to do is write an xhtml5.ddoc file which will give all of the necessary macro definitions to compile the DLang spec into tidy XHTML5 files. Unless I'm mistaken, I don't think that this is a very unreasonable goal.

However, I could be wrong and a solution like the one you suggest may be the only way to do this...

On Sun, May 26, 2013 at 03:03:30AM -0400, Andrei Alexandrescu wrote:
> On 5/26/13 2:03 AM, H. S. Teoh wrote:
> >I don't know, to me DDoc is still lacking a major feature: a mechanism for per-character translation. The problem is that many output formats have a different scheme of metacharacters, and some (most notably LaTeX) require special transcription of certain characters. Right now, the only way to handle this correctly in DDoc is very painful: write macros for every special character and logical entity (like mdash, nbsp, and the like), which makes it very hard to write. Your text would look like:
> >
> >	$(T)his is Mr$(DOT)$(NBSP)T$(APOS)s $(DOLLAR)0$(DOT)02
> >	recip$(EACUTE)$(MDASH)as seen on TV$(DOT)
> >
> >This problem is mostly evaded when you're targeting a single output format. Once you start targeting more than a single output format, the number of required macros grow exponentially. Making the DDoc source targetable to *arbitrary* output formats requires practically wrapping every character inside a macro, which is impractical.
> >
> >To work around this problem with the current version of DDoc, you'd need an external utility to do the transcriptions for you, which is a hassle.
> 
> ESCAPES has been recently defined to partially fix that.

Is it working now?


> Also, LaTeX has about the same limitation. Someone defined an "ActiveTeX" derivative in which each character was active (and therefore potentially definable as a macro). As far as I know it didn't catch up, which may be a sign that people were okay without that capability.
[...]

Oh? I thought TeX already had the capability. Well, at least, you could redefine the default escape character "\" to be basically anything, including a letter, so you can achieve strange things that way. I'm not saying that's a good design though.

What I'm more concerned with was how to write DDocs that targets output formats with incompatible metacharacters or different foreign character encodings. For example, if the docs contained a character like é, I'd like to be able to specify that it should be translated to \'e when targeting LaTeX, and left as-is in HTML, for example. I *could* define a macro $(EACUTE) for this purpose, of course, but it makes writing DDocs rather painful (why should I resort to $(EACUTE) if the DDoc input is already UTF-8 and can already represent such a character directly?).

Another annoyance, that somebody else already mentioned, is how to wrap paragraphs in $(P ...) correctly, as is required for (X)HTML. Currently we only have linebreaks, which does not reliably translate to <p> and </p> with the correct nesting. I've tried to hack around that but still cannot get it working correctly in all possible cases. This is rather disappointing, since DDoc itself already defines what a paragraph is (or at least claims to), yet it doesn't easily lend itself to correct <p> nesting. One shouldn't have to dictate the manual use of $(P) in code docs in order to generate correct output.

So in short, DDoc as it stands is quite a nice, clean, well-designed macro expansion system, but it falls a bit short of being a nice *documentation* generation system.


T

-- 
Blunt statements really don't have a point.