On 1/8/13 3:12 PM, Walter Bright wrote:
> On 1/8/2013 2:51 PM, Andrei Alexandrescu wrote:
>> I just pushed the first passable PDF documentation generated via LaTeX
>> with
>> ddoc: http://goo.gl/QIP4t. For now I'm only rendering the TOC and
>> lex.dd, but at
>> this time we have enough seed for anyone interested to convert other
>> files.
>>
>> Take a look at the generated PDF: http://erdani.com/d/dlangspec.pdf
>>
>>
>> Andrei
>
> Looks good to me.

I should add this is a rather large success story for ddoc. This is the first time ddoc has been used to generate a format that has nothing to do with HTML. LaTeX is different enough from HTML and finicky enough so we can claim if generating this was possible, pretty much any format is approachable.

> It's also good to keep in mind that when I prepared the kindle version,
> which has a small screen, I had to rather ruthlessly pare down the
> length of preformatted lines like the code examples.

Yah, for this proof of concept I'm using the stock article interior design. The production version should include beautiful PDFs designed purposefully for the Kindle.

In fact I just uploaded a Kindle-specialized pdf. I don't have the Kindle on me so I went blindly. Take a look and let me know!

http://erdani.com/d/dlangspec-kindle.pdf

Formatting code and tables will be a challenge but nothing really difficult.


Andrei

On 1/8/13 3:31 PM, Philippe Sigaud wrote:
> On Wed, Jan 9, 2013 at 12:12 AM, Walter Bright
> <newshound2@digitalmars.com>  wrote:
>>> Take a look at the generated PDF: http://erdani.com/d/dlangspec.pdf
>
>> Looks good to me.
>
> Yes, that has indeed quite a palatable appearance. I see you defined 2
> or 3-cols tables, that's a good idea. Too bad DDoc macros do not
> accept numerical arguments :) I guess in the end a 4-cols version will
> be necessary, if only for the big tables in std.algorithm.

Yah, though note that these macros were only necessary because I was doing unusual stuff subheadings for TABLE_3ROWS and multiline paragraph for TABLE_2ROWS. If special stuff is not required you can use TABLE2 for up to 10 columns.

> I also like your use of the `listings` package. How did you 'disable'
> the automatic colouring DMD insert in code samples?

I defined these guys:

D_COMMENT=$0
D_STRING=$0
D_KEYWORD=$0

>> It's also good to keep in mind that when I prepared the kindle version,
>> which has a small screen, I had to rather ruthlessly pare down the length of
>> preformatted lines like the code examples.
>
> Same here. I have a big tutorial on templates on github. Following
> your advice, Walter, I found ways to convert it into .epub or .mobi (I
> converted it from LaTex to Markdown). And, indeed, even 60-chars lines
> tend to be too long for these screens. That's a pretty harsh limit! I
> still have to scan the entire doc to find the longest lines.
>
> Maybe I can start converting it to a big Ddoc file...

I reached a sort of weird conclusion that ddoc is a redoubtable documentation source format. (I'm not sure whether it's better than Markdown or not as I'm not very fluent with Markdown.) One great thing about ddoc is that macros are infinitely flexible, and the expansion rules are not as weird as other macro systems such as M4.

The table use case is telling - as soon as I needed a special table form I could just encode that as a macro. I think with Markdown you get to generate one kind of table but if you want to fine-tune any particular table you just can't.


Andrei

On Tue, Jan 08, 2013 at 02:51:44PM -0800, Andrei Alexandrescu wrote:
> On 1/4/13 7:53 PM, Andrei Alexandrescu wrote:
> >Now that the issue of documentation came up, I wonder if there's interest in a high-quality PDF rendering of the language spec (e.g. similar to the interior design of TDPL itself). Is it worth the effort?
> >
> >Thanks,
> >
> >Andrei
> 
> I just pushed the first passable PDF documentation generated via LaTeX with ddoc: http://goo.gl/QIP4t. For now I'm only rendering the TOC and lex.dd, but at this time we have enough seed for anyone interested to convert other files.
> 
> Take a look at the generated PDF: http://erdani.com/d/dlangspec.pdf
[...]

Not bad.

Those single-column lists that span multiple pages are ugly, though. You might want to consider defining some LaTeX macros that split it up into multiple columns, depending on how many items there are. There should be a way to do this using things like the ifthen module (which AFAIK is included in standard LaTeX distributions). The macro can go into the doc preamble, and then the ddoc comment can just reference it.

I'd present an example of such a macro here, but my LaTeX is a bit rusty, and I'd need to look it up when I get home.


T

-- 
If a person can't communicate, the very least he could do is to shut up. -- Tom Lehrer, on people who bemoan their communication woes with their loved ones.

On 1/8/2013 3:57 PM, Andrei Alexandrescu wrote:
> One great thing about ddoc is that macros are
> infinitely flexible, and the expansion rules are not as weird as other macro
> systems such as M4.

I had one great advantage in it being the fifth text macro processor I'd designed :-)

I'd still like to see it done as a phobos module. Maybe an enhancement request?

On Wed, Jan 9, 2013 at 2:05 AM, Walter Bright <newshound2@digitalmars.com> wrote:

> I'd still like to see it done as a phobos module. Maybe an enhancement request?

So, this should:

- take a .ddoc file for macro definition, or a [name, code] range of pairs.
- read an input range
    - this range can be a standard D file, a .dd or whatever. That
means parsing D comments...
- find any macro definition
- find any $(NAME Args,) call, expand it recursively. Repeat until
nothing changes anymore. This should be done lazily (through the input
range interface), in case the user want to stop before the end.
- output the result as a range.

What I'm not clear about, is: if the input is a D source file, should the expander strip any D code from it? Should this return a full-fledged documentation (ie, find the documented symbols...) or just blindly expand DDoc macros.

On Wed, Jan 9, 2013 at 12:57 AM, Andrei Alexandrescu <SeeWebsiteForEmail@erdani.org> wrote:

> Yah, though note that these macros were only necessary because I was doing unusual stuff subheadings for TABLE_3ROWS and multiline paragraph for TABLE_2ROWS. If special stuff is not required you can use TABLE2 for up to 10 columns.

Oh, I did not see that, that's good to know. This should cover most needs.

>> I also like your use of the `listings` package. How did you 'disable' the automatic colouring DMD insert in code samples?
>
>
> I defined these guys:
>
> D_COMMENT=$0
> D_STRING=$0
> D_KEYWORD=$0

Simple and elegant :)

You introduce `listings`as a dependency. I guess it's a very common package, that should not cause problems. Maybe syntax highlighting can be disabled if LaTeX does not find listings.

> I reached a sort of weird conclusion that ddoc is a redoubtable documentation source format. (I'm not sure whether it's better than Markdown or not as I'm not very fluent with Markdown.) One great thing about ddoc is that macros are infinitely flexible, and the expansion rules are not as weird as other macro systems such as M4.

The thing is, markdown is just a mark-up language: it's 'passive': no
function, no expansion. I agree having some 'programmability' is nice.
LaTeX is showing its age there. Ddoc has the merit of being
wonderfully simple concerning its macro expansion algorithm.
But markdown is widely used, if only for sites like Github or StackOverflow.

I have code somewhere to produce a document using D only. That gave me access to the entire D power, so it's the other part of the spectrum.

auto doc = document( title("A title"),
section(...),
section(...),
section(...)
);

string latex = doc.as!"LaTeX";
string html = doc.as!"HTML";

and so on. Still embryonic and not a priority for me right now.



And, I'm slowly assembling small D parsers for markdown, xml, (a very limited subset of) LaTeX for a project I call Dialect, that would offer limited source-to-source translation for markup languages. Ddoc is also a target, but you just showed it can fend for itself quite well :)


> The table use case is telling - as soon as I needed a special table form I could just encode that as a macro. I think with Markdown you get to generate one kind of table but if you want to fine-tune any particular table you just can't.

Exactly. It's not a programming language. Maybe people use templates for that.

On 1/9/2013 2:40 AM, Philippe Sigaud wrote:
> On Wed, Jan 9, 2013 at 2:05 AM, Walter Bright
> <newshound2@digitalmars.com> wrote:
>
>> I'd still like to see it done as a phobos module. Maybe an enhancement
>> request?
>
> So, this should:
>
> - take a .ddoc file for macro definition, or a [name, code] range of pairs.
> - read an input range
>      - this range can be a standard D file, a .dd or whatever. That
> means parsing D comments...
> - find any macro definition
> - find any $(NAME Args,) call, expand it recursively. Repeat until
> nothing changes anymore. This should be done lazily (through the input
> range interface), in case the user want to stop before the end.
> - output the result as a range.
>
> What I'm not clear about, is: if the input is a D source file, should
> the expander strip any D code from it? Should this return a
> full-fledged documentation (ie, find the documented symbols...) or
> just blindly expand DDoc macros.
>

No no no. The module gets its input from an InputRange. Not a file. It gets its macro definitions, which are NAME=VALUE pairs, from an associative array of them.

On Wed, Jan 09, 2013 at 11:01:32AM -0800, Walter Bright wrote:
> On 1/9/2013 2:40 AM, Philippe Sigaud wrote:
> >On Wed, Jan 9, 2013 at 2:05 AM, Walter Bright <newshound2@digitalmars.com> wrote:
> >
> >>I'd still like to see it done as a phobos module. Maybe an enhancement request?
> >
> >So, this should:
> >
> >- take a .ddoc file for macro definition, or a [name, code] range of
> >  pairs.
> >- read an input range
> >     - this range can be a standard D file, a .dd or whatever. That
> >       means parsing D comments...
> >- find any macro definition
> >- find any $(NAME Args,) call, expand it recursively. Repeat until
> >  nothing changes anymore. This should be done lazily (through the
> >  input range interface), in case the user want to stop before the
> >  end.
> >- output the result as a range.
> >
> >What I'm not clear about, is: if the input is a D source file, should the expander strip any D code from it? Should this return a full-fledged documentation (ie, find the documented symbols...) or just blindly expand DDoc macros.
> >
> 
> No no no. The module gets its input from an InputRange. Not a file. It gets its macro definitions, which are NAME=VALUE pairs, from an associative array of them.

I agree, the interface should be as generic as possible. I think it would be nice to have a separate function for parsing macro definitions. So the API should look something like this:

	Result expandDdocMacros(I,Result)(I inputRange,
			string[string] macroDefs)
		if (isInputRange!I && isSomeChar!(ElementType!I))
	{ ... }

	string[string] parseMacroDefs(I)(I inputRange)
		if (isInputRange!I && isSomeChar!(ElementType!I))
	{ ... }

Then you can use it something like this:

	void main() {
		auto macrodefs = File(macroFile);
		auto inputfile = File(ddocFile);

		auto expanded = expandDdocMacros(inputfile.byChar(),
						macrodefs.byChar());
		static assert(isInputRange!(typeof(expanded)));

		stdout.write(expanded);
	}

You can also feed it all sorts of stuff, like network input:

	string[string] builtinMacros = ...;

	// Online Ddoc expander!
	void myCgiApp(HttpRequest req, ref HttpResponse resp) {
		static assert(isInputRange!(typeof(req.data)) &&
			isSomeChar!(ElementType!(typeof(req.data))));

		auto result = expandDdocMacros(req.data, builtinMacros);
		resp.put(result);
	}

Etc., etc.. D ranges are an extremely powerful concept.


T

-- 
Без труда не выловишь и рыбку из пруда.

On Wed, Jan 9, 2013 at 8:01 PM, Walter Bright <newshound2@digitalmars.com> wrote:

>> What I'm not clear about, is: if the input is a D source file, should the expander strip any D code from it? Should this return a full-fledged documentation (ie, find the documented symbols...) or just blindly expand DDoc macros.
>>
>
> No no no. The module gets its input from an InputRange. Not a file. It gets its macro definitions, which are NAME=VALUE pairs, from an associative array of them.

That's not what I'm talking about, sorry for not being clear. I don't care for the origin of the data, obviously. I meant a D module, not a file specifically. My question was if the data is a D module (code + comments), should it just expand macros? Or should it generate a documentation?

>From this and your other comments, I suppose the module does not
filter out comments specifically: it simply iterate until it finds a $( pattern.

On 1/9/2013 11:31 AM, H. S. Teoh wrote:
> I think it
> would be nice to have a separate function for parsing macro definitions.

No. Where the macro definitions come from is a completely separate problem, and the two should not be conflated.