On Tue, 2014-12-30 at 13:08 +0000, via Digitalmars-d wrote:
> On Tuesday, 30 December 2014 at 05:36:47 UTC, Andrei Alexandrescu wrote:
> > That should be possible (probably after a few improvements). I'm working on a few templates for alternate formats including LaTeX, plain text, and well, now json. -- Andrei
> 
> Would it not be easier to just do a raw convert to XML and use XSLT to transform into other formats?

And, of course, ASCIIDoc was invented to be a human usable input to such a tool chain. Though now with ASCIIDoctor there is a direct to PDF without using FOP or LaTeX.

Markdown is inadequate for more than single page HTML, XeLaTeX is incorrectly disliked by far too many people, ReStructured Text is perceived to be Python specific,… ASCIIDoc wins.

-- 
Russel. ============================================================================= Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder@ekiga.net 41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel@winder.org.uk London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder

On Tuesday, 30 December 2014 at 13:18:46 UTC, Russel Winder via Digitalmars-d wrote:
>
> On Tue, 2014-12-30 at 13:08 +0000, via Digitalmars-d wrote:
>> On Tuesday, 30 December 2014 at 05:36:47 UTC, Andrei Alexandrescu wrote:
>> > That should be possible (probably after a few improvements).
>> > I'm working on a few templates for alternate formats including LaTeX, plain text, and well, now json. -- Andrei
>> 
>> Would it not be easier to just do a raw convert to XML and use XSLT to transform into other formats?
>
> And, of course, ASCIIDoc was invented to be a human usable input to
> such a tool chain. Though now with ASCIIDoctor there is a direct to
> PDF without using FOP or LaTeX.
>
> Markdown is inadequate for more than single page HTML, XeLaTeX is
> incorrectly disliked by far too many people, ReStructured Text is
> perceived to be Python specific,… ASCIIDoc wins.

coming from Python, i'm in favour of rST, markdown and asciidoc.

About that user experience:

http://forum.dlang.org/post/nyclgpfzpkzemgitfyza@forum.dlang.org

On 12/29/14 10:49 PM, ketmar via Digitalmars-d wrote:
> On Mon, 29 Dec 2014 15:49:10 -0800
> Walter Bright via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>
>> On 12/29/2014 2:40 PM, Adam D. Ruppe wrote:
>>> Ddoc isn't too bad, but trying to document examples in dom.d turned into a mess
>>> of /// finds $(LT)foo/$(GT) quickly and I couldn't stand it.
>>
>> I'd make a macro:
>>
>>      XML=$(LT)$0/$(GT)
>>
>> I use custom macros all the time in Ddoc. If you aren't, you're not doing it
>> right :-)
> that's why ddoc is completely unusable either for reading "as is" or
> for generating separate documentation.
>
> i was very excited about built-in documentation generator in D, and now
> i'm not using it at all. i rarely generating stand-alone docs, they are
> just not handy for me. i prefer to read documentation right in the
> source (yet i still want to have an option to generate stand-alone
> files). did you tried to read Phobos documentation in Phobos sources?
> those macros are pure visual noise. i don't mind if D will understand
> one of the Markdown variants, or textile, or rss -- anything that is
> READABLE without preprocessing, yet can be easily processed to another
> format. i don't mind learning another markdown dialect if i can easily
> read it without preprocessing.
>
> that's why i'm not using doxygen too: it's noisy. seems that most
> document generator authors are sure that only generated documentation
> matters, so source documentation can be of any uglyness. yet if
> documentation is hard to read without preprocessor, it is hard to write
> it too! so people will tend to avoid writing it, and they will
> especially avoid polishing it, 'cause it's write-only, contaminated and
> hard to fix.
>
> D documentation WILL be bad until ddoc will start to understand some
> markdown-like mostly macro-free markup language.

Those are exactly my thoughts.

And here's that modified documentation generator with Markdown support (won't help anyone trying to contribute to Phobos, but maybe other projects):

http://forum.dlang.org/thread/itizuviesrhfaijyieex@forum.dlang.org

On Tuesday, 30 December 2014 at 01:50:01 UTC, ketmar via Digitalmars-d wrote:
> D documentation WILL be bad until ddoc will start to understand some markdown-like mostly macro-free markup language.

I honestly don't think the macros are the biggest problem though. I think a bigger deal is the lack of overviews.

Take a look here:

http://dlang.org/phobos/std_algorithm.html


There's some overview, and even a couple links. But the point about opaque types that are supposed to just work isn't easy to find.

Contrast it to what Microsoft wrote up for Windows:

http://msdn.microsoft.com/en-us/library/ms713499%28v=vs.85%29.aspx

There's conceptual overviews, real-world examples, and the references (which link back to the relevant concepts and examples).


std.algorithm could mention the concept of laziness, show examples of the opaque functions, have examples of the common (like seriously one of the most frequently asked questions I've seen) "how do I turn it into an array?", or show/explain how and why to avoid that.


That's mostly plain text that could be written up in the module explanation or as a separate page. I think that's more important than what macros are used.

On 28/12/2014 17:09, Nick Treleaven wrote:
> On 28/12/2014 14:22, Joseph Rushton Wakeling via Digitalmars-d wrote:
>> BTW is it just me, or are the various isSomething methods of std.traits
>> not having documentation generated properly?  See:
>> http://dlang.org/phobos/std_traits.html
>>
>> ... and try searching for, say, isBoolean, or isSomeString.
>
> Works with my newer dmd, I'm not sure why that link doesn't have them.

Also the pre-release docs seem to have them:
http://dlang.org/phobos-prerelease/std_traits.html#isSomeString

On 28/12/2014 17:11, Nick Treleaven wrote:
> But for some reason codePoints and codeUnits are in between .encode.3
> and .encode.4!

I made a start on fixing Walter's points, and fixed the above:
https://github.com/D-Programming-Language/phobos/pull/2825

On Mon, Dec 29, 2014 at 05:19:27PM -0800, Walter Bright via Digitalmars-d wrote:
> On 12/29/2014 4:45 PM, H. S. Teoh via Digitalmars-d wrote:
[...]
> >>But at least you can make it work with LaTeX. Whatcha gonna do with Markdown?
> >Again, I wasn't defending Markdown.
> 
> Then I'm a bit lost on what the point of complaining about Ddoc is. Are you arguing that Ddoc should implement LaTeX?

No, I'm saying that ddoc fails to be either (1) a human-readable source that can also be converted into a nicer format like HTML or LaTeX, or (2) a powerful-enough markup language that encompasses all necessary functionality that HTML/LaTeX/etc. encompass. Apparently point (2) fails by design, since you & Andrei repeatedly claim that ddoc should not be "too powerful". Point (1) fails because no matter what you do, you still have to insert markup *somewhere*. IOW, it's neither nice enough to be human-readable, nor powerful enough to justify the reduction in readability.


> >The only way to get it right is to turn your ddoc comments into tag soup. Are you seriously suggesting that we have to write ddoc tag soup while coding? Or that we first write in plain text then go back afterwards and wrap every paragraph in $(P ...) macros? The only reason zero source code changes were required, was because the ddoc comments were already written with the requisite tag soup to begin with. Which is OK, if that's the "correct" way to use ddoc... but in that case, the page on dlang.org that describes ddoc should be revised to not give the false impression that you can just write documentation comments in plain text format and expect to get nice output from it.
> 
> I think this is an unfair critique. The blank lines separating paragraphs work fine.

They certainly do not. Consider this:

	First section heading:
		Passing in a forward range that doesn't have .length may
		trigger some unexpected side effects.

		Some other description goes here.

	Second section heading:
		More descriptions follow here.

The correct XHTML markup would be something like this:

	<h2>First section heading:</h2>

	<p>Passing in a forward range that doesn't have .length may
	trigger some unexpected side effects.</p>

	<p>Some other description goes here.</p>

	<h2>Second section heading</h2>

	<p>More descriptions follow here.</p>

Now consider the structure of the input, as understood by ddoc:

	$(HEADING First section heading:)
	Passing in a forward range ... side effects.
	$(BLANKLINE)
	Some other description goes here.
	$(HEADING Second section heading:)
	More descriptions follow here.

How do you get from this structure, to the structure of the XHTML markup above? You can't. You *can* hack it so that <p> is inserted by $(HEADING), and $(BLANKLINE) prepends </p>, and $(BLANKLINE) expands to </p><p>, but this will fail at the transition to the second heading: the second paragraph won't be closed. Unless you also prepend </p> in $(HEADING), but then the first heading will acquire an extraneous </p>, which breaks the markup.

Things get worse if you now try to insert code blocks between paragraphs, or paragraphs within quoted blocks. Basically, the structure of $(BLANKLINE) does not perfectly align with tag boundaries, therefore no hack will cover all possible cases. The XHTML output will be malformed.

Basically, $(BLANKLINE) only works if you use <br/> for paragraph breaks, but that screws up CSS styling when you actually want semantically-tagged paragraphs.

Using blank lines to separate paragraphs would be fine, *if* ddoc processes them internally and wraps paragraphs in $(P...) automatically instead of inserting $(BLANKLINE). However, currently it doesn't. And I am loathe to change this because the PR will inevitably get rejected, since it will break every ddoc macro set that relies on $(BLANKLINE).


> Ddoc is not intended to be LaTeX. That it can't do everything a professional typesetting language can is not remarkable, no other markup language can, either.

The problem with this, is that ddoc is imposed upon all D users.  It tries to be readable, but fails to be beyond the most trivial of cases. Things like embedded code blocks and section headings have special meaning, and have a nice human-readable input syntax, but other common constructs such as lists and tables require ugly macro syntax. I'd rather that macro syntax is either used *everywhere* (make it a full-fledged markup language) or nowhere (make it a fully human-readable language), but what we have now is a halfway job.

And then, having included macros, which are deemed necessary, it doesn't go far enough either -- certain things (like autoindexing symbols in a module for making a table of contents, for example) aren't possible without lots of manual work and duplication. There are no capabilities for working with the text itself -- like capitalization so that you can use macros for extracting and placing text snippets in different places, inserting escape sequences for sensitive characters in the target format, extracting symbols to make a table of contents, etc.. This is apparently a deliberate design choice, which makes it feel like we're dangling the enticing capability to be a powerful documentation system, yet we withhold actual text-processing capabilities so that you either have to live with inferior output, or implement custom post-processing tools. Again, it feels like another halfway job, it's neither here nor there.

I'm not saying that the macro system sucks -- it's actually quite elegantly designed as far as macro languages go -- but the problem is that the ddoc system as a whole shies away from what it is. It pretends to be human-readable when it is not, and on the other hand it tries to be a powerful text macro system yet shies away from actual text-processing capabilities. I wish we would stop trying to be all things to everyone, and settle one way or the other. Either we cater to the human-readable-input crowd and make it fully human readable with markdown-like syntax (or whatever other syntax you care to design), or we go all-out with the macro system and endow it with the full text-processing capabilities that it currently lacks. Right now we're stuck in between, and do neither very well.


T

-- 
MAS = Mana Ada Sistem?

On Mon, Dec 29, 2014 at 05:45:08PM -0800, Andrei Alexandrescu via Digitalmars-d wrote:
> On 12/29/14 3:47 PM, H. S. Teoh via Digitalmars-d wrote:
> >On Mon, Dec 29, 2014 at 11:12:54PM +0000, Adam D. Ruppe via Digitalmars-d wrote:
> >>On Monday, 29 December 2014 at 23:11:09 UTC, Walter Bright wrote:
> >>>$(TROW all          , `all!"a > 0"([1, 2, 3, 4])` returns `true`  )
> >>
> >>Would that still work if the first column was something like (0,0) -
> >>including a comma?
> >>
> >>I've advocated nicer macros before, ddoc's recursive expansion makes for a lot of nice options, but the comma being a separator kinda worries me.
> >
> >We've already had to resort to $(COMMA) hacks to work around comma
> >issues in Phobos docs. :-(
> 
> That's to be expected from any macro system.
> 
> >Not to mention that as it stands, ddoc is only really convenient for HTML output; while it's certainly *possible* to target it for non-HTML output, it's a pain. Take LaTeX, for example.
> 
> I have successfully generated LaTeX from dlang.org and phobos.

Of what quality? Have you actually looked at the LaTeX output to see if it's actually correct? I doubt it's actually 100% correct. LaTeX is quite sensitive in some places to extra spaces, or special ways of writing certain text elements. If these details are not taken care of, sure it will still produce output, but there will be little niggling problems sprinkled throughout -- lines that don't line up properly, words that aren't spaced properly, etc..


> >You have things like "Mr.\ So-and-so" (i.e., backslash followed by a single space) for inserting space of the appropriate width after a period (.) that doesn't end a sentence, where a normal "." would introduce end-of-sentence spacing. The only way to fully support this in ddoc is to use $(DOT) or some such hack instead of writing a literal ".". Or take the various dashes: "--" for ndash, "---" for mdash in LaTeX, but "–" or "—" for HTML, respectively. Again, the only way to correctly support this in ddoc is to define $(MDASH) and $(NDASH) macros.
> 
> And what's the problem? This is exactly right, and fine. A macro system supports any formatter, present or future. One that's based on tricks needs to add a new trick for each formatter idiosyncrasy.

So we should stop the false advertisement that ddoc input is supposed to be "human readable", because it is not.


> >So instead of writing dashes in your ddocs, you now have to write the
> >much more verbose and far uglier $(MDASH) and $(NDASH). But it gets
> >worse.
> 
> No it does not get worse. That's pretty much it.
> 
> >Certain character sequences in LaTeX need to be escaped; for example "$" needs to be written as "\$", whereas it can appear literally in HTML.  Again, the only solution is $(DOLLAR).
> 
> Yah, as I said that's pretty much it. You need to encapsulate special stuff in macros, which is all uniform and simple.

Yes, and the system degrades into something no better than writing raw HTML or LaTeX, when every other element in your text is "special stuff". Do you seriously believe that the ". " in "Mr. So-and-So" is "special stuff"? And what if another formatter requires special markup for "-" because it has special meaning? So now, a simple string like:

	Mr. So-and-So

must be written as:

	Mr$(DOTSPACE)So$(HYPHEN)and$(HYPHEN)So

Which is OK if that's the way things are intended to work, but we should seriously stop selling ddoc as "human readable" because at this point, it most certainly is not.


> >Ditto for opening/closing quotation marks, which are `` and '' in LaTeX, and “ and ” in HTML. And non-breaking space, which is ~ in LaTeX, and   in HTML. So now, what was originally an easily-readable documentation comment:
> >
> >	Returns: A range of dollar amounts -- if the input is $100 or
> >	more, as stated in Dr. Watson's "A One" specs; otherwise an
> >	empty range.
> >
> >becomes this monstrosity:
> >
> >	Returns: A range of dollar amounts$(MDASH)if the input is
> >	$(DOLLAR)100 or more$(COMMA) as stated in Dr$(DOTSPACE)Watson's
> >	$(RDQUO)A$(NBSP)One$(LDQUO) specs; otherwise an empty range.
> 
> That's an extreme example. BTW it doesn't look much worse than TDPL's source.
[...]

Yes, and therefore we should stop selling ddoc as a human-readable input format, since the above example is anything but.


T

-- 
Caffeine underflow. Brain dumped.