On Tuesday, 23 January 2018 at 03:23:37 UTC, Walter Bright wrote:
> On 1/22/2018 10:14 AM, David Gileadi wrote:
>> Regarding multiple ways of doing headings, thematic breaks, unordered lists, etc. I'd really like to support as much standard Markdown as possible. My rationale is:
>
> Thanks for the reasonable approach, but I disagree and will try to explain.
>
> Markdown supports multiple ways of doing things because (I presume) it's trying to accommodate multiple diverse existing markdown schemes. We do not have that issue. When there are multiple ways to do X, then inevitably the arguments come as to which is the "approved" method. I do not wish to expend any time debating potayto-potahto. I've had plenty enough of them in my career :-)
>
> Having multiple equivalent ways to do something increases the size of the documentation, the size of the test suite, the potential for bugs, and the cognitive load on the user.
>
> Just pick the most reasonable method, and do that.
>
> (For example, using --- or ``` or ~~~ or 4 space indent for code blocks. Gahhh. Just use ---, because it works and looks fine, and we already do it.)

The prime directive of markdown is to be easy to *read* --- not necessarily easy to write. That is: to look good/natural, without looking hardly at all like markup. For example, you go through the extra trouble to indent subsequent lines in list items:

  * Some multi-line list item
    with subsequent lines indented
    to look good.

  * Second item. Lorem ipsum dolor
    sit amet, consectetur adipiscing
    elit,sed do eiusmod tempor.

instead of:

* Some multi-line list item
with subsequent lines not indented.
Doesn't look great.

* Second item. Lorem ipsum dolor
sit amet, consectetur adipiscing
elit,sed do eiusmod tempor.

because the former is much easier on the eyes. Though, both do indeed work...

There are two distinct reasons why Markdown supports multiple ways of doing some things:

 1. Because markdown also tries to be accomodating. For example, it allows some sloppy input and things still work; like allowing list items to have *up to* that 4 space indent, rather than *requiring* 4. Or allowing two _types_ of *italics*, **and** __bold__. Or all the different ways to make an `<hr/>`, or the three different list item markers. These accomodations are more for the occasional or even accidental markdown user, or for users with _olde_ typing habits that die hard. (Admittedly, some of them *do* sometimes make special cases look a little prettier.)

 2. As a compromise, to make things more convenient for people who use it all the time, but at the expense of it not looking *quite* as good. For example, allowing fenced (delimited) code blocks instead of only the indented code blocks (fenced are easier to copy/paste into). ATX-style headers are faster to type (and easier to autogenerate as well). "Lazy-style" list items (as the 2nd example above shows (and this current list item too) --- where subsequent lines aren't indented) and blockquotes.

It sounds like the kind of multiple-ways-of-doing-things you want to avoid is the former (the accomodations), not the latter (the conveniences).

On 2018-01-23 04:23, Walter Bright wrote:

> (For example, using --- or ``` or ~~~ or 4 space indent for code blocks. Gahhh. Just use ---, because it works and looks fine, and we already do it.)

The fenced code blocks (```) supports specifying a language as well:

```c++
class Foo {}
```

The already existing "---" only supports D. Although, it might not be that important to support.

-- 
/Jacob Carlborg

On 1/23/18 10:30 AM, Jacob Carlborg wrote:
> On 2018-01-23 04:23, Walter Bright wrote:
> 
>> (For example, using --- or ``` or ~~~ or 4 space indent for code blocks. Gahhh. Just use ---, because it works and looks fine, and we already do it.)
> 
> The fenced code blocks (```) supports specifying a language as well:
> 
> ```c++
> class Foo {}
> ```
> 
> The already existing "---" only supports D. Although, it might not be that important to support.

My current code does support it for ``` and ~~~ fences, but not for --- fences because that could change existing behavior. When there's a language it outputs an $(OTHER_CODE language, code) macro instead of a $(D_CODE code) macro.

On 2018-01-23 05:56, David Gileadi wrote:

> I'd been avoiding a PR because I've been rebasing my changes on master rather than merging, and I didn't want to cause confusion by rewriting history. Should I switch to merging changes from master now that I created a PR?

No, you should always rebase from master to get the latest changes from upstream. Branches for PRs are only temporary. Any changes that are performed in the PR after it's opened should be amended on the commit that introduced the code change.

-- 
/Jacob Carlborg

On 1/23/2018 9:30 AM, Jacob Carlborg wrote:
> The fenced code blocks (```) supports specifying a language as well:

Remember, Ddoc macros are not being removed. For the cases of using some other language,

   $(SOME_OTHER_LANGUAGE
       ...
   )

will still work, and will of course work with the inevitable languages that do not have syntax highlighting builtin to Ddoc.

On 1/23/2018 9:35 AM, David Gileadi wrote:
> but not for --- fences because that could change existing behavior.
Current Ddoc behavior is to not recognize --- followed by other characters as part of a fence. A language suffix could be added - I don't see that it is more of a breaking change than adding ~~~.

Please, don't support ``` or ~~~.

> When there's a language it outputs an $(OTHER_CODE language, code) macro instead of a $(D_CODE code) macro.

Clearly, you'll need to write a specification for what you're doing.

On 1/22/18 8:23 PM, Walter Bright wrote:
> On 1/22/2018 10:14 AM, David Gileadi wrote:
>> Regarding multiple ways of doing headings, thematic breaks, unordered lists, etc. I'd really like to support as much standard Markdown as possible. My rationale is:
> 
> Thanks for the reasonable approach, but I disagree and will try to explain.
> 
> Markdown supports multiple ways of doing things because (I presume) it's trying to accommodate multiple diverse existing markdown schemes. We do not have that issue. When there are multiple ways to do X, then inevitably the arguments come as to which is the "approved" method. I do not wish to expend any time debating potayto-potahto. I've had plenty enough of them in my career :-)

Markdown doesn't support multiple ways of doing things because of trying to accommodate multiple Markdown schemes--from the beginning the John Gruber designed the format with multiple ways of doing things [1]. As he says in his philosophy section, "Markdown is intended to be as easy-to-read and easy-to-write as is feasible" and "the single biggest source of inspiration for Markdown’s syntax is the format of plain text email." He intended it to support things that people already do, like start bullet lists with - or * or +, and have it just work.

We can pick out parts of Markdown to support, but it won't be true to the intended vision of Markdown and it will have the effect of annoying people who know Markdown because when they try to write it the way they're used to it won't do what they expect ("wait, I can only use + for lists? I always use dashes! And why aren't my headers working?") [2].

If we go this route then we shouldn't say we support Markdown.

> Having multiple equivalent ways to do something increases the size of the documentation, the size of the test suite, the potential for bugs, and the cognitive load on the user.

These things are all true.

I haven't yet made the corresponding changes to the documentation, but my intent is to show a few examples of Markdown like CommonMark's (very short) help page does [3], provide a link for more information about Markdown, and a (preferably short) list of items DDoc Markdown doesn't support, along with the reasons why.

If instead we support just few Markdown features, I believe we should avoid saying we support Markdown at all. This would probably result in somewhat longer documentation describing each feature.

As for the test suite, as I mentioned I'm generating it from CommonMark's test suite. This allows it to be comprehensive and accurate without building it from scratch. It also reduces the potential for bugs, although I can't argue with the fact that more code == more bugs.

The cognitive load on users depends on what kind of users we're talking about. For those who don't know Markdown the load is indeed greater, although Markdown's focus on supporting things people already do with plain text reduces its impact.

However for people who already know Markdown (and I suspect that's most developers who have used things like Github), the fewer standard Markdown features we support the greater the cognitive load is. Those developers will have to keep in mind both the standard features of Markdown and the list of things DDoc's Markdown doesn't support. The shorter we make that second list, the smaller the cognitive load.

>  > 2. There is already harboured-mod which melds Ddoc and Markdown, and I've been
>  > trying to match its list of differences from vanilla Markdown [1] so that it's
>  > straightforward to switch between DDoc generators.
> 
> What is proposed here shouldn't make life difficult for Habored-mod.

No, but it would make life more difficult for someone who currently generates their documents with harbored-mod and would like to switch to built-in DDoc instead. However this is probably not a large audience.

>  > 3. The additional code for implementing the same feature with different
>  > triggering characters is small.
> 
> Features should not be added because of "why not" or "they're easy to implement". They need to be justified. Ideally, a language should be a very small set of orthogonal building blocks from which anything can be constructed, not a panoply of marginally different overlapping features.

I agree. I mentioned it because I feel strongly about trying to support as much standard Markdown as we can, for the reasons I mentioned earlier. One argument against supporting many Markdown features is the additional code and its associated burdens, and I was attempting to argue in advance that it doesn't add large amounts of additional code.


[1] https://daringfireball.net/projects/markdown/syntax, see the two different styles of headers, two styles of links, and different triggering characters for list items and horizontal rules.

[2] I say this while fully aware that there are parts of Markdown that we won't support. However not supporting a feature because it conflicts with existing features is different than not supporting it for philosophical reasons.

[3] http://commonmark.org/help/

I agree completely with your reasoning that what we are doing should not be called "supporting Markdown". The farthest we should go is saying "inspired by Markdown." I also agree that we should fully document how this will work in Ddoc rather than referring to the Markdown spec.

Please be careful in copying any text, tests, or examples from the Markdown spec to ensure we fully comply with any licensing terms on their site. It may be more practical to just do our own tests and document how ours works in our own words. Frankly, I'd prefer doing the latter anyway so the D spec is 100% our work.

On Wed, 24 Jan 2018 16:55:12 -0700, David Gileadi wrote:

> The cognitive load on users depends on what kind of users we're talking about. For those who don't know Markdown the load is indeed greater, although Markdown's focus on supporting things people already do with plain text reduces its impact.

I actually think supporting multiple methods means the cognitive load may be reduced for people that don't already know Markdown. When I first read some Markdown spec, I quickly realized I could ignore the fact that there are three ways to do a bulleted list and just do what I already do - if there was only one way to do it, I'd have to remember that one way (which would probably not be _my_ way). Reading someone else's Markdown is still easy - '+' for bulleted lists is still obviously a list, even though I use '*'.

> However for people who already know Markdown (and I suspect that's most developers who have used things like Github), the fewer standard Markdown features we support the greater the cognitive load is. Those developers will have to keep in mind both the standard features of Markdown and the list of things DDoc's Markdown doesn't support. The shorter we make that second list, the smaller the cognitive load.

Agreed. It would also be easy to go back and add the alternative syntax later though, right? Start with one way to do it, and if/when people complain, add the alternative methods as needed.

>>  > 2. There is already harboured-mod which melds Ddoc and Markdown, and
>> I've been
>>  > trying to match its list of differences from vanilla Markdown [1] so
>> that it's
>>  > straightforward to switch between DDoc generators.
>> 
>> What is proposed here shouldn't make life difficult for Habored-mod.
> 
> No, but it would make life more difficult for someone who currently generates their documents with harbored-mod and would like to switch to built-in DDoc instead. However this is probably not a large audience.

Another advantage to supporting a variety of features is that support for
making README.md the generated index.html becomes possible (like Doxygen
can do). Supporting a limited subset greatly restricts that ability, as
the README would have to limit itself to the intersection of [GH-]Markdown
and DDocDown(?).

On Wednesday, 24 January 2018 at 23:55:12 UTC, David Gileadi wrote:
>
> We can pick out parts of Markdown to support, but it won't be true to the intended vision of Markdown and it will have the effect of annoying people who know Markdown because when they try to write it the way they're used to it won't do what they expect ("wait, I can only use + for lists? I always use dashes! And why aren't my headers working?") [2].
>
> If we go this route then we shouldn't say we support Markdown.

What do you think about supporting two modes of operation:

  * one with ddoc markup plus a conservative limited
    subset of CommonMark to enhance ddoc, and

  * one which just assumes doc comments are exclusively
    CommonMark Markdown

?

Not sure if this discussion is strictly limited to documenting D and phobos.

New users coming to D and creating modules for code.dlang.org will (do?) balk at having to learn yet another custom markup/macro format for their doc comments. It's hard enough to get people to write any docs at all, let alone ones in a language-specific format (see Racket's Scribble, Perl's POD, Python's ReST, Ruby's RDoc), *let alone* good docs. Would be great if they/we could just use common Markdown.

Another thing to note, since D is often compared to Rust: Rust doc comments are in markdown, and that has worked out very well for them.