On 12/11/2017 6:22 AM, Jakob Bornecrantz wrote:
> There are loads of implementations of CommonMark https://github.com/commonmark/CommonMark/wiki/List-of-CommonMark-Implementations 

They appear to be libraries that offer an implementation. Does the Markdown used in github, reddit, wikipedia, doxygen, etc., use any of those libraries?

> the one I have written is not listed. That covers 1 and 2.

I didn't know you wrote one. There are 3 Markdown packages on Dub:

    https://code.dlang.org/search?q=markdown

and I don't see your name on them. I strongly recommend you register yours with Dub (if it is in D) and on the commonmark site!

On 12/11/17 1:45 PM, Walter Bright wrote:
> On 12/11/2017 6:22 AM, Jakob Bornecrantz wrote:
>> There are loads of implementations of CommonMark https://github.com/commonmark/CommonMark/wiki/List-of-CommonMark-Implementations 
> 
> 
> They appear to be libraries that offer an implementation. Does the Markdown used in github, reddit, wikipedia, doxygen, etc., use any of those libraries?

It appears that github now uses their own fork of cmark, which adds support for tables, etc. [1]

[1] https://githubengineering.com/a-formal-spec-for-github-markdown/

On Monday, 11 December 2017 at 20:45:38 UTC, Walter Bright wrote:
> On 12/11/2017 6:22 AM, Jakob Bornecrantz wrote:
>> There are loads of implementations of CommonMark https://github.com/commonmark/CommonMark/wiki/List-of-CommonMark-Implementations
>
> They appear to be libraries that offer an implementation. Does the Markdown used in github, reddit, wikipedia, doxygen, etc., use any of those libraries?

Github in the past used something called Github Flavored Markdown, which was Markdown + whatever quirks where in the ruby implementation of Markdown + Tables. Not Markdown. The same with all of the others, they all had different quirks because the markdown "spec" was bad. Let me use a different technology as a analogy.

Walter: We should use HTML for DDOC!

Others: Cool, but can you use CommonTML (HTML5), it has this cool spec and actual tests that make sure it looks the same, and that the code you have written is robust. HTML isn't really a standard as such, you are not sure what you get and there is no agreed upon testsuit, so you aren't even sure you get the same results with different versions of your own code.

Walter: Is this used elsewhere? I really want HTML.

Others: Here is the spec and a bunch of implementations.

Walter: But HTML is more common, its used in Netscape Navigator and IE6.

Others: *collective headdesk*


What I'm trying to get at is, use the testsuit and spec, it will save you lot of other problems down the road. And you will be making the world of Markdown a better place because there will be one less implementation that does things slightly differently.

>
>> the one I have written is not listed. That covers 1 and 2.
>
> I didn't know you wrote one. There are 3 Markdown packages on Dub:
>
>     https://code.dlang.org/search?q=markdown
>
> and I don't see your name on them. I strongly recommend you register yours with Dub (if it is in D) and on the commonmark site!

It is not written in D, but the language is close enough in concepts that it can be mechanically ported into D, and is licensed under BOOST. Feel free to do what ever to it[1].

We first used the markdown parser from vibe.d, when we threw the CommonMark testsuit, 10 tests segfaulted and 1 infinite-spun in a loop somewhere in the code. We then rewrote from scratch using the recommended practices from the CommonMark spec and the XML output from cmark as a guide.

The code is used in our documentation system. Both doc-comments and outside documentation files are written in CommonMark. The doc-comments uses Doxygen tags which is then run through CommonMark, most of the time it does nothing to the comments, but if you want to write long comments it makes it much more natural and enjoyable.

[1] https://github.com/VoltLang/Watt/tree/master/markdown/src/watt/markdown

On 12/11/2017 2:30 PM, Jakob Bornecrantz wrote:
> What I'm trying to get at is, use the testsuit and spec, it will save you lot of other problems down the road. And you will be making the world of Markdown a better place because there will be one less implementation that does things slightly differently.

Perhaps you missed that I wrote this twice now:

"Don't get me wrong, I think commonmarkdown is a worthy effort, and is definitely in the running to be a standard. Certainly a lot more effort seems to have been put into it vs other markdowns. It is entirely reasonable to refer to it to answer questions about whether some detail should yin or yang."

On 12/11/2017 2:30 PM, Jakob Bornecrantz wrote:
> It is not written in D, but the language is close enough in concepts that it can be mechanically ported into D, and is licensed under BOOST. Feel free to do what ever to it[1].

Thank you for Boost licensing it!


> We first used the markdown parser from vibe.d, when we threw the CommonMark testsuit, 10 tests segfaulted and 1 infinite-spun in a loop somewhere in the code. We then rewrote from scratch using the recommended practices from the CommonMark spec and the XML output from cmark as a guide.
> 
> The code is used in our documentation system. Both doc-comments and outside documentation files are written in CommonMark. The doc-comments uses Doxygen tags which is then run through CommonMark, most of the time it does nothing to the comments, but if you want to write long comments it makes it much more natural and enjoyable.
> 
> [1] https://github.com/VoltLang/Watt/tree/master/markdown/src/watt/markdown

It's apparently written in Volt:

https://github.com/VoltLang/Watt

But the two links there to Volt are dead:

https://github.com/VoltLang/Watt/blob/master/volt-lang.org

On Tuesday, 12 December 2017 at 02:30:39 UTC, Walter Bright wrote:
> It's apparently written in Volt:
>
> https://github.com/VoltLang/Watt
>
> But the two links there to Volt are dead:
>
> https://github.com/VoltLang/Watt/blob/master/volt-lang.org

Ah, the joys of forgetting too ensure URLs in Markdown READMEs are absolute.

The correct link is volt-lang.org -- I have fixed that README also,
thanks for pointing it out.

On Monday, 11 December 2017 at 15:45:07 UTC, Guillaume Piolat wrote:
> On Monday, 11 December 2017 at 14:22:37 UTC, Jakob Bornecrantz wrote:
>>
>> And to add more, CommonMark on the other hand has a full spec written and several test that covers the difficult to get right parts of Markdown/CommonMark. I'm sure I don't need to tell you the virtues of a good test suit.
>>
>
> The CommonMark approach is to just take the union of all possible features and call it a day.
> Standards without opinions don't deserve to be implemented by anyone.

To me, it sounds more to that

https://xkcd.com/927/

On Tuesday, December 05, 2017 20:11:33 Walter Bright via Digitalmars-d wrote:
> https://help.github.com/articles/basic-writing-and-formatting-syntax/
>
> Anyone interested in picking up the flag?
>
> (I know this has come up before, and I've been opposed to it, but I've
> changed my mind.)

I confess that looking over that page, I don't see much reason to add any of it to ddoc. It all seems like stuff that we can already trivially do in ddoc, and most (all?) of it is stuff that already has built-in macros. What are we really trying to gain here?

In my experience, the problems with ddoc tend to stem from the fact that it's a general purpose macro language and not just intended for documentation generation - there's nothing built-in for generating links to symbols or a navigation bar like dlang.org has, and while some of it can be done in ddoc with a bit of work, some of it requires additional scripts or programs to do it (e.g. while you can add a macro to create a link for a symbol if you have a website layout designed for it, no macro will be able to generate a navigation bar with the list of modules in the project; you need a separate program to generate the list for you, even if that list is then in ddoc).

I really do like ddoc as a macro language, but the fact that that's all it is does hamper it for documentation generation, and while you can get decent, basic documentation out of the box, you can't get anything like doxygen or ddox gives you. You just have separate modules pages without any links to anything or anything indicating what the list of modules is. And you can't even take what's done for Phobos and sanely reuse it, because it's gotten so complicated and is so thoroughly tied to dlang.org.

At the moment, I'm actually trying to put together a basic set of ddoc files with a simple program for using dmd to generate the documentation (as opposed to using a separate generator like ddox does) so that I have something relatively simple that I can reuse across projects that will allow my to link documentation with macros like we do in Phobos and have the navigation bar out of the box - hopefully in a way that's reasonably understood and customizable as opposed to the mess that Phobos' documentation generation has become. I suspect that 90+% of folks that generate documentation just use ddox at this point, because making straight ddoc look good is a royal pain right now - at least if you want to be able to reasonably navigate among the documentation pages (or even within a page) as opposed to just reading the documentation for a single module from top to bottom.

As for markdown, I fail to see how adding any of it to ddoc would really help anything, since as far as I can tell, ddoc already does what markdown does just with macros instead of whatever combination of stray syntax that markdown uses for different things. And I certainly don't want to have to fight ddoc because it suddenly decided that some stray bit of syntax which isn't a macro is suddenly significant and does something special. There already complaints for how ddoc does special stuff under some circumstances instead of being purely a question of the macros used:

https://issues.dlang.org/show_bug.cgi?id=16068

That particular problem can be solved by using $(COLON), but the more magic syntax there is in ddoc, the more landmines it has. It reminds me of how annoying it is when a piece of software decides that it wants to translate any emojis you type into a graphical image, and it thinks that parts of a piece of code that you type are emojis, so your code gets messed up in the process. Obviously, we're not going to be looking to add emoji support to ddoc, but any magical syntax we add risks similar problems.

- Jonathan M Davis

On Tuesday, 12 December 2017 at 06:55:45 UTC, Jonathan M Davis wrote:
> On Tuesday, December 05, 2017 20:11:33 Walter Bright via Digitalmars-d wrote:
>> https://help.github.com/articles/basic-writing-and-formatting-syntax/
>>
>> Anyone interested in picking up the flag?
>>
>> (I know this has come up before, and I've been opposed to it, but I've
>> changed my mind.)
>
> I confess that looking over that page, I don't see much reason to add any of it to ddoc. It all seems like stuff that we can already trivially do in ddoc, and most (all?) of it is stuff that already has built-in macros. What are we really trying to gain here?
>

1) I avoid writing something like $(I foo) into a comment. Because other people may actually read my source code and not just read a generated documentation from a source code. So, I would prefer writing *foo* or _foo_. After all that's the point of markdown: Pure text _looks_ like intended and enables html-generation at the same time.

2) If someone learns the D programming language, I think it would be advantageous, if that person could just write down the already known markdown grammar

On Monday, 11 December 2017 at 00:54:00 UTC, Walter Bright wrote:
>
> I have a more pragmatic definition of a standard:
>
> 1. Products that implement it say they adhere to it and defer to it as the authority on correct behavior.
>
> 2. There's more than one such product.

You have to start somewhere. Somebody has to put the second product out there.

> 3. There's more products adhering to that standard than some other competing standard.

"In 2017, GitHub released a formal specification of their GitHub Flavored Markdown (GFM) that is based on CommonMark.[32] It follows the CommonMark specification exactly except for tables, strikethrough, autolinks and task lists, which the GitHub spec has added as extensions[33]. GitHub also changed the parser used on their sites accordingly, which required some documents to be changed, e.g. a space after the hash signs of a heading is now required."
(https://en.wikipedia.org/wiki/Markdown#GFM)

If GitHub is not some sort of "standard" or at least a good indicator of where things are going, I don't know what is.

So basically, CommonMark is the standard for GitHub. It merely added some extensions that are useful for GitHub related stuff. This means that millions of programmers are already familiar with CommonMark and use it on a daily basis. This in turn would make it easier for them to read and contribute to D source code. Sometimes simple things like that can be the straw that breaks the camel's back (and Ddocs certainly put people off (including me)).

> So far as I know, commonmarkdown satisfies zero of those.
>
> Don't get me wrong, I think commonmarkdown is a worthy effort, and is definitely in the running to be a standard. Certainly a lot more effort seems to have been put into it vs other markdowns. It is entirely reasonable to refer to it to answer questions about whether some detail should yin or yang.
>
> But implementing commonmarkdown in Ddoc is not what we're going to do, at least for the near term.

Hm. I'd really give it a second thought.