In continuing the series of rant posts about the website, this one will be about the documentation. This is a big one and no fixing css or pretty menus can amend this.

I've recently started learning Rust myself (a few weeks ago) and despite the alpha state of both rustc and rust-lang.org website, I found it great and exciting to learn the language. There is a well-thought-out official guide that was a breeze to read through -- I've read most of it while commuting to work on my phone. Then there's Rust by Example which is essentially a semi-organized collection of how-tos, gotchas, snippets and code samples. Rust is not the easiest language of all to learn, but they try to make it simple for those who try.

On the bright side, D has a book. Or rather, it has THE book and a few more books, some of which are free and some are not. However, I wouldn't ever start to read a 500-page manuscript just in order to get acquainted with the language and poke around. There needs to be an official Guide that is not too overly detailed and written in an easy language.

To give a few examples:
- http://doc.rust-lang.org/book/
- https://www.ruby-lang.org/en/documentation/quickstart/
- http://ocaml.org/learn/tutorials/basics.html

I personally see "learning" and "documentation" as two different classes of information. Learning resources might include tutorials, code examples, articles, books, whereas documentation would be a place for more formal (and sometimes computer-generated) information on the standard library, API reference, language grammar and maybe an official style guide. Ideally, the Learning section would be sufficient for a rookie to get started and that should be the go-to destination for all newcomers.

1.1) Documentation: D Reference

Is very hard to browse or search. I don't remember how many times I was looking for an "is" statement docs only to find again, buried under <h4>IsExpresssion</h4> in expression.html, a dozen of pages scrolling down from top. There are no TOCs in the files, there is no global TOC, the drop-down menu on the left is extremely long and non-nested and takes one screen vertically.

All in all though, the contents are OK -- it does require some polishing, it needs to be properly indexed and laid out but that's doable without changing the contents much.

1.2) Documentaiton: Standard Library

There's been some good progress on it recently by Andrei -- but it's auto-generated for the most part so the most of work is in hacking ddoc and styling. The only two issues I could point out is the landing page for "Standard Library" -- I would expect to see a structure of D's stdlib or a nice index and quick links rather than a "why this and why that" wall of text (which should stay as a separate page, but not as an entry point to the standard library pages, from the usability point of view). Another point: there needs to be a better on-site search. Apologies for mention Rust again, but: http://doc.rust-lang.org/std/index.html (type S, see what happens -- the search is not perfect but it's incremental and you can search for functions/traits separately). Since we are the ones who generate the docs, we can totally do this (in a simplest way, bake in the markers into the ddoc and generate a javascript hashmap to trace back to them).

1) D Learning

This is the most problematic part. It's not even obvious where to start.

Say I just landed on dlang.org via a google search and I want to find a quick user guide. I click D Reference (that seems the closest one to "beginner's guide" out of all options) and I see an "Introduction" page. Ok, looks promising, what do we have there... Ouch, Phases of Compilation, that went wrong fast. Let's try the next one, Lexical. Ouch, wall of text composed of ascii characters, unicode escapes and all other boring things.

Let's go back to Books & Articles. Now, Ali's book is sure very nice, but it's way overly detailed for introduction as it's trying to miss a single thing (which is sure a good thing! ... that is, if you actually decide on reading a BOOK in a first place). Plus, it's not on dlang.org so it doesn't feel "official". Going back to more books, we see some 5-7 year old books, some very recent ones (but not free), and a few tutorials on a specific areas (like Philip's) -- again, nothing really qualifies as an introduction guide. This section is definitely more well-suited for intermediate D programmers who already know what they want to learn.

Books & Articles -> Howtos (is that a book or an article?). This links to a wiki.dlang.org (yet another site?..) where we have an unorganized mess of links. Some of them sound useful but it's unlikely we are interested in voldemort types or unittest placement yet. In fact, I've just found out there's a bunch of information on wiki that I see myself for the first time -- if it took me several years, how long will it take a newcomer? It would be nice to move the best/finalized bits and examples into a sort of a "D by Example" official section on the website. Wiki is useful as a collaborative sandbox or a staging area, but it's fugly, feels foreign and not so nice for reading.

Books & Articles -> Floating Point. What does that article have to do with learning D and what exactly is it doing there (apart from being a nice article on floating point)?

Example: wc -> this is neither a book, nor an article and it belongs to a "D by Example" along with other examples from wiki. Plus, the code is quite ugly and inelegant.

Warnings: not a book nor an article. Belongs in Documentation.

C to D: this is especially confusing, since there are:
- Books & Articles > C to D
- Books & Articles > C++ to D
- Books & Articles > C Preprocessor to D
- D Reference > Interfacing to C
- D Reference > Interfacing to C++
which overlap somewhat (the "Books & Articles" ones) being more of a collection of recipes without a proper index. Quite hard to browse through or search for something specific. Not sure if this belongs to Learning or Documentation, perhaps the latter.

... that's all. At this point the disgruntled newbie closes the dlang.org website and goes on learning himself some Rust.

[NB] SUGGESTION: initiate work on an Official Guide and keep it up to date with the latest language features. It could even be largely a copy-paste from Ali's book and TDPL (upon Ali's and Andrei's permission), omitting unnecessary details (and rather pointing at a specific places in documentation where additional information could be found) and written in a friendly manner with a human touch. I think it's much simpler than it sounds if we do it collaboratively and it doesn't have to be feature-complete before released. If we choose to do it, it may be best to keep it in a separate github repo in order to make it easier to compile separately without having to compile dmd/druntime/etc.

Optionally, but highly suggested: initiate work on D by Example: a searchable collection of up-to-date self-documenting D examples, structured by the topic, like a book. Some of the examples from wiki could be moved there, some Rosetta examples by bearophile and many more. Could be even parts of some libraries or Phobos. The point is, we have more than we need, it's just scattered across the Internet, so we need to unscatter it.

On Fri, Jan 23, 2015 at 05:03:15PM +0000, aldanor via Digitalmars-d wrote: [...]
> I personally see "learning" and "documentation" as two different classes of information. Learning resources might include tutorials, code examples, articles, books, whereas documentation would be a place for more formal (and sometimes computer-generated) information on the standard library, API reference, language grammar and maybe an official style guide. Ideally, the Learning section would be sufficient for a rookie to get started and that should be the go-to destination for all newcomers.

+1. Our current documentation is *reference* documentation, i.e., something you look stuff up in *after* you've learned the language. It's not suitable for a tutorial or getting-started guide. We need more of those!

OTOH, do I hear the cry of a volunteer? ;-)  (I'm only half-joking... the thing is, if nobody steps up to write said tutorial, it isn't gonna materialize. The rest of us are already busy enough with whatever it is we're contributing to D. Having said that, though, I thought Ali's D book is pretty good in terms of serving as a beginner's tutorial to D? Or do we need a different one more geared towards seasoned programmers? (Ali's book is primarily targeted towards newbie programmers).)


> 1.1) Documentation: D Reference
> 
> Is very hard to browse or search. I don't remember how many times I was looking for an "is" statement docs only to find again, buried under <h4>IsExpresssion</h4> in expression.html, a dozen of pages scrolling down from top. There are no TOCs in the files, there is no global TOC, the drop-down menu on the left is extremely long and non-nested and takes one screen vertically.
> 
> All in all though, the contents are OK -- it does require some polishing, it needs to be properly indexed and laid out but that's doable without changing the contents much.

Yeah I've run into the same problem. Google search does not eliminate the need for a proper, well-thought-out, navigable index.


> 1.2) Documentaiton: Standard Library
> 
> There's been some good progress on it recently by Andrei -- but it's auto-generated for the most part so the most of work is in hacking ddoc and styling. The only two issues I could point out is the landing page for "Standard Library" -- I would expect to see a structure of D's stdlib or a nice index and quick links rather than a "why this and why that" wall of text (which should stay as a separate page, but not as an entry point to the standard library pages, from the usability point of view).

Actually, this has annoyed me so much that I've completely nuked the page in Phobos git HEAD, and replaced it with something hopefully more useful. You can see this new page under the prerelease tab:

	http://dlang.org/phobos-prerelease/index.html

So hopefully, by the next release, we will no longer be plagued by complaints about the current leading page.


> Another point: there needs to be a better on-site search. Apologies for mention Rust again, but: http://doc.rust-lang.org/std/index.html (type S, see what happens -- the search is not perfect but it's incremental and you can search for functions/traits separately). Since we are the ones who generate the docs, we can totally do this (in a simplest way, bake in the markers into the ddoc and generate a javascript hashmap to trace back to them).

I'm thinking perhaps an autogenerated alphabetical index of all symbols might be in order here?


> 1) D Learning
> 
> This is the most problematic part. It's not even obvious where to start.
> 
> Say I just landed on dlang.org via a google search and I want to find a quick user guide. I click D Reference (that seems the closest one to "beginner's guide" out of all options) and I see an "Introduction" page. Ok, looks promising, what do we have there... Ouch, Phases of Compilation, that went wrong fast. Let's try the next one, Lexical. Ouch, wall of text composed of ascii characters, unicode escapes and all other boring things.

Easy. We pick a suitable beginner's tutorial -- either Ali's excellent book or something you or some other volunteer writes up, and put a big fat link to it in a prominent place on the front page. Problem solved.


> Let's go back to Books & Articles. Now, Ali's book is sure very nice, but it's way overly detailed for introduction as it's trying to miss a single thing (which is sure a good thing! ... that is, if you actually decide on reading a BOOK in a first place). Plus, it's not on dlang.org so it doesn't feel "official". Going back to more books, we see some 5-7 year old books, some very recent ones (but not free), and a few tutorials on a specific areas (like Philip's) -- again, nothing really qualifies as an introduction guide. This section is definitely more well-suited for intermediate D programmers who already know what they want to learn.

If you're not happy with Ali's book, please contribute your own. I'm pretty sure the dlang.org maintainers will be more than glad to include it.


[...]
> [NB] SUGGESTION: initiate work on an Official Guide and keep it up to date with the latest language features. It could even be largely a copy-paste from Ali's book and TDPL (upon Ali's and Andrei's permission), omitting unnecessary details (and rather pointing at a specific places in documentation where additional information could be found) and written in a friendly manner with a human touch. I think it's much simpler than it sounds if we do it collaboratively and it doesn't have to be feature-complete before released. If we choose to do it, it may be best to keep it in a separate github repo in order to make it easier to compile separately without having to compile dmd/druntime/etc.

Would you like to step up and spearhead this effort?


> Optionally, but highly suggested: initiate work on D by Example: a searchable collection of up-to-date self-documenting D examples, structured by the topic, like a book. Some of the examples from wiki could be moved there, some Rosetta examples by bearophile and many more. Could be even parts of some libraries or Phobos. The point is, we have more than we need, it's just scattered across the Internet, so we need to unscatter it.

Good! So let's see the PR's. :-)


T

-- 
Stop staring at me like that! It's offens... no, you'll hurt your eyes!

On Friday, 23 January 2015 at 17:03:17 UTC, aldanor wrote:
> Since we are the ones who generate the docs, we can totally do this (in a simplest way, bake in the markers into the ddoc and generate a javascript hashmap to trace back to them).
I turns out that's exactly how it's done on the rust website, poor man's index engine :)

http://doc.rust-lang.org/search-index.js -- DDOC can totally generate something like this, just a huge dict of markers. Once this is done, the client-side is quite easy.

On Friday, 23 January 2015 at 17:19:44 UTC, H. S. Teoh wrote:
> OTOH, do I hear the cry of a volunteer? ;-)  (I'm only half-joking...
> the thing is, if nobody steps up to write said tutorial, it isn't gonna
> materialize. The rest of us are already busy enough with whatever it is
> we're contributing to D.
I could try in my spare time but I don't think I qualify as a D guru since there are some shady D areas I have yet to learn properly myself. It has to be a collaborative effort.

On Friday, 23 January 2015 at 17:19:44 UTC, H. S. Teoh wrote:
> Having said that, though, I thought Ali's D
> book is pretty good in terms of serving as a beginner's tutorial to D?
> Or do we need a different one more geared towards seasoned programmers?
> (Ali's book is primarily targeted towards newbie programmers).)
Ali's book is VERY good. The best part is that you can load it on Kindle / tablet / whatever (I did!) and take it with you. However, it's _not_ an official "30 minutes to D" guide. It's more like "1 month to D if you survive it" because it's very thorough and detailed. Come on, it's a bit too boring to only get to a for loop in chapter 10 if I'm just excited to see what the language is all about. For instance, I'm fairly certain that metaprogramming (at its simplest) should appear early in the guide.

> Yeah I've run into the same problem. Google search does not eliminate
> the need for a proper, well-thought-out, navigable index.

> I'm thinking perhaps an autogenerated alphabetical index of all symbols
> might be in order here?
Yep, see my post above re: the incremental index. It's absolutely doable with DDOC / client-side JS.

> Easy. We pick a suitable beginner's tutorial -- either Ali's excellent
> book or something you or some other volunteer writes up, and put a big
> fat link to it in a prominent place on the front page. Problem solved.
The problem is - I don't think we actually have one. And it really has to live on dlang.org to feel official and up to date. It has to be reasonably succinct but exciting, not too formal, well-styled, with links to official docs and "read more there and there" anchors.

> If you're not happy with Ali's book, please contribute your own. I'm
> pretty sure the dlang.org maintainers will be more than glad to include
> it.
I (personally) am happy with Ali's book of course! But as I've already said a link to the book != a proper _official_ introduction. It could largely overlap though, that's true.

> Would you like to step up and spearhead this effort?
Not alone by myself, that's for sure :) Ideally, someone who's already done a considerable amount of work on a book / docs, of course...

> Good! So let's see the PR's. :-)
This needs to be well thought through :) But if noone objects to the style of "Rust by Example" which I tend to like -- perhaps we could come up with something similar (and perhaps more interesting, especially when it gets to all the metaprogramming jazz).

On 1/23/15 9:03 AM, aldanor wrote:
> [NB] SUGGESTION: initiate work on an Official Guide and keep it up to
> date with the latest language features.

That would be fantastic, and something that might be parallelizable as well. Here's a related thought: I wonder what it would take to put together a nice slideshow-style progression, with editable code snippets and stuff. Then I'd be happy to work on the content. -- Andrei

On Friday, 23 January 2015 at 18:00:13 UTC, Andrei Alexandrescu wrote:
> On 1/23/15 9:03 AM, aldanor wrote:
>> [NB] SUGGESTION: initiate work on an Official Guide and keep it up to
>> date with the latest language features.
>
> That would be fantastic, and something that might be parallelizable as well. Here's a related thought: I wonder what it would take to put together a nice slideshow-style progression, with editable code snippets and stuff. Then I'd be happy to work on the content. -- Andrei

What do you mean by a slideshow style progression? Would you have an example? I thought of something simple http://rustbyexample.com/ as a guideline, but hosted officially on dlang.org and being part of the website. Is that close?

/* don't worry much about implementing the cool client-side UI stuff if that's what you mean, whatever it is, it's all doable (or, in all likelihood, it has already been done in a form of some js library) and it normally only takes one person to integrate it it; writing a cohesive exciting (!) user guide though is a big undertaking and responsibility. As you said though, it can be highly parallelizable! */

On 1/23/15 10:09 AM, aldanor wrote:
> On Friday, 23 January 2015 at 18:00:13 UTC, Andrei Alexandrescu wrote:
>> On 1/23/15 9:03 AM, aldanor wrote:
>>> [NB] SUGGESTION: initiate work on an Official Guide and keep it up to
>>> date with the latest language features.
>>
>> That would be fantastic, and something that might be parallelizable as
>> well. Here's a related thought: I wonder what it would take to put
>> together a nice slideshow-style progression, with editable code
>> snippets and stuff. Then I'd be happy to work on the content. -- Andrei
>
> What do you mean by a slideshow style progression? Would you have an
> example? I thought of something simple http://rustbyexample.com/ as a
> guideline, but hosted officially on dlang.org and being part of the
> website. Is that close?

Both are nice:

http://tour.golang.org/welcome/1
http://rustbyexample.com/

> /* don't worry much about implementing the cool client-side UI stuff if
> that's what you mean, whatever it is, it's all doable (or, in all
> likelihood, it has already been done in a form of some js library) and
> it normally only takes one person to integrate it it; writing a cohesive
> exciting (!) user guide though is a big undertaking and responsibility.
> As you said though, it can be highly parallelizable! */

I can get cranked on that. Just give me the ddoc macros to fill.


Andrei

On Fri, Jan 23, 2015 at 05:34:28PM +0000, aldanor via Digitalmars-d wrote:
> On Friday, 23 January 2015 at 17:19:44 UTC, H. S. Teoh wrote:
> >OTOH, do I hear the cry of a volunteer? ;-)  (I'm only half-joking... the thing is, if nobody steps up to write said tutorial, it isn't gonna materialize. The rest of us are already busy enough with whatever it is we're contributing to D.
>
> I could try in my spare time but I don't think I qualify as a D guru since there are some shady D areas I have yet to learn properly myself. It has to be a collaborative effort.

That's not a problem; post your ideas / drafts here and I'm sure you'll get lots of feedback and improvement suggestions.


> On Friday, 23 January 2015 at 17:19:44 UTC, H. S. Teoh wrote:
> >Having said that, though, I thought Ali's D book is pretty good in terms of serving as a beginner's tutorial to D?  Or do we need a different one more geared towards seasoned programmers?  (Ali's book is primarily targeted towards newbie programmers).)
>
> Ali's book is VERY good. The best part is that you can load it on Kindle / tablet / whatever (I did!) and take it with you. However, it's _not_ an official "30 minutes to D" guide. It's more like "1 month to D if you survive it" because it's very thorough and detailed. Come on, it's a bit too boring to only get to a for loop in chapter 10 if I'm just excited to see what the language is all about. For instance, I'm fairly certain that metaprogramming (at its simplest) should appear early in the guide.

Well, to be fair, the reason Ali's book is so detailed is because it's geared towards the newbie programmer who is learning how to program, possibly for the first time. So the pacing isn't really suited for an experienced programmer who just wants to learn D as an additional language. The material is excellent, no doubt, but probably needs repackaging to target the seasoned programmer.


[...]
> >I'm thinking perhaps an autogenerated alphabetical index of all symbols might be in order here?
>
> Yep, see my post above re: the incremental index. It's absolutely doable with DDOC / client-side JS.

I vote for ddoc / static generation. The site itself is static anyway, using client-side JS to do this seems to be overkill.


> >Easy. We pick a suitable beginner's tutorial -- either Ali's excellent book or something you or some other volunteer writes up, and put a big fat link to it in a prominent place on the front page. Problem solved.
>
> The problem is - I don't think we actually have one. And it really has to live on dlang.org to feel official and up to date. It has to be reasonably succinct but exciting, not too formal, well-styled, with links to official docs and "read more there and there" anchors.

If we don't have one, then the obvious first step is to begin writing one. :-)

And when I said you should spearhead this effort, I don't mean you should do everything yourself. That's not feasible for obvious reasons. Rather, the community can and should help -- but you do need to take the lead in the effort, otherwise there will be no focused direction and things will go nowhere after 2 months.

For example, you could come up with an initial draft or proof-of-concept and put it in a github fork where people can submit PRs to flesh out the skeletal ideas / outline you lay down as a first stab. Then you can solicit for feedback -- that's part of the role of spearheading the effort -- and oversee the overall direction. That way it's not just you who's put on the spot to do everything, but everyone can contribute and make it better than any single one of us can by ourselves.

Or, for an even less daunting start: start submitting PRs for draft tutorial pages and code examples, and encourage others to do the same. Nothing gets things moving more effectively than having actual code / docs sitting in the PR queue ready to be merged. OTOH, too many forum threads (well-intentioned, mind you) tend to just fizzle out after a while and nothing gets accomplished.


T

-- 
Give a man a fish, and he eats once. Teach a man to fish, and he will sit forever.

On Friday, 23 January 2015 at 18:17:30 UTC, H. S. Teoh wrote:
> Well, to be fair, the reason Ali's book is so detailed is because it's
> geared towards the newbie programmer who is learning how to program,
> possibly for the first time. So the pacing isn't really suited for an
> experienced programmer who just wants to learn D as an additional
> language. The material is excellent, no doubt, but probably needs
> repackaging to target the seasoned programmer.
Agreed. I think that's exactly the target audience (and exactly the one that we are missing the proper docs/guide for). After all, D is not a "learn me a ruby in 15 minutes" kind of language, so the chances are whoever's coming here has a vague idea of what they expect or what they want to learn about D. Just as an example, I've heard a lot of D talk on #rust-offtopic irc -- those folks would barely qualify as beginner programmers.

> I vote for ddoc / static generation. The site itself is static anyway,
> using client-side JS to do this seems to be overkill.
You need both. What I meant was that DDOC has to be able to generate an index (one huge dict) and store that in a text file/json. The client-side would just use that "database" to provide on-site incremental search. Quite easy.

> If we don't have one, then the obvious first step is to begin writing
> one. :-)
>
> And when I said you should spearhead this effort, I don't mean you
> should do everything yourself. That's not feasible for obvious reasons.
> Rather, the community can and should help -- but you do need to take the
> lead in the effort, otherwise there will be no focused direction and
> things will go nowhere after 2 months.
>
> For example, you could come up with an initial draft or proof-of-concept
> and put it in a github fork where people can submit PRs to flesh out the
> skeletal ideas / outline you lay down as a first stab. Then you can
> solicit for feedback -- that's part of the role of spearheading the
> effort -- and oversee the overall direction. That way it's not just you
> who's put on the spot to do everything, but everyone can contribute and
> make it better than any single one of us can by ourselves.
>
> Or, for an even less daunting start: start submitting PRs for draft
> tutorial pages and code examples, and encourage others to do the same.
> Nothing gets things moving more effectively than having actual code /
> docs sitting in the PR queue ready to be merged. OTOH, too many forum
> threads (well-intentioned, mind you) tend to just fizzle out after a
> while and nothing gets accomplished.
Thanks for clarifying. I too hope this thread doesn't just become another bikeshedding timesink :) I'll get some style-related drafts published on the weekend and then we'll see how it goes from there. Indeed, I won't mind to "spearhead" this (if I knew how!) since the whole documentation story is very sad, provides D with negative publicity and provides users like us with constant annoyance (but... it can be fixed -- we all know it).

Dne 23.1.2015 v 19:16 Andrei Alexandrescu via Digitalmars-d napsal(a):
> Both are nice:
> 
> http://tour.golang.org/welcome/1
> http://rustbyexample.com/

Or something along the lines of https://tryhaskell.org
With possible integration with D REPL.