On Tuesday, 12 November 2019 at 15:14:17 UTC, Steven Schveighoffer wrote:
> Alternatively, we could use something other than comments for documentation. e.g.:

lol

@ddoc!q{
/++

+/
}


that would actually work and the doc generator can recognize the pattern too.

But there's a big difference:

struct A {
   int a; /// does something
}

that is cute syntax and not repeatable with UDAs.

And of course, my big use case is to fetch docs for third party libs I'd rather not edit the source of anyway.


Now see, this is an argument against the fear that it will get abused though simply because the language has easier ways to do other things. Using UDAs for docs is doable but awkward, so we prefer to use /++ +/.

Similarly, abusing doc comments as UDAs is going to be more awkward than using the real thing - you can use the language to structure it and all. So why do things the hard way when there's an easy way?

On Tuesday, 12 November 2019 at 06:32:35 UTC, FeepingCreature wrote:
> You can already do the same degree of mess. You just have to make it ugly. All this restriction does is make good, useful code worse for little benefit.

And that's exactly the point: you have to make it ugly, so it isn't an attractive option.
Some things are ugly by design, such as __gshared, discouraging users from using it while still allowing it if needed.

Similarly, version() statements don't allow boolean logic, discouraging those annoying double negations and version formulas (see e.g. [1], [2]) and encouraging 'positive' identifier names that don't have 'no' in them. You can still make version spaghetti with `static if` if you want, but it won't be as appealing.

Technically you can still do C macro shenanigans like `#define BEGIN {` in D by putting your module in a q{} string, doing a string-replace in CTFE and mixing it in.
"You just have to make it ugly".
And that's exactly why D programmers don't tend to do that.

While I do see the value in __traits(docComment) (I would use it myself in the same way Adam wants to use it), I do have to agree with Steven that it just feels to powerful.
As soon as some hacker discovers `__traits(docComment)` he will have the 'brilliant' idea to inspect documentation and use parts of it for his code generation, and it will look elegant so it doesn't seem wrong (unlike some convoluted parse(import(__FILE__)) scheme).
And then a clueless contributor will be like 'how the heck did running dfmt on this codebase make the unittests fail'?

UDA's use string literals, so the rules of what string you get are exactly clear and a formatter won't touch it. How does __traits(docComment) behave on these two comments:

```
/** Word
 *  Word
 */

/******************
Word
Word
*/
```

What strings do they produce?

[1] https://github.com/PetteriAimonen/libfixmath/blob/c3017d7d74867398dcdcaa21d93c6a0b039b9bca/libfixmath/fix16.c#L8
[2] https://github.com/PetteriAimonen/libfixmath/blob/c3017d7d74867398dcdcaa21d93c6a0b039b9bca/libfixmath/fix16.c#L67

On Wednesday, 13 November 2019 at 15:22:23 UTC, Dennis wrote:
> UDA's use string literals, so the rules of what string you get are exactly clear and a formatter won't touch it. How does __traits(docComment) behave on these two comments:

This is actually defined in the D language specification

https://dlang.org/spec/ddoc.html

"The extra *'s and +'s on the comment opening, closing and left margin are ignored and are not part of the embedded documentation. Comments not following one of those forms are not documentation comments. "

It is a little bit ambiguous to get

/**
  Word
  * word
*/

Is that "* word" supposed to ignore the star? dmd ignores it which is probably most consistent with the spec. My adrdox does NOT ignore that and can use it for markdown-style lists...

But regardless, the __trait thing should do what the compiler already does for ddoc.

Though I would NOT parse the ddoc, just give the string as the compiler read it.

> ```
> /** Word
>  *  Word
>  */
>
> /******************
> Word
> Word
> */
> ```
>
> What strings do they produce?

" Word\n  Word\n "

and

"Word\nWord\n"

So not identical, but this is well defined by the D language spec!

On 11/12/19 11:23 AM, Adam D. Ruppe wrote:
> On Tuesday, 12 November 2019 at 15:14:17 UTC, Steven Schveighoffer wrote:
>> Alternatively, we could use something other than comments for documentation. e.g.:
> 
> lol
> 
> @ddoc!q{
> /++
> 
> +/
> }
> 
> 
> that would actually work and the doc generator can recognize the pattern too.
> 
> But there's a big difference:
> 
> struct A {
>     int a; /// does something
> }

This is really not much different:

struct A {
    @ddoc(does something)
    int a;
}

But I mean, we can do whatever we want. It doesn't have to fit into the world of UDA. It can be its own syntax.

struct A {
    int a; !ddoc does something
}

It just shouldn't be a comment, which the compiler is supposed to ignore.

> And of course, my big use case is to fetch docs for third party libs I'd rather not edit the source of anyway.

Well, assuming this isn't just a mental exercise, the answer would be, if you want it to be accessible to __traits(getDocs), you need to write it the other way. If that means you can't access the docs, then you can't access the docs. I would expect a dfix solution is braindead simple here.

> Now see, this is an argument against the fear that it will get abused though simply because the language has easier ways to do other things. Using UDAs for docs is doable but awkward, so we prefer to use /++ +/.
> 
> Similarly, abusing doc comments as UDAs is going to be more awkward than using the real thing - you can use the language to structure it and all. So why do things the hard way when there's an easy way?

The thought of comments *ever* affecting the code is just plain wrong to me. Comments are not code, we should keep that line bright.

-Steve

On Wednesday, 13 November 2019 at 16:03:55 UTC, Steven Schveighoffer wrote:
> It just shouldn't be a comment, which the compiler is supposed to ignore.

The D specification makes a distinction between ordinary comments and ddoc comments. ddoc comments are well-defined by the language and attached to symbols by parsing rules.

It currently doesn't expose it directly - it only processes it into separate files (not ignoring them!) - but I want to change that, and appealing to the authority of the status quo isn't much of a counterargument.

> The thought of comments *ever* affecting the code is just plain wrong to me. Comments are not code, we should keep that line bright.

I agree for regular comments, but ddoc is already a part of the language and processed by the compiler! All I want to do is let other parts of the language see it too.

And again, UDAs do a better job at embedding non-doc stuff than doc comments. So I really think the risk of people going crazy with them is quite small since the alternative is superior for this task.

On 11/13/19 11:12 AM, Adam D. Ruppe wrote:
> On Wednesday, 13 November 2019 at 16:03:55 UTC, Steven Schveighoffer wrote:
>> It just shouldn't be a comment, which the compiler is supposed to ignore.
> 
> The D specification makes a distinction between ordinary comments and ddoc comments. ddoc comments are well-defined by the language and attached to symbols by parsing rules.

Well defined, but it still compiles even if you don't follow the spec. These are comments still, and a secondary product from the binary output.

> It currently doesn't expose it directly - it only processes it into separate files (not ignoring them!) - but I want to change that, and appealing to the authority of the status quo isn't much of a counterargument.

In reality you could separate the ddoc parsing and generation from the compile step anyway. It's just that it's much more convenient to use the already parsed AST for the code to generate the documentation.

In this way, the *code* affects the documentation, not the other way around. The status quo should remain this way because code is complicated enough without requiring comments to "compile" correctly too.

However, I'm OK with something other than comments being used to generate the doc reading code, OR deal with it at runtime when the compiler is not involved.

Note that ddoc comments are still not ddoc data. You are missing a lot of stuff with just the string. Wouldn't it be nice to just import the JSON data generated by the compiler? That's how I would do it.

> 
>> The thought of comments *ever* affecting the code is just plain wrong to me. Comments are not code, we should keep that line bright.
> 
> I agree for regular comments, but ddoc is already a part of the language and processed by the compiler! All I want to do is let other parts of the language see it too.
> 
> And again, UDAs do a better job at embedding non-doc stuff than doc comments. So I really think the risk of people going crazy with them is quite small since the alternative is superior for this task.

I've seen lots of schemes where comments are significant to some process or another. IDE directives for instance.

But these schemes can play around in this realm *because* the compiler ignores them. I never want to see a case where comments are significant and break compilation or introduce bugs. We have enough to review in the code itself...

-Steve

On Wednesday, 13 November 2019 at 16:33:58 UTC, Steven Schveighoffer wrote:
> On 11/13/19 11:12 AM, Adam D. Ruppe wrote:
>> On Wednesday, 13 November 2019 at 16:03:55 UTC, Steven Schveighoffer wrote:
>>> It just shouldn't be a comment, which the compiler is supposed to ignore.
>> 
>> The D specification makes a distinction between ordinary comments and ddoc comments. ddoc comments are well-defined by the language and attached to symbols by parsing rules.
>
> Well defined, but it still compiles even if you don't follow the spec. These are comments still, and a secondary product from the binary output.

Easily fixed: make the trait fail if the comment violates the Ddoc guidelines.

>
>> It currently doesn't expose it directly - it only processes it into separate files (not ignoring them!) - but I want to change that, and appealing to the authority of the status quo isn't much of a counterargument.
>
> In reality you could separate the ddoc parsing and generation from the compile step anyway. It's just that it's much more convenient to use the already parsed AST for the code to generate the documentation.
>
> In this way, the *code* affects the documentation, not the other way around. The status quo should remain this way because code is complicated enough without requiring comments to "compile" correctly too.

We already require comments to compile correctly. That's what Ddoc *is*. Documentation should not be considered an optional part of functionality, especially if the user is trying to use Ddoc comments.

>
> However, I'm OK with something other than comments being used to generate the doc reading code, OR deal with it at runtime when the compiler is not involved.
>
> Note that ddoc comments are still not ddoc data. You are missing a lot of stuff with just the string. Wouldn't it be nice to just import the JSON data generated by the compiler? That's how I would do it.

That would be nice. However, imagine this usecase: Swagger is a documentation format for REST requests. It would be handy if this documentation could be served right alongside the requests themselves. However, this would require a two-step process where the code is compiled, the Ddoc comments are discarded; then the code is, for no good reason, compiled again with a separate tool that wraps the D compiler, having to ensure it gets the same compiler binary and same compiler flags, creating an additional point of error (for no reason!), then dynamically loaded back into a process and processed at runtime, when they could have been equally well evaluated via a trait at compiletime and processed in ctfe with std.json to generate static output with nicely integrated error checking with line numbers.

>
> I've seen lots of schemes where comments are significant to some process or another. IDE directives for instance.
>
> But these schemes can play around in this realm *because* the compiler ignores them. I never want to see a case where comments are significant and break compilation or introduce bugs. We have enough to review in the code itself...
>
> -Steve

Again: Ddoc comments are *already* a mandatory part of compilation. If code compiles but the comments violate Ddoc, it should under no circumstances be considered "working."
D is a static language. To me, that means: any runtime failure that can be a compiletime failure should be. Compilation failing is annoying; but, compilation failing *rather than* runtime failing is the entire point of static types.

On 11/13/19 12:19 PM, FeepingCreature wrote:
> On Wednesday, 13 November 2019 at 16:33:58 UTC, Steven Schveighoffer wrote:

>> I've seen lots of schemes where comments are significant to some process or another. IDE directives for instance.
>>
>> But these schemes can play around in this realm *because* the compiler ignores them. I never want to see a case where comments are significant and break compilation or introduce bugs. We have enough to review in the code itself...
>>
> 
> Again: Ddoc comments are *already* a mandatory part of compilation. If code compiles but the comments violate Ddoc, it should under no circumstances be considered "working."
> D is a static language. To me, that means: any runtime failure that can be a compiletime failure should be. Compilation failing is annoying; but, compilation failing *rather than* runtime failing is the entire point of static types.

I'm not convinced:

import std.stdio;

/// $(BADMACRO boo)
void main()
{
    writeln("compiles and runs!");
}

Even generates ddoc! Just that, @(BADMACRO boo) does nothing. It doesn't even produce an error.

Let's mess with the parsing:

/// $(BADMACRO boo

note the missing parentheses. No errors, even when compiled with -D. I get a correct html doc, but now I see "(BADMACRO boo" as the docs for main as if it were just a string (curiously missing the $ but whatevs, it's just comments and docs!)

I wouldn't consider this to be "mandatory" or even not "working". I get a working binary, and I get a working ddoc (I see main in the docs, and the junk that I stuck in there). You want to introduce into this a way to create *errors* when existing ddocs don't have to even parse correctly?

The D spec says what DDoc is, but doesn't say that anything has to conform to ddoc (indeed, someone might even use another doc system and have their comments written for it), or even that DDoc will fail to produce documentation if it's not correct!

-Steve

On Wednesday, 13 November 2019 at 15:40:34 UTC, Adam D. Ruppe wrote:
> So not identical, but this is well defined by the D language spec!

Well defined, but it requires the programmer to know those defined rules.
I wasn't sure whether in my first example there would be a double space between the two 'Word's, but it turns out there is, and now I need to account for that when printing the text after getting it with __traits(docComment).

I don't have this problem when doing documentation generation, I just do `dub build --build=docs` and I get html pages that look fine (no double spaces), without needing to know what happens under the hood.

> Is that "* word" supposed to ignore the star? dmd ignores it which is probably most consistent with the spec. My adrdox does NOT ignore that and can use it for markdown-style lists...

That's probably going to give new bug reports too. ("Why did my list bullets disappear with __traits(docComment)? I see it just fine when rendering with adrdox!")

On Wednesday, 13 November 2019 at 18:02:35 UTC, Dennis wrote:
> I don't have this problem when doing documentation generation, I just do `dub build --build=docs` and I get html pages that look fine (no double spaces), without needing to know what happens under the hood.

That's because HTML ignores extra spaces. If you were to do __traits(docComment) and output it to an HTML file, you'd see that same result. The space is still there, it is just rendered differently by the target environment.

All __traits(docComment) should do is return the source code after the compiler read it. (actually, I wouldn't even mind if it was the whole comment, in full, including the delimiters, but if the compiler already removes stars I'm ok with that too)

> That's probably going to give new bug reports too. ("Why did my list bullets disappear with __traits(docComment)? I see it just fine when rendering with adrdox!")

Well, adrdox is not compatible with ddoc exactly, so I'd just accept that.