November 13, 2019
On Wednesday, 13 November 2019 at 18:46:43 UTC, Adam D. Ruppe wrote:
> That's because HTML ignores extra spaces.

Of course, but the overall point I'm trying to make is that the trait is non-trivial and adds language complexity.

It gets implemented one way, and then someone comes around wanting it to be different. "please give me the html instead of raw comment, parsing it with CTFE is slow and cumbersome"
"please include the /** or /++, I'm using __traits(docComment) to statically assert my preferred documentation code style"
"is it a bug that leading/trailing whitespace is/isn't trimmed?"
"the previous changes broke my existing CTFE documentation parser"
"__traits(docComment) does not work when the comment is in a mixin that is seen later in semantic analysis"
"I want only my doc comment of my function to depend on the build type, but putting the comment inside version blocks does not work! How to fix?"

This can of worms can stay closed by just sticking with good old predictable and flexible UDA strings.
November 13, 2019
On Monday, 11 November 2019 at 17:16:20 UTC, Steven Schveighoffer wrote:
> On 11/8/19 9:00 AM, Adam D. Ruppe wrote:
>> Writing code to wrap some Phobos functions for a script language and need documentation to be displayed at runtime...
>> 
>> Would be really nice to just say if(other_docs.length == 0) return __traits(docComment, item).
>> 
>> A UDA can populate other_docs in some functions, but when wrapping libraries you can't add that. Having the doc comment available through reflection means we can reuse these existing items.
>
> I still am very VERY leery of having anything in comments affect the code in any way.
>
> What about a compromise? Provide access to the docs but ONLY at runtime. That is, you will not know anything about the string at compiletime, just at runtime, and you can make your decisions there.
>
> Should work for your use case.
>
> -Steve

Compiler flag as we do with string imports?  Most projects won't use this feature.  If it does end up being used then at least it will be clear.  It's difficult to save people from themselves completely - and I think there are many more likely ways to end up abusing the powerful features of D.

Suppose in scenario 1 you use doc comment traits and in scenario 2 you have an extra build stage to achieve the same effect; and suppose through unfortunate choices the documentation ends up affecting the runtime behaviour of the code in non-documentation related ways.  Are you better off in scenario 1 or 2?  It's not clear, but 1 might be better.

D has goto - people don't habitually abuse it.
November 13, 2019
On Wednesday, 13 November 2019 at 19:15:56 UTC, Dennis wrote:
> Of course, but the overall point I'm trying to make is that the trait is non-trivial and adds language complexity.

It doesn't have to be. All I want is the existing string.

The ONLY addition I am calling for is that a trait docComment (or whatever) returns the ddoc string attached to the declaration (and if the existing implementation strips delimiters, so be it, and if not, so be it, the spec does define stripping them so i assume it does but I don't really care).

Everything else is already defined or out of scope, so we really shouldn't be worrying about them.

> It gets implemented one way, and then someone comes around wanting it to be different. "please give me the html instead of raw comment

Impossible, since that's undefined.

> "is it a bug that leading/trailing whitespace is/isn't trimmed?"

Clear answer to this: no, it is not a bug. That's defined to be there.

> "the previous changes broke my existing CTFE documentation parser"

ddoc is part of the spec and it has had breaking changes, we just deal with it.

> "__traits(docComment) does not work when the comment is in a mixin that is seen later in semantic analysis"

This is similarly already an open question with the language. Should __traits(allMembers) return stuff from mixins?

> "I want only my doc comment of my function to depend on the build type, but putting the comment inside version blocks does not work! How to fix?"

This is also already well-defined, version applies to declarations, doc comments are part of a declaration, not a separate entity.
November 13, 2019
On Wednesday, 13 November 2019 at 19:44:38 UTC, Adam D. Ruppe wrote:
> The ONLY addition I am calling for is that a trait docComment (or whatever) returns the ddoc string attached to the declaration

That may be the only thing _you_ want, but then someone else comes along who wants something different.
All Manu wanted was a mangle-only `extern(C++, "ns")` as an additional option, so simple. Why was Walter initially so resistant for such a harmless addition? Then it got merged.

https://github.com/dlang/dmd/pull/8667

Then the issues come in.

https://issues.dlang.org/show_bug.cgi?id=19278
https://issues.dlang.org/show_bug.cgi?id=19339
https://issues.dlang.org/show_bug.cgi?id=19557
https://issues.dlang.org/show_bug.cgi?id=19574
https://issues.dlang.org/show_bug.cgi?id=19893
https://issues.dlang.org/show_bug.cgi?id=19920

And someone wants to deprecate the old way.

https://github.com/dlang/dmd/pull/10031

I'm not saying it wasn't worth it in that case, but those fictional remarks about __traits(docComment) were supposed to show that the trait will be ANOTHER source of enhancement requests, introduces ANOTHER thing that can get queried during any point of semantic analysis, probably making it ANOTHER source of DMD bugs, and it will be ANOTHER source of confusion / material for D learners to swallow.

That's just how it goes despite how simple it may look, and the question is whether the new trait justifies this cost. Not every code annoyance warrants a new feature, sometimes it's better to e.g. duplicate a few lines of code instead of introducing another C-macro.

Likewise, in this case having some duplication in UDA's or having another build step may be the easier solution in the end.
November 13, 2019
On 11/13/19 2:30 PM, Laeeth Isharc wrote:
> On Monday, 11 November 2019 at 17:16:20 UTC, Steven Schveighoffer wrote:
>> I still am very VERY leery of having anything in comments affect the code in any way.
>>
>> What about a compromise? Provide access to the docs but ONLY at runtime. That is, you will not know anything about the string at compiletime, just at runtime, and you can make your decisions there.
>>
>> Should work for your use case.
>>
> 
> Compiler flag as we do with string imports?  Most projects won't use this feature.  If it does end up being used then at least it will be clear.  It's difficult to save people from themselves completely - and I think there are many more likely ways to end up abusing the powerful features of D.

I'm trying to save the reviewers and the maintainers. Code review shouldn't include reviewing stuff that isn't compiled. If you are doing weird imports, that's an unusual situation and requires unusual knowledge.

If you put this into traits, then it's a usual and easy tool, and now you have to verify comments are correctly written.

If all you want is to fetch the documentation for a function, it can be done in better ways.

> 
> Suppose in scenario 1 you use doc comment traits and in scenario 2 you have an extra build stage to achieve the same effect; and suppose through unfortunate choices the documentation ends up affecting the runtime behaviour of the code in non-documentation related ways.  Are you better off in scenario 1 or 2?  It's not clear, but 1 might be better.

For sure, it's possible already to run extra steps to have *anything* affect the code. You could use a preprocessor that replaces ddocs with UDAs for the same effect. But in that case, you are altering the input to the compiler, not using the compiler to build something that isn't normal compiler input.

Of course, you can already do such things with string imports.

> D has goto - people don't habitually abuse it.

This is more akin to mixins, not goto. Goto is actually easy to review (automatic rejection! /jk), you know exactly what it does.

And yes, mixins are abused. I just did some work to the diet-ng project to make it possible to update non-interpolated HTML and have it figured out at runtime[1]. The way diet imports non-D code and changes it into D code to mixin, it's cool, but I'd consider it an abuse, and really difficult to comprehend. I'm getting to the point where I just think an external tool is better[2].

-Steve

[1] https://github.com/rejectedsoftware/diet-ng/pull/70
[2] https://github.com/schveiguy/dietpc
November 13, 2019
On Wednesday, 13 November 2019 at 22:53:49 UTC, Steven Schveighoffer wrote:
> I'm trying to save the reviewers and the maintainers. Code review shouldn't include reviewing stuff that isn't compiled.

Documentation is a key component of code review. And any code reviewer would surely be skeptical of a __trait(getDoc) that does anything other than use it for documentation.

(and you could actually even static assert it is written reasonably with the trait but i could see that being hated too lol.)
November 13, 2019
On 11/13/19 6:03 PM, Adam D. Ruppe wrote:
> On Wednesday, 13 November 2019 at 22:53:49 UTC, Steven Schveighoffer wrote:
>> I'm trying to save the reviewers and the maintainers. Code review shouldn't include reviewing stuff that isn't compiled.
> 
> Documentation is a key component of code review. And any code reviewer would surely be skeptical of a __trait(getDoc) that does anything other than use it for documentation.

Where it starts becoming a problem is when users of a library say "you changed your docs, and it broke my code". This is always the argument against named parameters (you changed parameter X's name, and it broke my code).

Anything that becomes observable at compile-time now becomes part of the API.

Theoretically, even a runtime string could cause this. But there is much less possibility for abuse, and at least the compiler doesn't write *different* binary instructions based on comments.

> (and you could actually even static assert it is written reasonably with the trait but i could see that being hated too lol.)

Isn't that what commit hooks are for? ;)

-Steve
November 14, 2019
On Wednesday, 13 November 2019 at 20:38:07 UTC, Dennis wrote:
> On Wednesday, 13 November 2019 at 19:44:38 UTC, Adam D. Ruppe wrote:
>> The ONLY addition I am calling for is that a trait docComment (or whatever) returns the ddoc string attached to the declaration
>
> That may be the only thing _you_ want, but then someone else comes along who wants something different.
> All Manu wanted was a mangle-only `extern(C++, "ns")` as an additional option, so simple. Why was Walter initially so resistant for such a harmless addition? Then it got merged.
>
> https://github.com/dlang/dmd/pull/8667
>
> Then the issues come in.
>
> https://issues.dlang.org/show_bug.cgi?id=19278
> https://issues.dlang.org/show_bug.cgi?id=19339
> https://issues.dlang.org/show_bug.cgi?id=19557
> https://issues.dlang.org/show_bug.cgi?id=19574
> https://issues.dlang.org/show_bug.cgi?id=19893
> https://issues.dlang.org/show_bug.cgi?id=19920

IIRC there was a WIP pull request accompanied by a DIP, the pull request was merged while it was still being ironed out along with the DIP. Those were all easy fixes foir the most part. Hell there's still bugs being fixed regarding extern(C). With that kind of justification no new features should ever be added. Not to mention bug fixes that just cause no bugs. We should just stop making changes all together. Point to one of D's biggest mistakes, even then I would still say that's not a good enough reason to not make changes.

> And someone wants to deprecate the old way.
>
> https://github.com/dlang/dmd/pull/10031


They probably want to, because the old way is impeding the new method, you can't use CTFE to construct a string.


> I'm not saying it wasn't worth it in that case, but those fictional remarks about __traits(docComment) were supposed to show that the trait will be ANOTHER source of enhancement requests, introduces ANOTHER thing that can get queried during any point of semantic analysis, probably making it ANOTHER source of DMD bugs, and it will be ANOTHER source of confusion / material for D learners to swallow.
>
> That's just how it goes despite how simple it may look, and the question is whether the new trait justifies this cost. Not every code annoyance warrants a new feature, sometimes it's better to e.g. duplicate a few lines of code instead of introducing another C-macro.

Here's a counter example to your example: https://github.com/dlang/dmd/pull/10013

I wouldn't really say it's a new "feature". It'd just be exposing the information that is already available


> Likewise, in this case having some duplication in UDA's or

It's not just duplication, you'd have to modifying other people's source code. How would you duplicate this doc:

/// ditto

The implementation then becomes much more convoluted.

> having another build step may be the easier solution in the end.

Using dub? That's going to be a pain in the ass honestly. This sort of implementation you'd have to do isn't that simple. You'd have to effectively run a dub command twice, once to generate the doc output and then run it again to compile the code with the now available documentation. Dub wasn't designed for that, and build systems for C/C++ are often overly complicated because the languages don't expose information that the compiler already has available to it. Basically of what D does right with it's compile time features. __traits already has an atrocious eye catching name. If you need to find code somewhere that might be doing something with comments that it shouldn't be doing, it's not going to be that difficult to find.


November 14, 2019
On Wednesday, 13 November 2019 at 22:53:49 UTC, Steven Schveighoffer wrote:
> On 11/13/19 2:30 PM, Laeeth Isharc wrote:
>> 
>> Compiler flag as we do with string imports?  Most projects won't use this feature.  If it does end up being used then at least it will be clear.  It's difficult to save people from themselves completely - and I think there are many more likely ways to end up abusing the powerful features of D.
>
> I'm trying to save the reviewers and the maintainers. Code review shouldn't include reviewing stuff that isn't compiled. If you are doing weird imports, that's an unusual situation and requires unusual knowledge.

Our code review already includes Ddoc.

That was my point, really.

The compiler might not complain right now about invalid Ddoc, but I'd be happy if it did. It would not create extra effort, but save us effort.

November 14, 2019
On Thursday, 14 November 2019 at 03:17:38 UTC, Steven Schveighoffer wrote:
> On 11/13/19 6:03 PM, Adam D. Ruppe wrote:
>> On Wednesday, 13 November 2019 at 22:53:49 UTC, Steven Schveighoffer wrote:
>>> I'm trying to save the reviewers and the maintainers. Code review shouldn't include reviewing stuff that isn't compiled.
>> 
>> Documentation is a key component of code review. And any code reviewer would surely be skeptical of a __trait(getDoc) that does anything other than use it for documentation.
>
> Where it starts becoming a problem is when users of a library say "you changed your docs, and it broke my code". This is always the argument against named parameters (you changed parameter X's name, and it broke my code).
>

I'm in the camp of having parameters names "private by default" and requiring libraries to opt-in to exposing them.  By my estimate, named parameters are only useful about 15% of the time.  Up till now, D code is written assuming that parameter names are private.  They're inconsistent and not well thought out because they were never part of the function interface. We could come up with naming standards and go through all the code to fix this, which would be a big effort, but it's hard to justify if only a small percentage of functions would actually benefit from named parameters.  By making them "opt-in" you can fix these things before you expose them.

That being said, Ddoc comments differ from named parameters. DDoc comments are already exposed to the compiler and external tools.  Unlike parameter names, they are already written to be publicly consumed by external components.  Adding another access point via __traits is less significant since this information is already exposed to other external components. DDoc comments are also "opt-in".  You can use regular comments if you don't want to expose/maintain well-formatted metadata about your code.

I'd also be careful about thinking about DDoc comments like they are "comments". DDoc comments may be called comments but they don't fit the criteria of normal comments.  They have their own format (a mini language within D) and have lexical significance to the compiler. They actually behave more like user-defined attributes rather than normal comments.  I have a feeling that if they were called something else like "DDoc Metadata" rather than comments then there wouldn't be much resistance to exposing them via __traits.