On Mon, 24 Jan 2011 16:07:38 -0500, Andrej Mitrovic <andrej.mitrovich@gmail.com> wrote:

> It's often the case that you want documentation examples to be short,
> but also correct. But you still want to write complex unittests that
> you don't want to put in the documentation. Sounds like a perfect
> candidate for named unittests:
>
> unittest(ddoc)
> {
>    // outputted in documentation
> }
>
> Here "ddoc" would be a predefined identifier, kind of like X86 is for
> version() statements.
>
> unittest // our own complex unittest that we don't want to output in
> the documentation
> {
>     // ...
> }

This would require a language change.

I like the ddoc comment version Andrei posted better.

-Steve

On 1/24/11 3:36 PM, Steven Schveighoffer wrote:
> On Mon, 24 Jan 2011 16:03:24 -0500, Andrei Alexandrescu
> <SeeWebsiteForEmail@erdani.org> wrote:
>> I find documented unittests attractive mainly because they're
>> _simple_. As soon as we start to add that kind of stuff... exponential
>> decay.
>
> It's only not simple if you want it to be. The /** Example: */ simple
> method is also covered. Let's also not forget that the end result is
> generated documentation, not the comments. All this 'non-simplicity' is
> going to be hidden there.

I think I worked too much with Walter because I'm almost thinking on his behalf. The thing is, at this point Walter (and me too) has a sympathy for language changes that remove undue limitations, and an aversion for language changes that introduce new stuff that the user would learn.

So if we go to Walter with: "Hey, we can currently document a variety of declaration, but not unittests. Let's allow documenting unittests as well" he'd be like, "heh, that sounds great... okay". But if we go to him with "hey, here's this new feature that needs these syntactic additions and these semantics and and has several nontrivial effects and..." then the chance of adoptions are seriously harmed.

Let's stick with KISS. Though I agree there are many improvements that can be brought to ddoc, I don't want a new feature, only to be able to document unittests.


Andrei

Steven Schveighoffer <schveiguy@yahoo.com> wrote:

> If DIP9 is accepted (writeTo), then showing examples of how the format specifiers work would certainly look less confusing via writefln.

I thought this was what std.format was for.

-- 
Simen

On Mon, 24 Jan 2011 16:36:21 -0500, Steven Schveighoffer <schveiguy@yahoo.com> wrote:

> On Mon, 24 Jan 2011 16:03:24 -0500, Andrei Alexandrescu <SeeWebsiteForEmail@erdani.org> wrote:

>> Yah, that is an issue. For examples that do non-unittesty stuff (e.g. writeln, use sockets etc.) we can still use the old-style documentation.
>
> That sounds reasonable, although I still think we need to be able to compile these to prevent doc rot.

Something that might work (should be compiled during ddoc, right?)

/**
Example
*/

unittest
{
  version(DDoc) // should not be outputted to example!
  {
      auto s = new Socket;
      s.connect("192.168.50.5");
      ...
  }
}

-Steve

On 01/25/2011 12:16 AM, Nick Sabalausky wrote:
> "Andrej Mitrovic"<andrej.mitrovich@gmail.com>  wrote in message
> news:mailman.910.1295903266.4748.digitalmars-d@puremagic.com...
>> It's often the case that you want documentation examples to be short,
>> but also correct. But you still want to write complex unittests that
>> you don't want to put in the documentation. Sounds like a perfect
>> candidate for named unittests:
>>
>> unittest(ddoc)
>> {
>>    // outputted in documentation
>> }
>>
>> Here "ddoc" would be a predefined identifier, kind of like X86 is for
>> version() statements.
>>
>> unittest // our own complex unittest that we don't want to output in
>> the documentation
>> {
>>     // ...
>> }
>
> That also provides a good solution for unittests that belong in the examples
> of more than one item:
>
> unittest(foo, bar)
> {
>    // Use both foo and bar
>    // outputted in documentation
> }
>
>
>
...and now you need to also maintain annotation list in addition to unittest body, and keep both in sync. Reminds me a prototype-implementation idiom from some languages.

Andrei Alexandrescu <SeeWebsiteForEmail@erdani.org> wrote:

>> Documentation is a reference, not a novel. If someone looked up the
>> documentation for "bar", why make them jump over to "foo" (and make sure
>> they know to do so) to see bar's examples?
>
> Then there better are two examples, one focused on foo and the other on bar. Anyway, I don't see the need for such a feature. All I want is to have certain unittests appear in the generated documentation.

Consider fopen and family - would you not say that an example like the
below is good enough for both fopen and fclose?

FILE* pFile;
pFile = fopen( "myfile.txt", "w" );
if ( pFile != null ) {
    fputs ("fopen example",pFile);
    fclose (pFile);
}

-- 
Simen

On Mon, 24 Jan 2011 16:43:39 -0500, Andrei Alexandrescu <SeeWebsiteForEmail@erdani.org> wrote:

> On 1/24/11 3:36 PM, Steven Schveighoffer wrote:
>> On Mon, 24 Jan 2011 16:03:24 -0500, Andrei Alexandrescu
>> <SeeWebsiteForEmail@erdani.org> wrote:
>>> I find documented unittests attractive mainly because they're
>>> _simple_. As soon as we start to add that kind of stuff... exponential
>>> decay.
>>
>> It's only not simple if you want it to be. The /** Example: */ simple
>> method is also covered. Let's also not forget that the end result is
>> generated documentation, not the comments. All this 'non-simplicity' is
>> going to be hidden there.
>
> I think I worked too much with Walter because I'm almost thinking on his behalf. The thing is, at this point Walter (and me too) has a sympathy for language changes that remove undue limitations, and an aversion for language changes that introduce new stuff that the user would learn.
>
> So if we go to Walter with: "Hey, we can currently document a variety of declaration, but not unittests. Let's allow documenting unittests as well" he'd be like, "heh, that sounds great... okay". But if we go to him with "hey, here's this new feature that needs these syntactic additions and these semantics and and has several nontrivial effects and..." then the chance of adoptions are seriously harmed.
>
> Let's stick with KISS. Though I agree there are many improvements that can be brought to ddoc, I don't want a new feature, only to be able to document unittests.

My proposed enhancement is fully backwards-compatible, so it's no biggie if it's not in the first iteration.  I was simply thinking in terms of my projects where I use unit tests to test several functions that interact with eachother, those would make nice examples.  But I could only attach them to one function (or repeat them).

-Steve

On Monday, January 24, 2011 13:35:22 Andrei Alexandrescu wrote:
> On 1/24/11 3:16 PM, Nick Sabalausky wrote:
> > "Andrej Mitrovic"<andrej.mitrovich@gmail.com>  wrote in message news:mailman.910.1295903266.4748.digitalmars-d@puremagic.com...
> > 
> >> It's often the case that you want documentation examples to be short, but also correct. But you still want to write complex unittests that you don't want to put in the documentation. Sounds like a perfect candidate for named unittests:
> >> 
> >> unittest(ddoc)
> >> {
> >> 
> >>    // outputted in documentation
> >> 
> >> }
> >> 
> >> Here "ddoc" would be a predefined identifier, kind of like X86 is for
> >> version() statements.
> >> 
> >> unittest // our own complex unittest that we don't want to output in
> >> the documentation
> >> {
> >> 
> >>     // ...
> >> 
> >> }
> > 
> > That also provides a good solution for unittests that belong in the examples of more than one item:
> > 
> > unittest(foo, bar)
> > {
> > 
> >    // Use both foo and bar
> >    // outputted in documentation
> > 
> > }
> 
> Why make everything complicated? The simplest feature request becomes a syntactic and semantic clusterfrak.

I'm _very_ interested in having name unit tests, but I see no reason to conflate named unit tests and ddoc examples. I think that that would very quickly make named unit tests far less useful than they would be otherwise - especially if you go so far as to try and make the unittest block name have any connection to the name of the function that the block is supposed to be the example fore.

Simply doing something like /++ Ditto +/ only now it's /++ Examples +/ or /++ Example +/ seems far better to me. There's no need to complicate things here or conflate to separate concepts.

- Jonathan M Davis

Steven Schveighoffer napisał:

>> BTW I consider this a very important topic. We have _plenty_ of examples that don't work and are not mechanically verifiable. The reasons range from minor typos to language changes to implementation limitations. Generally this is what they call "documentation rot". This is terrible PR for the language.
>>
>> Changing ddoc to recognize documentation unittests would fix this matter once and forever.
>>
>> Last but not least, the "----" separators for code samples are awful because no editor recognizes them for anything - they confuse the hell out of Emacs for one thing.
> 
> This only makes sense if:
> 
> 1. The unit test immediately follows the item being documented 2. The unit test *only* tests that item.
> 
> The second one could be pretty annoying.  Consider cases where several functions interact (I've seen this many times on Microsoft's Documentation), and it makes sense to make one example that covers all of them.  Having them 'testable' means creating several identical unit tests.
> 
> One way to easily fix this is to allow an additional parameter to the comment:
> 
> /**
> Example(Foo.foo(int), Foo.bar(int)):
> */
> unittest
> {
>     auto foo = new Foo;
>     foo.foo(5);
>     foo.bar(6);
>     assert(foo.toString() == "bazunga!");
> }
> 
> The above means, copy the example to both Foo.foo(int) and
> Foo.bar(int)
> 
> An alternative that is more verbose, but probably more understandable:
> 
> /**
> Example:
> Covers Foo.foo(int)
> Covers Foo.bar(int)
> */
> 
> Of course, a lack of target just means it applies to the item just documented.

Although coming from good intentions, it's just.. too much. The original idea is very compelling without add-ons.

Often the interacting functions are members of the same class or at least same module, so it's enough to place the unittest appropriately. To cover remaining cases an artificial declaration may be introduced.

/// Uses of Foo.foo(int) and Foo.bar(int)
struct foo_and_bar_examples;

/// Example:
unittest { ... }

Both functions would simply link to the artificial symbol in their ddocs.

> One other thing, using writefln is considered bad form in unit tests (you want *no* output if the unit test works).  But many examples might want to demonstrate how e.g. an object interacts with writefln.  Any suggestions? The assert line above is not very pretty for example...

I was thinking of mockFile.writefln(obj) but not sure if std.stdio can
handle it.

--
Tomek

On Mon, Jan 24, 2011 at 3:43 PM, Andrei Alexandrescu < SeeWebsiteForEmail@erdani.org> wrote:

> On 1/24/11 3:36 PM, Steven Schveighoffer wrote:
>
>> On Mon, 24 Jan 2011 16:03:24 -0500, Andrei Alexandrescu <SeeWebsiteForEmail@erdani.org> wrote:
>>
>>> I find documented unittests attractive mainly because they're _simple_. As soon as we start to add that kind of stuff... exponential decay.
>>>
>>
>> It's only not simple if you want it to be. The /** Example: */ simple method is also covered. Let's also not forget that the end result is generated documentation, not the comments. All this 'non-simplicity' is going to be hidden there.
>>
>
> I think I worked too much with Walter because I'm almost thinking on his behalf. The thing is, at this point Walter (and me too) has a sympathy for language changes that remove undue limitations, and an aversion for language changes that introduce new stuff that the user would learn.
>
> So if we go to Walter with: "Hey, we can currently document a variety of declaration, but not unittests. Let's allow documenting unittests as well" he'd be like, "heh, that sounds great... okay". But if we go to him with "hey, here's this new feature that needs these syntactic additions and these semantics and and has several nontrivial effects and..." then the chance of adoptions are seriously harmed.
>
> Let's stick with KISS. Though I agree there are many improvements that can be brought to ddoc, I don't want a new feature, only to be able to document unittests.


Here's another approach:
When you think about it, what we're after is tagging unit tests as code
examples, so why not do exactly that with annotations?

@example(Foo.bar)
unittest {
//...
}

It means there are no new keywords to worry about, and this feels more like a job for metadata than a language feature to me. We'd probably have to formalize adding parameters to annotations, but that's pretty much inevitable if they're ever going to reach a useful state.