June 20, 2013
On 03/18/2013 01:00 AM, Walter Bright wrote:
> With the new -cov=nnn (not pulled yet) and -main, we can now add to the build process a minimum bar for unit test coverage.
> 
> The current state looks like this, the percentages are the current coverage amounts. It's not bad, but there's a lot of low hanging fruit ready for pull requests!
> 
>         $(DMD) -cov=79 -unittest -main -run std\random.d

I've submitted a pull request raising that to 91% :-)

Can I suggest tweaking the make files to print code coverage percentages when running the Phobos unittests?  Would be useful to check if there are any unexpected drops or just to have a quick glance where could be useful to try and help out.
June 20, 2013
On 6/20/13 5:19 PM, Joseph Rushton Wakeling wrote:
> On 03/18/2013 01:00 AM, Walter Bright wrote:
>> With the new -cov=nnn (not pulled yet) and -main, we can now add to the build
>> process a minimum bar for unit test coverage.
>>
>> The current state looks like this, the percentages are the current coverage
>> amounts. It's not bad, but there's a lot of low hanging fruit ready for pull
>> requests!
>>
>>          $(DMD) -cov=79 -unittest -main -run std\random.d
>
> I've submitted a pull request raising that to 91% :-)

Awes.

> Can I suggest tweaking the make files to print code coverage percentages when
> running the Phobos unittests?  Would be useful to check if there are any
> unexpected drops or just to have a quick glance where could be useful to try and
> help out.

Nice! Could you please submit an enhancement request?


Andrei
June 20, 2013
On 06/20/2013 10:59 PM, Andrei Alexandrescu wrote:
> Awes.

Auto-testing failed, but I think not due to me :-(

    setting up remote topull -> https://github.com/WebDrake/phobos.git

    fetching contents of https://github.com/WebDrake/phobos.git
    error: RPC failed; result=22, HTTP code = 405
    fatal: The remote end hung up unexpectedly

Does it re-try in the event of pull failures like this?

>> Can I suggest tweaking the make files to print code coverage percentages when running the Phobos unittests?  Would be useful to check if there are any unexpected drops or just to have a quick glance where could be useful to try and help out.
> 
> Nice! Could you please submit an enhancement request?

Done: http://d.puremagic.com/issues/show_bug.cgi?id=10430

June 20, 2013
On Thursday, June 20, 2013 23:14:42 Joseph Rushton Wakeling wrote:
> On 06/20/2013 10:59 PM, Andrei Alexandrescu wrote:
> > Awes.
> 
> Auto-testing failed, but I think not due to me :-(
> 
> setting up remote topull -> https://github.com/WebDrake/phobos.git
> 
> fetching contents of https://github.com/WebDrake/phobos.git
> error: RPC failed; result=22, HTTP code = 405
> fatal: The remote end hung up unexpectedly
> 
> Does it re-try in the event of pull failures like this?

It'll retry eventually (at least after changes have been merged into the main repository), but if it's been failing, then it's at the bottom of the queue.

- Jonathan M Davis
June 20, 2013
On 06/20/2013 11:41 PM, Jonathan M Davis wrote:
> It'll retry eventually (at least after changes have been merged into the main repository), but if it's been failing, then it's at the bottom of the queue.

Meh, no worries.  I can always add a patch removing those superfluous
assert(false)s and take it back to the top, right? :-)

No, not right away.  If by tomorrow afternoon it's still languishing I might think about it. :-P

June 20, 2013
On Thursday, June 20, 2013 23:50:00 Joseph Rushton Wakeling wrote:
> On 06/20/2013 11:41 PM, Jonathan M Davis wrote:
> > It'll retry eventually (at least after changes have been merged into the main repository), but if it's been failing, then it's at the bottom of the queue.
> Meh, no worries. I can always add a patch removing those superfluous
> assert(false)s and take it back to the top, right? :-)

I believe that new commits in the pull (and possibly rebasing the pull) do cause a pull request to go up in priority in the pull tester.

- Jonathan M Davis
June 21, 2013
On 2013-06-20 23:19, Joseph Rushton Wakeling wrote:

> I've submitted a pull request raising that to 91% :-)
>
> Can I suggest tweaking the make files to print code coverage percentages when
> running the Phobos unittests?  Would be useful to check if there are any
> unexpected drops or just to have a quick glance where could be useful to try and
> help out.

I thought that flag was supposed to make the test fail if the coverage gets lower than specified? It would be nice to have the value printed to see if the coverage is increased.

-- 
/Jacob Carlborg
June 21, 2013
On Monday, 18 March 2013 at 01:00:43 UTC, Walter Bright wrote:
> The current state looks like this, the percentages are the current coverage amounts. It's not bad, but there's a lot of low hanging fruit ready for pull requests!
>
>         $(DMD) -cov=57 -unittest -main -run std\uri.d

Took that is a chance to become a contributor to Phobos and submitted a pull request [0]. Am I supposed to file something in Bugzilla or just wait for someone to look at my request?


[0] https://github.com/D-Programming-Language/phobos/pull/1359
June 21, 2013
On 06/21/2013 09:33 AM, Jacob Carlborg wrote:
> I thought that flag was supposed to make the test fail if the coverage gets lower than specified? It would be nice to have the value printed to see if the coverage is increased.

-cov=nnn tests aren't (AFAICS) implemented as part of make unittest, nor is
plain -cov.

I think a minimum acceptable threshold is necessary but not sufficient -- say your minimum code coverage is 85%, it's still most likely unacceptable if your coverage drops (say) from 92% to 87%.

Anyway, the main benefit I see in printing the percentages isn't for testing purposes (though it's handy) but in advertising the existence and usefulness of code coverage analysis, and giving developers a nudge as to where and what to work on :-)
June 21, 2013
On 2013-06-21 11:26, Joseph Rushton Wakeling wrote:

> -cov=nnn tests aren't (AFAICS) implemented as part of make unittest, nor is
> plain -cov.

I thought that was the idea. Perhaps it's not finished yet.

> I think a minimum acceptable threshold is necessary but not sufficient -- say
> your minimum code coverage is 85%, it's still most likely unacceptable if your
> coverage drops (say) from 92% to 87%.

I would say that if the -cov=nnn flag works as I described you need to update the percent as soon as you increase the coverage. In the above case the flag should have said -cov=92 and not 85.

> Anyway, the main benefit I see in printing the percentages isn't for testing
> purposes (though it's handy) but in advertising the existence and usefulness of
> code coverage analysis, and giving developers a nudge as to where and what to
> work on :-)

I don't mind it :) It's better if it can be enforced.

-- 
/Jacob Carlborg