On Fri, Mar 23, 2018 at 07:46:34PM +0000, Johan Engelen via Digitalmars-d-announce wrote:
> On Thursday, 22 March 2018 at 15:18:40 UTC, Jacob Carlborg wrote:
[...]
> > I completely agree. Although my reason is mostly because there will be too much code in a single file if the regular code and unit tests are mixed in the same file.
> 
> Fully agree with this "too much code in a single file" point. I am confident that part of the reason of Phobos unittesting being very incomplete, is that adding unittests further clutters the codebase. Moving all unittests to the bottom of the file (pulling them out of classes too) would resolve this issue in part.
[...]

I beg to differ.  From my experience with working on Phobos, file size or perceived "clutter" simply isn't an issue.  Good chunks of Phobos have long lists of unittests following an overload set, and often even interspersed between overload sets, and we have never shied away from adding more unittests to the list as bug fixes get added, nor have we ever felt the pressure to move the unittests elsewhere.

Using a code editor with modern capabilities helps to a great degree, of course.  If your primary way of navigating a source file is by paging up/down or by pulling a scrollbar, IMO you're doing it wrong.  You should rather be using actually useful navigation, like Vim's search / fold / jump to matching brace/parenthesis / jump to bookmark / ctags + jump to tag, etc., or whatever a GUI IDE's equivalents of these are. Dragging scrollbars around is so 90's, and simply isn't up to task for navigating and working with large files.

In my own projects, I regularly deal with 3000-line source files, and the size of the file has never been an issue as far as navigation is concerned.  Of course, there are other considerations as to why stuffing everything in one file might not be such a good idea, such as encapsulation, but certainly I have never been deterred because of the fear that adding unittests might make a file "too big".  When I do split a file into multiple modules, it's purely for encapsulation reasons rather than file size considerations.  In fact, I prefer to keep related code, e.g. code + unittest for that code, together in the same file, because otherwise things tend to go out-of-sync.  It's always easier to put related things together in one place, rather than scatter them across multiple places and then have to worry about keeping each piece in sync with the others.  If a well-encapsulated module contains 1000 lines of code and 5000 lines of unittests, then so much the better.

But of course, YMMV.


T

-- 
What did the alien say to Schubert? "Take me to your lieder."

On Fri, Mar 23, 2018 at 03:56:03PM -0400, Steven Schveighoffer via Digitalmars-d-announce wrote: [..]
> Note that a frequent complaint of std.datetime (at least when it was one module) is that the file was too big. While it does hold a lot of functionality, the majority of the file size is unittests. This means that it can be hard to surf the file for functionality.

Actually, I didn't have a problem with std.datetime being a huge file. What I *did* have a problem with was that it took too much memory to compile and the unittests took too long to run.  And also that it consisted of a bunch of somewhat loosely-coupled chunks that could be more meaningfully encapsulated as submodules.


> But on the flip side, there aren't a lot of datetime bugs!

Exactly.


> I personally believe that there should be unit tests for every function, and be right next to the function. I don't want to go on a search for such things, or have to rely on manual documentation to know what is testing what. It would be nice to have your editor hide the unit tests unless you want to work on them.

It's just a matter of configuring folding in an editor that supports it. Or dump your editor if it doesn't support folding, 'cos it sux and you deserve better. :-D  I mean, this is 2018, not 1995, we shouldn't have to be stuck with the handicap of navigating by scrollbar and paging up/down anymore.


> I've worked on a project where the testing was separated from the code, and it was a liability IMO. Things would get missed and not tested properly.
[...]

Yep.  As I mentioned elsewhere, recently I've had to resort to external testing for one of my projects, and I'm still working on that right now. And already, I'm seeing a liability: rather than quickly locating a unittest immediately following a particular function, now I have to remember "oh which subdirectory was it that the tests were put in? and which file was it that a particular test of this function was done?". It's an additional mental burden to have to keep doing the mapping between current source location <-> test code location (even if it's a 1-to-1 mapping), and a physical burden to have to continually open external files (and typing a potentially long path for them) rather than just "bookmark, jump to end of function, navigate unittest blocks" in the same file.

Fortunately, this problem is somewhat alleviated by having cumulative code coverage (see other thread), which can quickly identify what hasn't been tested yet.  Currently, with the help of coverage, I've managed to get 100% coverage for 4 of my modules, and 90%+ for another 5 modules, and working on the remainder right now.  It's an uphill battle at times, but already I've been reaping the benefits: buggy corner cases that I missed were discovered, fixed, and now have a corresponding test case / unittest where possible.


T

-- 
I've been around long enough to have seen an endless parade of magic new techniques du jour, most of which purport to remove the necessity of thought about your programming problem.  In the end they wind up contributing one or two pieces to the collective wisdom, and fade away in the rearview mirror. -- Walter Bright

On Friday, March 23, 2018 13:43:15 H. S. Teoh via Digitalmars-d-announce wrote:
> Yep.  As I mentioned elsewhere, recently I've had to resort to external testing for one of my projects, and I'm still working on that right now. And already, I'm seeing a liability: rather than quickly locating a unittest immediately following a particular function, now I have to remember "oh which subdirectory was it that the tests were put in? and which file was it that a particular test of this function was done?". It's an additional mental burden to have to keep doing the mapping between current source location <-> test code location (even if it's a 1-to-1 mapping), and a physical burden to have to continually open external files (and typing a potentially long path for them) rather than just "bookmark, jump to end of function, navigate unittest blocks" in the same file.

When I've done unit testing in C++, I've had the tests in separate files, and when I do that, I usually put all of the test files in the same place (e.g. include/ for the .h files, source/ for the .cpp files and tests/ for the .h and .cpp files with the tests) and have a 1-to-1 relationship between the .h/.cpp pair containing the code and the .h/.cpp pair containing the tests. Also, the test functions are usually named after the function that they're testing. So, it's all fairly organized, but it's still way more of a pain than just having the unittest block right after the function being tested, and it makes it easy for folks to just ignore the tests and easy for you to miss that something wasn't tested.

Having dealt with both that and putting all of the unit tests next to stuff in D, I find having the tests right after the functions to be _far_ more maintainable.

In fact, in std.datetime.interval, most of the tests are outside the templated types that they're testing in order to avoid having the tests included in every template instantiation, and that's turned out to be _way_ more of a pain to maintain than having the tests right after the functions. And that doesn't even involve a separate file.

Obviously, YMMV, but in my experience, having the tests _immediately_ after what they're testing is vastly more maintainable than having the tests elsewhere.

- Jonathan M Davis

On Friday, 23 March 2018 at 20:43:15 UTC, H. S. Teoh wrote:
>
>> I've worked on a project where the testing was separated from the code, and it was a liability IMO. Things would get missed and not tested properly.

That's where Test Driven Development comes in.

>
> Yep.  As I mentioned elsewhere, recently I've had to resort to external testing for one of my projects, and I'm still working on that right now. And already, I'm seeing a liability: rather than quickly locating a unittest immediately following a particular function, now I have to remember "oh which subdirectory was it that the tests were put in? and which file was it that a particular test of this function was done?". It's an additional mental burden to have to keep doing the mapping between current source location <-> test code location (even if it's a 1-to-1 mapping), and a physical burden to have to continually open external files (and typing a potentially long path for them) rather than just "bookmark, jump to end of function, navigate unittest blocks" in the same file.

There are pluses and minuses to both approaches, but I don't think that a separate file approach is as difficult as you are suggesting. The naming is typically identical to the project entities being tested,  with a prefix like "Test_" tacked onto the front of the project, modules, classes and functions, making finding things straightforward. And most modern editors/IDEs will allow multiple files and projects to be open at the same time, allowing test code to be opened only once per coding session.

On Fri, Mar 23, 2018 at 09:45:33PM +0000, Tony via Digitalmars-d-announce wrote:
> On Friday, 23 March 2018 at 20:43:15 UTC, H. S. Teoh wrote:
> > 
> > > I've worked on a project where the testing was separated from the code, and it was a liability IMO. Things would get missed and not tested properly.
> 
> That's where Test Driven Development comes in.

That's not an option when you have an existing codebase that you have to work with.  You basically have to start out with tons of code and no tests, and incrementally add them.  Having to also maintain a separate test tree mirroring the source tree is simply far too much overhead to be worth the effort.


[...]
> > Yep.  As I mentioned elsewhere, recently I've had to resort to external testing for one of my projects, and I'm still working on that right now.  And already, I'm seeing a liability: rather than quickly locating a unittest immediately following a particular function, now I have to remember "oh which subdirectory was it that the tests were put in? and which file was it that a particular test of this function was done?".  It's an additional mental burden to have to keep doing the mapping between current source location <-> test code location (even if it's a 1-to-1 mapping), and a physical burden to have to continually open external files (and typing a potentially long path for them) rather than just "bookmark, jump to end of function, navigate unittest blocks" in the same file.
> 
> There are pluses and minuses to both approaches, but I don't think that a separate file approach is as difficult as you are suggesting. The naming is typically identical to the project entities being tested,  with a prefix like "Test_" tacked onto the front of the project, modules, classes and functions, making finding things straightforward.

Well, in my case it wasn't so simple, because it's an external testsuite. Meaning the mapping of tests to function names is non-trivial, because you have to go through the interface language in order to get at the function at all.


> And most modern editors/IDEs will allow multiple files and projects to be open at the same time, allowing test code to be opened only once per coding session.
[...]

Being able to open multiple files or not, isn't the problem. (An editor that couldn't do that would be worthless as a coding editor!)  The problem lies in having to open multiple files in the first place, for what's essentially the same piece of functionality.  When I'm working with a large project, I already have enough files open to cross-reference stuff that depends on each other.  Having to open yet another file just to work with the tests is an additional, and IMO unnecessary, burden.  Even if your IDE can open 50 files at the same time, you still end up having to scroll through the list of 50 open files in order to find the one you're looking for -- which is not much different from having to open a new window and opening a currently-unopened file.

Whereas if the unittest is already right there *in the current file*, and better yet, *right next* to the function you're writing, you can just go to it immediately.  No additional fuss of typing filenames or navigating big lists of open files (and locating, within said additional file, the right place where the test is found).

Now of course, a sufficiently-powerful editor like Vim could let you define macros for automatically finding and opening the test file corresponding with your current location, and finding the unittest defined therein for you.  But still, it begs the question of why bother with such encumbrances when it's really not necessary in the first place?


T

-- 
When solving a problem, take care that you do not become part of the problem.

On Friday, 23 March 2018 at 22:32:50 UTC, H. S. Teoh wrote:
> On Fri, Mar 23, 2018 at 09:45:33PM +0000, Tony via Digitalmars-d-announce wrote:
>> On Friday, 23 March 2018 at 20:43:15 UTC, H. S. Teoh wrote:
>> > On Friday, 23 March 2018 at 19:56:03 UTC, Steven Schveighoffer wrote:
>> > > I've worked on a project where the testing was separated from the code, and it was a liability IMO. Things would get missed and not tested properly.
>> 
>> That's where Test Driven Development comes in.
>
> That's not an option when you have an existing codebase that you have to work with.  You basically have to start out with tons of code and no tests, and incrementally add them.  Having to also maintain a separate test tree mirroring the source tree is simply far too much overhead to be worth the effort.
>
I think that you could "Test Driven Develop" the code you are adding or changing.

On Friday, March 23, 2018 22:42:34 Tony via Digitalmars-d-announce wrote:
> On Friday, 23 March 2018 at 22:32:50 UTC, H. S. Teoh wrote:
> > On Fri, Mar 23, 2018 at 09:45:33PM +0000, Tony via
> >
> > Digitalmars-d-announce wrote:
> >> On Friday, 23 March 2018 at 20:43:15 UTC, H. S. Teoh wrote:
> >> > On Friday, 23 March 2018 at 19:56:03 UTC, Steven
> >> >
> >> > Schveighoffer wrote:
> >> > > I've worked on a project where the testing was separated
> >> > > from the code, and it was a liability IMO. Things would
> >> > > get missed and not tested properly.
> >>
> >> That's where Test Driven Development comes in.
> >
> > That's not an option when you have an existing codebase that you have to work with.  You basically have to start out with tons of code and no tests, and incrementally add them.  Having to also maintain a separate test tree mirroring the source tree is simply far too much overhead to be worth the effort.
>
> I think that you could "Test Driven Develop" the code you are adding or changing.

Insisting on writing the tests before writing the code doesn't help with the kind of situation that H. S. Teoh is describing. And arguably it exacerbates the problem. Regardless, it doesn't help when the code has already been written.

- Jonathan M Davis

On Saturday, 24 March 2018 at 00:12:23 UTC, Jonathan M Davis wrote:
> On Friday, March 23, 2018 22:42:34 Tony via Digitalmars-d-announce wrote:
>> On Friday, 23 March 2018 at 22:32:50 UTC, H. S. Teoh wrote:
>> > On Fri, Mar 23, 2018 at 09:45:33PM +0000, Tony via
>> >
>> > Digitalmars-d-announce wrote:
>> >> On Friday, 23 March 2018 at 20:43:15 UTC, H. S. Teoh wrote:
>> >> > On Friday, 23 March 2018 at 19:56:03 UTC, Steven
>> >> >
>> >> > Schveighoffer wrote:
>> >> > > I've worked on a project where the testing was separated
>> >> > > from the code, and it was a liability IMO. Things would
>> >> > > get missed and not tested properly.
>> >>
>> >> That's where Test Driven Development comes in.
>> >
>> > That's not an option when you have an existing codebase that you have to work with.  You basically have to start out with tons of code and no tests, and incrementally add them.  Having to also maintain a separate test tree mirroring the source tree is simply far too much overhead to be worth the effort.
>>
>> I think that you could "Test Driven Develop" the code you are adding or changing.
>
> Insisting on writing the tests before writing the code doesn't help with the kind of situation that H. S. Teoh is describing. And arguably it exacerbates the problem. Regardless, it doesn't help when the code has already been written.

I don't see how it exacerbates it and I don't see how it doesn't help. The point of Test-Driven Development it to make sure you have written a test for all your code. You can also do test-driven development in unittest blocks.

But as far as whether or not it can be done with maintenance code, my original reply that mentioned it was to someone who appeared to be talking about a new project not getting everything tested, not a maintenance project. So saying "can't do it for maintenance" doesn't even apply to my reply.

On Saturday, March 24, 2018 00:51:07 Tony via Digitalmars-d-announce wrote:
> On Saturday, 24 March 2018 at 00:12:23 UTC, Jonathan M Davis
>
> wrote:
> > On Friday, March 23, 2018 22:42:34 Tony via
> >
> > Digitalmars-d-announce wrote:
> >> On Friday, 23 March 2018 at 22:32:50 UTC, H. S. Teoh wrote:
> >> > On Fri, Mar 23, 2018 at 09:45:33PM +0000, Tony via
> >> >
> >> > Digitalmars-d-announce wrote:
> >> >> On Friday, 23 March 2018 at 20:43:15 UTC, H. S. Teoh wrote:
> >> >> > On Friday, 23 March 2018 at 19:56:03 UTC, Steven
> >> >> >
> >> >> > Schveighoffer wrote:
> >> >> > > I've worked on a project where the testing was separated
> >> >> > > from the code, and it was a liability IMO. Things would
> >> >> > > get missed and not tested properly.
> >> >>
> >> >> That's where Test Driven Development comes in.
> >> >
> >> > That's not an option when you have an existing codebase that you have to work with.  You basically have to start out with tons of code and no tests, and incrementally add them. Having to also maintain a separate test tree mirroring the source tree is simply far too much overhead to be worth the effort.
> >>
> >> I think that you could "Test Driven Develop" the code you are adding or changing.
> >
> > Insisting on writing the tests before writing the code doesn't help with the kind of situation that H. S. Teoh is describing. And arguably it exacerbates the problem. Regardless, it doesn't help when the code has already been written.
>
> I don't see how it exacerbates it and I don't see how it doesn't help. The point of Test-Driven Development it to make sure you have written a test for all your code. You can also do test-driven development in unittest blocks.

TDD makes it worse, because when you do TDD, you're constantly hopping between your code and the tests instead of just writing the code and then writing the tests. That still involves some back and forth, but it avoids having to hop back and forth while you're still designing the function, whereas with TDD, you have to go change the tests any time you change the function before the function is even done. But whatever. If you prefer TDD, then use it. A number of us have nothing good to say about TDD. You'd pretty much have to hold a gun to my head before I'd ever do anything along those lines. I'm all for writing tests, but I hate TDD.

Regardless, having the tests right next to the functions reduces how much hopping around you have to do to edit the code and tests, because you have everything in one place instead of in separate files.

> But as far as whether or not it can be done with maintenance code, my original reply that mentioned it was to someone who appeared to be talking about a new project not getting everything tested, not a maintenance project. So saying "can't do it for maintenance" doesn't even apply to my reply.

You were replying to H. S. Teoh talking about adding tests to an existing project, in which case, it's very much about maintenance.

- Jonathan M Davis

On Saturday, 24 March 2018 at 01:15:46 UTC, Jonathan M Davis wrote

> A number of us have nothing good to say about TDD.

That's fine. That's why they have menus in restaurants. But saying it is an inferior method is different than saying it won't work or can't be used in a maintenance situation.

On Saturday, 24 March 2018 at 01:15:46 UTC, Jonathan M Davis wrote:
>
>> But as far as whether or not it can be done with maintenance code, my original reply that mentioned it was to someone who appeared to be talking about a new project not getting everything tested, not a maintenance project. So saying "can't do it for maintenance" doesn't even apply to my reply.
>
> You were replying to H. S. Teoh talking about adding tests to an existing project, in which case, it's very much about maintenance.

I said my "original reply", meaning the one where I first mentioned Test-Driven Development. That was to something that Steven Schveighoffer said (although I did not reply directly to his message, but replied to his comment that was still in H.S. Teoh's message):

"I've worked on a project where the testing was separated from the code, and it was a liability IMO. Things would get missed and not tested properly."

He doesn't explicitly specify development or maintenance, but I assume it was development.