On Friday, 5 December 2014 at 13:43:51 UTC, H. S. Teoh via Digitalmars-d wrote:
> On Fri, Dec 05, 2014 at 09:27:15AM +0000, Paulo Pinto via Digitalmars-d wrote:
>> On Friday, 5 December 2014 at 02:25:20 UTC, Walter Bright wrote:
> [...]
> you have to hire humans to
> sit all day repeating the same sequence of mouse clicks just to make
> sure the latest dev build is still working properly. That's grossly
> inefficient and a waste of money spent hiring the employee.
>

Since this is a public forum, the best I allowed to say is search for sweet shop coding.

Over a beer I would say something else.

--
Paulo

On Friday, 5 December 2014 at 13:06:14 UTC, H. S. Teoh via Digitalmars-d wrote:
> On Fri, Dec 05, 2014 at 02:39:07AM +0000, deadalnix via Digitalmars-d wrote:
>> On Friday, 5 December 2014 at 02:25:20 UTC, Walter Bright wrote:
> [...]
>> >From the article:
>> >
>> >"Most importantly, the kinds of bugs that people introduce most often
>> >aren’t the kind of bugs that unit tests catch. With few exceptions
>> >(such as parsers), unit tests are a waste of time."
>> >
>> >Not my experience with unittests, repeated over decades and with
>> >different languages. Unit tests are a huge win, even with statically
>> >typed languages.
>> 
>> Well truth to be said, if you don't test, you don't know there is a
>> bug. Therefore there is no bug.
>
> Yeah, back in my C/C++ days, I also thought unittests were a waste of
> time. But after having been shamed into writing unittests in D ('cos
> they are just sooo easy to write I ran out of excuses not to), I started
> realizing to my horror at how many bugs are actually in my code -- all
> kinds of corner cases that I missed, typos that slipped past compiler
> checks, etc.. More times than I'm willing to admit, I've revised and
> revised my code to perfection and "proven" (in my head) that it's
> correct, only to run it and have it fail the unittests because my brain
> has unconsciously tuned out a big glaring typo staring me right in the
> face. Had this been in C/C++, the bug wouldn't have been discovered
> until much later.
>
> That said, though, for unittests to be actually useful, you sometimes
> need to change your coding style. Certain kinds of coding style doesn't
> lend itself well to unittesting -- for example, deeply-nested loops that
> are very hard to reach into from a unittest, because it may not be
> immediately obvious how a unittest might test a rare, tricky
> if-condition buried 3 levels inside nested loops. Usually, such code is
> actually *never* tested because it's too hard to test -- it's a rare
> error-condition that doesn't happen with good input (and how many times
> we succumbed to the temptation of thinking the program is only ever
> given well-formed input, with disastrous results), too rare to justify
> the effort of crafting a unittest that would actually trigger it.
>
> This is where range-based component programming becomes an extremely
> powerful idiom -- separating out the logical parts of a complex piece of
> code so that there are no longer deeply-nested loops with hard-to-reach
> conditions, but everything is brought to the forefront where they can be
> easily verified with simple unittests.
>
> But, some people may not be willing to change the way they think about
> their coding problem in order to code in a testable way like this. So
> they may well resort to trying to rationalize away the usefulness of
> unittests. Well, the loss is their own, as the lack of unittesting will
> only result in poorer quality of their code, whereas those who are less
> arrogant will benefit by developing a much better track record of code
> correctness. :-)
>
>
> T

I wish it was arrogance or unwillingness to change, because you can work on that. If I were arrogant or unwilling to learn, I wouldn't have gone for D which keeps kicking me in the backend relentlessly.

I introduced uint tests a while ago (because it's sooo easy in D) but I've failed to maintain them. I found out that most of the bugs weren't caught in the unit tests but where somewhere else further down in the logic (more on that later). I repeat my point that we put into the unit tests what think will or won't work, just as you said:

> I've revised and
> revised my code to perfection and "proven" (in my head) that it's
> correct, only to run it and have it fail the unittests because my brain
> has unconsciously tuned out a big glaring typo staring me right in the
> face.

Unit tests are just another way  of "proving in your head". In D I work a lot with components and ranges, and I f**king love it. While it is true that this approach to programming makes each unit easily testable, I've found out that they are no guard against major f**k ups further down in a program's logic. At the end of the day, you have to design a general test suit for the whole program and see if it works. So why bother to test every unit, which may work perfectly fine on its own, when you have to test the whole shebang anyway.

As I said, I'm not against unit tests and I use them where they make sense (difficult output, not breaking existing tested code). But I often don't bother with them when they tell me what I already know.

assert(addNumbers(1,1) == 2);

I've found myself in the position when unit tests give me a false sense of security.

What Russel said, that we should think about breaking the code. It's true, but extremely hard to do when you _create_ something. You create something to make it work and not to destroy it. It also kills your imagination. It's like writing a thesis when you spend most of the time preparing for possible attacks from the examiners rather than thinking innovatively. I don't think any technology would have been invented, if people had thought of how to destroy it at the same time. That's what you (and others) think about after you've created it.

On 12/5/14, 8:53 AM, Chris wrote:
> On Friday, 5 December 2014 at 09:27:16 UTC, Paulo  Pinto wrote:
>> On Friday, 5 December 2014 at 02:25:20 UTC, Walter Bright wrote:
>>> On 12/4/2014 5:32 PM, ketmar via Digitalmars-d wrote:
>
> Now is the right time to confess. I hardly ever use unit tests although
> it's included (and encouraged) in D. Why? When I write new code I "unit
> test" as I go along, with
>
> debug writefln("result %s", result);
>
> and stuff like this. Stupid? Unprofessional? I don't know. It works.

You should trying writing a compiler without unit tests.

On 12/5/14, 9:42 AM, Chris wrote:
> On Friday, 5 December 2014 at 12:06:55 UTC, Nemanja Boric wrote:
>>> The good thing about unit tests is that they tell you when you break
>>> existing code.
>>
>> That's the great thing about unittests, and the reason why I write
>> unittests. I work on a fairly complex code base and every now and then
>> there's a new feature requested. Implementing features involves
>> several to dozen of modules to be changed, and there's no way that I
>> could guarantee that feature implementation didn't change behaviour of
>> the existing code. I both hate and love when I `make` compiles and
>> unittest fails.
>>
>>> But you'll realize that soon enough anyway.
>>
>> This is not good enough for me. Sometimes "soon enough" means week or
>> two before somebody actually notice the bug in the implementation
>> (again, very complex project that's simply not hand-testable), and
>> that's definitively not soon enough keeping in mind amount of $$$ that
>> you wasted into air.
>>
>>
>>
>> On Friday, 5 December 2014 at 11:53:11 UTC, Chris wrote:
>>> On Friday, 5 December 2014 at 09:27:16 UTC, Paulo  Pinto wrote:
>>>> On Friday, 5 December 2014 at 02:25:20 UTC, Walter Bright wrote:
>>>>> On 12/4/2014 5:32 PM, ketmar via Digitalmars-d wrote:
>>>>>>> http://www.teamten.com/lawrence/writings/java-for-everything.html
>>>>>> i didn't read the article, but i bet that this is just another
>>>>>> article
>>>>>> about his language of preference and how any other language he tried
>>>>>> doesn't have X or Y or Z. and those X, Y and Z are something like
>>>>>> "not
>>>>>> being on market for long enough", "vendor ACME didn't ported
>>>>>> ACMElib to
>>>>>> it", "out staff is trained in G but not in M" and so on. boring.
>>>>>>
>>>>>
>>>>> From the article:
>>>>>
>>>>> "Most importantly, the kinds of bugs that people introduce most
>>>>> often aren’t the kind of bugs that unit tests catch. With few
>>>>> exceptions (such as parsers), unit tests are a waste of time."
>>>>>
>>>>> Not my experience with unittests, repeated over decades and with
>>>>> different languages. Unit tests are a huge win, even with
>>>>> statically typed languages.
>>>>
>>>> Yes, but they cannot test everything. GUI code is specially ugly as
>>>> it requires UI automation tooling.
>>>>
>>>> They do exist, but only enterprise customers are willing to pay for it.
>>>>
>>>> This is why WPF has UI automation built-in.
>>>>
>>>> The biggest problem with unit tests are managers that want to see
>>>> shiny reports, like those produced by tools like Sonar.
>>>>
>>>> Teams than spend ridiculous amount of time writing superfluous unit
>>>> tests just to match milestone targets.
>>>>
>>>> Just because code has tests, doesn't mean the tests are testing what
>>>> they should. But if they reach the magical percentage number then
>>>> everyone is happy.
>>>>
>>>> --
>>>> Paulo
>>>
>>> Now is the right time to confess. I hardly ever use unit tests
>>> although it's included (and encouraged) in D. Why? When I write new
>>> code I "unit test" as I go along, with
>>>
>>> debug writefln("result %s", result);
>>>
>>> and stuff like this. Stupid? Unprofessional? I don't know. It works.
>>> I once started to write unit tests only to find out that indeed they
>>> don't catch bugs, because you only put into unit tests what you know
>>> (or expect) at a given moment (just like the old writefln()). The
>>> bugs I, or other people, discover later would usually not be caught
>>> by unit tests simply because you write for your own expectations at a
>>> given moment and don't realize that there are millions of other ways
>>> to go astray. So the bugs are usually due to a lack of imagination or
>>> a tunnel vision at the moment of writing code. This will be reflected
>>> in the unit tests as well. So why bother? You merely enshrine your
>>> own restricted and circular logic in "tests". Which reminds me of
>>> maths when teachers would tell us "And see, it makes perfect sense!",
>>> yeah, because they laid down the rules themselves in the first place.
>>>
>>> The same goes for comparing your output to some "gold standard". The
>>> program claims to have an accuracy of 98%. Sure, because you wrote
>>> for the gold standard and not for the real world where it drastically
>>> drops to 70%.
>>>
>>> The good thing about unit tests is that they tell you when you break
>>> existing code. But you'll realize that soon enough anyway.
>
> Yes, yes, yes. Unit tests can be useful in cases like this. But I don't
> think that they are _the_ way to cope with bugs. It's more like "stating
> the obvious", and bugs are hardly ever obvious, else they wouldn't be bugs.

Unit tests are not for detecting bugs. They are only useful for:

1. Making sure things work (a bit).
2. Making sure things continue to work when you refactor or introduce new code.
3. When a new bug is found you can write a test for it that will make that bug impossible to ever resurrect.
4. Show how code is supposed to be used.

Again, their purpose is not to detect bugs, but to build more robust software.

On Friday, 5 December 2014 at 15:03:39 UTC, Ary Borenszweig wrote:
> On 12/5/14, 9:42 AM, Chris wrote:
>> On Friday, 5 December 2014 at 12:06:55 UTC, Nemanja Boric wrote:
>>>> The good thing about unit tests is that they tell you when you break
>>>> existing code.
>>>
>>> That's the great thing about unittests, and the reason why I write
>>> unittests. I work on a fairly complex code base and every now and then
>>> there's a new feature requested. Implementing features involves
>>> several to dozen of modules to be changed, and there's no way that I
>>> could guarantee that feature implementation didn't change behaviour of
>>> the existing code. I both hate and love when I `make` compiles and
>>> unittest fails.
>>>
>>>> But you'll realize that soon enough anyway.
>>>
>>> This is not good enough for me. Sometimes "soon enough" means week or
>>> two before somebody actually notice the bug in the implementation
>>> (again, very complex project that's simply not hand-testable), and
>>> that's definitively not soon enough keeping in mind amount of $$$ that
>>> you wasted into air.
>>>
>>>
>>>
>>> On Friday, 5 December 2014 at 11:53:11 UTC, Chris wrote:
>>>> On Friday, 5 December 2014 at 09:27:16 UTC, Paulo  Pinto wrote:
>>>>> On Friday, 5 December 2014 at 02:25:20 UTC, Walter Bright wrote:
>>>>>> On 12/4/2014 5:32 PM, ketmar via Digitalmars-d wrote:
>>>>>>>> http://www.teamten.com/lawrence/writings/java-for-everything.html
>>>>>>> i didn't read the article, but i bet that this is just another
>>>>>>> article
>>>>>>> about his language of preference and how any other language he tried
>>>>>>> doesn't have X or Y or Z. and those X, Y and Z are something like
>>>>>>> "not
>>>>>>> being on market for long enough", "vendor ACME didn't ported
>>>>>>> ACMElib to
>>>>>>> it", "out staff is trained in G but not in M" and so on. boring.
>>>>>>>
>>>>>>
>>>>>> From the article:
>>>>>>
>>>>>> "Most importantly, the kinds of bugs that people introduce most
>>>>>> often aren’t the kind of bugs that unit tests catch. With few
>>>>>> exceptions (such as parsers), unit tests are a waste of time."
>>>>>>
>>>>>> Not my experience with unittests, repeated over decades and with
>>>>>> different languages. Unit tests are a huge win, even with
>>>>>> statically typed languages.
>>>>>
>>>>> Yes, but they cannot test everything. GUI code is specially ugly as
>>>>> it requires UI automation tooling.
>>>>>
>>>>> They do exist, but only enterprise customers are willing to pay for it.
>>>>>
>>>>> This is why WPF has UI automation built-in.
>>>>>
>>>>> The biggest problem with unit tests are managers that want to see
>>>>> shiny reports, like those produced by tools like Sonar.
>>>>>
>>>>> Teams than spend ridiculous amount of time writing superfluous unit
>>>>> tests just to match milestone targets.
>>>>>
>>>>> Just because code has tests, doesn't mean the tests are testing what
>>>>> they should. But if they reach the magical percentage number then
>>>>> everyone is happy.
>>>>>
>>>>> --
>>>>> Paulo
>>>>
>>>> Now is the right time to confess. I hardly ever use unit tests
>>>> although it's included (and encouraged) in D. Why? When I write new
>>>> code I "unit test" as I go along, with
>>>>
>>>> debug writefln("result %s", result);
>>>>
>>>> and stuff like this. Stupid? Unprofessional? I don't know. It works.
>>>> I once started to write unit tests only to find out that indeed they
>>>> don't catch bugs, because you only put into unit tests what you know
>>>> (or expect) at a given moment (just like the old writefln()). The
>>>> bugs I, or other people, discover later would usually not be caught
>>>> by unit tests simply because you write for your own expectations at a
>>>> given moment and don't realize that there are millions of other ways
>>>> to go astray. So the bugs are usually due to a lack of imagination or
>>>> a tunnel vision at the moment of writing code. This will be reflected
>>>> in the unit tests as well. So why bother? You merely enshrine your
>>>> own restricted and circular logic in "tests". Which reminds me of
>>>> maths when teachers would tell us "And see, it makes perfect sense!",
>>>> yeah, because they laid down the rules themselves in the first place.
>>>>
>>>> The same goes for comparing your output to some "gold standard". The
>>>> program claims to have an accuracy of 98%. Sure, because you wrote
>>>> for the gold standard and not for the real world where it drastically
>>>> drops to 70%.
>>>>
>>>> The good thing about unit tests is that they tell you when you break
>>>> existing code. But you'll realize that soon enough anyway.
>>
>> Yes, yes, yes. Unit tests can be useful in cases like this. But I don't
>> think that they are _the_ way to cope with bugs. It's more like "stating
>> the obvious", and bugs are hardly ever obvious, else they wouldn't be bugs.
>
> Unit tests are not for detecting bugs. They are only useful for:
>
> 1. Making sure things work (a bit).
> 2. Making sure things continue to work when you refactor or introduce new code.
> 3. When a new bug is found you can write a test for it that will make that bug impossible to ever resurrect.
> 4. Show how code is supposed to be used.
>
> Again, their purpose is not to detect bugs, but to build more robust software.

I completely agree with all your points. Point 4 I forgot to mention, I often look at the unit tests in D source code to see how it is supposed to be used. All I'm saying is that sometimes unit tests are sold as the be all end all anti-bug design. I think they should be used sensibly not everywhere.

On 12/5/14, 12:11 PM, Chris wrote:
> On Friday, 5 December 2014 at 15:03:39 UTC, Ary Borenszweig wrote:
>> On 12/5/14, 9:42 AM, Chris wrote:
>>> On Friday, 5 December 2014 at 12:06:55 UTC, Nemanja Boric wrote:
>>>>> The good thing about unit tests is that they tell you when you break
>>>>> existing code.
>>>>
>>>> That's the great thing about unittests, and the reason why I write
>>>> unittests. I work on a fairly complex code base and every now and then
>>>> there's a new feature requested. Implementing features involves
>>>> several to dozen of modules to be changed, and there's no way that I
>>>> could guarantee that feature implementation didn't change behaviour of
>>>> the existing code. I both hate and love when I `make` compiles and
>>>> unittest fails.
>>>>
>>>>> But you'll realize that soon enough anyway.
>>>>
>>>> This is not good enough for me. Sometimes "soon enough" means week or
>>>> two before somebody actually notice the bug in the implementation
>>>> (again, very complex project that's simply not hand-testable), and
>>>> that's definitively not soon enough keeping in mind amount of $$$ that
>>>> you wasted into air.
>>>>
>>>>
>>>>
>>>> On Friday, 5 December 2014 at 11:53:11 UTC, Chris wrote:
>>>>> On Friday, 5 December 2014 at 09:27:16 UTC, Paulo  Pinto wrote:
>>>>>> On Friday, 5 December 2014 at 02:25:20 UTC, Walter Bright wrote:
>>>>>>> On 12/4/2014 5:32 PM, ketmar via Digitalmars-d wrote:
>>>>>>>>> http://www.teamten.com/lawrence/writings/java-for-everything.html
>>>>>>>> i didn't read the article, but i bet that this is just another
>>>>>>>> article
>>>>>>>> about his language of preference and how any other language he
>>>>>>>> tried
>>>>>>>> doesn't have X or Y or Z. and those X, Y and Z are something like
>>>>>>>> "not
>>>>>>>> being on market for long enough", "vendor ACME didn't ported
>>>>>>>> ACMElib to
>>>>>>>> it", "out staff is trained in G but not in M" and so on. boring.
>>>>>>>>
>>>>>>>
>>>>>>> From the article:
>>>>>>>
>>>>>>> "Most importantly, the kinds of bugs that people introduce most
>>>>>>> often aren’t the kind of bugs that unit tests catch. With few
>>>>>>> exceptions (such as parsers), unit tests are a waste of time."
>>>>>>>
>>>>>>> Not my experience with unittests, repeated over decades and with
>>>>>>> different languages. Unit tests are a huge win, even with
>>>>>>> statically typed languages.
>>>>>>
>>>>>> Yes, but they cannot test everything. GUI code is specially ugly as
>>>>>> it requires UI automation tooling.
>>>>>>
>>>>>> They do exist, but only enterprise customers are willing to pay
>>>>>> for it.
>>>>>>
>>>>>> This is why WPF has UI automation built-in.
>>>>>>
>>>>>> The biggest problem with unit tests are managers that want to see
>>>>>> shiny reports, like those produced by tools like Sonar.
>>>>>>
>>>>>> Teams than spend ridiculous amount of time writing superfluous unit
>>>>>> tests just to match milestone targets.
>>>>>>
>>>>>> Just because code has tests, doesn't mean the tests are testing what
>>>>>> they should. But if they reach the magical percentage number then
>>>>>> everyone is happy.
>>>>>>
>>>>>> --
>>>>>> Paulo
>>>>>
>>>>> Now is the right time to confess. I hardly ever use unit tests
>>>>> although it's included (and encouraged) in D. Why? When I write new
>>>>> code I "unit test" as I go along, with
>>>>>
>>>>> debug writefln("result %s", result);
>>>>>
>>>>> and stuff like this. Stupid? Unprofessional? I don't know. It works.
>>>>> I once started to write unit tests only to find out that indeed they
>>>>> don't catch bugs, because you only put into unit tests what you know
>>>>> (or expect) at a given moment (just like the old writefln()). The
>>>>> bugs I, or other people, discover later would usually not be caught
>>>>> by unit tests simply because you write for your own expectations at a
>>>>> given moment and don't realize that there are millions of other ways
>>>>> to go astray. So the bugs are usually due to a lack of imagination or
>>>>> a tunnel vision at the moment of writing code. This will be reflected
>>>>> in the unit tests as well. So why bother? You merely enshrine your
>>>>> own restricted and circular logic in "tests". Which reminds me of
>>>>> maths when teachers would tell us "And see, it makes perfect sense!",
>>>>> yeah, because they laid down the rules themselves in the first place.
>>>>>
>>>>> The same goes for comparing your output to some "gold standard". The
>>>>> program claims to have an accuracy of 98%. Sure, because you wrote
>>>>> for the gold standard and not for the real world where it drastically
>>>>> drops to 70%.
>>>>>
>>>>> The good thing about unit tests is that they tell you when you break
>>>>> existing code. But you'll realize that soon enough anyway.
>>>
>>> Yes, yes, yes. Unit tests can be useful in cases like this. But I don't
>>> think that they are _the_ way to cope with bugs. It's more like "stating
>>> the obvious", and bugs are hardly ever obvious, else they wouldn't be
>>> bugs.
>>
>> Unit tests are not for detecting bugs. They are only useful for:
>>
>> 1. Making sure things work (a bit).
>> 2. Making sure things continue to work when you refactor or introduce
>> new code.
>> 3. When a new bug is found you can write a test for it that will make
>> that bug impossible to ever resurrect.
>> 4. Show how code is supposed to be used.
>>
>> Again, their purpose is not to detect bugs, but to build more robust
>> software.
>
> I completely agree with all your points. Point 4 I forgot to mention, I
> often look at the unit tests in D source code to see how it is supposed
> to be used. All I'm saying is that sometimes unit tests are sold as the
> be all end all anti-bug design. I think they should be used sensibly not
> everywhere.

This is very true. Specially when mocks come into play, sometimes test become duplicated code and every time you make changes in your codebase you have to go and change the expected behaviour of mocks, which is just tedious and useless.

On Friday, 5 December 2014 at 15:25:19 UTC, Ary Borenszweig wrote:
> On 12/5/14, 12:11 PM, Chris wrote:
>> On Friday, 5 December 2014 at 15:03:39 UTC, Ary Borenszweig wrote:
>>> On 12/5/14, 9:42 AM, Chris wrote:
>>>> On Friday, 5 December 2014 at 12:06:55 UTC, Nemanja Boric wrote:
>>>>>> The good thing about unit tests is that they tell you when you break
>>>>>> existing code.
>>>>>
>>>>> That's the great thing about unittests, and the reason why I write
>>>>> unittests. I work on a fairly complex code base and every now and then
>>>>> there's a new feature requested. Implementing features involves
>>>>> several to dozen of modules to be changed, and there's no way that I
>>>>> could guarantee that feature implementation didn't change behaviour of
>>>>> the existing code. I both hate and love when I `make` compiles and
>>>>> unittest fails.
>>>>>
>>>>>> But you'll realize that soon enough anyway.
>>>>>
>>>>> This is not good enough for me. Sometimes "soon enough" means week or
>>>>> two before somebody actually notice the bug in the implementation
>>>>> (again, very complex project that's simply not hand-testable), and
>>>>> that's definitively not soon enough keeping in mind amount of $$$ that
>>>>> you wasted into air.
>>>>>
>>>>>
>>>>>
>>>>> On Friday, 5 December 2014 at 11:53:11 UTC, Chris wrote:
>>>>>> On Friday, 5 December 2014 at 09:27:16 UTC, Paulo  Pinto wrote:
>>>>>>> On Friday, 5 December 2014 at 02:25:20 UTC, Walter Bright wrote:
>>>>>>>> On 12/4/2014 5:32 PM, ketmar via Digitalmars-d wrote:
>>>>>>>>>> http://www.teamten.com/lawrence/writings/java-for-everything.html
>>>>>>>>> i didn't read the article, but i bet that this is just another
>>>>>>>>> article
>>>>>>>>> about his language of preference and how any other language he
>>>>>>>>> tried
>>>>>>>>> doesn't have X or Y or Z. and those X, Y and Z are something like
>>>>>>>>> "not
>>>>>>>>> being on market for long enough", "vendor ACME didn't ported
>>>>>>>>> ACMElib to
>>>>>>>>> it", "out staff is trained in G but not in M" and so on. boring.
>>>>>>>>>
>>>>>>>>
>>>>>>>> From the article:
>>>>>>>>
>>>>>>>> "Most importantly, the kinds of bugs that people introduce most
>>>>>>>> often aren’t the kind of bugs that unit tests catch. With few
>>>>>>>> exceptions (such as parsers), unit tests are a waste of time."
>>>>>>>>
>>>>>>>> Not my experience with unittests, repeated over decades and with
>>>>>>>> different languages. Unit tests are a huge win, even with
>>>>>>>> statically typed languages.
>>>>>>>
>>>>>>> Yes, but they cannot test everything. GUI code is specially ugly as
>>>>>>> it requires UI automation tooling.
>>>>>>>
>>>>>>> They do exist, but only enterprise customers are willing to pay
>>>>>>> for it.
>>>>>>>
>>>>>>> This is why WPF has UI automation built-in.
>>>>>>>
>>>>>>> The biggest problem with unit tests are managers that want to see
>>>>>>> shiny reports, like those produced by tools like Sonar.
>>>>>>>
>>>>>>> Teams than spend ridiculous amount of time writing superfluous unit
>>>>>>> tests just to match milestone targets.
>>>>>>>
>>>>>>> Just because code has tests, doesn't mean the tests are testing what
>>>>>>> they should. But if they reach the magical percentage number then
>>>>>>> everyone is happy.
>>>>>>>
>>>>>>> --
>>>>>>> Paulo
>>>>>>
>>>>>> Now is the right time to confess. I hardly ever use unit tests
>>>>>> although it's included (and encouraged) in D. Why? When I write new
>>>>>> code I "unit test" as I go along, with
>>>>>>
>>>>>> debug writefln("result %s", result);
>>>>>>
>>>>>> and stuff like this. Stupid? Unprofessional? I don't know. It works.
>>>>>> I once started to write unit tests only to find out that indeed they
>>>>>> don't catch bugs, because you only put into unit tests what you know
>>>>>> (or expect) at a given moment (just like the old writefln()). The
>>>>>> bugs I, or other people, discover later would usually not be caught
>>>>>> by unit tests simply because you write for your own expectations at a
>>>>>> given moment and don't realize that there are millions of other ways
>>>>>> to go astray. So the bugs are usually due to a lack of imagination or
>>>>>> a tunnel vision at the moment of writing code. This will be reflected
>>>>>> in the unit tests as well. So why bother? You merely enshrine your
>>>>>> own restricted and circular logic in "tests". Which reminds me of
>>>>>> maths when teachers would tell us "And see, it makes perfect sense!",
>>>>>> yeah, because they laid down the rules themselves in the first place.
>>>>>>
>>>>>> The same goes for comparing your output to some "gold standard". The
>>>>>> program claims to have an accuracy of 98%. Sure, because you wrote
>>>>>> for the gold standard and not for the real world where it drastically
>>>>>> drops to 70%.
>>>>>>
>>>>>> The good thing about unit tests is that they tell you when you break
>>>>>> existing code. But you'll realize that soon enough anyway.
>>>>
>>>> Yes, yes, yes. Unit tests can be useful in cases like this. But I don't
>>>> think that they are _the_ way to cope with bugs. It's more like "stating
>>>> the obvious", and bugs are hardly ever obvious, else they wouldn't be
>>>> bugs.
>>>
>>> Unit tests are not for detecting bugs. They are only useful for:
>>>
>>> 1. Making sure things work (a bit).
>>> 2. Making sure things continue to work when you refactor or introduce
>>> new code.
>>> 3. When a new bug is found you can write a test for it that will make
>>> that bug impossible to ever resurrect.
>>> 4. Show how code is supposed to be used.
>>>
>>> Again, their purpose is not to detect bugs, but to build more robust
>>> software.
>>
>> I completely agree with all your points. Point 4 I forgot to mention, I
>> often look at the unit tests in D source code to see how it is supposed
>> to be used. All I'm saying is that sometimes unit tests are sold as the
>> be all end all anti-bug design. I think they should be used sensibly not
>> everywhere.
>
> This is very true. Specially when mocks come into play, sometimes test become duplicated code and every time you make changes in your codebase you have to go and change the expected behaviour of mocks, which is just tedious and useless.

Thanks for saying that. That's my experience too, especially when a module is under heavy development with frequent changes.

On Friday, 5 December 2014 at 14:53:43 UTC, Chris wrote:
>
> As I said, I'm not against unit tests and I use them where they make sense (difficult output, not breaking existing tested code). But I often don't bother with them when they tell me what I already know.
>
> assert(addNumbers(1,1) == 2);
>
> I've found myself in the position when unit tests give me a false sense of security.
>
Sure, you need to test the obvious things, but I find the real gains come from being able to verify the behaviour of edge cases and pathological input; and, critically, ensuring that that behaviour doesn't change as you refactor.  (My day job involves writing and maintaining legacy network libraries and parsers in pure C.  D's clean and easy unit tests would be a godsend for me.)

-Wyatt

On Friday, 5 December 2014 at 13:56:27 UTC, H. S. Teoh via Digitalmars-d wrote:
> On Fri, Dec 05, 2014 at 11:53:10AM +0000, Chris via Digitalmars-d wrote:

>
> At my day job, you'd be shocked to know how many times things flat-out
> break in the nastiest, most obvious ways, yet people DO NOT EVEN
> NOTICE!!!! QA has developed this bad habit of only testing the feature
> they were asked to test, and the developers have become complacent over
> time and blindly trusting that QA has done their job,

That's not trust or complaceny. That's a worldwide conspiracy to ensure good paid jobs for poor programmers & testers like us...

On Friday, 5 December 2014 at 15:44:35 UTC, Wyatt wrote:
> On Friday, 5 December 2014 at 14:53:43 UTC, Chris wrote:
>>
>> As I said, I'm not against unit tests and I use them where they make sense (difficult output, not breaking existing tested code). But I often don't bother with them when they tell me what I already know.
>>
>> assert(addNumbers(1,1) == 2);
>>
>> I've found myself in the position when unit tests give me a false sense of security.
>>
> Sure, you need to test the obvious things,

Everywhere? For each function? It may be desirable but hard to maintain. Also, unit tests break when you change the behavior of a function, then you have to redesign the unit test for this particular function. I prefer unit tests for bigger chunks.

> but I find the real gains come from being able to verify the behaviour of edge cases and pathological input; and, critically, ensuring that that behaviour doesn't change as you refactor.  (My day job involves writing and maintaining legacy network libraries and parsers in pure C.  D's clean and easy unit tests would be a godsend for me.)
>
> -Wyatt

True, true. Unfortunately, the edge cases are usually spotted when using the software, not in unit tests. They can be included later, but new pathological input keeps coming up (especially if you write for third party software).

Now don't get me wrong, I wouldn't want to miss unit tests in D, but I use them more carefully now, not everywhere.