On Friday, 8 May 2015 at 14:16:27 UTC, Kagamin wrote:
> On Friday, 8 May 2015 at 09:52:06 UTC, Jonathan M Davis wrote:
>> That's not to say that there's not value in some of those features, but I think that the language itself has a very good foundation for unit testing with covers the 90% case, and if others want to create their own unit testing frameworks to handle more complex cases, they're certainly free to do so.
>
> My estimation is 20% - they apply to simplistic self-contained libraries like phobos and warp. My impression is not much code falls in this category. Even dmd doesn't, and it's already rather simplistic.

When the tests aren't self-contained, you're not really doing unit tests anymore. That's getting more into component test territory and the like, and that's a whole other ballgame.

- Jonathan M Davis

On 2015-05-08 12:43, Walter Bright wrote:

> I've never seen any as easy to use as D's, and that includes Ruby. Easy
> to use is what makes it a game changer, because people are much more
> likely to use it.

As I've said, it's available in Ruby standard library, but perhaps that's not enough built-in for you.

Here's a simple Ruby unit test:

require 'minitest/autorun'

class TestMeme < MiniTest::Unit::TestCase
  def test_foo
    assert 1 == 1
  end
end

Run with "ruby foo.rb"

The D version:

module foo;

unittest
{
    assert(1 == 1);
}

Run with "dmd -main -unittest -run foo.d"

The Ruby version is slightly more complicated to type but easier to run. It's the opposite with D. The Ruby version will give you, out of the box:

* A nice report
* A bunch of assertions
* Setup and tear down methods
* Runs all tests even if a previous one failed

And some other stuff as well.

> I know from long experience that when a test framework comes with a
> manual, people tend to not use it. It's just the way people are. If it's
> not builtin, people also tend not to use it.
>
> I'm not claiming D has invented unit testing, nor that it is
> sophisticated. But it is trivial to use, and getting people to actually
> use it is what matters. It's like exercise - just getting out and doing
> something, anything, regularly is 90% of it.
>
> It doesn't matter how sophisticated a testing and coverage tool is if
> people aren't using it as a matter of course. I've seen enough tool
> manuals sitting on programmers' shelves for years still in their shrink
> wrap.
>
> For example, a well-known Ruby project, Rails. Picking a file pretty
> much at random:
>
> https://github.com/rails/rails/blob/master/actionmailer/lib/action_mailer/base.rb
>
>
> No unit tests in it. Poking around, I found this:
>
> https://github.com/rails/rails/blob/master/actionmailer/test/base_test.rb
>
> in another directory. I presume those are the tests for base.rb.

Yes.

> Why can't the tests be in base.rb?

It's the convention used in Ruby. It's perfectly possible to put the unit tests in the same file as the implementation, there's no technical limitation.

What do you do in D when it comes to integration and functional tests that test functionality across several modules?

In Ruby the convention is already to put the tests in the "test" directory so there's no special considerations in Ruby.

> I assume there is some 3rd file that connects the two.

Rails uses something called auto loading. If it can't find a class it will automatically try to require a file based on a convention. For plain Ruby you would just require whatever you want to test, as you would do in D, if you used a separate module.

> I really like having the unit tests adjacent to the
> function under test, for the same reason that the Ddoc is adjacent. It
> sounds trite, but having them all together and not in some other file
> hierarchy is a game changer. It certainly changed things in D (before
> Ddoc, Phobos documentation was an utter disaster).

I never liked to have the tests in the same file as the implementation. There's also all other kinds of tests that doesn't have a obvious implementation module to put them in. But Phobos doesn't have those kind of tests so the problem is avoided.

> DMD has a set of source files in one directory, and test files in
> another. There's no obvious correspondence between code and its
> corresponding test.

No, not the way those tests are written. But those are not unit tests, they're full stack tests.

> We don't actually know how good the test coverage
> is. I'm very much looking forward to ddmd in order to address this. I
> expect this will result in a large increase in quality.


-- 
/Jacob Carlborg

Thanks for the explanation.

On 5/8/2015 7:00 AM, Chris wrote:
> The only drawback is that sometimes the logic of a program does not allow to
> test every little bit, especially when handling files is concerned.

This is an interesting problem, one that I faced with Warp.

The solution was to make the function being tested a function template. Then, in the unit test I replace the 'file' argument to the function with a static array of test data.

The term for it is 'type mocking', and it's fairly well explored territory. What I find most intriguing about it is it results in D programs consisting largely of template functions rather than plain functions.

On 5/8/2015 7:03 AM, Chris wrote:
> The funny thing is that people keep complaining about the lack of tools for D,
> and when a tool is built into the language they say "That tool shouldn't be part
> of the language". Yet, if it were omitted, people would say "Why doesn't D have
> this tool built in?". Human nature, I guess.

I see it slightly differently. If the tool is built in to the language, people do not regard it as a tool anymore when preparing a mental checklist of "available tooling".

---- Warning! Another Boring Walter Cutaway -------------

It reminds me of back when we were selling the Zortech C++ compiler, we included complete runtime library source with the compiler. This was back in the days when most compilers' library source code was a closely held trade secret.

Nobody noticed that we included the runtime library source.

Then, one day, Borland decided to make their previously trade secret library source code available as a separate purchase. They did an amazing job marketing this, and journalists everywhere celebrated the forward thinking breakthrough. Even in magazine compiler roundup reviews, the journalists would breathlessly note that one could now buy Borland's library source code, but Zortech C++ including it for free was never mentioned.

We threw in the towel, and made the library source code a separately priced add on. This was a big success for us!

No, I'm not suggesting we unbundle unit testing, Ddoc, coverage analysis, profiling, etc., into separate tools for marketing purposes. I'm just bemused by how perceptions work.

------------------------------------------------------

On 5/8/2015 8:47 AM, Vladimir Panteleev wrote:
> On Thursday, 7 May 2015 at 22:27:57 UTC, Walter Bright wrote:
>> But let's not forget the meat and potatoes on our plate while looking at our
>> neighbor's salad dressing.
>
> I decided to line up our potatoes on a nice new wiki page:
>
> http://wiki.dlang.org/Development_tools

:-)

Thanks for doing this, it's nice.

On 5/8/2015 7:24 AM, Nick Sabalausky wrote:
>> a formatter.
>
> Granted, not built-in, but a pretty minor thing IMO.

I use detab and tolf as part of my checkin script :-)

Sometime, one got to be harsh, but the truth is that D have many mostly implemented features, often falling short in some cases or working in a way that make no sense.

On Thursday, 7 May 2015 at 22:27:57 UTC, Walter Bright wrote:
> On 5/7/2015 5:16 AM, d user wrote:
>> the truth is, one of the biggest things holding D back from mass adoption is the
>> complete lack of tooling compared to  basically every other mainstream language.
>
> D has some excellent tools that are generally nonstandard, klunky or nonexistent in other languages:
>
> 1. unit testing

The way D runs tests before running the app does not make any sense. The way you can't run them for a module without a main makes no sense. The way you end runnign all kind of test with the ones you are interested in makes no sense.

Yes D has easy syntax to make unitests. Meaning it is a great language in that regard.

But it has horrible tooling that can be regarded at best as a glorified hack that is mostly useless at scale.

> 2. documentation generation

The fact this is backed into the language the way it is mostly show a lack of separation of concerns. That is a side effect of the fact the frontend is completely monolithic.

You don't need to bake that into the language when the compiler can feed symbol information to 3rd party tools.

> 3. coverage analysis

I'm not sure how Go and Rust stand on that one, but all mainstream languages have descent code coverage, with the added bonus that this is very well integrated with the rest of the tooling (tolling which is non existent in D). I would not put this has a string point of D. Not bad either, but definitively what you'd expect from any serious language.

Last but not least, things like:
if (foo && bar) ...;

Contains 2 branches. The output we have to not provide data about this, which makes the coverage infos lacking in some regard.

In java for instance, I could run tests (and not the whole program with test at startup) and get all the coverage infos directly in my editor with colors and stuff, include things like short circuit for boolean operators, which we don't have. We are miles (kilometers damn it !) away from this level of support.

> 4. profiler

Same things, this is a standard tool nowaday for any language worth considering. In fact, even if we had no specific support in D for it, C/C++ tooling could probably do it for us to some extent.

> 5. and as of last week, a memory usage profiler
>

Good memory allocator like tcmalloc and/or jemalloc have detailed output. And if we go back to java, you can even observe things in a graphic way in real time concurrently to the application running ! Once again, we are not even remotely close to anything like this.

On 5/8/2015 2:51 PM, deadalnix wrote:
>> 1. unit testing
> The way D runs tests before running the app does not make any sense.

It works perfectly fine and obviates the need to create a separate test harness.


> The way you can't run them for a module without a main makes no sense.

    dmd foo -unittest -main


> The way you end runnign all kind of test with the ones you are interested in makes no sense.

   dmd std/path -unittest -main

runs just the unit tests in std/path. You can run tests in some modules, but not others, with:

   dmd -c a b c -unittest
   dmd d e f a.o
   ./d


>> 2. documentation generation
>
> The fact this is backed into the language the way it is mostly show a lack of
> separation of concerns.

It means it is always there and always the correct version.


> That is a side effect of the fact the frontend is completely monolithic.

Has nothing to do with how usable a tool is.


> You don't need to bake that into the language when the compiler can feed symbol
> information to 3rd party tools.

D worked with 3rd party doc tools before Ddoc. The result was that maybe 1 or 2 people ever used such tools. You weren't using D then - the Phobos documentation was all of:

1. nonexistent
2. wrong
3. had no correlation with the APIs on the functions at all

Ddoc fixed all that. It's fine to gripe about this or that, but the results are undeniably a seismic improvement for D.


>> 3. coverage analysis
>
> I'm not sure how Go and Rust stand on that one, but all mainstream languages
> have descent code coverage, with the added bonus that this is very well
> integrated with the rest of the tooling (tolling which is non existent in D). I
> would not put this has a string point of D. Not bad either, but definitively
> what you'd expect from any serious language.

Try using gcov without going back to consult the manuals on it. Even if you have it installed. Coverage analyzers in other languages that I checked all required finding and installing some extra package, then trying to figure out how to hook it in.


> Last but not least, things like:
> if (foo && bar) ...;
>
> Contains 2 branches. The output we have to not provide data about this, which
> makes the coverage infos lacking in some regard.

Check before assuming -cov does it wrong. You'll find it counts the branches separately. It does sum them for the line count, but writing it as:

   if (foo &&
       bar)
        ....

you'll get two counts. What I am disappointed in is the repeated default assumption, without bothering to check, that D's tools are guilty until proven innocent.


> In java for instance, I could run tests (and not the whole program with test at
> startup) and get all the coverage infos directly in my editor with colors and
> stuff, include things like short circuit for boolean operators, which we don't
> have. We are miles (kilometers damn it !) away from this level of support.

Here's a slice of a D coverage report:


       |static if (allSatisfy!(hasMobileElements, R))
       |{
       |    RvalueElementType moveAt(size_t index)
       |    {
      6|        foreach (i, Range; R)
       |        {
       |            static if (isInfinite!(Range))
       |            {
0000000|                return .moveAt(source[i], index);
       |            }
       |            else
       |            {
      3|                immutable length = source[i].length;
      5|                if (index < length) return .moveAt(source[i], index);
      1|                    index -= length;
       |            }
       |        }
0000000|        assert(false);
       |    }
       |}
std\range\package.d is 91% covered

It's not in color, I concede that. Saying this report is "miles behind" is ludicrous, besides the incorrect statement that short circuits are not supported.


>> 4. profiler
> Same things, this is a standard tool nowaday for any language worth considering.
> In fact, even if we had no specific support in D for it, C/C++ tooling could
> probably do it for us to some extent.

Yah, I know about gprof. Try it (with a C or C++ program), without spending time with the manuals. Here's the manual for D:

   dmd -profile foo


>> 5. and as of last week, a memory usage profiler
> Good memory allocator like tcmalloc and/or jemalloc have detailed output.

They don't come with C or C++. Pray you can find one that works with your version of the compiler on your platform, and there are no conflicts with 3rd party libraries. No such worries with D compilers.


> And if we go back to java, you can even observe things in a graphic way in real time
> concurrently to the application running ! Once again, we are not even remotely
> close to anything like this.

It's designed so that you can write one without changing anything in the compiler. All you gotta do is override the default one in druntime, which merely aggregates the statistics and prints a report. That report is 90% of what anyone needs. If you want to change the report generator to produce color html output, probably 30 min work, feel free.

On 5/8/15 3:58 PM, Walter Bright wrote:
> On 5/8/2015 2:51 PM, deadalnix wrote:
>>> 1. unit testing
>> The way D runs tests before running the app does not make any sense.
>
> It works perfectly fine and obviates the need to create a separate test
> harness.
>
>
>> The way you can't run them for a module without a main makes no sense.
>
>      dmd foo -unittest -main
>
>
>> The way you end runnign all kind of test with the ones you are
>> interested in makes no sense.
>
>     dmd std/path -unittest -main
>
> runs just the unit tests in std/path. You can run tests in some modules,
> but not others, with:
>
>     dmd -c a b c -unittest
>     dmd d e f a.o
>     ./d

The issues with unittests are legit, albeit fixable. It's goofy to run the program after unittests, and unittests should have names that can be introspected, selected etc. I couldn't find much merit with the rest of the list. -- Andrei