Lately, I find myself wondering, if I should add parameterized unit tests to std.string, because the last few bugs I fixed where not caught by tests, as the test-data was not good enough. I know random data is not perfect either, but it would be good addition IMO.

Additionally, I thought these unit tests could be used to benchmark the performance of the functions and the compilers. Thus allowing continues monitoring of performance changes.

I'm thinking of something like:

version(unittest_benchmark) {
unittest {
    auto ben = Benchmark("bench_result_file", numberOfRuns);
    auto values = ValueGen!(StringGen(0, 12), IntGen(2, 24))(ben);

    foreach(string rStr, int rInt; values) {
        auto rslt = functionToTest(rStr, rInt);
        // some asserts
    }
}
}

The bench_result_file would be a csv with e.g. date,runtime,...

ideas, suggestions?

On 10/8/14, 2:37 PM, Robert burner Schadek wrote:
> version(unittest_benchmark) {
> unittest {

No need for the outer braces :o). -- Andrei

On Wednesday, 8 October 2014 at 23:31:59 UTC, Andrei Alexandrescu wrote:
> On 10/8/14, 2:37 PM, Robert burner Schadek wrote:
>> version(unittest_benchmark) {
>> unittest {
>
> No need for the outer braces :o). -- Andrei

I just love my braces.
If that's gone be the most negative comment, I will have a PR ready before next week.

On 10/8/14, 4:44 PM, Robert burner Schadek wrote:
> On Wednesday, 8 October 2014 at 23:31:59 UTC, Andrei Alexandrescu wrote:
>> On 10/8/14, 2:37 PM, Robert burner Schadek wrote:
>>> version(unittest_benchmark) {
>>> unittest {
>>
>> No need for the outer braces :o). -- Andrei
>
> I just love my braces.

If you love your braces you gotta love your indentation. They come together... -- Andrei

On Thursday, 9 October 2014 at 00:13:25 UTC, Andrei Alexandrescu wrote:

> If you love your braces you gotta love your indentation. They come together... -- Andrei

I guess you're right, but back to the topic.
Has anyone tried something similar for phobos? (I couldn't find anything)
Does anyone else consider the idea worth pursuing?

On 10/9/14, 1:01 AM, Robert burner Schadek wrote:
> On Thursday, 9 October 2014 at 00:13:25 UTC, Andrei Alexandrescu wrote:
>
>> If you love your braces you gotta love your indentation. They come
>> together... -- Andrei
>
> I guess you're right, but back to the topic.
> Has anyone tried something similar for phobos? (I couldn't find anything)
> Does anyone else consider the idea worth pursuing?

Not yet but I think we need to create a task and a plan for adding the old std.benchmark to std.experimental.benchmark, and use it internally for Phobos. -- Andrei

Am Thu, 09 Oct 2014 08:11:33 -0700
schrieb Andrei Alexandrescu <SeeWebsiteForEmail@erdani.org>:

> On 10/9/14, 1:01 AM, Robert burner Schadek wrote:
> > On Thursday, 9 October 2014 at 00:13:25 UTC, Andrei Alexandrescu wrote:
> >
> >> If you love your braces you gotta love your indentation. They come together... -- Andrei
> >
> > I guess you're right, but back to the topic.
> > Has anyone tried something similar for phobos? (I couldn't find
> > anything) Does anyone else consider the idea worth pursuing?
> 
> Not yet but I think we need to create a task and a plan for adding the old std.benchmark to std.experimental.benchmark, and use it internally for Phobos. -- Andrei

+1, std.benchmark should be revived. There's still the constructor cycles issue but we can probably find workarounds. In the worst case we do not generate constructors but runBenchmark functions and call these manually via some runAllBenchmarks("std.xyz", "std.abc") template magic)

On 10/9/14, 9:36 AM, Johannes Pfau wrote:
> Am Thu, 09 Oct 2014 08:11:33 -0700
> schrieb Andrei Alexandrescu <SeeWebsiteForEmail@erdani.org>:
>
>> On 10/9/14, 1:01 AM, Robert burner Schadek wrote:
>>> On Thursday, 9 October 2014 at 00:13:25 UTC, Andrei Alexandrescu
>>> wrote:
>>>
>>>> If you love your braces you gotta love your indentation. They come
>>>> together... -- Andrei
>>>
>>> I guess you're right, but back to the topic.
>>> Has anyone tried something similar for phobos? (I couldn't find
>>> anything) Does anyone else consider the idea worth pursuing?
>>
>> Not yet but I think we need to create a task and a plan for adding
>> the old std.benchmark to std.experimental.benchmark, and use it
>> internally for Phobos. -- Andrei
>
> +1, std.benchmark should be revived. There's still the constructor
> cycles issue but we can probably find workarounds. In the worst
> case we do not generate constructors but runBenchmark functions and
> call these manually via some runAllBenchmarks("std.xyz", "std.abc")
> template magic)

My suggestion is to first use it in std.internal for a couple of months, and only then propose it for inclusion. -- Andrei

On Thursday, 9 October 2014 at 16:36:35 UTC, Johannes Pfau wrote:>
> +1, std.benchmark should be revived.


-1, This really does not solve my problem. I need a parameterized unit test facility aka. Haskell QuickCheck with the additional benchmarking feature that allows me/us to track performance over time aka. benchmark result dump to file in csv style manner.

On 10/9/14, 9:57 AM, Robert burner Schadek wrote:
> On Thursday, 9 October 2014 at 16:36:35 UTC, Johannes Pfau wrote:>
>> +1, std.benchmark should be revived.
>
>
> -1, This really does not solve my problem. I need a parameterized unit
> test facility aka. Haskell QuickCheck with the additional benchmarking
> feature that allows me/us to track performance over time aka. benchmark
> result dump to file in csv style manner.

version (xyz) unittest
{
   ...
}

is a good idiom. I noticed you feel strongly about bracing but I unrecommend it (and we should disallow it for phobos) as it forces you to either double-indent everything (net loss) or create an exception with regard to indentation. Also the opening extra brace has a "garden path sentence" liability, it suggests more stuff will be encompassed by the brace but then only at the closing brace there's the realization there's only encompassed entity.

The idiom is already used, see e.g. https://github.com/D-Programming-Language/phobos/blob/master/std/file.d#L322.


Andrei