On Saturday, 3 May 2014 at 18:16:52 UTC, Andrei Alexandrescu wrote:
> On 5/3/14, 4:54 AM, Atila Neves wrote:
>> So I tried using unit-threaded to run Phobos unit tests
> [snip]
>
> Thanks. Are you using thread pooling (a limited number of threads e.g. 1.5 * cores running all unittests)? -- Andrei

I'm using parallel and taskPool from std.parallelism. I was under the impression it gave me a ready-to-use pool with as many threads as I have cores.

On Saturday, 3 May 2014 at 18:26:37 UTC, Walter Bright wrote:
> On 5/3/2014 10:22 AM, Atila Neves wrote:
>> I can reproduce the slower-with-threads issue without using my library. I've
>> included the source file below and would like to know if other people see the
>> same thing.
>
> I haven't investigated this, but my suspicions are:
>
> 1. thread creation/destruction is dominating the times.

In the current measurements probably since the whole run takes less than a second. But the first ones I did were dozens of seconds long, so I don't think so.

>
> 2. since very few of the unittests block, there is no speed advantage from having more threads than cores. If you limit the number of threads to the number of cores on your machine, you might see a speedup.

Like I mentioned above, unless I'm mistaken taskPool should be using a correct number of threads for my machine already.

03-May-2014 21:22, Atila Neves пишет:
> I can reproduce the slower-with-threads issue without using my library.
> I've included the source file below and would like to know if other
> people see the same thing.
>
> The Phobos modules are all called "ustd" because I couldn't/didn't know
> how to get this to work otherwise. So I copied the std/*.d files to a
> directory called ustd and changed their module declarations. Silly but
> it works. I'd love to know how to do this properly.

[snip]

>      if(single) {
>          foreach(test; tests) {
>              test();
>          }
>      } else {
>          foreach(test; tests.parallel) {

Try different batch size:
test.parallel(1), test.parallel(2) etc.



-- 
Dmitry Olshansky

>>     if(single) {
>>         foreach(test; tests) {
>>             test();
>>         }
>>     } else {
>>         foreach(test; tests.parallel) {
>
> Try different batch size:
> test.parallel(1), test.parallel(2) etc.

So as to not have thread creation be disproportionately represented, I repeated the module list over and over again, making the number of tests run equal to 9990. This takes 5s on my machine to run in on thread and 12s in multiple. Here are the things I tried:

1. Created my own TaskPool so I could decide how many threads to use
2. Changed the batch size in parallel from 1 to 10 to 100 to 1000
3. Explicitly spawn two threads and tell each to do a foreach on half of the tests


None of them made it go any faster. I had similar results using unit-threaded on my own projects. This is weird.

Atila

gdc gave _very_ different results. I had to use different modules because at some point tests started failing, but with gdc the threaded version runs ~3x faster.

On my own unit-threaded benchmarks, running the UTs for Cerealed over and over again was only slightly slower with threads than without. With dmd the threaded version was nearly 3x slower.

Atila

On Saturday, 3 May 2014 at 21:14:29 UTC, Atila Neves wrote:
>>>    if(single) {
>>>        foreach(test; tests) {
>>>            test();
>>>        }
>>>    } else {
>>>        foreach(test; tests.parallel) {
>>
>> Try different batch size:
>> test.parallel(1), test.parallel(2) etc.
>
> So as to not have thread creation be disproportionately represented, I repeated the module list over and over again, making the number of tests run equal to 9990. This takes 5s on my machine to run in on thread and 12s in multiple. Here are the things I tried:
>
> 1. Created my own TaskPool so I could decide how many threads to use
> 2. Changed the batch size in parallel from 1 to 10 to 100 to 1000
> 3. Explicitly spawn two threads and tell each to do a foreach on half of the tests
>
>
> None of them made it go any faster. I had similar results using unit-threaded on my own projects. This is weird.
>
> Atila

Same thing with unit_threaded on Phobos, 3x faster even without repeating the modules (0.1s vs 0.3s). Since the example is shorter than the other one, I'll post it here in case anyone else wants to try:

import unit_threaded.runner;

int main(string[] args) {
    return args.runTests!(
        "ustd.array",
        "ustd.ascii",
        "ustd.base64",
        "ustd.bigint",
        "ustd.bitmanip",
        "ustd.concurrency",
        "ustd.container",
        "ustd.cstream",
        );
}


On Saturday, 3 May 2014 at 21:42:13 UTC, Atila Neves wrote:
> gdc gave _very_ different results. I had to use different modules because at some point tests started failing, but with gdc the threaded version runs ~3x faster.
>
> On my own unit-threaded benchmarks, running the UTs for Cerealed over and over again was only slightly slower with threads than without. With dmd the threaded version was nearly 3x slower.
>
> Atila
>
> On Saturday, 3 May 2014 at 21:14:29 UTC, Atila Neves wrote:
>>>>   if(single) {
>>>>       foreach(test; tests) {
>>>>           test();
>>>>       }
>>>>   } else {
>>>>       foreach(test; tests.parallel) {
>>>
>>> Try different batch size:
>>> test.parallel(1), test.parallel(2) etc.
>>
>> So as to not have thread creation be disproportionately represented, I repeated the module list over and over again, making the number of tests run equal to 9990. This takes 5s on my machine to run in on thread and 12s in multiple. Here are the things I tried:
>>
>> 1. Created my own TaskPool so I could decide how many threads to use
>> 2. Changed the batch size in parallel from 1 to 10 to 100 to 1000
>> 3. Explicitly spawn two threads and tell each to do a foreach on half of the tests
>>
>>
>> None of them made it go any faster. I had similar results using unit-threaded on my own projects. This is weird.
>>
>> Atila

On 5/3/14, 2:42 PM, Atila Neves wrote:
> gdc gave _very_ different results. I had to use different modules
> because at some point tests started failing, but with gdc the threaded
> version runs ~3x faster.
>
> On my own unit-threaded benchmarks, running the UTs for Cerealed over
> and over again was only slightly slower with threads than without. With
> dmd the threaded version was nearly 3x slower.

Sounds like a severe bug in dmd or dependents. -- Andrei

On Sat, 2014-05-03 at 19:37 +0000, Atila Neves via Digitalmars-d wrote: […]
> I'm using parallel and taskPool from std.parallelism. I was under the impression it gave me a ready-to-use pool with as many threads as I have cores.

There is a default, related to the number of cores the OS thinks there is (*), but you can also set the number manually.  std.parallelism could do with some work to make it better than it already is.


(*) Physical cores are not necessarily the number reported by the OS due to core hyperthreads. Quad core no hyperthreads, and dual core, two hyperthreads per core, both get reported as four processor systems. However if you benchmark them you get very, very different performance characteristics.

-- 
Russel. ============================================================================= Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder@ekiga.net 41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel@winder.org.uk London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder

On Saturday, 3 May 2014 at 22:46:03 UTC, Andrei Alexandrescu wrote:
> On 5/3/14, 2:42 PM, Atila Neves wrote:
>> gdc gave _very_ different results. I had to use different modules
>> because at some point tests started failing, but with gdc the threaded
>> version runs ~3x faster.
>>
>> On my own unit-threaded benchmarks, running the UTs for Cerealed over
>> and over again was only slightly slower with threads than without. With
>> dmd the threaded version was nearly 3x slower.
>
> Sounds like a severe bug in dmd or dependents. -- Andrei

Seems like it. Just to be sure I swapped ld.gold for ld.bfd and the problem was still there.

I'm not entirely sure how to file this bug: with just my simple example above?

Atila

Like I mentioned afterwards, I tried a different number of threads. On my machine, at least, std.parallelism.totalCPUs returns 8, the number of virtual cores. As it should.

Atila

On Sunday, 4 May 2014 at 07:49:51 UTC, Russel Winder via Digitalmars-d wrote:
> On Sat, 2014-05-03 at 19:37 +0000, Atila Neves via Digitalmars-d wrote:
> […]
>> I'm using parallel and taskPool from std.parallelism. I was under the impression it gave me a ready-to-use pool with as many threads as I have cores.
>
> There is a default, related to the number of cores the OS thinks there
> is (*), but you can also set the number manually.  std.parallelism could
> do with some work to make it better than it already is.
>
>
> (*) Physical cores are not necessarily the number reported by the OS due
> to core hyperthreads. Quad core no hyperthreads, and dual core, two
> hyperthreads per core, both get reported as four processor systems.
> However if you benchmark them you get very, very different performance
> characteristics.