August 28, 2023
On 8/28/23 15:37, Joe@bloow.edu wrote:

> Basically everything is hard coded to use totalCPU's

parallel() is a function that dispatches to a default TaskPool object, which uses totalCPUs. It's convenient but as you say, not all problems should use it.

In such cases, you would create your own TaskPool object and call .parallel on it:

  https://youtu.be/dRORNQIB2wA?t=1611s

Ali

August 29, 2023

On 29.08.23 00:37, Joe@bloow.edu wrote:

>

Well, I have 32 cores so that would spawn 64-1 threads with hyper threading so not really a solution as it is too many simultaneous downs IMO.

"These properties get and set the number of worker threads in the TaskPool instance returned by taskPool. The default value is totalCPUs - 1. Calling the setter after the first call to taskPool does not changes number of worker threads in the instance returned by taskPool. "

I guess I could try to see if I can change this but I don't know what the "first call" is(and I'm using parallel to create it).
The call that is interesting in your case is the call to taskPool in your foreach loop. So if you call defaultPoolThreads(8) before your loop you should be good.

>

Seems that the code should simply be made more robust. Probably a just a few lines of code to change/add at most. Maybe the constructor and parallel should take an argument to set the "totalCPUs" which defaults to getting the total number rather than it being hard coded.

I currently don't need or have 32+ downlaods to test ATM so...

   this() @trusted
    {
        this(totalCPUs - 1);
    }

    /**
    Allows for custom number of worker threads.
    */
    this(size_t nWorkers) @trusted
    {

Basically everything is hard coded to use totalCPU's and that is the ultimate problem. Not all tasks should use all CPU' >
What happens when we get 128 cores? or even 32k at some poin >
It shouldn't be a hard coded value, it's really that simple and where the problem originates because someone didn't think ahead.
You have the option to not use the default value.

Kind regards,
Christian

August 29, 2023
On Monday, 28 August 2023 at 22:43:56 UTC, Ali Çehreli wrote:
> On 8/28/23 15:37, Joe@bloow.edu wrote:
>
> > Basically everything is hard coded to use totalCPU's
>
> parallel() is a function that dispatches to a default TaskPool object, which uses totalCPUs. It's convenient but as you say, not all problems should use it.
>
> In such cases, you would create your own TaskPool object and call .parallel on it:
>
>   https://youtu.be/dRORNQIB2wA?t=1611s
>
> Ali

Thanks. Seems to work. Didn't realize it was that easy ;)
1 2
Next ›   Last »