This should be a not so long question to answer, I hope.

I took an example from the "Programming in D" book, chapter "Message Passing Concurrency", around page 550. The question of interest was, how many threads I can have spawned at the same time.
So I made an array of robot objects from the example and found out: the maximum number is different on different PCs I have.
For example:
- on my mac I can have 2048 threads spawned at the same time
- on my linux machine the maximum number is 32192.
The numbers are quite fixed, however there were some small fluctuations on the linux machine.

The questions still remains: how do I know, what maximum number of threads I can have? Is it possible to calculate this value during runtime of my program? The two machines I have available for testing are very different, what is the main parameter which controls the number of possible threads? (CPU maybe? Number of cores? RAM is not I think at the moment...)

Thanks in advance
Alex

On Thursday, 24 September 2015 at 08:55:25 UTC, Alex wrote:
> This should be a not so long question to answer, I hope.
>
> I took an example from the "Programming in D" book, chapter "Message Passing Concurrency", around page 550. The question of interest was, how many threads I can have spawned at the same time.
> So I made an array of robot objects from the example and found out: the maximum number is different on different PCs I have.
> For example:
> - on my mac I can have 2048 threads spawned at the same time
> - on my linux machine the maximum number is 32192.
> The numbers are quite fixed, however there were some small fluctuations on the linux machine.
>
> The questions still remains: how do I know, what maximum number of threads I can have? Is it possible to calculate this value during runtime of my program? The two machines I have available for testing are very different, what is the main parameter which controls the number of possible threads? (CPU maybe? Number of cores? RAM is not I think at the moment...)
>
> Thanks in advance
> Alex

From the numbers (2048 = 2^11, 32192 = ca 2^15), it's clear that you're not running out of memory, but that these are artificial limits imposed by the OS.

This stackoverflow answer describes the situation on Linux:
http://unix.stackexchange.com/a/47600/34768

On Thursday, September 24, 2015 08:55:22 Alex via Digitalmars-d-learn wrote:
> This should be a not so long question to answer, I hope.
>
> I took an example from the "Programming in D" book, chapter
> "Message Passing Concurrency", around page 550. The question of
> interest was, how many threads I can have spawned at the same
> time.
> So I made an array of robot objects from the example and found
> out: the maximum number is different on different PCs I have.
> For example:
> - on my mac I can have 2048 threads spawned at the same time
> - on my linux machine the maximum number is 32192.
> The numbers are quite fixed, however there were some small
> fluctuations on the linux machine.
>
> The questions still remains: how do I know, what maximum number of threads I can have? Is it possible to calculate this value during runtime of my program? The two machines I have available for testing are very different, what is the main parameter which controls the number of possible threads? (CPU maybe? Number of cores? RAM is not I think at the moment...)

It's entirely system specific. Not only does every OS handle it differently, but it con depend on how your machine is configured. In general though, I believe that it's a hard limit in the OS and has nothing to do with the number of cores or amount of RAM in your machine (though even if you haven't hit the limit, if the OS doesn't have enough resources to create another thread, then it will fail).

So, if you want to know for a particular OS, you're probably going to have to google for it. In the case of linux, a quick search seems to indicate that getrlimit will do the trick:

http://linux.die.net/man/2/getrlimit

But you can find as much pretty quickly googling yourself. Maybe someone else can tell you what to use on other OSes off the top of their head though.

Regardless, D doesn't provid any special way to do this, so to figure it out, you'd be calling whatever the C functions are that will tell you.

- Jonathan M Davis

On Thursday, 24 September 2015 at 08:55:25 UTC, Alex wrote:
> - on my mac I can have 2048 threads spawned at the same time
> - on my linux machine the maximum number is 32192.
> The numbers are quite fixed, however there were some small fluctuations on the linux machine.

I only know of `cat /proc/sys/kernel/threads-max` on linux, which will give you the max number of threads system-wide.

Offtop: i think if number of threads > number of real cores, than there's something wrong with your design. Maybe fibers suit better ?

On Thursday, September 24, 2015 12:38:39 Temtaime via Digitalmars-d-learn wrote:
> Offtop: i think if number of threads > number of real cores, than there's something wrong with your design. Maybe fibers suit better ?

That depends on what the threads are doing. If they're all CPU-intensive, then yeah, having more than about the number of real cores isn't going to work very well, but in many programs, a lot of the threads aren't doing much (e.g. waiting for something to happen and sitting idly in the meantime). If anything, I'd guess that it's far more common to have a lot more threads than cores if a program isn't single-threaded. I'd say that whether the number of cores matters particularly depends heavily on what your program is doing. But there are certainly cases where using fibers instead of threads makes a lot of sense.

- Jonathan M Davis

On Thursday, 24 September 2015 at 12:38:41 UTC, Temtaime wrote:
> Offtop: i think if number of threads > number of real cores, than there's something wrong with your design. Maybe fibers suit better ?

Well... you got my idea :) So it is not so far offtop, as you think )))
Fibers DO suit better as an idea of light workers. But the problem I have to manage is, that my program has to incorporate unpredictability in the work cycle of my workers. It is one of the main idea of the process itself. So the fact that a fiber yields at fixed points of his work is just the opposite from what I want.

But the portability of code is an issue too... and my tries show, that I can't rely on this with threads...

Having said that, I got an idea today, how I could try to incorporate fibers for my needs. If I manage to hide the yields calls of my workers in some random environment, I could express, that a worker can but must not give up his control at a specific point of his work. I have to try this idea next days.
Using fibers instead of threads would save me from a lot of troubles anyway... ;)

Thanks to everybody for hints about my question!