| |
 | Posted by eris
in reply to dsimcha |  Permalink Reply |
|
eris 
Posted in reply to dsimcha
| dsimcha Wrote:
> When I do parallel programming in D, it's nice to have a sane default for the number of threads. The ideal number is the number of cores that can see the current address space. core.cpuid provides coresPerCPU, but it doesn't appear to provide nPhysialCPUs or something similar, which would allow the total number of physical CPUs in the system to be determined. Is there any easy way to determine this in D2?
dsimcha,
This is a relatively difficult problem in general to do portably due to hardware differences, topology differences, changes to hardware, OS variations. Even the pthreads library doesn't reliably implement it in a portable manner.
I came to the conclusion that the people most motivated to keep up to date on a portable CPU core topology are the national supercomputing labs. INRIA and various US labs came up with "Portable Hardware Locality" library. It gives you *everything* you need to discover the number of CPU sockets, memory architecture, number of cores per socket, control cpu affinity etc.
The HWLoc C libraries are written by the open-mpi project here: http://www.open-mpi.org/projects/hwloc/
I created an interface to their C library called HWLoc.d and I haven't published it anywhere. I just needed it for my multi-core Dendrite system. Give me an email you want it sent to and it's yours. I could post it here, but it would be a lot of noise in the discussion. I'll probably post it on dsource later.
My license on all of my Dendrite code is zlib (which means you can do almost anything except claim that you wrote the original) and the open-mpi C libraries are new BSD, so have at it.
You'll obviously have to download and install the hwloc c libraries also.
Here's some output from "hwhello.d" on a 2 socket system:
*** Objects at level 0
CPU Set: 00000003
Index 0: System(1002MB)
*** Objects at level 1
CPU Set: 00000001
Index 0: Socket#0
CPU Set: 00000002
Index 1: Socket#2
*** Objects at level 2
CPU Set: 00000001
Index 0: L2(2048KB)
CPU Set: 00000002
Index 1: L2(2048KB)
*** Objects at level 3
CPU Set: 00000001
Index 0: L1(16KB)
CPU Set: 00000002
Index 1: L1(16KB)
*** Objects at level 4
CPU Set: 00000001
Index 0: Core#0
CPU Set: 00000002
Index 1: Core#0
*** Objects at level 5
CPU Set: 00000001
Index 0: P#0
CPU Set: 00000002
Index 1: P#1
*** Printing overall tree
System(1002MB)
Socket#0
L2(2048KB)
L1(16KB)
Core#0
P#0
Socket#2
L2(2048KB)
L1(16KB)
Core#0
P#1
*** 2 socket(s)
depth is now: 4
number objects at this depth: 2
ob 0: 00000001
ob 0: 00000002
Last CPU Set: 00000002
Singleified CPU Set: 00000002
burning CPU
... and here's the code snippet from hwloc.d that generates the last part of that output:
/* Get last level. */
obj = hwloc_get_obj_by_depth(topology, depth,
hwloc_get_nbobjs_by_depth(topology, depth) - 1);
if (obj) {
/* Get a copy of its cpuset that we may modify. */
cpuset = hwloc_cpuset_dup(obj.cpuset);
// convert it to a string
nchars = hwloc_cpuset_snprintf(string.ptr, string.sizeof, cpuset);
Stdout("Last CPU Set: ")(string[0..nchars]).newline;
/* Get only one logical processor (in case the core is
SMT/hyperthreaded). */
hwloc_cpuset_singlify(cpuset);
nchars = hwloc_cpuset_snprintf(string.ptr, string.sizeof, cpuset);
Stdout("Singleified CPU Set: ")(string[0..nchars]).newline;
/* And try to bind ourself there. */
if (hwloc_set_cpubind(topology, cpuset, 0)) {
char* str;
nchars = hwloc_cpuset_asprintf(&str, obj.cpuset);
Stdout.formatln("Couldn't bind to cpuset {}", str[0..nchars]);
free(str);
}
Stdout("burning CPU").newline;
/* do something we can see in htop */
for (j=0; j<100000000; j++) f+=f/2;
/* Free our cpuset copy */
hwloc_cpuset_free(cpuset);
}
.... finally here's the output from the 'lstopo' utility in the hwloc distro that displays the topology visually (mine is a 4 socket, hexacore NUMA machine. Use a fixed font to view it).
┌────────────────────────────────────────────────────────────────┐
│ System(4096MB) │
│ │
│ ┌────────────────────────────────────────────────────────────┐ │
│ │ ┌────────────────────────────────────────────────────────┐ │ │
│ │ │ Node#0(1024MB) │ │ │
│ │ └────────────────────────────────────────────────────────┘ │ │
│ │ │ │
│ │ ┌──────┐ ┌──────┐ ┌──────┐ ┌──────┐ ┌──────┐ ┌──────┐ │ │
│ │ │ P#0 │ │ P#1 │ │ P#2 │ │ P#3 │ │ P#4 │ │ P#5 │ │ │
│ │ └──────┘ └──────┘ └──────┘ └──────┘ └──────┘ └──────┘ │ │
│ └────────────────────────────────────────────────────────────┘ │
│ │
│ ┌────────────────────────────────────────────────────────────┐ │
│ │ ┌────────────────────────────────────────────────────────┐ │ │
│ │ │ Node#1(1024MB) │ │ │
│ │ └────────────────────────────────────────────────────────┘ │ │
│ │ │ │
│ │ ┌──────┐ ┌──────┐ ┌──────┐ ┌──────┐ ┌──────┐ ┌──────┐ │ │
│ │ │ P#6 │ │ P#7 │ │ P#8 │ │ P#9 │ │ P#10 │ │ P#11 │ │ │
│ │ └──────┘ └──────┘ └──────┘ └──────┘ └──────┘ └──────┘ │ │
│ └────────────────────────────────────────────────────────────┘ │
│ │
│ ┌────────────────────────────────────────────────────────────┐ │
│ │ ┌────────────────────────────────────────────────────────┐ │ │
│ │ │ Node#2(1024MB) │ │ │
│ │ └────────────────────────────────────────────────────────┘ │ │
│ │ │ │
│ │ ┌──────┐ ┌──────┐ ┌──────┐ ┌──────┐ ┌──────┐ ┌──────┐ │ │
│ │ │ P#12 │ │ P#13 │ │ P#14 │ │ P#15 │ │ P#16 │ │ P#17 │ │ │
│ │ └──────┘ └──────┘ └──────┘ └──────┘ └──────┘ └──────┘ │ │
│ └────────────────────────────────────────────────────────────┘ │
│ │
│ ┌────────────────────────────────────────────────────────────┐ │
│ │ ┌────────────────────────────────────────────────────────┐ │ │
│ │ │ Node#3(1024MB) │ │ │
│ │ └────────────────────────────────────────────────────────┘ │ │
│ │ │ │
│ │ ┌──────┐ ┌──────┐ ┌──────┐ ┌──────┐ ┌──────┐ ┌──────┐ │ │
│ │ │ P#18 │ │ P#19 │ │ P#20 │ │ P#21 │ │ P#22 │ │ P#23 │ │ │
│ │ └──────┘ └──────┘ └──────┘ └──────┘ └──────┘ └──────┘ │ │
│ └────────────────────────────────────────────────────────────┘ │
└────────────────────────────────────────────────────────────────┘
draco%
Have fun and let me know if you need anything...
eris
|
