Permalink
Reply
Went through some interesting debugging with someone on Discord.
Long story short -- Fibers manage a resource that is limited. On Linux, the maximum number of concurrent fibers you can have by default is 2^15.
In practice, this means you really shouldn't be depending on the GC to clean up your fibers for you. The person on discord had a simple test:
void main () {
import std.stdio;
import core.thread;
import core.memory;
int n = 0;
foreach (a; 0..10) {
foreach (tick; 0..100_000) {
auto gt1 = new Fiber(() {
foreach (i; 0..5) {
n += 10;
Fiber.yield;
}
});
auto gt2 = new Fiber(() {
foreach (i; 0..5) {
n += 10;
Fiber.yield;
}
});
while (gt1.state != Fiber.State.TERM && gt2.state != Fiber.State.TERM) {
gt1.call;
gt2.call;
}
assert(gt1.state == Fiber.State.TERM && gt2.state == Fiber.State.TERM);
}
GC.stats.writeln;
}
writeln(n);
}
This results in a coredump after about 3 writes of the GC stats on my system.
What ends up happening is the number of fibers goes up to the limit, and the OS runs out of mmap slots in the kernel.
My recommendation to him is not to manage fibers using the GC, but to proactively destroy them.
But shouldn't D's standard library have a way to do this easily? Like some sort of fiber pool you can use to manage one-use fibers (that get returned to the pool when terminated)? It also might be able to limit fiber use so instead of an ugly abort OS error, you get a nice message.
Is there something like this already on code.dlang.org?
-Steve
