The default size of the runtime stack for a Fiber is 4*PAGESIZE which is very small, and a quick test shows that a Fiber suffers a stack overflow that doesn't lead to a clean termination when this limit is exceeded.

This makes it difficult to simulate deterministic alternation where the stack size needed is unpredictable because complex deterministic computations are going on inside Fibers.

In contrast, the Go programming language's goroutines can extend their stacks as needed at runtime, and so can be used to simulate deterministic alternation without this limitation, and yet be initially executed with each having only a small stack size.

There seems to be a claim that all that's needed to add D-routines (goroutines for D) is a scheduler and a Channel type, on top of Fiber.
http://forum.dlang.org/thread/lphnen$1ml7$1@digitalmars.com
See the initial post, point 7., as well as supporting remarks in later replies.

Am I missing something? Is there a clean and simple way to get Fiber to no longer suffer a stack overflow when implementing D-routines?

Infinite stack comes with a cost. Every function call first checks the state of the stack. This should be done with the help of compiler. And such a "tradeoff" could scare bare-metal programmers away from D. Though maybe there's a chance of making this stack check controllable, but not that i can think of.

On Thursday, 14 August 2014 at 07:46:29 UTC, Carl Sturtivant wrote:
>
> The default size of the runtime stack for a Fiber is 4*PAGESIZE which is very small, and a quick test shows that a Fiber suffers a stack overflow that doesn't lead to a clean termination when this limit is exceeded.
>
> This makes it difficult to simulate deterministic alternation where the stack size needed is unpredictable because complex deterministic computations are going on inside Fibers.
>
> In contrast, the Go programming language's goroutines can extend their stacks as needed at runtime, and so can be used to simulate deterministic alternation without this limitation, and yet be initially executed with each having only a small stack size.
>
> There seems to be a claim that all that's needed to add D-routines (goroutines for D) is a scheduler and a Channel type, on top of Fiber.
> http://forum.dlang.org/thread/lphnen$1ml7$1@digitalmars.com
> See the initial post, point 7., as well as supporting remarks in later replies.
>
> Am I missing something? Is there a clean and simple way to get Fiber to no longer suffer a stack overflow when implementing D-routines?

On 64 bit, reserve a huge chunk of memory, set a SEGV handler and commit more as needed. Basically how kernel thread stacks work. I've been meaning to do this but haven't gotten around to it yet.

On Thursday, 14 August 2014 at 07:46:29 UTC, Carl Sturtivant wrote:
>
> The default size of the runtime stack for a Fiber is 4*PAGESIZE which is very small, and a quick test shows that a Fiber suffers a stack overflow that doesn't lead to a clean termination when this limit is exceeded.
>
> This makes it difficult to simulate deterministic alternation where the stack size needed is unpredictable because complex deterministic computations are going on inside Fibers.
>
> In contrast, the Go programming language's goroutines can extend their stacks as needed at runtime, and so can be used to simulate deterministic alternation without this limitation, and yet be initially executed with each having only a small stack size.
>
> There seems to be a claim that all that's needed to add D-routines (goroutines for D) is a scheduler and a Channel type, on top of Fiber.
> http://forum.dlang.org/thread/lphnen$1ml7$1@digitalmars.com
> See the initial post, point 7., as well as supporting remarks in later replies.
>
> Am I missing something? Is there a clean and simple way to get Fiber to no longer suffer a stack overflow when implementing D-routines?

Segmented stacks come with a cost. Rust abandoned them for reasons you can read about here:

https://mail.mozilla.org/pipermail/rust-dev/2013-November/006314.html

I believe Go has taken steps to help mitigate stack thrashing but I don't know if they have been successful yet.

On Thursday, 14 August 2014 at 18:52:00 UTC, Sean Kelly wrote:
> On 64 bit, reserve a huge chunk of memory, set a SEGV handler and commit more as needed. Basically how kernel thread stacks work. I've been meaning to do this but haven't gotten around to it yet.

I think using some sort of thread-local shared heap pool is better approach in general as it does need any SEGV handling overhead and simplifies fiber implementation (all fibers are guaranteed to take fixed amount of memory). It is not a silver bullet and forces you to think much more about application memory layout but I believe this is much better approach for high performance services than segmented stack like in Go.

On Thursday, 14 August 2014 at 18:52:00 UTC, Sean Kelly wrote:
> On 64 bit, reserve a huge chunk of memory, set a SEGV handler and commit more as needed. Basically how kernel thread stacks work. I've been meaning to do this but haven't gotten around to it yet.

AFAIK, OS already provides this transparently: when you allocate memory, OS only reserves the memory range and commits pages when they are accessed.

http://msdn.microsoft.com/en-us/library/windows/desktop/aa366887%28v=vs.85%29.aspx
> Allocates memory charges (from the overall size of memory and the paging files on disk) for the specified reserved memory pages. The function also guarantees that when the caller later initially accesses the memory, the contents will be zero. Actual physical pages are not allocated unless/until the virtual addresses are actually accessed.

On Thursday, 14 August 2014 at 07:46:29 UTC, Carl Sturtivant wrote:
> The default size of the runtime stack for a Fiber is 4*PAGESIZE which is very small, and a quick test shows that a Fiber suffers a stack overflow that doesn't lead to a clean termination when this limit is exceeded.

Pass a bigger stack size to the Fiber constructor?

On Friday, 15 August 2014 at 08:36:34 UTC, Kagamin wrote:
> http://msdn.microsoft.com/en-us/library/windows/desktop/aa366887%28v=vs.85%29.aspx
>> Allocates memory charges (from the overall size of memory and the paging files on disk) for the specified reserved memory pages. The function also guarantees that when the caller later initially accesses the memory, the contents will be zero. Actual physical pages are not allocated unless/until the virtual addresses are actually accessed.

Oh handy, so there's basically no work to be done on Windows.  I'll have to check the behavior of mmap on Posix.

On Friday, 15 August 2014 at 08:41:30 UTC, Kagamin wrote:
> On Thursday, 14 August 2014 at 07:46:29 UTC, Carl Sturtivant wrote:
>> The default size of the runtime stack for a Fiber is 4*PAGESIZE which is very small, and a quick test shows that a Fiber suffers a stack overflow that doesn't lead to a clean termination when this limit is exceeded.
>
> Pass a bigger stack size to the Fiber constructor?

Won't that kind of kill the purpose of Fiber as low-cost context abstraction? Stack size does add up for thousands of fibers.