Jump to page: 1 24  
Page
Thread overview
Stack frames larger than 4K should be rejected, but what if I want more
Jun 27, 2021
IGotD-
Jun 27, 2021
Dennis
Jun 27, 2021
Dennis
Jun 27, 2021
IGotD-
Jun 28, 2021
Luis
Jun 28, 2021
IGotD-
Jun 28, 2021
SealabJaster
Jun 28, 2021
IGotD-
Jun 27, 2021
Walter Bright
Jun 28, 2021
ag0aep6g
Jun 28, 2021
Walter Bright
Jun 28, 2021
ag0aep6g
Jun 27, 2021
IGotD-
Jun 27, 2021
Walter Bright
Jun 28, 2021
Elronnd
Jun 27, 2021
rikki cattermole
Jun 28, 2021
IGotD-
Jun 28, 2021
IGotD-
Jun 28, 2021
sighoya
Jun 29, 2021
Vladimir Panteleev
June 27, 2021

This is a breakout thread from this thread.

https://forum.dlang.org/thread/ghodronzxeirokyoqeag@forum.dlang.org

According to the comment from Walter.

As per Walter's recent comment in that thread, he asserts that stack allocations beyond 4k should be rejected.

in this issue:
https://issues.dlang.org/show_bug.cgi?id=17566

In general I can agree with the rationale to not allow stack frames larger than 4K when it comes to normal programming, it makes sense. However, since D is supposed to be a "systems programming language" the language should not dictate what the programmer should be able to do with the stack. It also assumes things that all systems have 4K page sizes, that the stack should be used moderately. There might be cases when the programmer wants to go nuts with stack and do a lot storage on it, for whatever reason. Therefore I think this limit should not be a language definition.

First this 4K limit should be configurable per operating system. Also there should be an option to override this limit, for example with a compiler option.

June 27, 2021

On Sunday, 27 June 2021 at 18:56:57 UTC, IGotD- wrote:

>

In general I can agree with the rationale to not allow stack frames larger than 4K when it comes to normal programming, it makes sense.

It makes no sense and would kill a system level language. The stack depth for Linux is 8MiB. 4KiB isn't even enough to fit a commonly sized FFT buffer.

June 27, 2021

On Sunday, 27 June 2021 at 18:56:57 UTC, IGotD- wrote:

>

In general I can agree with the rationale to not allow stack frames larger than 4K when it comes to normal programming, it makes sense.

It makes no sense and would kill a system level language. The stack depth for Linux is 8MiB. 4KiB isn't even enough to fit a commonly sized FFT buffer, anything less than 16KiB is a joke IMHO.

June 28, 2021
On 28/06/2021 6:56 AM, IGotD- wrote:
> First this 4K limit should be configurable per operating system. Also there should be an option to override this limit, for example with a compiler option.

You can set it for fibers too!
June 27, 2021

On Sunday, 27 June 2021 at 19:41:40 UTC, Ola Fosheim Grøstad wrote:

>

It makes no sense and would kill a system level language.

While it's not specified in "The compiler should reject any stack frame that's larger than 4K", I think it's only meant to apply to @safe functions, not @system or @trusted ones.

Also, instead of straight up rejecting large stack frames in @safe code, the compiler could also start such a function with probing the guard page by making writes at intervals of 4K.

June 27, 2021

On Sunday, 27 June 2021 at 19:41:40 UTC, Ola Fosheim Grøstad wrote:

>

It makes no sense and would kill a system level language. The stack depth for Linux is 8MiB. 4KiB isn't even enough to fit a commonly sized FFT buffer, anything less than 16KiB is a joke IMHO.

Yes kind of, and I can think of use cases when you want big arrays on the stack. The question is if the approach is correct. Walter wants to catch potential memory corruption but limiting the stack usage (per function I assume) but in this case the real solution would be reading the stack limit (probably with an expensive system API but can be stored as a TLS variable).

On 64-bit systems there usually enough virtual space to put several megabytes of guard regions. On 32-bit systems it is more cramped and perhaps only 4K. The question is if this is something that should be dealt by the language.

June 27, 2021

On Sunday, 27 June 2021 at 20:17:48 UTC, Dennis wrote:

>

I think it's only meant to apply to @safe functions, not @system or @trusted ones.

Actually, that's hard to realize, since the check for @safe is a semantic check in the frontend, while final stack sizes are only known by the backend. Making the frontend guess an upper bound is hard because of tail calls and/or inlining, e.g:

void f(ubyte[] x) {
    ubyte[4000] bufA = void;
    g(bufA[]);
}

void g(ubyte[] bufA) {
    ubyte[4000] bufB = void;
    h(bufA, bufB);
}

void h(ubyte[] x, ubyte[] y);

With ldc -O3, the stack frame of f is 8008 bytes because it has g inlined.

June 27, 2021

On Sunday, 27 June 2021 at 22:01:22 UTC, Dennis wrote:

>

Actually, that's hard to realize, since the check for @safe is a semantic check in the frontend, while final stack sizes are only known by the backend. Making the frontend guess an upper bound is hard because of tail calls and/or inlining, e.g:

void f(ubyte[] x) {
    ubyte[4000] bufA = void;
    g(bufA[]);
}

void g(ubyte[] bufA) {
    ubyte[4000] bufB = void;
    h(bufA, bufB);
}

void h(ubyte[] x, ubyte[] y);

With ldc -O3, the stack frame of f is 8008 bytes because it has g inlined.

That's a good observation. Does this mean that the point of the suggested 4K limit falls?

In practice, if you want to prevent stack overflow and be sure about it I think you need a check for every new frame. This has a performance impact but safety usually has a that.

June 27, 2021
On 6/27/2021 2:08 PM, IGotD- wrote:
> On Sunday, 27 June 2021 at 19:41:40 UTC, Ola Fosheim Grøstad wrote:
>>
>> It makes no sense and would kill a system level language. The stack depth for Linux is 8MiB. 4KiB isn't even enough to fit a commonly sized FFT buffer, anything less than 16KiB is a joke IMHO.
> 
> Yes kind of, and I can think of use cases when you want big arrays on the stack. The question is if the approach is correct. Walter wants to catch potential memory corruption but limiting the stack usage (per function I assume) but in this case the real solution would be reading the stack limit (probably with an expensive system API but can be stored as a TLS variable).
> 
> On 64-bit systems there usually enough virtual space to put several megabytes of guard regions. On 32-bit systems it is more cramped and perhaps only 4K. The question is if this is something that should be dealt by the language.

There's a huge difference between the virtual address range set aside for the entire thread stack, and the amount of stack consumed by one function invocation.

Large amounts of stack allocated for one function's frame is almost always the result of an inexperienced system coder who doesn't realize what it means to allocate memory on the stack, how guard pages work, and how physical memory gets allocated as the stack grows.
June 27, 2021
On 6/27/2021 1:17 PM, Dennis wrote:
> Also, instead of straight up rejecting large stack frames in @safe code, the compiler could also start such a function with probing the guard page by making writes at intervals of 4K.

dmd already does that. But very few programmers are aware of this, and it's pretty inefficient.

« First   ‹ Prev
1 2 3 4