On Tuesday, 21 July 2020 at 13:23:32 UTC, Adam D. Ruppe wrote:
>
> But the array isn't initialized in the justification scenario. It is accessed through a null pointer and the type system thinks it is fine because it is still inside the static limit.
>
> At run time, the cpu just sees access to memory address 0 + x, and if x is sufficient large, it can bypass those guard pages.

I'm not that convinced. This totally depends on how the virtual memory for the process looks like. Some operating systems might have a gap between 0 - 16MB but some others don't. This is also a subject that can change between versions of the OS and even more uncertain as address space randomization becomes popular. Safety based on assumptions aren't really worth it.

I don't personally care about the 16MB limit as I would never use it for any foreseeable future but the motivation for it is kind of vague.

On Monday, 20 July 2020 at 22:05:35 UTC, WhatMeWorry wrote:
> 1) The D Language Reference says:
>
> "There are four kinds of arrays..." with the first example being
> "type*	Pointers to data"  and "int* p;  etc.
>
> At the risk of sounding overly nitpicky, isn't a pointer to an integer simply a pointer to an integer?  How does that pertain to an array?

I agree. "type*" being an array makes no sense from a D language point of view.

> 2) "The total size of a static array cannot exceed 16Mb" What limits this? And with modern systems of 16GB and 32GB, isn't 16Mb excessively small?   (an aside: shouldn't that be 16MB in the reference instead of 16Mb? that is, Doesn't b = bits and B = bytes)

This doesn't make sense either. Where did you find this in the documentation? It should be removed, as it is easily proven to work (`ubyte[170_000_000] s; void main(){s[160_000_000] = 1;}`).

> 3) Lastly, In the following code snippet, is arrayA and arrayB both allocated on the stack? And how does their scopes and/or lifetimes differ?
>
> ==== module1 =====
> int[100] arrayA;
> void foo() // changed from main to foo for clarity
> {
>     int[100] arrayB;
>     // ...
> }
> ==== module1 =====

"The stack" is not a D language thing, a better way of looking at it is that local storage is implemented by all D compilers by using the "stack" (on x86).

arrayA is not allocated on the stack, lifetime is whole duration of program, one array per thread.
arrayB is indeed allocated on the stack (local storage), lifetime is only from start to end of foo(), one array per call to foo (!).

Because arrayB is on the stack, you are limited by stack size which is set by the OS (but can be overridden). The array would be competing with all other things that are put on the stack, such as function call return addresses and temporary values, both of which you as coder cannot see. What maximum size of arrayB you can get away with heavily depends on the rest of your program (and the stack size allocated by the OS, which is somewhere in the 4MB, 8MB, 16MB range), thus best to avoid putting large arrays on the stack alltogether.
arrayA is allocated together with other global/TLS variables in a section for which I don't think there really is a size limit.

-Johan

On Tuesday, 21 July 2020 at 13:42:15 UTC, Steven Schveighoffer wrote:
> On 7/21/20 8:34 AM, Adam D. Ruppe wrote:
>
>> The others aren't wrong about stack size limits playing some role, but the primary reason is that it is a weird hack for @safe, believe it or not.
> ...
>> I don't recall exactly when this was discussed but it came up in the earlier days of @safe, I'm pretty sure it worked before then.
>
> I think this was discussed, but was not the reason for the limitation. The limitation exists even in D1, which is before @safe: https://digitalmars.com/d/1.0/arrays.html#static-arrays
>
> I have stressed before that any access of a pointer to a large object in @safe code should also check that the base of the object is not within the null page (this is not currently done). This is the only way to ensure safety.

It seems the limitation was introduced in DMD 0.123, in May 2005:
https://forum.dlang.org/post/d61jpa$1m0l$1@digitaldaemon.com
Walter gives some justification in the post immediately following:

> 1) Gigantic static arrays are often either the result of a typo or are a
> newbie mistake.
> 2) Such require a lot of memory for the compiler to handle. Before the OS
> officially runs out of memory, it goes to greater and greater lengths to
> scavenge memory for the compiler, often bringing the computer to its knees
> in desperation.
> 3) D needs to be a portable language, and by capping the array size a
> program is more likely to be portable.
> 4) Giant arrays are reflected in a corresponding giant size for the exe
> file.
> 5) There simply isn't a need I can think of for such arrays. There shouldn't
> be a problem with allocating them dynamically.

I admit I thought it was an old optlink limitation, but it seems it's basically arbitrary.

--
  Simen

On Tuesday, 21 July 2020 at 19:20:28 UTC, Simen Kjærås wrote:
> Walter gives some justification in the post immediately following:

whelp proves my memory wrong!