I am allocating 2d array as:

double[gridSize][gridSize] gridInfo;

which works for small dimension, but for large dimension, 16Mb limit comes.

Would you please tell me how do allocate a large 2d array (which has to be done as a dynamic array)? It is a square grid and dimensions are already known.

I also searched internet but could not find an answer. Thanks.

Sparsh Mittal:

> I am allocating 2d array as:
>
> double[gridSize][gridSize] gridInfo;
>
> which works for small dimension, but for large dimension, 16Mb limit comes.

Walter has decided to introduce a very low limit for the size of static arrays. (Both Clang and G++ support larger ones).


> Would you please tell me how do allocate a large 2d array (which has to be done as a dynamic array)? It is a square grid and dimensions are already known.

This allocates your array, filled with NaNs:

auto gridInfo = new double[][](gridSize, gridSize);


If you want to skip most or all the initialization then take a look at two functions in std.array. In most cases the "partially initialized" function is enough, and it's quite safer.

Bye,
bearophile

Thanks for your prompt reply. It was very helpful.

On Monday, 4 February 2013 at 15:23:20 UTC, Sparsh Mittal wrote:
> I am allocating 2d array as:
>
> double[gridSize][gridSize] gridInfo;
>
> which works for small dimension, but for large dimension, 16Mb limit comes.
>
> Would you please tell me how do allocate a large 2d array (which has to be done as a dynamic array)? It is a square grid and dimensions are already known.
>
> I also searched internet but could not find an answer. Thanks.

If all (but last of) the dimensions are known at compile time, then you can dynamically allocate an array of fixed sized arrays:

//----
enum size_t gridSize = 4_000;
enum size_t total   = gridSize * gridSize;
static assert (total == 16_000_000); //16 million doubles total
static assert (total * double.sizeof == 128_000_000); //126 Megs allocated

void main()
{
    double[gridSize][] gridInfo = new double[gridSize][](gridSize);
}
//----

This will give you a dense array: Eg: all the data is contiguous in memory.

You can also use the "convenient 2D notation":

//----
import std.stdio;
enum gridSize = 4;
enum total   = gridSize * gridSize;

void main()
{
    double[][] gridInfo = new double[][](gridSize, gridSize);
}
//----

DO NOTE though that in this case, you have a (probably compact but) jagged array. What this means is that instead of having one single solid block of data, you have a top level array, that references a second level of arrays.

EG: It is (should be) equivalent to:
//----
enum size_t gridSize = 4_000;
enum size_t total   = gridSize * gridSize;

void main()
{
    double[][] gridInfo  = new double[][](gridSize); //Create an array of empty array;

    //Initialize gridInfo. Scope block to avoid polution
    {
        double[]   megaBlock = new double[](total); //Create all the data
        foreach (i ; 0 .. gridSize) //Iterate on each gridInfo entry
        {
            size_t offset = i * gridSize;
            gridInfo[i] = megaBlock[offset .. offset + gridSize];//Connect the arrays;
        }
    }
}
//----

It's not a big deal, but indexing *might* be a little slower with this scheme.

monarch_dodra:

> If all (but last of) the dimensions are known at compile time, then you can dynamically allocate an array of fixed sized arrays:
>
> //----
> enum size_t gridSize = 4_000;
> enum size_t total   = gridSize * gridSize;
> static assert (total == 16_000_000); //16 million doubles total
> static assert (total * double.sizeof == 128_000_000); //126 Megs allocated
>
> void main()
> {
>     double[gridSize][] gridInfo = new double[gridSize][](gridSize);
> }
> //----
>
> This will give you a dense array: Eg: all the data is contiguous in memory.

Nice. This idiom should be added to the D docs.

Bye,
bearophile

> It's not a big deal, but indexing *might* be a little slower with this scheme.
Thanks a lot for your reply. It was extremely useful, since I am optimizing for performance.

On Mon, 04 Feb 2013 10:58:36 -0500, bearophile <bearophileHUGS@lycos.com> wrote:

> monarch_dodra:
>
>> If all (but last of) the dimensions are known at compile time, then you can dynamically allocate an array of fixed sized arrays:
>>
>> //----
>> enum size_t gridSize = 4_000;
>> enum size_t total   = gridSize * gridSize;
>> static assert (total == 16_000_000); //16 million doubles total
>> static assert (total * double.sizeof == 128_000_000); //126 Megs allocated
>>
>> void main()
>> {
>>     double[gridSize][] gridInfo = new double[gridSize][](gridSize);
>> }
>> //----
>>
>> This will give you a dense array: Eg: all the data is contiguous in memory.
>
> Nice. This idiom should be added to the D docs.

Wow, this is something I didn't know was possible.  Very useful!

BTW, "all (but last of) the dimensions" isn't correct, you know them all in your example, and it looks great!

-Steve

Steven Schveighoffer:

> Wow, this is something I didn't know was possible.  Very useful!

It's it cute when you use a language almost daily for few years,
and then you see a new way to allocate built-in arrays? :-)

Bye,
bearophile

On 2013-13-04 17:02, Steven Schveighoffer <schveiguy@yahoo.com> wrote:

> BTW, "all (but last of) the dimensions" isn't correct, you know them all in your example, and it looks great!

I believe his point was that this also works if you don't know the last
dimension beforehand.

-- 
Simen

On Monday, 4 February 2013 at 16:54:37 UTC, bearophile wrote:
> Steven Schveighoffer:
>
>> Wow, this is something I didn't know was possible.  Very useful!
>
> It's it cute when you use a language almost daily for few years,
> and then you see a new way to allocate built-in arrays? :-)
>
> Bye,
> bearophile

Technically, the straight up "C - style" syntax also does this just as well:

"int[2][] a = new int[2][2];"

Since the last "[2]" gets  rewritten to "[](2)"

But when written that way, I find it is not very clear what happens.

Ideally, I wish we could allocate static arrays on the heap easily:

"int[2]* p = new int[2]()"

Anybody know why this doesn't work? In a GC language like D, that has to track "capacity/usage" of its dynamic arrays, there is a real gain to being able to specify: I want my damn array to be fixed in size: Don't track any growing information for me.