so at this link: https://dlang.org/spec/arrays.html

it indicates an array of type[] is of type 'Dynamic array'.

with that in mind, I ask, is this below a 'Dynamic array'.

If not, why not?


int[][] mArr2 = array(iota(1, 9).chunks(2).map!array.array);

On 1/15/22 20:09, forkit wrote:
> so at this link: https://dlang.org/spec/arrays.html
>
> it indicates an array of type[] is of type 'Dynamic array'.

I have a problem with calling type[] a dynamic array because it is a slice, which may be providing access to the elements of a dynamic array.

One complexity in D's slices is that they will start providing access to a dynamic array upon appending or concatenation.

Here is the proof:

void main() {
  int[2] data;
  int[] slice = data[];
}

'slice' has *nothing* to do with any dynamic array. Period!

It will provide access to one with the following operation:

  slice ~= 42;

Again, 'slice' is not a dynamic array; it provides access to the elements of one. (That dynamic array is owned by the D runtime (more precisely, the GC).)

Despite that reality, people want dynamic arrays in the language and call slices dynamic arrays. Ok, I feel better now. :)

> with that in mind, I ask, is this below a 'Dynamic array'.

Nothing different from what I wrote above.

> If not, why not?
>
>
> int[][] mArr2 = array(iota(1, 9).chunks(2).map!array.array);

mArr2 is a slice of slices. We can see from the initialization that there are many dynamic arrays behind the scenes.

The outermost array() call is unnecessary because array() of a slice will produce another slice with the same type (perhaps e.g. a 'const' might be dropped) and elements. This is the same:

  int[][] mArr2 = iota(1, 9).chunks(2).map!array.array ;

Ali

On Sunday, 16 January 2022 at 04:58:21 UTC, Ali Çehreli wrote:
>
> I have a problem with calling type[] a dynamic array because it is a slice, which may be providing access to the elements of a dynamic array.
>

Yes. A more useful way of describing [] would be to say:

"[] represents a dynamic array except when it represents a slice - in which case it is merely shorthand for a slice that is referencing data stored somewhere else."

Something that still confuese me in my example then:
 mArr[] is actually a slice, and not a dynamic array. That is, my slice is referencing data located somewhere else. But where exactly is that other data located? It seems to me, I have a slice of data that doesn't actually exist once I have that slice.

On 1/16/22 01:43, forkit wrote:
> On Sunday, 16 January 2022 at 04:58:21 UTC, Ali Çehreli wrote:
>>
>> I have a problem with calling type[] a dynamic array because it is a
>> slice, which may be providing access to the elements of a dynamic array.
>>
>
> Yes. A more useful way of describing [] would be to say:
>
> "[] represents a dynamic array except when it represents a slice

I still don't agree with that. :) To me, [] is always a slice. (Again, some other members of the community like to name it a "dynamic array" instead "slice".)

> - in
> which case it is merely shorthand for a slice that is referencing data
> stored somewhere else."

A slice always references data stored somewhere else.

> Something that still confuese me in my example then:
>   mArr[] is actually a slice, and not a dynamic array. That is, my slice
> is referencing data located somewhere else. But where exactly is that
> other data located?

Can be anywhere:

a) On the stack (as in my earlier example of int[2] static array)

b) In "dynamic memory" (i.e. "GC memory")

c) In malloc'ed memory (e.g. allocated by a C function)

d) etc.

> It seems to me, I have a slice of data that doesn't
> actually exist once I have that slice.

It would be a bug if the slice references non-existing data. Slice consist of two things:

1) A pointer to the beginning of data

2) The number of elements that are being referenced at that location

Going back to the three example of data I listed above:

a) The data is on the stack and will die when the scope is exited. It would be a bug to refer to that data longer that its lifetime:

int[] foo() {
  int[2] dataOnTheStack;
  return dataOnTheStack[];
}

Luckily, dmd catches that bug in that case:

Error: returning `dataOnTheStack[]` escapes a reference to local variable `dataOnTheStack`

b) This is the most common case: The data is in dynamic memory. This is owned and managed by druntime. druntime includes the GC, which occasionally performs a cleanup to free unused memory.

int[] bar(int[] input) {
  int[] output = input ~ 42;
  return output;
}

void main() {
  bar([ 1, 2 ]);
}

The first line of bar() appends 42 to an existing slice. That operation causes druntime to allocate new memory, copy the existing elements of 'input' there and also write 42 at the end of those elements. After bar's first line, this is a picture of 'input' and 'output' during bar():

input.ptr --> [ (some data) ]

output.ptr --> [ (copy of 'input's data in dynamic memory) 42 ]

Note that while 'input's elements may be e.g. on the stack, 'output's elements are definitely in dynamic memory. The data that is in dynamic memory will be alive as long as there is at least one reference to it.

c) The data may be allocated by malloc:

import std.stdio;
import core.stdc.stdlib;

void main() {
  enum count = 7;

  // Allocate some memory
  void* rawData = malloc(int.sizeof * count);

  // Access that data as int[]
  int[] slice = (cast(int*)rawData)[0..count];

  // Yet another slice of half of those
  int[] half = slice[0..$/2];

  // Free the memory
  free(rawData);

  // Ouch! Accessing dead data
  writeln(slice);
  writeln(half);
}

So, in all three examples it is the same D feature, a slice, that references data but the data is managed in different ways.

Ali

On 1/16/22 07:32, Salih Dincer wrote:
> On Sunday, 16 January 2022 at 11:43:40 UTC, Ali Çehreli wrote:
>>
>> void main() {
>>   enum count = 7;
>>
>>   // Allocate some memory
>>   void* rawData = malloc(int.sizeof * count);

In practice, malloc'ed memory is cleared e.g. by memset(). Or, there is calloc() which returns memory filled with zeros. It takes two parameters:

  void* rawData = calloc(count, int.sizeof);

(Of course, one also needs to check that the returned value is not null before using it.)

> If count is not equal to 8 I get weird results! The reason of course, is
> the free():
> // [93947717336544, 1, 2, 3, 4, 5, 6]

I didn't know free wrote into the freed buffer but since it's undefined behavior, we shouldn't even be able to know whether it did or not. :/

> [Here](https://run.dlang.io/is/yFmXwO) is my test code.

You have the following loop there:

  foreach(i, ref s; slice) s = cast(int)i;

Note that the assignment operator into a malloc'ed buffer works only for some types like the fundamental ones. Otherwise, the opAssign() of a user-defined type may be very unhappy with random or zero data of 'this' object. (Note that there really wouldn't be any user-defined type in the buffer; we would have just cast'ed it to fool ourselves.) The reason is, opAssign() may have to destroy existing members of 'this' during the assignment and destroying random or zero data may not work for a particular type.

So, one would have to emplace() an object in that memory. Here is my explanation:

  http://ddili.org/ders/d.en/memory.html#ix_memory.construction,%20emplace

Ok, this is too much theory and those are parts of the reasons why we don't generally use calloc or malloc. :)

Ali

On Sunday, 16 January 2022 at 11:43:40 UTC, Ali Çehreli wrote:
>
> So, in all three examples it is the same D feature, a slice, that references data but the data is managed in different ways.
>
> Ali

Well, it's fair to say, that 'range-based programming' is kinda new to me.

With this statement:
 int[][] mArr = iota(1, 9).chunks(2).map!array.array;

- one would intuitively expect, that at the end, you end up with a dynamically allocated array, intialised with it's argument.

That is, you would expect some GC allocation to be going on, similar to what would happen with this code:

int[][] mArr2 = [[1, 2], [3, 4], [5, 6], [7, 8]]; // GC allocation

But it turns out:

int[][] mArr = iota(1, 9).chunks(2).map!array.array; // no GC allocation going on at all, not anywhere.

How can this be I asked myself?

Then I watched this, and learn about 'memory disallocation'.

http://dconf.org/2015/talks/bright.html

Now I understand.. I think ;-)

On 1/16/22 14:43, forkit wrote:

> Well, it's fair to say, that 'range-based programming' is kinda new to me.

I hope it will be useful to you. :)

> int[][] mArr2 = [[1, 2], [3, 4], [5, 6], [7, 8]]; // GC allocation
>
> But it turns out:
>
> int[][] mArr = iota(1, 9).chunks(2).map!array.array; // no GC allocation
> going on at all, not anywhere.

That's not correct. There are many range algorithms that are lazy to defer memory allocation but array() is not one of those. array() does eagerly allocate memory, which is it's whole purpose:

  https://dlang.org/phobos/std_array.html#array

"Allocates an array and initializes it with copies of the elements of range r."

So, that range expression has the same allocations as your "GC allocation" line above.

> Then I watched this, and learn about 'memory disallocation'.
>
> http://dconf.org/2015/talks/bright.html
>
> Now I understand.. I think ;-)

That applies to most other range algorithms. array() is an expensive one. :)

Ali

On Sunday, 16 January 2022 at 23:03:49 UTC, Ali Çehreli wrote:
>
> That's not correct. There are many range algorithms that are lazy to defer memory allocation but array() is not one of those. array() does eagerly allocate memory, which is it's whole purpose:
>
>   https://dlang.org/phobos/std_array.html#array
>
> "Allocates an array and initializes it with copies of the elements of range r."
>
> So, that range expression has the same allocations as your "GC allocation" line above.
>

Honestly, that is what I thought, and expected.

However, when I compiled with: -profile=gc  .. it indicated no GC allocation going on. That's really why I got confused - because, as you say, the array statement (as I understand it)will result in a dynamically allocated array (i.e. allocated on GC heap).

But -profile=gc .. says no. It's not.

int[][] mArr = [[1, 2], [3, 4], [5, 6], [7, 8]]; // GC allocation occurs.
int[][] mArr = iota(1, 9).chunks(2).map!array.array; // No GC allocation occurs.

On 1/16/22 15:14, forkit wrote:

> But -profile=gc .. says no. It's not.
>
> int[][] mArr = [[1, 2], [3, 4], [5, 6], [7, 8]]; // GC allocation occurs.
> int[][] mArr = iota(1, 9).chunks(2).map!array.array; // No GC allocation
> occurs.

Definitely a -profile=gc bug. Here are the existing ones:

  https://issues.dlang.org/buglist.cgi?quicksearch=profile%20gc

Ali