I'm a c++ programmer trying to understand how memory allocation works in D.

I created a struct and added a destructor to it. My understanding is that structs have deterministic destructors - they are called when the struct goes out of scope (unless it is allocated with new).

Now if I put instances of the struct in a fixed size array

data[6] d;
d[3] = data(1, 2, 3);

then the destructor on all the contents is called when the array goes out of scope.

However if I add them to a dynamic array...

data[] d;
d ~= data(1, 2, 3)

Then the destructor appears to be called at some random time later. So it looks like it's the garbage collection that is doing this. That seems to go against the specification of how struct works... I'm not creating the item with "new" and as far as I can tell the array is storing instances of objects, not pointers to objects?

Is my understanding correct?
Is it documented anywhere how memory allocation works for this?

Is a dynamic array in fact storing an array of GC'd pointers to the structs? Or something else...

Hi,

On Thursday, 27 August 2015 at 10:05:31 UTC, John Burton wrote:
> understanding is that structs have deterministic destructors - they are called when the struct goes out of scope

Yes, when they are declared right at the scope, and not contained in something that might live past the current scope.

> However if I add them to a dynamic array...
> Then the destructor appears to be called at some random time later. So it looks like it's the garbage collection that is doing this.

Yeah.

> That seems to go against the specification of how struct works... I'm not creating the item with "new" and as far as I can tell the array is storing instances of objects, not pointers to objects?

The array is storing the full structs, yes. The array is GC-ably allocated and will therefore not go out of scope at end of scope. Because the array doesn't vanish here, the structs inside will not go out of scope either; they treat the array as their scope.

(I'm sure others could explain this more formally.)

There is no reference left to the array, so the GC may at a random later time deallocate the array, and thereby call ~this() on each struct.

> Is my understanding correct?

Explicit "new" is not the only way to put objects on the GC'ed heap. Putting them in a GC-ed array like this is another way. Or having them as a component in a class, then instantiating that class.

> Is it documented anywhere how memory allocation works for this?

I'll leave this for others, too.

> Is a dynamic array in fact storing an array of GC'd pointers to the structs?

No, it's the full struct.

-- Simon

On Thursday, 27 August 2015 at 10:05:31 UTC, John Burton wrote:
> I'm a c++ programmer trying to understand how memory allocation works in D.
>
> I created a struct and added a destructor to it. My understanding is that structs have deterministic destructors - they are called when the struct goes out of scope (unless it is allocated with new).
>
> Now if I put instances of the struct in a fixed size array
>
> data[6] d;
> d[3] = data(1, 2, 3);
>
> then the destructor on all the contents is called when the array goes out of scope.
>
> However if I add them to a dynamic array...
>
> data[] d;
> d ~= data(1, 2, 3)
>
> Then the destructor appears to be called at some random time later. So it looks like it's the garbage collection that is doing this. That seems to go against the specification of how struct works... I'm not creating the item with "new" and as far as I can tell the array is storing instances of objects, not pointers to objects?
>
> Is my understanding correct?
> Is it documented anywhere how memory allocation works for this?
>
> Is a dynamic array in fact storing an array of GC'd pointers to the structs? Or something else...

With a local scope,
- a static array  of data will have the destructors called on exit because memory for the memebers is not allocated on the GC-hep but on the stack frame.
- a dynamic array  of data will have the destructors called on next GC collection because the memory for the memebers is allocated on the GC-heap.
- a dynamic array of pointer to data will have the destructors called on next GC collection because the memory for the memebers is allocated on the GC-heap.

you can see this in this small program. deactivate to commented GC.collect to see the difference:

---
struct Foo {
    long v0, v1;
    ~this(){writeln(typeof(this).stringof);}
}

void localteststatic(){
    Foo[1] fl;
}

void localtestdynamic1(){
    Foo[] fl;
    fl.length = 1;
    fl.length = 0;
}

void localtestdynamic2(){
    Foo* [] fl;
    fl ~= new Foo(1);
}

void localtestdynamic3(){
    Foo[] fl;
    fl.length = 1;
    fl.length = 0;
}

void main(string[] args)
{
    import core.memory;
    localteststatic; writeln("done local test static");
    localtestdynamic3; writeln("done local test dynamic 3");
    //GC.collect;
    localtestdynamic1; writeln("done local test dynamic 1");
    //GC.collect;
    localtestdynamic2;  writeln("done local test dynamic 2");
    //GC.collect;
}
---

Also for the second question:
* fl[]: each element has a .sizeof 16 (long .size_of * 2)
* fl* []: each element has a .sizeof size_t.sizeof (this a pointer so 4 or 8).

Ok that's great thank you.

It's quite hard trying to get a proper understanding of memory allocation in D after years of C++ / RAII / unique_ptr / shared_ptr . I understand the details of course but it's going to take a while to really know it.

Is there any way to explicitly free a dynamic array when I am done with it?
(I'm more interested in the contained items destructors being called than in the memory / gc aspects)

I fear I'm trying to write C++ in D and need to get my mind around different ways to do things...

Thanks again for the updates. I've experimented some more and believe I understand.

To be honest I'm finding it very hard to find the right idioms in D for safe and efficient programming when I'm so used to C++ / RAII everywhere. I'll adapt though :P

On Thursday, 27 August 2015 at 10:49:02 UTC, John Burton wrote:
> Thanks again for the updates. I've experimented some more and believe I understand.
>
> To be honest I'm finding it very hard to find the right idioms in D for safe and efficient programming when I'm so used to C++ / RAII everywhere. I'll adapt though :P

There's also std.container.array with deterministic memory usage not reliant on the GC, as an alternative to the built-in arrays.

http://dlang.org/phobos/std_container_array.html

On Thursday, 27 August 2015 at 10:49:02 UTC, John Burton wrote:
>
> To be honest I'm finding it very hard to find the right idioms in D for safe and efficient programming when I'm so used to C++ / RAII everywhere. I'll adapt though :P

This is true for every new language you learn. You first stick to the idioms you already know before getting the right style.
My first Python code looked like C and my first D code looked like Java.

BTW, if you need a growable array with deterministic destruction, you should have a look at std.container.array.Array(T), which is equivalent to a shared_ptr<vector<T>>.

On Thursday, 27 August 2015 at 10:49:02 UTC, John Burton wrote:
> Thanks again for the updates. I've experimented some more and believe I understand.
>
> To be honest I'm finding it very hard to find the right idioms in D for safe and efficient programming when I'm so used to C++ / RAII everywhere. I'll adapt though :P

You can also do explicit
* memory allocation/deallocation
* class construction/destruction
* struct/union construction/destruction

Personally my background is Object-Pascal/Delphi which use similar memory managment to C++ (though more RAII with ownership than RAII with refcounting). For example I have those routines in my home-cooked general library:

https://github.com/BBasile/iz/blob/master/import/iz/types.d#L125
https://github.com/BBasile/iz/blob/master/import/iz/types.d#L150
https://github.com/BBasile/iz/blob/master/import/iz/types.d#L191

they allow to do manual memory managment and you'll find similar routines in several other user libraries, for example
* https://github.com/etcimon/memutils
* https://github.com/Dgame/m3
* etc...

On Thursday 27 August 2015 13:15, BBasile wrote:

> https://github.com/BBasile/iz/blob/master/import/iz/types.d#L125 https://github.com/BBasile/iz/blob/master/import/iz/types.d#L150 https://github.com/BBasile/iz/blob/master/import/iz/types.d#L191

Your use of @trusted is wrong and dangerous. @trusted functions are supposed to be memory-safe, but you're marking unsafe functions with it.

Things like a @trusted `free` [1] are just plain wrong. `free` isn't memory- safe.

The problems with @trusted templates can be more subtle. Even if the template body itself doesn't do anything unsafe, the template arguments are being trusted, too. So if the template ever calls any code from the arguments (including constructors, destructors, postblits, ...), then it cannot be marked @trusted.


[1] https://github.com/BBasile/iz/blob/master/import/iz/types.d#L112

On Thursday, 27 August 2015 at 11:45:14 UTC, anonymous wrote:
> On Thursday 27 August 2015 13:15, BBasile wrote:
>
>> https://github.com/BBasile/iz/blob/master/import/iz/types.d#L125 https://github.com/BBasile/iz/blob/master/import/iz/types.d#L150 https://github.com/BBasile/iz/blob/master/import/iz/types.d#L191
>
> Your use of @trusted is wrong and dangerous. @trusted functions are supposed to be memory-safe, but you're marking unsafe functions with it.
>
> Things like a @trusted `free` [1] are just plain wrong. `free` isn't memory- safe.
>
> The problems with @trusted templates can be more subtle. Even if the template body itself doesn't do anything unsafe, the template arguments are being trusted, too. So if the template ever calls any code from the arguments (including constructors, destructors, postblits, ...), then it cannot be marked @trusted.
>
>
> [1] https://github.com/BBasile/iz/blob/master/import/iz/types.d#L112

the pointer is checked before the call. Yes it can be dangling but free goes in pair with the newPtr funct. I plan to do better when Andrei's allocators will be released:
https://github.com/BBasile/phobos/blob/showcase-construct/std/experimental/allocator/showcase.d#L105

Anyway. I cheat a bit with attributes but as long as it's only for me...I know this kinds of functions are not phobos-level.