October 30, 2021

On Saturday, 30 October 2021 at 16:55:03 UTC, Steven Schveighoffer wrote:

>

auto v = k in aa;
aa.remove(k);

How can the GC/compiler work out that there is still a reference?

??? The same way it does for all other references.

I think either you misunderstood me, or I misunderstood you.

October 30, 2021

On 10/30/21 1:38 PM, Stanislav Blinov wrote:

>

On Saturday, 30 October 2021 at 16:55:03 UTC, Steven Schveighoffer wrote:

>

auto v = k in aa;
aa.remove(k);

How can the GC/compiler work out that there is still a reference?

??? The same way it does for all other references.

I think either you misunderstood me, or I misunderstood you.

You said "deallocating unreferenced elements". I thought you meant elements unreferenced by the AA.

What I mean is, the AA isn't going to change implementations where it now deallocates values that may still have existing references. If that's the case, we can state that in the docs.

e.g.:

Removing a key does not deallocate the value that was removed. The value's lifetime is managed by the GC and will be alive until there are no references to that value.

-Steve

October 30, 2021

On Saturday, 30 October 2021 at 17:45:57 UTC, Steven Schveighoffer wrote:

>

You said "deallocating unreferenced elements". I thought you meant elements unreferenced by the AA.

Yup, I misunderstood you :)

>

What I mean is, the AA isn't going to change implementations where it now deallocates values that may still have existing references. If that's the case, we can state that in the docs.

e.g.:

Removing a key does not deallocate the value that was removed. The value's lifetime is managed by the GC and will be alive until there are no references to that value.

:whateveritistodoathumbsuphere:

October 30, 2021

On Saturday, 30 October 2021 at 00:52:23 UTC, Imperatorn wrote:

>

On Saturday, 30 October 2021 at 00:49:04 UTC, Stanislav Blinov wrote:

>

On Friday, 29 October 2021 at 21:00:48 UTC, Steven Schveighoffer wrote:

>

This is incorrect, the buckets are each heap allocated. Just the array of bucket pointers would change.

In addition, AAs do not deallocate the key/value pairs ever. You are safe to obtain a pointer to a value and it will stay there, even if you remove the key.

Who's going to document these implementation details? ;) I mean, if no one, then the above shouldn't be stated. Wouldn't you agree?

Given the premise of the question at hand, it does seem useful to know these. But at least one should stress what is and isn't subject to change (even if unlikely).

This should be documented for sure

I did small test and it printed the same values three times so even rehash doesn't change the address of the value:

            long[long] aa = [0:0];
            writeln(&aa[0]);
            foreach(i; 0 .. 100_000_000)
                    aa[i]=i;
            writeln(&aa[0]);
            aa.rehash;
            writeln(&aa[0]);

So it seems pretty safe to store a pointer to a value in AA. And I agree that this should definitely be documented.

October 30, 2021

On Saturday, 30 October 2021 at 18:31:16 UTC, Andrey Zherikov wrote:

>

I did small test and it printed the same values three times so even rehash doesn't change the address of the value:

>

So it seems pretty safe to store a pointer to a value in AA. And I agree that this should definitely be documented.

Address itself may change though. While the AA won't move stuff around, a GC might. I don't think current GC moves anything, but it's definitely allowed to. Which would be transparent to your code so long as you don't depend on the value of address itself :)

https://dlang.org/spec/garbage.html#pointers_and_gc

October 30, 2021

On Saturday, 30 October 2021 at 20:05:17 UTC, Stanislav Blinov wrote:

>

On Saturday, 30 October 2021 at 18:31:16 UTC, Andrey Zherikov wrote:

>

I did small test and it printed the same values three times so even rehash doesn't change the address of the value:

>

So it seems pretty safe to store a pointer to a value in AA. And I agree that this should definitely be documented.

Address itself may change though. While the AA won't move stuff around, a GC might. I don't think current GC moves anything, but it's definitely allowed to. Which would be transparent to your code so long as you don't depend on the value of address itself :)

https://dlang.org/spec/garbage.html#pointers_and_gc

What test could be written to verify the behaviour?

October 30, 2021

On Saturday, 30 October 2021 at 20:19:58 UTC, Imperatorn wrote:

> >

https://dlang.org/spec/garbage.html#pointers_and_gc

What test could be written to verify the behaviour?

Assuming the GC was moving?

You'd need a loop allocating different sizes, storing the addresses somewhere the GC won't see (i.e. in memory allocated not with the GC: malloc, VirtualAlloc, mmap...), orphaning some allocations, repeating a bunch of times, and then comparing addresses of remaining allocations with stored ones.

Behavior would very much depend on the GC implementation, so would a concrete test. Point is, it's allowed to move so we have to assume it would even if it doesn't, and relying on it not moving (i.e. depending on concrete addresses) is UB.

October 30, 2021

On 10/30/21 2:31 PM, Andrey Zherikov wrote:

>

On Saturday, 30 October 2021 at 00:52:23 UTC, Imperatorn wrote:

>

On Saturday, 30 October 2021 at 00:49:04 UTC, Stanislav Blinov wrote:

>

On Friday, 29 October 2021 at 21:00:48 UTC, Steven Schveighoffer wrote:

>

This is incorrect, the buckets are each heap allocated. Just the array of bucket pointers would change.

In addition, AAs do not deallocate the key/value pairs ever. You are safe to obtain a pointer to a value and it will stay there, even if you remove the key.

Who's going to document these implementation details? ;) I mean, if no one, then the above shouldn't be stated. Wouldn't you agree?

Given the premise of the question at hand, it does seem useful to know these. But at least one should stress what is and isn't subject to change (even if unlikely).

This should be documented for sure

I did small test and it printed the same values three times so even rehash doesn't change the address of the value:

             long[long] aa = [0:0];
             writeln(&aa[0]);
             foreach(i; 0 .. 100_000_000)
                     aa[i]=i;
             writeln(&aa[0]);
             aa.rehash;
             writeln(&aa[0]);

So it seems pretty safe to store a pointer to a value in AA. And I agree that this should definitely be documented.

Here is the rehash code, it definitely just moves around the buckets:

https://github.com/dlang/druntime/blob/20963f956c550b7b1b52849e5cf41f436d0df788/src/rt/aaA.d#L144-L157

That free at the end is just for the original Bucket[] array.

A Bucket is just a hash and a void pointer to the actual key/value pair. When you get a pointer to an AA value, it's to the key/value pair block (there is no struct for this, it's done via runtime type information, which is horrific legacy).

-Steve

October 30, 2021
On Saturday, 30 October 2021 at 21:20:15 UTC, Stanislav Blinov wrote:
> On Saturday, 30 October 2021 at 20:19:58 UTC, Imperatorn wrote:
>
>>> https://dlang.org/spec/garbage.html#pointers_and_gc
>>
>> What test could be written to verify the behaviour?
>
> Assuming the GC was moving?

If the GC were moving, it would also have to move the pointers you took to AA elements.  You would never get stale pointers in any event.
October 30, 2021
On Saturday, 30 October 2021 at 22:47:57 UTC, Elronnd wrote:

> If the GC were moving, it would also have to move the pointers you took to AA elements.  You would never get stale pointers in any event.

Who said you would?..