After a while my program crashes.

I'm inspecting in the debugger that some strings are overwritten after a struct is assigned to an associative array.

- I have disabled the GC.
- All happens in the same thread.
- The strings belong to an object collection inside an object created from a d-DLL.
- The object returned by the DLL function is added to the GC with GC.addRoot().
- This object also lives in a static array the whole time.
- Not all objects are affected but many.
- The struct itself looks okay also the key for the associative array has normal form.

The data is not overwritten by another Thread (only one is running) but by the compiler. I'm watching it by memory location. It gets always visible first after that assignment. But how is this even possible? In theory, how could I ran into this issue?

On Monday, 25 January 2021 at 11:15:28 UTC, frame wrote:
> After a while my program crashes.
>
> I'm inspecting in the debugger that some strings are overwritten after a struct is assigned to an associative array.
>
> - I have disabled the GC.
> - All happens in the same thread.
> - The strings belong to an object collection inside an object created from a d-DLL.
> - The object returned by the DLL function is added to the GC with GC.addRoot().
> - This object also lives in a static array the whole time.
> - Not all objects are affected but many.
> - The struct itself looks okay also the key for the associative array has normal form.
>
> The data is not overwritten by another Thread (only one is running) but by the compiler. I'm watching it by memory location. It gets always visible first after that assignment. But how is this even possible? In theory, how could I ran into this issue?

That should really not be possible.

I suspect the memory used by the original data got reused for the associative array somehow. But if the GC is off from program start, that should really not occur. Do you maybe turn the GC off before the AA assignment, but after it's already marked that memory freed? Try turning it off completely from the commandline with --DRT-gcopt=gc:manual

On Monday, 25 January 2021 at 11:25:56 UTC, FeepingCreature wrote:

> I suspect the memory used by the original data got reused for the associative array somehow. But if the GC is off from program start, that should really not occur. Do you maybe turn the GC off before the AA assignment, but after it's already marked that memory freed? Try turning it off completely from the commandline with --DRT-gcopt=gc:manual

With that option the bytes behind the strings look different, so it has an impact. But sadly it does not help.

- I have comment out all GC.free().
- GC.disable() is called in main.
- GC.profileStats.numCollections is 0.

On Monday, 25 January 2021 at 13:44:52 UTC, frame wrote:
> On Monday, 25 January 2021 at 11:25:56 UTC, FeepingCreature wrote:
>
>> I suspect the memory used by the original data got reused for the associative array somehow. But if the GC is off from program start, that should really not occur. Do you maybe turn the GC off before the AA assignment, but after it's already marked that memory freed? Try turning it off completely from the commandline with --DRT-gcopt=gc:manual
>
> With that option the bytes behind the strings look different, so it has an impact. But sadly it does not help.
>
> - I have comment out all GC.free().
> - GC.disable() is called in main.
> - GC.profileStats.numCollections is 0.

Is the object returned from dll GC allocated?

On Monday, 25 January 2021 at 14:34:23 UTC, vitamin wrote:

> Is the object returned from dll GC allocated?

The object is created on the default way. No alternating allocation. Before the object is returned it's added to GC.addRoot() which should be enough but may I'm wrong. I also tried to add each object that holds the string member with GC.addRoot().

On Monday, 25 January 2021 at 15:46:15 UTC, frame wrote:
> On Monday, 25 January 2021 at 14:34:23 UTC, vitamin wrote:
>
>> Is the object returned from dll GC allocated?
>
> The object is created on the default way. No alternating allocation. Before the object is returned it's added to GC.addRoot() which should be enough but may I'm wrong. I also tried to add each object that holds the string member with GC.addRoot().

If created on the default way mean allocated with new (=> GC) then I don't known where is problem, but if the object is allocated with other way, for example malloc, some allocator then you need tell GC about that object with GC.addRange.

On Monday, 25 January 2021 at 16:14:05 UTC, vitamin wrote:

> If created on the default way mean allocated with new (=> GC) then I don't known where is problem, but if the object is allocated with other way, for example malloc, some allocator then you need tell GC about that object with GC.addRange.

Yes with simple new operator.
Forgot to mention: the DLL itself calls a DLL.


I made following observation, don't know if it makes any sense:

Assuming that executable and DLL have their own GC instance, it seems that the --DRT option is ignored by DLLs or they do not see the command line, idk.

But using

extern (C) __gshared string[] rt_options = ["gcopt=gc:manual"];

in main and DLLs, keeps the memory intact.

Setting gc:conservative on a DLL corrupts again. Also GC.profileStats.numCollections says that there are cycles when the memory gets corrupted.

So that means that GC.addRoot() isn't "global" and must be called after the DLL function has returned the object to adapt it or within the DLL itself? or both?

On 1/25/21 8:14 AM, vitamin wrote:

> If created on the default way mean allocated with new (=> GC)

I had the same thought. The following would be the "default way" for me but passing that object's address to addRoot would be wrong:

import core.memory;

struct S {
}

void main() {
  auto a = S();
  GC.addRoot(&a); // Wrong: 'a' is on the stack
}

I'm pretty sure frame knows this but still...

Ali

On Monday, 25 January 2021 at 16:44:40 UTC, frame wrote:
> On Monday, 25 January 2021 at 16:14:05 UTC, vitamin wrote:
>
>> [...]
>
> Yes with simple new operator.
> Forgot to mention: the DLL itself calls a DLL.
>
> [...]

If there are separated GCs for DLLs, then you must call GC.addRoot() from DLL where you allocate that object.

On Monday, 25 January 2021 at 16:54:42 UTC, vitamin wrote:
> On Monday, 25 January 2021 at 16:44:40 UTC, frame wrote:
>> On Monday, 25 January 2021 at 16:14:05 UTC, vitamin wrote:
>>
>>> [...]
>>
>> Yes with simple new operator.
>> Forgot to mention: the DLL itself calls a DLL.
>>
>> [...]
>
> If there are separated GCs for DLLs, then you must call GC.addRoot() from DLL where you allocate that object.

Yes, I directly calling on every function that returns an object:

T fun(T)(T object) {
  GC.addRoot(cast(void*) object);
}
...
extern (C) export SomeObject bar() {
    return fun(new SomeObject);
}

Wrong way?