In a toy project I am working on with D v2.065, I came to the following situation:

Node path = solver.find (map, start, end);
if (path !is null) {
    path.writeContents();  <-- Access Violation
}

Apparently between the test and trying to use the class, the reference becomes null.

Quite strange in a single threaded application.

Any idea what might be happening or should I delve into assembly?

--
Paulo

On Thursday, 10 April 2014 at 09:24:51 UTC, Paulo Pinto wrote:
> In a toy project I am working on with D v2.065, I came to the following situation:
>
> Node path = solver.find (map, start, end);
> if (path !is null) {
>     path.writeContents();  <-- Access Violation
> }
>
> Apparently between the test and trying to use the class, the reference becomes null.
>
> Quite strange in a single threaded application.
>
> Any idea what might be happening or should I delve into assembly?
>
> --
> Paulo

Delve delve delve. Is this in an optimised build?

On Thursday, 10 April 2014 at 09:24:51 UTC, Paulo Pinto wrote:
> In a toy project I am working on with D v2.065, I came to the following situation:
>
> Node path = solver.find (map, start, end);
> if (path !is null) {
>     path.writeContents();  <-- Access Violation
> }
>
> Apparently between the test and trying to use the class, the reference becomes null.
>
> Quite strange in a single threaded application.
>
> Any idea what might be happening or should I delve into assembly?
>
> --
> Paulo

Can you post writeContents?

On Thursday, 10 April 2014 at 09:31:30 UTC, John Colvin wrote:
> On Thursday, 10 April 2014 at 09:24:51 UTC, Paulo Pinto wrote:
>> In a toy project I am working on with D v2.065, I came to the following situation:
>>
>> Node path = solver.find (map, start, end);
>> if (path !is null) {
>>    path.writeContents();  <-- Access Violation
>> }
>>
>> Apparently between the test and trying to use the class, the reference becomes null.
>>
>> Quite strange in a single threaded application.
>>
>> Any idea what might be happening or should I delve into assembly?
>>
>> --
>> Paulo
>
> Delve delve delve. Is this in an optimised build?

No, debug build.

On Thursday, 10 April 2014 at 10:37:58 UTC, Rene Zwanenburg wrote:
> On Thursday, 10 April 2014 at 09:24:51 UTC, Paulo Pinto wrote:
>> In a toy project I am working on with D v2.065, I came to the following situation:
>>
>> Node path = solver.find (map, start, end);
>> if (path !is null) {
>>    path.writeContents();  <-- Access Violation
>> }
>>
>> Apparently between the test and trying to use the class, the reference becomes null.
>>
>> Quite strange in a single threaded application.
>>
>> Any idea what might be happening or should I delve into assembly?
>>
>> --
>> Paulo
>
> Can you post writeContents?

Sure, but only today's evening. At work now.

This is a simple A* search program I was porting from C++ to D, and I plan to make it visible on Github anyway.

But it shouldn't be related to writeContents, I guess, as the access violation takes place regardless of the operation I try to apply to path.


--
Paulo

On Thursday, 10 April 2014 at 09:31:30 UTC, John Colvin wrote:
> On Thursday, 10 April 2014 at 09:24:51 UTC, Paulo Pinto wrote:
>> In a toy project I am working on with D v2.065, I came to the following situation:
>>
>> Node path = solver.find (map, start, end);
>> if (path !is null) {
>>    path.writeContents();  <-- Access Violation
>> }
>>
>> Apparently between the test and trying to use the class, the reference becomes null.
>>
>> Quite strange in a single threaded application.
>>
>> Any idea what might be happening or should I delve into assembly?
>>
>> --
>> Paulo
>
> Delve delve delve. Is this in an optimised build?

Well, I got into something but it will require more research.

The above code is


0x0040223a e8ed360100       call        solver.find (0041592c)
0x0040223f 8945f0           mov         dword ptr [path],eax
0x00402242 83c40c           add         esp,0c
0x00402245 85c0             test        eax,eax
0x00402247 7408             je          D main+00000241 (00402251)
0x00402249 8b45f0           mov         eax,dword ptr [path]
0x0040224c 8b08             mov         ecx,dword ptr [eax]
0x0040224e ff5138           call        dword ptr [ecx+38]
0x00402251 ff3564e14400     push        dword ptr [__xt_z+000000f4 (0044e164)]
0x00402257 ff3560e14400     push        dword ptr [__xt_z+000000f0 (0044e160)]
0x0040225d 837df000         cmp         dword ptr [path],00
0x00402261 0f95c0           setne       al
0x00402264 e86b8d0000       call        std.stdio.writefln!(string, bool).writefln (0040afd4)

After find() gets called, eax = 0x0030f900 which points to nowhere valid.

Hence, ecx will contain a null address and [ecx + 38] as vtable lookup points to 0x00000038, which lands in no man's land and boom.

Trying to get deep into this will require a bit more time.

I wonder if this is caused by the nodes being stored into a BinaryHeap with an Array store, coupled with a RedBlackTree as well. Maybe some interaction between the manual memory management and the GC.

More info later on.

--
Paulo

On Thursday, 10 April 2014 at 09:31:30 UTC, John Colvin wrote:
> On Thursday, 10 April 2014 at 09:24:51 UTC, Paulo Pinto wrote:
>> In a toy project I am working on with D v2.065, I came to the following situation:
>>
>> Node path = solver.find (map, start, end);
>> if (path !is null) {
>>    path.writeContents();  <-- Access Violation
>> }
>>
>> Apparently between the test and trying to use the class, the reference becomes null.
>>
>> Quite strange in a single threaded application.
>>
>> Any idea what might be happening or should I delve into assembly?
>>
>> --
>> Paulo
>
> Delve delve delve. Is this in an optimised build?

I found out the cause, apparently std.container.Array destroys the memory used by reference types as well (e.g. classes).

The small example below reproduces the error.

Apparently the way Array manages the memory on its own invalidates the reference pointed by node, as I assume bad == node, but its memory is invalid.

import std.stdio;
import std.container;


class Node {
}

Node indentity (Node n)
{
    return n;
}

Node badIndentity (Node n)
{
    Array!Node data;
    data.insert(n);
    return data.front();
}

int  main(string[] args)
{
    auto node = new Node();
    auto n = indentity (node);
    writefln("Hello the address is %s", n);
    auto bad = badIndentity (node);
    writefln("Hello the address is %s", bad);

    return 0;
}

On Friday, 11 April 2014 at 06:51:01 UTC, Paulo Pinto wrote:
> On Thursday, 10 April 2014 at 09:31:30 UTC, John Colvin wrote:
>> On Thursday, 10 April 2014 at 09:24:51 UTC, Paulo Pinto wrote:
>>> In a toy project I am working on with D v2.065, I came to the following situation:
>>>
>>> Node path = solver.find (map, start, end);
>>> if (path !is null) {
>>>   path.writeContents();  <-- Access Violation
>>> }
>>>
>>> Apparently between the test and trying to use the class, the reference becomes null.
>>>
>>> Quite strange in a single threaded application.
>>>
>>> Any idea what might be happening or should I delve into assembly?
>>>
>>> --
>>> Paulo
>>
>> Delve delve delve. Is this in an optimised build?
>
> I found out the cause, apparently std.container.Array destroys the memory used by reference types as well (e.g. classes).
>
> The small example below reproduces the error.
>
> Apparently the way Array manages the memory on its own invalidates the reference pointed by node, as I assume bad == node, but its memory is invalid.
>
> import std.stdio;
> import std.container;
>
>
> class Node {
> }
>
> Node indentity (Node n)
> {
>     return n;
> }
>
> Node badIndentity (Node n)
> {
>     Array!Node data;
>     data.insert(n);
>     return data.front();
> }
>
> int  main(string[] args)
> {
>     auto node = new Node();
>     auto n = indentity (node);
>     writefln("Hello the address is %s", n);
>     auto bad = badIndentity (node);
>     writefln("Hello the address is %s", bad);
>
>     return 0;
> }

This reminds me of the question I had about the use of appender. (http://forum.dlang.org/thread/xfnvtlzyolmtncsmmqqi@forum.dlang.org)

The internal memory management of containers and appender can be the source of subtle bugs. For cases like your badIndentity function, Objective-C has autorelease, so it's only released when the program is done with the value.
Out of interest, why would you write a function like badIndentity this way? Or is it just a proof of concept?

On Friday, 11 April 2014 at 09:32:29 UTC, Chris wrote:
> On Friday, 11 April 2014 at 06:51:01 UTC, Paulo Pinto wrote:
>> On Thursday, 10 April 2014 at 09:31:30 UTC, John Colvin wrote:
>>> On Thursday, 10 April 2014 at 09:24:51 UTC, Paulo Pinto wrote:
>>>> In a toy project I am working on with D v2.065, I came to the following situation:
>>>>
>>>> Node path = solver.find (map, start, end);
>>>> if (path !is null) {
>>>>  path.writeContents();  <-- Access Violation
>>>> }
>>>>
>>>> Apparently between the test and trying to use the class, the reference becomes null.
>>>>
>>>> Quite strange in a single threaded application.
>>>>
>>>> Any idea what might be happening or should I delve into assembly?
>>>>
>>>> --
>>>> Paulo
>>>
>>> Delve delve delve. Is this in an optimised build?
>>
>> I found out the cause, apparently std.container.Array destroys the memory used by reference types as well (e.g. classes).
>>
>> The small example below reproduces the error.
>>
>> Apparently the way Array manages the memory on its own invalidates the reference pointed by node, as I assume bad == node, but its memory is invalid.
>>
>> import std.stdio;
>> import std.container;
>>
>>
>> class Node {
>> }
>>
>> Node indentity (Node n)
>> {
>>    return n;
>> }
>>
>> Node badIndentity (Node n)
>> {
>>    Array!Node data;
>>    data.insert(n);
>>    return data.front();
>> }
>>
>> int  main(string[] args)
>> {
>>    auto node = new Node();
>>    auto n = indentity (node);
>>    writefln("Hello the address is %s", n);
>>    auto bad = badIndentity (node);
>>    writefln("Hello the address is %s", bad);
>>
>>    return 0;
>> }
>
> This reminds me of the question I had about the use of appender. (http://forum.dlang.org/thread/xfnvtlzyolmtncsmmqqi@forum.dlang.org)
>
> The internal memory management of containers and appender can be the source of subtle bugs. For cases like your badIndentity function, Objective-C has autorelease, so it's only released when the program is done with the value.
> Out of interest, why would you write a function like badIndentity this way? Or is it just a proof of concept?


This was only to show you how to reproduce the error.

On my application I use Array as a backing store for a BinaryHeap used to store the nodes that are available on the A* open list.

If this usage of Array is correct and the destroy method is misbehaving, I can create a bug for it.

I assume Array should manage its own memory but not touch the memory that belongs to reference types stored on it. At least the documentation doesn't say otherwise.

--
Paulo

On Friday, 11 April 2014 at 11:25:16 UTC, Paulo Pinto wrote:
> This was only to show you how to reproduce the error.
>
> On my application I use Array as a backing store for a BinaryHeap used to store the nodes that are available on the A* open list.
>
> If this usage of Array is correct and the destroy method is misbehaving, I can create a bug for it.
>
> I assume Array should manage its own memory but not touch the memory that belongs to reference types stored on it. At least the documentation doesn't say otherwise.

Yes, it's bug. Trivial to fix too. Please file it.