I almost finished my -gc for Natvis experiment (to use MS C++ Debugger from VS Code, fully translating type names from what comes from DMD frontend to valid C++ type name so Natvis works), but then I encountered dAssocArray, which has form of: void* as CodeView structure member, KeyType __key_t and ValueType __val_t as nested types.

As for someone who just found about CodeView today: what is nested type, what is the deal with void*, what to do with __key_t and __val_t to get values.

On 25/09/2018 22:08, Void-995 wrote:
> I almost finished my -gc for Natvis experiment (to use MS C++ Debugger from VS Code, fully translating type names from what comes from DMD frontend to valid C++ type name so Natvis works), but then I encountered dAssocArray, which has form of: void* as CodeView structure member, KeyType __key_t and ValueType __val_t as nested types.
> 
> As for someone who just found about CodeView today: what is nested type, what is the deal with void*, what to do with __key_t and __val_t to get values.

Nested types are types declared in another type, e.g. in C++

struct dAssocArray
{
	typedef int __key_t;
	typedef long __val_t;
	void* ptr;
};

The AA data structures are not exposed by the compiler, but the mago debugger rebuilds them from these types.

On Wednesday, 26 September 2018 at 07:37:26 UTC, Rainer Schuetze wrote:
>
>
> On 25/09/2018 22:08, Void-995 wrote:
>> I almost finished my -gc for Natvis experiment (to use MS C++ Debugger from VS Code, fully translating type names from what comes from DMD frontend to valid C++ type name so Natvis works), but then I encountered dAssocArray, which has form of: void* as CodeView structure member, KeyType __key_t and ValueType __val_t as nested types.
>> 
>> As for someone who just found about CodeView today: what is nested type, what is the deal with void*, what to do with __key_t and __val_t to get values.
>
> Nested types are types declared in another type, e.g. in C++
>
> struct dAssocArray
> {
> 	typedef int __key_t;
> 	typedef long __val_t;
> 	void* ptr;
> };
>
> The AA data structures are not exposed by the compiler, but the mago debugger rebuilds them from these types.

Makes sense. I've tried to move them to "member" but they obviously did not point anywhere, which is logical if that is just "typedef". Is there any way to expose real key/values from backend or get them via some Natvis magic? That's basically only thing that stops from using MS C/C++ plug-in for debugging from VS Code. Natvis support may be valuable for Linux and Mac as well, as they've added minimal support for Natvis to GDB and LLDB via their plug-in.

On 26/09/2018 09:45, Void-995 wrote:
> On Wednesday, 26 September 2018 at 07:37:26 UTC, Rainer Schuetze wrote:
>>
>>
>> On 25/09/2018 22:08, Void-995 wrote:
>>> I almost finished my -gc for Natvis experiment (to use MS C++ Debugger from VS Code, fully translating type names from what comes from DMD frontend to valid C++ type name so Natvis works), but then I encountered dAssocArray, which has form of: void* as CodeView structure member, KeyType __key_t and ValueType __val_t as nested types.
>>>
>>> As for someone who just found about CodeView today: what is nested type, what is the deal with void*, what to do with __key_t and __val_t to get values.
>>
>> Nested types are types declared in another type, e.g. in C++
>>
>> struct dAssocArray
>> {
>>     typedef int __key_t;
>>     typedef long __val_t;
>>     void* ptr;
>> };
>>
>> The AA data structures are not exposed by the compiler, but the mago debugger rebuilds them from these types.
> 
> Makes sense. I've tried to move them to "member" but they obviously did not point anywhere, which is logical if that is just "typedef". Is there any way to expose real key/values from backend or get them via some Natvis magic? That's basically only thing that stops from using MS C/C++ plug-in for debugging from VS Code. Natvis support may be valuable for Linux and Mac as well, as they've added minimal support for Natvis to GDB and LLDB via their plug-in.

Not sure if you can define new structs in NatVis, but the void pointer points to rt.aaA.Impl which contains an array of Buckets, and their entry member points to a std::pair<__key_t,__val_t>.

On Wednesday, 26 September 2018 at 18:06:48 UTC, Rainer Schuetze wrote:
>
>
> On 26/09/2018 09:45, Void-995 wrote:
>> On Wednesday, 26 September 2018 at 07:37:26 UTC, Rainer Schuetze wrote:
>>> [...]
>> 
>> Makes sense. I've tried to move them to "member" but they obviously did not point anywhere, which is logical if that is just "typedef". Is there any way to expose real key/values from backend or get them via some Natvis magic? That's basically only thing that stops from using MS C/C++ plug-in for debugging from VS Code. Natvis support may be valuable for Linux and Mac as well, as they've added minimal support for Natvis to GDB and LLDB via their plug-in.
>
> Not sure if you can define new structs in NatVis, but the void pointer points to rt.aaA.Impl which contains an array of Buckets, and their entry member points to a std::pair<__key_t,__val_t>.

Thanks, I did something like that meanwhile: https://www.dropbox.com/s/h70m4v63jxp79g1/d.natvis?dl=0

Need to take a look at other ideas as well. Of course it won't hurt to clean that up, but this one works.

On Wednesday, 26 September 2018 at 18:48:27 UTC, Void-995 wrote:
> On Wednesday, 26 September 2018 at 18:06:48 UTC, Rainer Schuetze wrote:
>>
>>
>> On 26/09/2018 09:45, Void-995 wrote:
>>> On Wednesday, 26 September 2018 at 07:37:26 UTC, Rainer Schuetze wrote:
>>>> [...]
>>> 
>>> Makes sense. I've tried to move them to "member" but they obviously did not point anywhere, which is logical if that is just "typedef". Is there any way to expose real key/values from backend or get them via some Natvis magic? That's basically only thing that stops from using MS C/C++ plug-in for debugging from VS Code. Natvis support may be valuable for Linux and Mac as well, as they've added minimal support for Natvis to GDB and LLDB via their plug-in.
>>
>> Not sure if you can define new structs in NatVis, but the void pointer points to rt.aaA.Impl which contains an array of Buckets, and their entry member points to a std::pair<__key_t,__val_t>.
>
> Thanks, I did something like that meanwhile: https://www.dropbox.com/s/h70m4v63jxp79g1/d.natvis?dl=0
>
> Need to take a look at other ideas as well. Of course it won't hurt to clean that up, but this one works.

My C++ naming for NatVis experiment has been failed right now. Somewhere around memory allocations which are all that kind of spaghetti right now in backend of DMD. But! For x86_mscoff and x86_64 associative arrays are named just as `dAssocArray`, while slices are using full/real name from D. It's pretty easy to get it to be named as `dArray`, just as it was before. Of course, adding additional nested type to allow NatVis to know about actual type would be great as well, as for no reason it ignores pointer type and thinks it's `void*`, so we would need an additional cast in NatVis. With that simple change, next NatVis file would totally work as you expect: https://www.dropbox.com/s/t5yejma3w7cbna9/dlang.natvis?dl=0

Which might be a best we can get to make VS Code experience on Windows with DMD and debugging more comfortable. I hope it will be useful addition to D ecosystem for others as well.

On 27/09/2018 18:52, Void-995 wrote:
> On Wednesday, 26 September 2018 at 18:48:27 UTC, Void-995 wrote:
>> On Wednesday, 26 September 2018 at 18:06:48 UTC, Rainer Schuetze wrote:
>>> Not sure if you can define new structs in NatVis, but the void pointer points to rt.aaA.Impl which contains an array of Buckets, and their entry member points to a std::pair<__key_t,__val_t>.
>>
>> Thanks, I did something like that meanwhile: https://www.dropbox.com/s/h70m4v63jxp79g1/d.natvis?dl=0
>>
>> Need to take a look at other ideas as well. Of course it won't hurt to clean that up, but this one works.
> 
> My C++ naming for NatVis experiment has been failed right now. Somewhere around memory allocations which are all that kind of spaghetti right now in backend of DMD. But! For x86_mscoff and x86_64 associative arrays are named just as `dAssocArray`, while slices are using full/real name from D. It's pretty easy to get it to be named as `dArray`, just as it was before. Of course, adding additional nested type to allow NatVis to know about actual type would be great as well, as for no reason it ignores pointer type and thinks it's `void*`, so we would need an additional cast in NatVis. With that simple change, next NatVis file would totally work as you expect: https://www.dropbox.com/s/t5yejma3w7cbna9/dlang.natvis?dl=0
> 
> Which might be a best we can get to make VS Code experience on Windows with DMD and debugging more comfortable. I hope it will be useful addition to D ecosystem for others as well.

Reminds me of autoexp.dat, the predecessor of NatVis. With a little help from cv2pdb I managed to show slices and AAs in the C++ debugger aswell, but it was horrible to figure out as the debugger crashed if there was something unexpected in the visualizer macros, needing a bunch of workarounds. I guess this has improved with NatVis, has it?

You could also capture the type of template arguments in the type name. Doesn't NatVis feature this, too? That should allow casting the slice pointer (though the debug info should be correct, too).

autoexp.dat macros were also pretty slow, so showing a larger AA could cause the debugger to grind to a halt. How does NatVis perform?

For a more advanced problem: show the value of `aa["7"]` in the watch window ;-)

On Friday, 28 September 2018 at 16:32:36 UTC, Rainer Schuetze wrote:
>
>
> On 27/09/2018 18:52, Void-995 wrote:
>> On Wednesday, 26 September 2018 at 18:48:27 UTC, Void-995 wrote:
>>> On Wednesday, 26 September 2018 at 18:06:48 UTC, Rainer Schuetze wrote:
>>>> Not sure if you can define new structs in NatVis, but the void pointer points to rt.aaA.Impl which contains an array of Buckets, and their entry member points to a std::pair<__key_t,__val_t>.
>>>
>>> Thanks, I did something like that meanwhile: https://www.dropbox.com/s/h70m4v63jxp79g1/d.natvis?dl=0
>>>
>>> Need to take a look at other ideas as well. Of course it won't hurt to clean that up, but this one works.
>> 
>> My C++ naming for NatVis experiment has been failed right now. Somewhere around memory allocations which are all that kind of spaghetti right now in backend of DMD. But! For x86_mscoff and x86_64 associative arrays are named just as `dAssocArray`, while slices are using full/real name from D. It's pretty easy to get it to be named as `dArray`, just as it was before. Of course, adding additional nested type to allow NatVis to know about actual type would be great as well, as for no reason it ignores pointer type and thinks it's `void*`, so we would need an additional cast in NatVis. With that simple change, next NatVis file would totally work as you expect: https://www.dropbox.com/s/t5yejma3w7cbna9/dlang.natvis?dl=0
>> 
>> Which might be a best we can get to make VS Code experience on Windows with DMD and debugging more comfortable. I hope it will be useful addition to D ecosystem for others as well.
>
> Reminds me of autoexp.dat, the predecessor of NatVis. With a little help from cv2pdb I managed to show slices and AAs in the C++ debugger aswell, but it was horrible to figure out as the debugger crashed if there was something unexpected in the visualizer macros, needing a bunch of workarounds. I guess this has improved with NatVis, has it?
>
> You could also capture the type of template arguments in the type name. Doesn't NatVis feature this, too? That should allow casting the slice pointer (though the debug info should be correct, too).
>
> autoexp.dat macros were also pretty slow, so showing a larger AA could cause the debugger to grind to a halt. How does NatVis perform?
>
> For a more advanced problem: show the value of `aa["7"]` in the watch window ;-)

NatVis pretty fast from my experience, at least for my current needs.

After some additional digging into the topic I'm thinking about DIP or something to "standardize" debug type names. CodeView for `x86_mscoff` and `x86_64` on Windows works in one way, DWARF code path on Linux seems to generate totally another names, LDC2 on Windows says that AA is `unsigned char *`.

With help from WebFreak, the author of VS Code plug-in for D, we found that the only thing that was really missing on MIEngine, that powers GDB/LLDB debugging in Microsoft's plug-in, is the support for `CustomListItems` tag that I'm using for AA. I'm even willing to add `CustomListItems` support for MIEngine by getting a bit dirty with C#. But there a problem with debug types naming:
+ DWARF:
- `int[]` - `_Array_int`
- `int[int]` - `_AArray_int_int`
+ CV8 (CV4?):
- `int[]` - `int[]`
- `int[int]` - `dAssocArray`
+ What actually works if we'll change CV8 (CV4?):
- `int[]` - `darray.ptrid:423` (423 is an internal type ID for example, can work as `darray.int[]` as well, whatever, needs prefix `whatever.` to make NatVis happy)
- `int[int]` - `dassocarray.keyid:423_valueid:423` (again, 423 is an internal type ID for example, it would be great to do dassocarray.int[int], but setting identifier inside of DMD's backend for AA's type crashes DMD, while working perfectly fine when same is done to Slice)

And now to the fun part... If I will change CV8 to what works - I will break Mago in VisualD. And I would have to change naming in DWARF as well. Which means to make D debugging with DMD working in VS Code everywhere (Windows, Linux, Mac), I need to:
- Change DWARF naming in DMD
- Add support of `CustomListItems` to MIEngine
- Change CV8 naming (CV4?) in DMD
- Make sure Mago would use prefix checking with new names instead of total matching for AA (and same for Slices).

So my proposal (at least at this moment) for `int[]` and `int[int]` is to generate/use `darray.int[]` and `dassocarray.int[int]` with `dassocarray.keyid:423_valueid:423` as temporary workaround (that will work as well) for DMD. LDC2/GDC may use normal variant without workaround. This one will allow NatVis be happy on Windows and with patching MIEngine - on Linux and Mac as well. I guess it shouldn't interfere with how Pretty Printer is working for native GDB as well (LLDB too?).

Why the prefix + suffix? More like why adding suffix for AA, you might ask. Because NatVis is caching how it handles the type, and second time he will see `dAssocArray` - he will think that this AA has the same type as previous one, while __key_t and __val_t could be different.