Hi D community!

I'm here again, to describe what I've done during the fifth week of
Symmetry
Autumn of Code.

## LLVM Patches follow up

During the discussion on some patches, two other patches have been
raised,
refactoring the usage of string utilities shared among other
demanglers.

- https://reviews.llvm.org/D111948
- https://reviews.llvm.org/D111947

## Second milestone kickoff

This week I started working on the second milestone, mostly covering
DWARF
issues that I discovered. The initial idea is to research a bit about
what
debug information is provided by each compiler and understand what can
be done,
not only on the LLDB side but also improve information generated by the
compilers.

During this week I also took a look at the DWARF5 standard to
understand what
to improve. I discovered very interesting tags that can help generalize
the
arrays and strings interaction with the debug info.

To give a bit of context, and more deeply into the DWARF standard,
`DW_TAG_array_type` exists to represent array types. Similarly, there
is also a
tag to specifically represent string types. These tags can be used with
`DW_TAG_subrange_type` to represent a so-called "slice" and attributes
like
`DW_AT_count` or `DW_AT_upper_bound` can be used to represent the slice
length.
Interestingly those attributes can refer to runtime variables and
therefore
create debug information for a "dynamic array". I checked other
compilers and
only C VLAs are using it.

I wrote a document describing some of the issues and comparing the
DWARF
capabilities of the 3 most known D compilers. You can read it
[here](https://gist.github.com/ljmf00/0194d03f6f30303feec9f58eb9f0c7df)
.

Some of those issues I managed to fix in the compiler backend, others I
filed a
bug. Here is a list of patches and bug reports I created:

- https://github.com/dlang/dmd/pull/13196
- https://issues.dlang.org/show_bug.cgi?id=22423
- https://github.com/dlang/dmd/pull/13200
- https://issues.dlang.org/show_bug.cgi?id=22426
- https://github.com/dlang/dmd/pull/13202
- https://github.com/dlang/dmd/pull/13204
- https://bugs.llvm.org/show_bug.cgi?id=52223

I have an upcoming patch to the DMD backend that I'm currently working
on to
refactor some of the DMD DWARF generator, specifically to the
abbreviation
code. My motivation for the refactor is the fact that I spent a lot of
time
looking into the backend codebase, implementing PR #13202. Later I
realized,
when looking into the DWARF standard, that DWARF abbreviations are all
LEB128
encoded and the code is not respecting that. I believe some of the
errors
produced by the compiler is due to that.

## LLVM Plugin

To have pretty-printing capabilities I will need to either implement
some of
those missing DWARF tags or/and write an LLDB language plugin. The last
bug
report on the list above triggers a discussion in that regard. Very
soon I'm
going to write a thread on the LLVM mailing list describing the goals
of the
plugin and discuss the possibility of upstreaming it, however, I'm not
too
confident that will happen, because Rust among other language plugins
being
removed from the upstream for "lacking maintainability".

I already started working on some D integration specifically on the
LLDB side,
although I haven't pushed any of this code, since some stuff is still
janky
experiments and not properly split up.

## What is next?

Next week, now that I have some scope about DWARF, I'm going to tackle
majorly
on the LLDB integration, since that is what the milestone is focused
on. I'm
going to clean up the idea I have of the language plugin and hopefully
show
some interesting practical results.

You can also read this on my blog,
[here](https://lsferreira.net/posts/d-saoc-2021-05).

-- 
Sincerely,
Luís Ferreira @ lsferreira.net

On Thursday, 21 October 2021 at 01:48:32 UTC, Luís Ferreira wrote:
> Hi D community!
>
> [...]

I love it! Are you planning to do more D related stuff after SAoC too?

When I made my LLDB pretty printer plugin I implemented the array and string types, which will probably become obsolete once your suggested DWARF changes are in the compilers. Additionally if you are planning to support D as an LLDB plugin, I think the Phobos types would be a great addition in the future (JSONValue, Variant, sumtype, DateTime, SysTime, Duration, Appender...) and especially if it's somehow possible to detect ranges and evaluate them.

On Thu, 2021-10-21 at 08:46 +0000, WebFreak001 via Digitalmars-d wrote:
> On Thursday, 21 October 2021 at 01:48:32 UTC, Luís Ferreira wrote:
> > Hi D community!
> > 
> > [...]
> 
> I love it! Are you planning to do more D related stuff after SAoC too?
> 

Yes. I have some ideas to improve D ecosystem and also some proposals in mind for D itself.

- One of the things I'm writing is a DIP to be able to ignore coverage.
The idea is to be able to mark code as not intended to be covered, like
dummy statements. I'm not sure if it is needed a forma DIP, but I'm
doing it anyway to have some rationale.
- Related to coverage I recently created a library/tool to convert
various coverage formats, ready to adapt on a coverage service. I want
to expand it for other well known formats and implement a C API.
- Have a stalled DIP implementation on the compiler about throw STC.
- I'm restructuring the runtime demangler to be faster and exception-
free.
- Have plans to work on a 'silly' package based test runner to support
BetterC and fancy colored diffs with green and red color diffs when
comparing actual and expected sides and syntatic sugar assert
variations for that purpose.
- Have a stale implementation of parser combinators library CTFE ready
to parse protobufs and generically other things more easily.
- I would like to, at some point, contribute to libdparse and dscanner
to implement more code smell and add the possibility of ignoring false
positives or "trust" certain code.

I have planty of other projects that I want to contribute/do but either my time is limited by other priorities or sometimes not motivated to do some of them. Normally I do all of them at the same time, and therefore suffer from the problem of implementing none.

I have my TODO list open-source, so you can check it out: https://tasks.lsferreira.net/ . I also plan to dramatically improve the interface of it, and probably D will be involved on that :P

> When I made my LLDB pretty printer plugin I implemented the array and string types, which will probably become obsolete once your suggested DWARF changes are in the compilers. Additionally if you are planning to support D as an LLDB plugin, I think the Phobos types would be a great addition in the future (JSONValue, Variant, sumtype, DateTime, SysTime, Duration, Appender...) and especially if it's somehow possible to detect ranges and evaluate them.

About that, I think implementing support for array and strings is straightforward and acceptable, because the memory layout of them are specified on the D ABI spec. Things like associative arrays and other data structures are not, and hardcoding the fields is ok but at some point in time it can break, unfortunetly. I think if we plan to upstream the plugin too (which I think will not going to happen, because Rust created a fork due to being denied by the upstream) we should not include those "ambiguous" additions, although, would be cool to find another way to export information for the debuggers. If this end up not being upstreamed, I think we can think about syncing that with the reference compiler/runtime/stdlib implementation and possibly support other major implementations. Another possibility is to standardize the memory layout of those data structures, but I think that is difficult and restrictive.

But yeah, for now, I'm going to focus on the "achievable" things and then improve it when I have some "stable" implementation.

-- 
Sincerely,
Luís Ferreira @ lsferreira.net

On Thursday, 21 October 2021 at 08:46:12 UTC, WebFreak001 wrote:
> On Thursday, 21 October 2021 at 01:48:32 UTC, Luís Ferreira wrote:
>> Hi D community!
>>
>> [...]
>
> I love it! Are you planning to do more D related stuff after SAoC too?
>
> When I made my LLDB pretty printer plugin I implemented the array and string types, which will probably become obsolete once your suggested DWARF changes are in the compilers. Additionally if you are planning to support D as an LLDB plugin, I think the Phobos types would be a great addition in the future (JSONValue, Variant, sumtype, DateTime, SysTime, Duration, Appender...) and especially if it's somehow possible to detect ranges and evaluate them.

What are the planned DWARF changes?

On Thu, 2021-10-21 at 20:50 +0000, deadalnix via Digitalmars-d wrote:
> On Thursday, 21 October 2021 at 08:46:12 UTC, WebFreak001 wrote:
> > On Thursday, 21 October 2021 at 01:48:32 UTC, Luís Ferreira wrote:
> > > Hi D community!
> > > 
> > > [...]
> > 
> > I love it! Are you planning to do more D related stuff after SAoC too?
> > 
> > When I made my LLDB pretty printer plugin I implemented the array and string types, which will probably become obsolete once your suggested DWARF changes are in the compilers. Additionally if you are planning to support D as an LLDB plugin, I think the Phobos types would be a great addition in the future (JSONValue, Variant, sumtype, DateTime, SysTime, Duration, Appender...) and especially if it's somehow possible to detect ranges and evaluate them.
> 
> What are the planned DWARF changes?

You can briefly check them on the document I mentioned. Most of the things I'm going to try to implement on the DMD, which is things implemented by either GDC or LDC.

The other things are just possibilities. We need to study them to see what is the best fit. I'm specifically talking about the arrays and strings. DWARF has a type for arrays and strings. I describe that on the post as well.

-- 
Sincerely,
Luís Ferreira @ lsferreira.net

On Thursday, 21 October 2021 at 20:50:22 UTC, deadalnix wrote:
> On Thursday, 21 October 2021 at 08:46:12 UTC, WebFreak001 wrote:
>> On Thursday, 21 October 2021 at 01:48:32 UTC, Luís Ferreira wrote:
>>> Hi D community!
>>>
>>> [...]
>>
>> I love it! Are you planning to do more D related stuff after SAoC too?
>>
>> When I made my LLDB pretty printer plugin I implemented the array and string types, which will probably become obsolete once your suggested DWARF changes are in the compilers. Additionally if you are planning to support D as an LLDB plugin, I think the Phobos types would be a great addition in the future (JSONValue, Variant, sumtype, DateTime, SysTime, Duration, Appender...) and especially if it's somehow possible to detect ranges and evaluate them.
>
> What are the planned DWARF changes?

I was referring to the linked github gist: https://gist.github.com/ljmf00/0194d03f6f30303feec9f58eb9f0c7df