Hi D community!

I'm here again, to describe what I've done during the eighth week of Symmetry Autumn of Code. Sorry for being a bit late.

## LLVM upstream follow up update

The following patches got merged:
- https://reviews.llvm.org/D111414
- https://reviews.llvm.org/D111432
- https://reviews.llvm.org/D110578

This includes the most important patch of the Milestone 1, which is initial support for D demangling. After a conversation with Chris Lattner (LLVM lead developer and creator) I managed to get acceptance on merging that change, leaving relicensing issues aside from my task list. This also means that minimal support for D in LLDB is now on the official tree.

Right after the merge, I submitted a patch to Google OSS Fuzz, to start fuzzing D demangler: https://github.com/google/oss-fuzz/pull/6811 . Some more patches are maybe required, since the last docker is really out-of-date and is using deprecated features of the LLVM build system.

## Fixed issues and bug hunting

### Types with wrong exported names in DWARF

As reported in the previous week, I pushed a fix for [issue 22469](https://issues.dlang.org/show_bug.cgi?id=22469) this week, in which you can find [here](https://github.com/dlang/dmd/pull/13274). I also fixed a nit specification issue, https://github.com/dlang/dlang.org/pull/3119 .

So TL;DR, now debug info with DMD should report the following type names instead:

```
wchar_t -> wchar
long double -> real
_Bool -> bool
long long -> long
uint long long -> ulong
imaginary float -> ifloat
imaginary double -> idouble
imaginary long double -> ireal
complex float -> cfloat
complex double -> cdouble
complex long double -> creal
```

### Delegates debug info now conform with the specification on LDC

This trivial change on LDC was required in order to provide the current
member
name, according to the specification:
https://github.com/ldc-developers/ldc/pull/3866 .

### LDC should generate type qualifier tags

I though that LDC issues was trivial to fix, but some required upstream
support. This particular issue, is not a blocker, since only immutable
tag is missing on the upstream. So, I made a collection of patches to
fix that there:
- https://reviews.llvm.org/D113632
- https://reviews.llvm.org/D113633
- https://reviews.llvm.org/D113634

You can follow up this issue
[here](https://github.com/ldc-developers/ldc/issues/3867) and
[here](https://bugs.llvm.org/show_bug.cgi?id=52471).

### LDC should generate column DECL attributes on debug info

This is also an issue that requires upstream changes. I though those
changes were trivial, but apparently, they require bytecode changes. I
tried to make a fix, that is half backed
(https://github.com/ljmf00/llvm-project/tree/add-di-column-type) and
currently freezed, due to lack of knowledge on bytecode read/write on
the LLVM part. You can follow up the discussion of this issue in either
LDC and LLVM bugzilla:
- https://github.com/ldc-developers/ldc/issues/3865
- https://bugs.llvm.org/show_bug.cgi?id=52470

## Calling convention in DMD/LDC is wrong

Calling convention in D compilers is currently behaving wrong, and we need to fix this.

I decided to investigate how D calling convention works, in order to thinker with calling a function on debuggers. I thought D was using the standard calling convention with additional features like hidden parameters for context pointers, but I was wrong.

I tested a few things on [godbolt](https://godbolt.org/z/sWz4x37bb), and realized that the parameters are passed to the CPU registers in the reverse order. I pushed a fix on the specification, thinking that we use a custom calling convention, since LDC is also relying on this behaviour, but apparently all the implementations are not conformant and the specification is right.

### What does this means in terms of debugging?

Well, with the wrong calling convention, debuggers can't reliably call functions and undefined behaviour happens, since the parameters are passed to the wrong registers. This is not a blocker for what I'm currently planning, but should definitely be fixed, in order to go forward with that feature, plus, binaries generated by different compilers can't be reliably linked.

You can check the specification PR I made and appreciate discussion about that topic [here](https://github.com/dlang/dlang.org/pull/3120) .

## Other reported issues and trivial patches

I also reported the following issues:

- https://issues.dlang.org/show_bug.cgi?id=22492
- https://issues.dlang.org/show_bug.cgi?id=22493

I've made some other trivial patches not worth much attention:
- https://reviews.llvm.org/D113604
- https://reviews.llvm.org/D113605
- https://reviews.llvm.org/D113572
- https://reviews.llvm.org/D113631

## What is next?

Regarding all these DWARF issues, I'm going to continue fixing some of them in parallel and probably going to start implementing the D TypeSystem and DWARFFASTParser, required for the LLDB plugin. These are things not trivial to me, but I'm going to try to have some output.

Read this on my blog
[here](https://lsferreira.net/posts/d-saoc-2021-08).

-- 
Sincerely,
Luís Ferreira @ lsferreira.net

On Thursday, 11 November 2021 at 16:20:57 UTC, Luís Ferreira wrote:
> Hi D community!
>
> I'm here again, to describe what I've done during the eighth week of Symmetry Autumn of Code. Sorry for being a bit late.
>
> [...]

Well done 👏

On Thursday, 11 November 2021 at 16:20:57 UTC, Luís Ferreira wrote:
> Hi D community!
>
> I'm here again, to describe what I've done during the eighth week of Symmetry Autumn of Code. Sorry for being a bit late.
>
> [...]

really nice improvements being made here! The next LLDB release will definitely be a must-have and the debug improvements in the compilers already add tons of value.

On Thursday, 11 November 2021 at 16:20:57 UTC, Luís Ferreira wrote:
> Hi D community!
>
> I'm here again, to describe what I've done during the eighth week of Symmetry Autumn of Code. Sorry for being a bit late.
>
> ## LLVM upstream follow up update
>
> The following patches got merged:
> - https://reviews.llvm.org/D111414
> - https://reviews.llvm.org/D111432
> - https://reviews.llvm.org/D110578
>
> This includes the most important patch of the Milestone 1, which is initial support for D demangling. After a conversation with Chris Lattner (LLVM lead developer and creator) I managed to get acceptance on merging that change, leaving relicensing issues aside from my task list. This also means that minimal support for D in LLDB is now on the official tree.

Congratulations Luis, indeed a major milestone! You made it to the LLVM weekly newsletter!

> Right after the merge, I submitted a patch to Google OSS Fuzz, to start fuzzing D demangler: https://github.com/google/oss-fuzz/pull/6811 .

Shameless plug: fuzzing also works with LDC, but I don't think we have any fuzzing of D projects (the stdlib?) at OSS Fuzz yet... ;)
https://johanengelen.github.io/ldc/2018/01/14/Fuzzing-with-LDC.html

cheers,
  Johan

On Mon, 2021-11-15 at 22:12 +0000, Johan via Digitalmars-d wrote:
> On Thursday, 11 November 2021 at 16:20:57 UTC, Luís Ferreira wrote:
> > Hi D community!
> > 
> > I'm here again, to describe what I've done during the eighth week of Symmetry Autumn of Code. Sorry for being a bit late.
> > 
> > ## LLVM upstream follow up update
> > 
> > The following patches got merged:
> > - https://reviews.llvm.org/D111414
> > - https://reviews.llvm.org/D111432
> > - https://reviews.llvm.org/D110578
> > 
> > This includes the most important patch of the Milestone 1, which is initial support for D demangling. After a conversation with Chris Lattner (LLVM lead developer and creator) I managed to get acceptance on merging that change, leaving relicensing issues aside from my task list. This also means that minimal support for D in LLDB is now on the official tree.
> 
> Congratulations Luis, indeed a major milestone! You made it to the LLVM weekly newsletter!
> 

Thanks! I didn't know about LLVM weekly newsletter, and should definitely follow it!

> > Right after the merge, I submitted a patch to Google OSS Fuzz, to start fuzzing D demangler: https://github.com/google/oss-fuzz/pull/6811 .
> 
> Shameless plug: fuzzing also works with LDC, but I don't think we have any fuzzing of D projects (the stdlib?) at OSS Fuzz yet... ;) https://johanengelen.github.io/ldc/2018/01/14/Fuzzing-with-LDC.html
> 

Thanks for the valuable resources.

Yes. That is on one of my side plans, but first we need to fix some
stuff in the compiler/standard library/druntime:
- Memory leaks in the compiler invocations even with `-lowmem`.
- ASAN/UBSAN reports stack overflow when CTFE stack overflows, which we
should check before "exploding".
- There is some heap usage after free() call and other issues reported
by ASAN/UBSAN with the current testsuite. See
https://issues.dlang.org/show_bug.cgi?id=22450 for context.
- Some standard library/runtime parts requires LDC-specific changes to
run successfully, especially in the standard library with inline asm
blocks present on math code (e.g. x87 implementations of
cos/sin/tan/...).

I have done some fixes on the standard library upstream to try to run the test suite with LDC and a custom test runner. That is why I discovered some problems related with LDC.

We can and should still do some fuzzing even with limited heuristics. We can also consider doing that already with the LDC fork, which probably doesn't have most of these prolems (maybe still some memory leaks due to sharing the frontend).


-- 
Sincerely,
Luís Ferreira @ lsferreira.net