Thread overview
Compiler-assisted code reflection for IDEs
Jul 16, 2021
SealabJaster
Jul 16, 2021
Per Nordlöw
Jul 16, 2021
rikki cattermole
Jul 16, 2021
SealabJaster
Jul 16, 2021
H. S. Teoh
Jul 17, 2021
SealabJaster
Jul 18, 2021
russhy
Jul 20, 2021
SealabJaster
Jul 22, 2021
JN
July 16, 2021

A common flaw with things like code-d and (in my limited experience) Visual-D, these plugins are unable to resolve even simple things like:

struct S(T)
{
    T a;
}

...

S!int.init.// No autocompletion from this point.

I was wondering if functionality can be added onto the compiler (DMD at least, due to its speed) which allows for external tooling to gain accurate information about the code?

Take the above struct for example, maybe when the plugin detects the following:

S!int myS;

It could make a call to the compiler like: dmd myapp.d -reflect "myapp.S!(int)", where the compiler would return all the info it can (It's a struct, it has a member of int called a, etc.)

It's just a rough idea, but I hope you can see what I'm trying to suggest, and why I feel it's at least worth asking about.

In other words, build some much needed language service functionality into the compiler, to allow the actual language servers to provide a much higher quality experience. Naively assuming it's as simple as that :3

July 16, 2021

On Friday, 16 July 2021 at 10:02:06 UTC, SealabJaster wrote:

>

I was wondering if functionality can be added onto the compiler (DMD at least, due to its speed) which allows for external tooling to gain accurate information about the code?

Take the above struct for example, maybe when the plugin detects the following:

S!int myS;

It could make a call to the compiler like: dmd myapp.d -reflect "myapp.S!(int)", where the compiler would return all the info it can (It's a struct, it has a member of int called a, etc.)

It's just a rough idea, but I hope you can see what I'm trying to suggest, and why I feel it's at least worth asking about.

In other words, build some much needed language service functionality into the compiler, to allow the actual language servers to provide a much higher quality experience. Naively assuming it's as simple as that :3

That's absolutely possible. DMD is available as a library. You basically take the source code feed it to the engine, apply whatever compiler passes you want, and accept the results using a visitor of your choice. There are already visitors available; I'd suggest starting with SemanticTimeTransitiveVisitor because it's a transitive visitor (less boiler-plate) that should be applied after all the semantic phases has been applied.

I suggest you first get in contact with the developers of Visual-D (Rainer Schutze) code-d, DCD (D completion daemon) and others to see what's already been done. They likely have similar visions.

Good luck!

July 17, 2021
On 17/07/2021 2:38 AM, Per Nordlöw wrote:
> I suggest you first get in contact with the developers of Visual-D (Rainer Schutze) code-d, DCD (D completion daemon) and others to see what's already been done. They likely have similar visions.

VisualD is the only one of importance in terms of discussion as it is the only one that uses dmd-fe for completion.

Everything else that is active pretty much only uses DCD and we have known for ages it needs a rewrite to use dmd-fe. But once DCD is rewritten, all existing IDE's that support it will just work in terms of switching over.
July 16, 2021

On Friday, 16 July 2021 at 14:38:07 UTC, Per Nordlöw wrote:

>

...

I wasn't quite suggesting doing it myself. While I could try I know I wouldn't be able to get anywhere, at least for quite a long time.

I was more seeking discussion on the possibility and silently hoping someone with more experience would take up the mantle >:D

>

But once DCD is rewritten, all existing IDE's that support it will just work in > terms of switching over.

So an ETA of roughly 5 years minimum then. Doesn't dmd's frontend have the issue of poorly managing its memory (issue for longer running tools), or is that mostly fixed now?

July 16, 2021
On Fri, Jul 16, 2021 at 10:11:37PM +0000, SealabJaster via Digitalmars-d wrote: [...]
> So an ETA of roughly 5 years minimum then. Doesn't dmd's frontend have the issue of poorly managing its memory (issue for longer running tools), or is that mostly fixed now?

There's an option to turn on the GC for DMDFE.  But the fact remains that the code was written with single execution in mind, meaning that once you call it to compile a code snippet, you cannot call it again without running into problems. There's global state and stuff that doesn't get cleaned up (this is part of why dmd is so fast: it doesn't bother cleaning up anything, just does its job ASAP then lets the OS clean up afterwards).

So you have to reboot the DMDFE component for each compilation request anyway, meaning memory management is mostly moot (just load it up as a subprocess, do its thing, then let the OS reclaim the subprocess memory).  It was never meant to be a long-running process.

The code may have been updated more recently to make it more friendly for reuse, but AFAIK the bulk of the code still assumes single invocation only.


T

-- 
Today's society is one of specialization: as you grow, you learn more and more about less and less. Eventually, you know everything about nothing.
July 17, 2021

On Friday, 16 July 2021 at 22:11:37 UTC, SealabJaster wrote:

>

Spent a good half a day learning and testing out dmd-fe.

T_T My head hurts slightly. I feel I'm definitely not the right person to try and do this.

July 18, 2021

That's one of the issue with IDEs in D, everything works nice and great, including debugging, but when you play with generics, then nothing works (code completion)

Generics/Templates is one of D's strength, lot to gain from making sure that works too, setting up a work group to tackle it would a great investment imo

But it's hard to find people with expertise in the subject, and it is harder to motivate people to do the work, because it's not an easy job and requires allocating significant amount of free time

July 20, 2021

On Sunday, 18 July 2021 at 22:14:50 UTC, russhy wrote:

>

But it's hard to find people with expertise in the subject, and it is harder to motivate people to do the work, because it's not an easy job and requires allocating significant amount of free time

I could only wish to be even a fraction as educated and knowledgeable as a lot of the people on this forums.

Sadly though, that'd likely mean I'd no longer have the free time to do anything!
... except I don't have the expertise to really do anything either :D!

July 22, 2021

On Sunday, 18 July 2021 at 22:14:50 UTC, russhy wrote:

>

But it's hard to find people with expertise in the subject, and it is harder to motivate people to do the work, because it's not an easy job and requires allocating significant amount of free time

I think expertise might not be enough. C++ has been trying to solve this problem for decades with much more manpower thrown at it and IDEs still mostly give up when encountering templates.