How can we make it easier to experiment with the compiler? (page 12)

On Thursday, 27 May 2021 at 06:08:53 UTC, Ola Fosheim Grostad wrote: > Which is fair enough. Just don't pretend it aspires to be more than that, because that takes reorganization and restructuring. And let me add thar reorganization and restructuring is not a sign of failure. It is a sign of professionalism. Well run projects have this built into the process so that these important activities are not put aside or put on hold.

On 5/27/21 1:36 AM, Nicholas Wilson wrote: > Yes, a README is strictly better than nothing. It does not substitute for having organised files. Neither does well organised files substitute for a lack of README. Razvan found in https://github.com/dlang/dmd/pull/12560 a number of imports of backend modules that shouldn't be there. I wonder if this convention could be enforced by using package-level protection in backend (and elsewhere) in such a way that would have made it impossible for those imports to work. That would be a good way forward because as it goes (and went in the past) the discussion remains sterile. Once there is a demonstrable improvement brought about by packages and (self-evidently) you can't get package-level protection without packages, the case will be much easier to make. The overarching point is that better modularization should predate, inform, and motivate division in packages, not follow it.

May 27, 2021

Re: How can we make it easier to experiment with the compiler?

Posted by Ola Fosheim Grostad
in reply to Mathias LANG

Permalink

Ola Fosheim Grostad

Posted in reply to Mathias LANG

Permalink

On Thursday, 27 May 2021 at 07:14:25 UTC, Mathias LANG wrote:

On Thursday, 27 May 2021 at 06:08:53 UTC, Ola Fosheim Grostad wrote:

If that is it, then we might as well close the thread and conclude that dmd is a hobby project.

Which is fair enough. Just don't pretend it aspires to be more than that, because that takes reorganization and restructuring.

Don't forget about the many contributors that have invested thousands of hours to understand and improve the code base :)

Yes, let us not forget that they wasted many unproductive hours on trying to understand... Let us put a number on that in dollars...

While I disagree with many of Walter's arguments here, a refactorning has a lower barrier of entry than a bugfix, and is more prone to be subjective.

I dont want to enter that territory, neither bugfixes or micro level refactoring has much of an impact on people wanting to experiment. To enable that we have to look at the macro level and establish stable well designed interfaces so that changes in compiler internals have small impact on experimental components.

Well designed interfaces can hook up to internals you want to change at a later stage, but now you do at least not get more dependecies tied to things you want to replace.

So in essence, if there is a mess, first step is not to clean up the mess (could be too costly), but to hide it so that people stop depending on it.

My impression is that Walter is arguing that everything should be cleaned up first. That is not realistic.

Now to talk about what can be done to improve the DMD codebase, it's fairly obvious: ELIMINATE ALL CASTS. But not by replacing cast(XXX)e with e.isXXX(), but by actually using proper encapsulation.

Does not enable experimentation. Only a good macro level architecture enables experimentation.

The internals can to some extent be a mess, with little impact, it does not matter unless you want to change templating or type system features.

Many interesting experiments can be done by combining parser mods, runtime mods and post frontend mods.

Other interesting improvements can be done if one identifies areas in the compiler that can be isolated from the whole and where new features could be enabled. I suspect this is needed to get a solver that provides proper type unification, but I havent looked at this...

Some stuff being messy is not the big picture issue.

The big picture is to get clean points in the codebase where you can inject your own component. And to put those injection points where they have most potential for enabling experimentation.

An easy first step is to put a separation layer between frontend and backend.

On 5/27/2021 12:14 AM, Mathias LANG wrote: > Now to talk about what can be done to improve the DMD codebase, it's fairly obvious: ELIMINATE ALL CASTS. But not by replacing `cast(XXX)e` with `e.isXXX()`, but by actually using proper encapsulation. > > What I mean is that instead of switching on types, like this: > ```D > if (auto tf = t.isTypeFunction()) > (cast(FunctionDeclaration)t.sym).something(); > else if (auto td = t.isTypeDelegate()) > (cast(FunctionDeclaration)(cast(TypeFunction)t).sym).something(); > else > // Something else > ``` The isXXX() functions also make for safe casting. Your example would be: if (auto tf = t.isTypeFunction()) tf.sym.something(); else if (auto td = t.isTypeDelegate()) t.isTypeFunction().sym.isFunctionDeclaration().something(); > There are simple areas where one can start, for example making all aggregate have the most specialized type possible: `TypeDelegate.nextOf()` should return a `TypeFunction`, not a `Type`. `FunctionDeclaration.type()` should be a property that gives you a `TypeFunction`, not a `Type`, etc... FunctionDeclaration.type() can also give you a TypeError.

On 5/25/2021 3:16 AM, Dibyendu Majumdar wrote: > Wow - that's pretty fundamental. How does such code get in? I assume you own the DMD code - all changes should be vetted and approved by you? There are many people who have pull privileges.

On Thursday, 27 May 2021 at 08:41:49 UTC, Walter Bright wrote: > On 5/27/2021 12:14 AM, Mathias LANG wrote: >> Now to talk about what can be done to improve the DMD codebase, it's fairly obvious: ELIMINATE ALL CASTS. But not by replacing `cast(XXX)e` with `e.isXXX()`, but by actually using proper encapsulation. >> >> What I mean is that instead of switching on types, like this: >> ```D >> if (auto tf = t.isTypeFunction()) >> (cast(FunctionDeclaration)t.sym).something(); >> else if (auto td = t.isTypeDelegate()) >> (cast(FunctionDeclaration)(cast(TypeFunction)t).sym).something(); >> else >> // Something else >> ``` > > The isXXX() functions also make for safe casting. And this is actually the only way to dyncast cast nodes as DMD AST is extern(C++)... But TBH I think that all the isXXX family of functions should be free functions, not members funcs. All these isXXX calls are virtuals but they dont need to (although often devirtualized).

On Thursday, 27 May 2021 at 08:50:32 UTC, Basile B. wrote: > On Thursday, 27 May 2021 at 08:41:49 UTC, Walter Bright wrote: >> On 5/27/2021 12:14 AM, Mathias LANG wrote: >>> Now to talk about what can be done to improve the DMD codebase, it's fairly obvious: ELIMINATE ALL CASTS. But not by replacing `cast(XXX)e` with `e.isXXX()`, but by actually using proper encapsulation. >>> >>> What I mean is that instead of switching on types, like this: >>> ```D >>> if (auto tf = t.isTypeFunction()) >>> (cast(FunctionDeclaration)t.sym).something(); >>> else if (auto td = t.isTypeDelegate()) >>> (cast(FunctionDeclaration)(cast(TypeFunction)t).sym).something(); >>> else >>> // Something else >>> ``` >> >> The isXXX() functions also make for safe casting. > > And this is actually the only way to dyncast cast nodes as DMD AST is extern(C++)... But TBH I think that all the isXXX family of functions should be free functions, not members funcs. All these isXXX calls are virtuals but they dont need to (although often devirtualized). Other advantage of module scope isXXX functions is that the base Expression node would not need to know about all the derived. We would have a real astbase module with just Type, Statement, DSymbol, Expression. The isXXXX would be in the module that declare the XXXX class.

On 5/27/2021 1:50 AM, Basile B. wrote: > All these isXXX calls are virtuals but they dont need to (although often devirtualized). They're all `final` meaning not virtual. The intent is them being inlined.

On 5/25/2021 4:22 AM, jmh530 wrote: > I feel like this forum has ADHD sometimes. A week ago it was all up in arms about ImportC, now it's fcused on this, two weeks from now this will be forgotten and on to something else... Meanwhile, Iain and I are putting out PRs on ImportC.

On Thursday, 27 May 2021 at 09:53:19 UTC, Walter Bright wrote: > On 5/25/2021 4:22 AM, jmh530 wrote: >> I feel like this forum has ADHD sometimes. A week ago it was all up in arms about ImportC, now it's fcused on this, two weeks from now this will be forgotten and on to something else... > > Meanwhile, Iain and I are putting out PRs on ImportC. Yes, 2 people run ahead while 10 equally capable people throw their hands up in the air then walks off and start writing their own compilers.

Forums