How can we make it easier to experiment with the compiler? (page 3)

On 5/24/2021 2:55 AM, Alexandru Ermicioi wrote: > It will be a huge help if they are though. At minimum it will organize the things into packages that have one purpose, which will help in understanding the structure of dmd, and also make navigation and search of desired functionality easier, compared to one flat package. It establishes a fake hierarchy that is *not* expressed in the code. Poor encapsulation is the problem, and this does nothing to solve it. > This can actually be the first step at unwinding all the mess with imports you're mentioning, since packages are not just folders, but logical organization of a set of modules that are somewhat related to the purpose the package has. It's backwards. Fix the rust on the car, then repaint it.

On Monday, 24 May 2021 at 09:41:12 UTC, Walter Bright wrote: > On 5/24/2021 1:40 AM, poffer wrote: >> A good enhancement to the language would be adding some sort of module declaration that just states the admitted import packages or modules. I know that could be done by an external tool, but I feel that this one is a common problem. > > Importing unused modules is a problem, but a minor one. The larger problem is needing those modules. No. What I mean is a declaration that for example, allows only import from dmd in dmd/backend, of declare that imports from dmd/root are forbidden. Aren't you the guy pushing from declarations over conventions? Conventions do not scale.

On 5/24/2021 2:46 AM, Iain Buclaw wrote: > To be fair, most of this is imported because a function needs the definition of one or more symbols. This can be made better by either: > > 1. Making these selective imports, or... That doesn't really help, the dependencies are still there. > 2. Moving type definitions of AST nodes into modules that _only_ contain definitions. It is not critical that we fix target.d. It's just that it would be better if its API was not AST nodes, but just values. Let the caller construct the AST node from the information provided. Like what we did for the C parser. I was happy to have it not indirectly import everything in dmd when all it needed was a couple values. I'm not saying any of this is easy.

On Monday, 24 May 2021 at 10:15:53 UTC, Walter Bright wrote: > On 5/24/2021 2:55 AM, Alexandru Ermicioi wrote: >> It will be a huge help if they are though. At minimum it will organize the things into packages that have one purpose, which will help in understanding the structure of dmd, and also make navigation and search of desired functionality easier, compared to one flat package. > > It establishes a fake hierarchy that is *not* expressed in the code. > > Poor encapsulation is the problem, and this does nothing to solve it. It _is_ in the code. FFS, the AST is literally a hierarchy!

On Monday, 24 May 2021 at 09:39:42 UTC, Walter Bright wrote: > On 5/24/2021 1:35 AM, Nicholas Wilson wrote: >> Did you read _literally nothing else_ that I wrote? > > I read it, my response was to the entire posting. Then I can only conclude you have absolutely no perspective for someone who has little or no experience with the DMD codebase. I have had multiple GSoC/SAoC students and I have spoken to perhaps two dozen people at various dconfs who I consider to be well versed in D and all of them have complained that the lack of organisation of the files in DMD to be a significant hinderance to contribution to the point where many simply do not bother. Many of these people are regular commits to phobos and druntime.

On 5/24/2021 2:44 AM, Alexandru Ermicioi wrote: > They are not simple for new volunteers to dmd. You're right, they are not. They're optimized for the people who spend thousands of hours working on it. This inevitably happens with every profession, every discipline, and every project. A jargon specific to it grows up around it, for the convenience of the people who work on it every day. If the jargon is consistent and reasonably logical, it can be a great aid to understanding once one gets familiar with it. Unfortunately, I have failed at my original design goal of making DMD a simple compiler. Reshuffling files around and renaming things will not help. What will help is better encapsulation - unfortunately, that is hard to do. There are some reasonably well-encapsulated parts. The lexer, the parser, and the files in the root package. To understand the compiler, I'd start there.

On Monday, 24 May 2021 at 09:53:42 UTC, Walter Bright wrote: > But here we run into our rule that dmd shouldn't rely on Phobos. Compromise is inevitable. Outbuffer isn't a spike we need to impale ourselves on. There are plenty of other encapsulation problems that could be improved, like target.d. *ahem* https://github.com/dlang/dmd/pull/12574 It's a start at least.

On Monday, 24 May 2021 at 10:04:49 UTC, Walter Bright wrote: > On 5/24/2021 2:39 AM, Walter Bright wrote: >> On 5/24/2021 1:35 AM, Nicholas Wilson wrote: >>> Did you read _literally nothing else_ that I wrote? >> >> I read it, my response was to the entire posting. > > To be a little clearer, if the files are all merely reshuffled into various packages, then they all violate the rule: > > *** Never import a file from an uplevel directory *** > > and understanding is not increased at all. And it isn't even just an uplevel directory, it's up then sideways then down. Putting the files into directories would make those violations obvious and serve as documentation on how the deps should be. Than all others can start work towards that goal.

May 24, 2021

Re: How can we make it easier to experiment with the compiler?

Posted by Johan Engelen
in reply to Walter Bright

Permalink

Johan Engelen

Posted in reply to Walter Bright

Permalink

On Monday, 24 May 2021 at 09:53:42 UTC, Walter Bright wrote:

On 5/23/2021 10:55 PM, Andrei Alexandrescu wrote:

One problem with that is code duplication.
Sure. But in the outbuffer case, the duplication stems from backend being used in multiple projects.

It's hard to have perfection, and if getting to perfection means driving off a cliff (!) it's better to just live with a bit of imperfection here and there.

I don't like having two outbuffers, but the cure is worse for that particular case.

There's even another implementation of outbuffer in Phobos (because I thought outbuffer was generally very useful):

https://dlang.org/phobos/std_outbuffer.html

But here we run into our rule that dmd shouldn't rely on Phobos. Compromise is inevitable. Outbuffer isn't a spike we need to impale ourselves on. There are plenty of other encapsulation problems that could be improved, like target.d.

Outbuffer is a case of a data structure that is useful throughout the compiler. So it is put in a package of the compiler that is OK to be imported from other packages (and should avoid importing other packages). I think the dmd.root package is exactly like such a package (compare with ADT in LLVM). From that standpoint, I don't see why the dmd.backend package cannot import dmd.root. If dmd.backend is to be used in different projects, then those should also use dmd.root and that's where the dependency chain stops.
Better: if it is in Phobos, great, use that!
If you need a certain data structure you know where to look: Phobos or dmd.root. Is it not in there? Don't create a new structure elsewhere, add it to dmd.root and import it.

In LDC, we use the C++ stdlib and we use Phobos, because why not? We are programming in D after all, and it is the standard library that is available in all bootstrapping compiler required versions. We do take care not to rely on latest Phobos, but on Phobos from the oldest bootstrapping D compiler version supported up to the latest version. Same for the C++ stdlib (C++20 is not ok, but C++14 is much encouraged).

For example: LDC uses MD5 hashing for its machine codegen cache. import std.digest.md; auto md5hash = md5Of(slice); Done.
LLVM does the same, e.g. the project removed its own unique_ptr implementation (OwningPtr), and now uses std::unique_ptr.

My standpoint on the original topic of "make it easier to experiment with the compiler": I disagree with making the code more stable. If anything, we should be refactoring much more aggressively, changing function names etc. Nicholas mentions that it is a pain to keep up with LLVM, where even function names are renamed from "SortSomeName" to "SortSomeNames" (made-up example), because plural is correct. The pain would be much more if all unfixed small incorrectness/clumsiness/etc. accumulates over time and you end up with a convoluted codebase...
The main stumbling block is already mentioned: ownership. If you want contributors, you have to give up some ownership and be willing to make compromises between your own and the new contributors' viewpoints. The lack of willingness to give up ownership is what keeps me out (and I suspect, indirectly, others too). The frontend source code is not nice, but I'm not drawn to fix it at all (even if paid for) because I am not ashamed by it as I would be if I would have some shared 'ownership' of it.

-Johan

Forums