On 5/24/21 5:46 AM, Iain Buclaw wrote:
> On Monday, 24 May 2021 at 02:56:14 UTC, Walter Bright wrote:
>>   import dmd.argtypes_x86;
>>   import dmd.argtypes_sysv_x64;
>>   import core.stdc.string : strlen;
>>   import dmd.cond;
>>   import dmd.cppmangle;
>>   import dmd.cppmanglewin;
>>   import dmd.dclass;
>>   import dmd.declaration;
>>   import dmd.dscope;
>>   import dmd.dstruct;
>>   import dmd.dsymbol;
>>   import dmd.expression;
>>   import dmd.func;
>>   import dmd.globals;
>>   import dmd.id;
>>   import dmd.identifier;
>>   import dmd.mtype;
>>   import dmd.statement;
>>   import dmd.typesem;
>>   import dmd.tokens : TOK;
>>   import dmd.root.ctfloat;
>>   import dmd.root.outbuffer;
>>   import dmd.root.string : toDString;
>>
>> If I want to understand the code, I have to understand half of the rest of the compiler. On a more abstract level, why on earth would a target abstraction need to know about AST nodes? At least half of these imports shouldn't be here, and if they are, the code needs to be redesigned.
>>
> 
> To be fair, most of this is imported because a function needs the definition of one or more symbols.  This can be made better by either:
> 
> 1. Making these selective imports, or...
> 2. Moving type definitions of AST nodes into modules that _only_ contain definitions.

Yes, and these are good incremental steps that help a lot, are low cost, and inform larger refactorings. There should be active work on pushing imports down to where they're used.

My dream: top-level imports will become an antipattern in large D code.

On 5/24/21 6:47 AM, Johan Engelen wrote:
> On Monday, 24 May 2021 at 09:53:42 UTC, Walter Bright wrote:
>> On 5/23/2021 10:55 PM, Andrei Alexandrescu wrote:
>>> One problem with that is code duplication.
>> Sure. But in the outbuffer case, the duplication stems from backend being used in multiple projects.
>>
>> It's hard to have perfection, and if getting to perfection means driving off a cliff (!) it's better to just live with a bit of imperfection here and there.
>>
>> I don't like having two outbuffers, but the cure is worse for that particular case.
>>
>> There's even another implementation of outbuffer in Phobos (because I thought outbuffer was generally very useful):
>>
>> https://dlang.org/phobos/std_outbuffer.html
>>
>> But here we run into our rule that dmd shouldn't rely on Phobos. Compromise is inevitable. Outbuffer isn't a spike we need to impale ourselves on. There are plenty of other encapsulation problems that could be improved, like target.d.
> 
> Outbuffer is a case of a data structure that is useful throughout the compiler. So it is put in a package of the compiler that is OK to be imported from other packages (and should avoid importing other packages). I think the `dmd.root` package is exactly like such a package  (compare with `ADT` in LLVM). From that standpoint, I don't see why the `dmd.backend` package cannot import `dmd.root`. If `dmd.backend` is to be used in different projects, then those should also use `dmd.root` and that's where the dependency chain stops.

Thanks. I set out to write pretty much exactly that. To add to it:

A. High-level modules should not depend on low-level modules. Both should depend on abstractions (e.g., interfaces).
B. Abstractions should not depend on details. Details (concrete implementations) should depend on abstractions.

(Source: https://en.wikipedia.org/wiki/Dependency_inversion_principle)

Applied here:

A. The back-end should not depend on the front end. Both should depend on abstractions (e.g., interfaces) such as OutBuffer.
B. OutBuffer should not depend on memory-mapped minutia. Memory-mapped work should be done to serve OutBuffer.

> Better: if it is in Phobos, great, use that!
> If you need a certain data structure you know where to look: Phobos or `dmd.root`. Is it not in there? Don't create a new structure elsewhere, add it to `dmd.root` and import it.

I am sympathetic to the cause of not addding a large number of moving pieces to the compiler codebase. But yes the point stands.

Hopefully good versioning could help a lot with all that.

> In LDC, we use the C++ stdlib and we use Phobos, because why not? We are programming in D after all, and it is the standard library that is available in all bootstrapping compiler required versions. We do take care not to rely on _latest_ Phobos, but on Phobos from the oldest bootstrapping D compiler version supported up to the latest version. Same for the C++ stdlib (C++20 is not ok, but C++14 is much encouraged).
> 
> For example: LDC uses MD5 hashing for its machine codegen cache. `import std.digest.md; auto md5hash = md5Of(slice);`  Done.
> LLVM does the same, e.g. the project removed its own unique_ptr implementation (`OwningPtr`), and now uses `std::unique_ptr`.

Cool stuff!

> My standpoint on the original topic of "make it easier to experiment with the compiler": I disagree with making the code more stable. If anything, we should be refactoring much more aggressively, changing function names etc.

Doesn't aggressive refactoring require massive unittests?

On 5/24/21 8:38 AM, zjh wrote:
> Small refactoring is useless.
> `Big changes` are necessary.

That evokes the couple who had problems in their relationship, so they decided to solve them by getting married.

On 5/24/21 9:53 AM, 12345swordy wrote:
> On Monday, 24 May 2021 at 10:34:35 UTC, Walter Bright wrote:
>> On 5/24/2021 2:44 AM, Alexandru Ermicioi wrote:
>>> They are not simple for new volunteers to dmd.
>>
>> You're right, they are not. They're optimized for the people who spend thousands of hours working on it.
>>
>> This inevitably happens with every profession, every discipline, and every project. A jargon specific to it grows up around it, for the convenience of the people who work on it every day. If the jargon is consistent and reasonably logical, it can be a great aid to understanding once one gets familiar with it.
>>
>> Unfortunately, I have failed at my original design goal of making DMD a simple compiler. Reshuffling files around and renaming things will not help. What will help is better encapsulation - unfortunately, that is hard to do.
>>
>> There are some reasonably well-encapsulated parts. The lexer, the parser, and the files in the root package. To understand the compiler, I'd start there.
> 
> I seriously question the "Optimized for people who spend thousands of hours working on it" line, as I had a very intelligent person posted on slacks asking what does this function do, as there is no comments for said functions.

Adding documentation would be another good investment with terrific dividends. Again it minds my boggle that people talk about big changes (and no doubt would be willing to try them) but can't be bothered to make small changes with disproportionately good impact.

On Monday, 24 May 2021 at 20:45:11 UTC, Andrei Alexandrescu wrote:
> On 5/24/21 8:38 AM, zjh wrote:
>> Small refactoring is useless.
>> `Big changes` are necessary.
>
> That evokes the couple who had problems in their relationship, so they decided to solve them by getting married.

If that meant that they encapsulated their problems and had a united front towards the rest of the world, then that is the right approach for dmd. Arranged marriages are underappreciated...

On Monday, 24 May 2021 at 20:47:39 UTC, Andrei Alexandrescu wrote:
> On 5/24/21 9:53 AM, 12345swordy wrote:
>> On Monday, 24 May 2021 at 10:34:35 UTC, Walter Bright wrote:
>>> [...]
>> 
>> I seriously question the "Optimized for people who spend thousands of hours working on it" line, as I had a very intelligent person posted on slacks asking what does this function do, as there is no comments for said functions.
>
> Adding documentation would be another good investment with terrific dividends. Again it minds my boggle that people talk about big changes (and no doubt would be willing to try them) but can't be bothered to make small changes with disproportionately good impact.

Where do you start? i.e. there's always work to be done but unless you enforce change from the top you're blocking a river at the mouth (to play devil's advocate)

On Monday, 24 May 2021 at 21:01:05 UTC, Max Haughton wrote:
> On Monday, 24 May 2021 at 20:47:39 UTC, Andrei Alexandrescu wrote:
>> On 5/24/21 9:53 AM, 12345swordy wrote:
>>> On Monday, 24 May 2021 at 10:34:35 UTC, Walter Bright wrote:
>>>> [...]
>>> 
>>> I seriously question the "Optimized for people who spend thousands of hours working on it" line, as I had a very intelligent person posted on slacks asking what does this function do, as there is no comments for said functions.
>>
>> Adding documentation would be another good investment with terrific dividends. Again it minds my boggle that people talk about big changes (and no doubt would be willing to try them) but can't be bothered to make small changes with disproportionately good impact.
>
> Where do you start? i.e. there's always work to be done but unless you enforce change from the top you're blocking a river at the mouth (to play devil's advocate)

The reason I put documentation low on my list is that it has a high maintenance cost if you are going to redesign.

Also, it has not been a hindrance for experimentation for me. Probably a hindrance for fixing bugs, but that is not the topic..

In general, let us try too focus on macro issues, there is no need for dmd to be perfect in order to better support experimentation. Partitioning and interfacing is more important than statement and block level issues. Micro issues such as imports and number of OutBuffer implementations are low impact issues, those are more aesthetical in nature...

On 5/24/2021 3:51 AM, Iain Buclaw wrote:
> For instance, how else would we be able to infer `isReturnOnStack` without a `TypeFunction`?
Challenge accepted!

Let's see. The only things the TypeFunction for are:

1. the return type
2. the function linkage
3. if the function returns a ref

Pass those in instead, and the need for TypeFunction goes away.

https://github.com/dlang/dmd/blob/master/src/dmd/target.d#L762

(1) can be further broken down into "is it a POD", etc.

Breaking all this info out of a TypeFunction takes some code, but this "decomposition" can be done by a wrapper function in another module.

The end result is target.d can be completely independent of the compiler's internal AST structures.

But wait! There's more!

Notice how isReturnOnStack depends on several random global variables like os, is64bit, and isPOSIX. They can be passed in as arguments, too (or passed in as a const ref to a struct containing those as members).

isReturnOnStack() then becomes a pure function! (and safe, nogc, nothrow, all that good stuff)

Those initialize() functions go away, too.

The beauty now becomes that we can (at last!) easily and correctly write unittests for it. target.d now becomes INDEPENDENT of the rest of the compiler.

How sweet it will be!

On 5/24/2021 1:35 PM, Dukc wrote:
> On Monday, 24 May 2021 at 02:56:14 UTC, Walter Bright wrote:
>> A good rule of thumb is:
>>
>>     *** Never import a file from an uplevel directory ***
>>
>> Import sideways and down, never up.
> 
> You may want to reconsider what you just said.
> 
> Do you really insist that `std.stdio` should copy-paste the CLib headers instead of importing `core.stdc.stdio`?

I knew someone would bring that up. :-) It's a good question.

`core` is a separate library in its own, independent hierarchy, it is not in the `std` hierarchy. It is not "up sideways and down". So it's good.

Now, if std.stdio imported core.stdc.stdio, and core.stdc.stdio imported std.stdio, then you've got a really bad design.