DMD Technical Debt

As what happens with all long term projects, code accumulates cruft and technical
debt. People often ask what to do to improve things, so here's a list:

1. Adding attributes const/nothrow/pure/scope/@safe/private/etc.

2. Adding correct Ddoc function comment blocks. Do not use Ddoc comments for
overrides unless the overriding function does something different (as far as
the caller is concerned) than the overridden function. Ddoc comment blocks
are often overkill for nested functions and function literals; use ordinary
comments for those.

3. Get rid of strlen/strcmp and using D arrays instead.

4. Use `ref` instead of raw pointers.

5. Use nested functions to get rid of rats' nests of goto's.

6. Look for duplicative code and factor out into functions.

7. Declare local variables as `const` as much as possible.

8. Use Single Assignment for local variables:

    T t = x;
    ...
    t = y;
    ...

   becomes:

    T tx = x;
    ...
    T ty = y;
    ...

9. "Shrinkwrap" the scope of local variables as tightly as possible
around their uses.

10. Similar to (8), use distinct variable names for non-overlapping uses.

11. Avoid use of mutable globals as much as practical. Consider passing them
in as parameters.

12. Avoid use of default parameters. Spell them out.

13. Minimize use of overloading.

14. Avoid clever code. Anybody can write clever code. It takes genius to write simple code.

15. Try to reduce cyclomatic complexity, i.e. think about how to make the code work
without control flow statements.

16. Try not to mix functions that "answer a question" with functions that "mutate the data".
I was able to successfully do this in escape.d, it wasn't easy, but it was well worth it.

17. Try to eliminate reliance on `global.errors`.

18. For aggregates that expose public access to fields, think hard about why this is
necessary and if it can be done better. Merely replacing them with read/write properties
accomplishes nothing. The more of its internal details can be made private, the better.

19. Try to use function prefixes:

'is' is the parameter in a certain category?
'has' does the parameter have a certain feature?
'can' for can I do X with the parameter?

Such functions should not be mutating the data nor issuing error messages.

20. The function return value variable should be named `result`.

21. The more constrained the scope of a name is, the shorter it should be.



NOT On The Table:

1. Grand renamings of variables and functions

2. Shuffling all the code about

3. Creating packages

4. As a general rule, any improvement that is implemented by using sed scripts
across the source tree is likely to be disruptive and unlikely to provide
significant improvement.

5. Changes of more than 500 lines need buyoff from Andrei or myself.

6. Reformatting.



Keep in mind that nearly all of this technical debt is MY fault.

As always, treating the above as sacred writ is a huge mistake. Use
your best judgement on a case-by-case basis. Blindly doing things just
adds more technical debt.

On Friday, 23 February 2018 at 08:11:41 UTC, Walter Bright wrote:
> DMD Technical Debt
>
> As what happens with all long term projects, code accumulates cruft and technical
> debt. People often ask what to do to improve things, so here's a list:
>

If you don't mind, I'll as some side notes / technical points.


> 6. Look for duplicative code and factor out into functions.
>

One easy win here are visitor functions that return a result.

Instead of having code like this all over the place:
---
if (foo)
{
    this.result = e;
    return;
}
else
{
    this.result = new ErrorExp();
    return;
}
---

Make it a function!
---
if (foo)
    return setResult(e);
else
    return errorExp();

// (Names of functions are just explanatory) ...

void setResult(Expression e)
{
    this.result = e;
}

void errorExp()
{
    this.result = new ErrorExp();
}
---


> 14. Avoid clever code. Anybody can write clever code. It takes genius to write simple code.
>

14b. Rather than taking the approach of "first make it work, then make it pretty", try to go for the second stage when making a PR.

I can not stress this enough when it comes to making the language implementation as cross-platform independent as possible.

> 17. Try to eliminate reliance on `global.errors`.
>

17b. Try to elimate reliance on global gagging.

>
> 21. The more constrained the scope of a name is, the shorter it should be.
>
>

22. Try not to ask questions such as "am I Windows and 64bit?" in the decision making process of semantic analysis, instead ask questions like "do I reverse the vtable array?", which doesn't tie itself to any particular platform, or lock an implementing compiler to an ABI which is incompatible to the one its targeting.


23. Consider what is currently a public or extern(C++) method or function, and try to make it private or extern(D).  The backend / glue layer doesn't need all access to the front-end AST.  And having a stable API / abstraction layer would ultimately mean less breakages between C++ <-> D boundaries.


>
> Keep in mind that nearly all of this technical debt is MY fault.
>
> As always, treating the above as sacred writ is a huge mistake. Use
> your best judgement on a case-by-case basis. Blindly doing things just
> adds more technical debt.

I will add an extra note here, "this seems to work" counts as blindly doing things.


Iain.

On Friday, 23 February 2018 at 08:11:41 UTC, Walter Bright wrote:
> DMD Technical Debt
>
> As what happens with all long term projects, code accumulates cruft and technical
> debt. People often ask what to do to improve things, so here's a list:
>
> [...]

Very nice list.  I pretty much agree with all of this and I'm glad you've taken the time to put it all in one place.

On Friday, 23 February 2018 at 08:11:41 UTC, Walter Bright wrote:
> DMD Technical Debt
>
> As what happens with all long term projects, code accumulates cruft and technical
> debt. People often ask what to do to improve things, so here's a list:

I've been considering contributing to Dlang, and I might have the free time soon. Do you have a list of files / modules where you feel some of these issues are particularly severe?

Priorities for potential contributors, along the lines of "If you're not sure where to start, files X and Y could use some cleaning up"?

On 4/21/2018 6:58 AM, Olivier FAURE wrote:
> I've been considering contributing to Dlang, and I might have the free time soon. Do you have a list of files / modules where you feel some of these issues are particularly severe?
> 
> Priorities for potential contributors, along the lines of "If you're not sure where to start, files X and Y could use some cleaning up"?

Just pick any file and start reading it.

On 2018-04-21 15:58, Olivier FAURE wrote:

> I've been considering contributing to Dlang, and I might have the free time soon. Do you have a list of files / modules where you feel some of these issues are particularly severe?
> 
> Priorities for potential contributors, along the lines of "If you're not sure where to start, files X and Y could use some cleaning up"?

I would say the backend, because that's still written in C++. But it will require more work to port it to D.

-- 
/Jacob Carlborg

On 2018-04-21 15:58, Olivier FAURE via Dlang-internal wrote:
> On Friday, 23 February 2018 at 08:11:41 UTC, Walter Bright wrote:
>> DMD Technical Debt
>> 
>> As what happens with all long term projects, code accumulates cruft and technical
>> debt. People often ask what to do to improve things, so here's a list:
> 
> I've been considering contributing to Dlang, and I might have the free
> time soon. Do you have a list of files / modules where you feel some
> of these issues are particularly severe?
> 
> Priorities for potential contributors, along the lines of "If you're
> not sure where to start, files X and Y could use some cleaning up"?

BTW if you are interested in contributing to the standard library Phobos too, there are a few currently running cleanup projects too:

https://github.com/dlang/phobos/projects

On 4/23/2018 4:21 AM, Jacob Carlborg wrote:
> I would say the backend, because that's still written in C++. But it will require more work to port it to D.

Once https://github.com/dlang/dmd/pull/8112 passes, I will convert the backend to D.