So this is the second thread in my series to explore potential GSoC projects for students and provide more detailed insights, ideas, impression and advise for them.

The first thread can be found here [1] and the list of all potential GSoC ideas is still at [2].

This thread I want to devote to DMD as it's obviously one (if not the) most important piece in D's infrastructure.

The wiki already suggests a few ideas for DMD:

- header generation for C/C++ [3]
- language server protocol for D based on DMD as a library [4]

And there's also DIP1014 [5] which could be implemented as part of a GSoC project.

Anyhow, I make a start and list my personal favorite shortcomings of DMD:

- better error messages (see e.g. Adam's recent thread [6])
- multiple alias this (see [7] for the last attempt)
- string interpolation (see [8] for the last attempt)

There are a few more DMD goals defined in last year's survey (see e.g. [9]), though some of them are too much work for the three months of GSoC.

@community: what other features do you miss the most from DMD that could be done in the span of a GSoC?

[1] https://forum.dlang.org/post/eftttpylxanvxjhoigqu@forum.dlang.org
[2] https://wiki.dlang.org/GSOC_2019_Ideas
[3] https://wiki.dlang.org/GSOC_2019_Ideas#Header_generation_for_C.2FC.2B.2B
[4] https://wiki.dlang.org/GSOC_2019_Ideas#Language_Server_Protocol_for_D
[5] https://github.com/dlang/DIPs/blob/master/DIPs/accepted/DIP1014.md
[6] https://forum.dlang.org/post/amiprwpqkkmbdbhaggua@forum.dlang.org
[7] https://github.com/dlang/dmd/pull/8378
[8] https://github.com/dlang/dmd/pull/7988
[9] https://rawgit.com/wilzbach/state-of-d/master/report.html#180373345

On Thursday, 7 March 2019 at 09:53:13 UTC, Seb wrote:
> @community: what other features do you miss the most from DMD that could be done in the span of a GSoC?

I would like to have an additional coverage report format. Instead of having multiple
lst files, I would like to have 1 file. Either in XML or JSON format. Something
like Cobertura format would be great:

<coverage lines-valid="463" lines-covered="299" line-rate="0.6458">
<packages>
<package name="runtime.shell" line-rate="0.01666">
<class name="runtime.shell.app" filename="source\runtime\shell\app.d" line-rate="1">
<lines>
<line number="9" hits="1"/>
</lines>
</class>
<class name="runtime.shell.command" filename="source\runtime\shell\command.d" line-rate="0">
<lines>
<line number="17" hits="0"/>
<line number="19" hits="0"/>
<line number="29" hits="0"/>
<line number="30" hits="0"/>
<line number="37" hits="0"/>

The luxury solution of course would be, if I could hook into the coverage report logic of DMD and could write my own coverage reporter.

Kind regards
André

On Thursday, 7 March 2019 at 09:53:13 UTC, Seb wrote:
> @community: what other features do you miss the most from DMD that could be done in the span of a GSoC?

Convert runtime hooks [1] to templates

Example
--------
void main()
{
    auto bArray = new byte[5];
    bArray.length = 8;  // gets lowered to `_d_arraysetlengthT`
}

`bArray.length = 8` gets lowered by the compiler to the runtime hook `_d_arraysetlengthT` [2].  If you look at the signature for `_d_arraysetlengthT`, you'll see that it takes a runtime `TypeInfo` object. That is unfortunate because everything the compiler needs for the implementation is known at compile-time. Therefore, `_d_arraysetlengthT` should be converted to a template, and the compiler should be changed to lower `bArray.length = 8` to the template implementation.

Lucia Cojocaru led the charge on this in 2017.  She gave a great talk about it at DConf 2017 [3].  Her motivations for doing this work came accross much different than mine, but still valid and beneficial to D.  For me, one of the problems with `TypeInfo` is that it is a class, and clesses require a great deal of runtime support in D.  If the runtime hooks have a dependency on `TypeInfo` classes, then that means every D language feature that gets lowered to a runtime hook also depends on runtime support for classes.  Runtime support for classes includes the GC, and that doesn't scale well to bare-metal and resource-constrained platforms that neither want, nor need, that runtime overhead.  Implementing the runtime hooks as templates leverages D's all-star features such as design by introspection, metaprogramming, and CTFE which can yield performance benefits as demonstrated by Lucia.

Although it was before my time, my understanding for why things are currently implemented with runtime `TypeInfo` when everything is known at compile-time is because the runtime hooks were written before D had templates. Now that D has templates, it seems, at least to me, that templates is the obvious choice.

There are challenges with this effort, however. In the compiler, the lowerings are performed after the semantic phase in e2ir.d [4]. This means that currently, for example, setting the length of a dynamic array can be done in a `@safe`, `pure`, and `nothrow` context [5], but `_d_arraysetlengthT` is neither `@safe`, `pure`, nor `nothrow` [2]. The compiler is lying to us. So, if you convert these runtime hooks to templates, the lowering will be done in the semantic phase, and suddenly, you'll be forced to be honest. After a conversation with Walter, I was told to make use of `pureMalloc` and friends [6] to work around the `pure` constraint. `trusted` can be used to work around the `@safe` constraint, and instead of throwing exceptions in `nothrow` code, `assert`ions can be used.

See [7] for an example implementation.

Mike

[1] https://wiki.dlang.org/Runtime_Hooks
[2] https://github.com/dlang/druntime/blob/c86e4a0e541dc63f9e9f2ee09c7efd325c0be2d5/src/rt/lifetime.d#L1443
[3] https://www.youtube.com/watch?v=endKC3fDxqs
[4] https://github.com/dlang/dmd/blob/f4f29c3e47ee30b2646cede35ed0f9bd823f0add/src/dmd/e2ir.d#L2611
[5] https://run.dlang.io/is/2aefNu
[6] https://github.com/dlang/druntime/blob/29f495b5b3571484bcf4d0af2fe211d6b7d86830/src/core/memory.d#L862
[7] https://github.com/dlang/dmd/pull/8531

On Thursday, 7 March 2019 at 09:53:13 UTC, Seb wrote:
> So this is the second thread in my series to explore potential GSoC projects for students and provide more detailed insights, ideas, impression and advise for them.
>
> [...]

Compile time float to string conversion would also be great. See thread here

https://forum.dlang.org/thread/uytbknagemrzwlhawxud@forum.dlang.org

Kind regards
Andre

On Thursday, 7 March 2019 at 09:53:13 UTC, Seb wrote:
> @community: what other features do you miss the most from DMD that could be done in the span of a GSoC?

`alloca` cannot be used without linking in druntime [1], and `scope` arrays should allocate on the stack [2]

In DMD `alloca` is implemented in druntime, so to do dynamic stack allocation, one must link in druntime.  -betterC does not link in the druntime, so dynamic stack allocation cannot be done in -betterC compiled code.  This is only a limitation of DMD, however; GDC and LDC can both do dynamic stack allocation without linking in druntime.

Also, it is odd that `scope` classes are allocated on the stack, but `scope` arrays are not.  My understanding is that allocating `scope` classes on the stack is done only as an optimization, but I don't see why that optimization wouldn't apply equally well to `scope` arrays.

Mike

[1] https://issues.dlang.org/show_bug.cgi?id=19159
[2] https://issues.dlang.org/show_bug.cgi?id=18788

On Thu, Mar 07, 2019 at 09:53:13AM +0000, Seb via Digitalmars-d wrote: [...]
> @community: what other features do you miss the most from DMD that could be done in the span of a GSoC?
[...]

Enable the GC in DMD and have it not crash and/or otherwise produce wrong results / behaviour.

This is critical for making dmd (and D in general) *not* look like laughing stock on low-memory machines.


T

-- 
What are you when you run out of Monet? Baroque.

On 2019-03-07 10:53, Seb wrote:

> The wiki already suggests a few ideas for DMD:

Library related:

* Source code transformations

* Incremental compilation. I'm not talking about recompiling a whole module and relink. I'm talking about lexing and parsing exactly only the text that has changed in a file since last time. Then run the semantic phases and code generation on those changes

New features to the language:

* Struct initialization that works everywhere [1]
* Better support for tuples [2]

[1] https://github.com/dlang/DIPs/pull/71
[2] https://github.com/tgehr/DIPs/blob/tuple-syntax/DIPs/DIP1xxx-tg.md

-- 
/Jacob Carlborg

On Thursday, 7 March 2019 at 09:53:13 UTC, Seb wrote:
> So this is the second thread in my series to explore potential GSoC projects for students and provide more detailed insights, ideas, impression and advise for them.

This [1] is a good starting point to make wishes come true.

[1] https://dlang.typeform.com/report/H1GTak/PY9NhHkcBFG0t6ig

On Thursday, 7 March 2019 at 09:53:13 UTC, Seb wrote:

> @community: what other features do you miss the most from DMD that could be done in the span of a GSoC?

There is currently an unfinished project in the DMD repository that is intended to replace all makefiles with a script written in D [1].  It was started by Seb [2] (Thanks! Seb) and I tried to move it along [3].

There are a number of reasons for doing this.

1. The current build system relies on Posix-y and Nix-y tooling to work. Much of that tooling is not available by default on Windows, so a number of 3rd party dependencies need to be installed first to get a build on Windows.  This increases the barrier to entry for those wishing to contribute to D or build one's own compiler.

2. The build system is difficult to understand, further heightening the barrier to entry.  The one thing all contributors to D have in common is the D language itself.  It will be much easier to understand and debug a build system written in D than one written with makefiles.

3. Why in the world do we work so hard building extremely powerful programming languages like D to automate our lives, and then resort to inferior tools like makefiles to build our software? rdmd makes scripting in D easy an convenient.

build.d currently works for Linux, but it needs more work for Windows.  It's almost there; it just needs a little more work.

It currently mimics the makefiles, as it was ported from them, but I believe that once build.d is working, it can be refactored to something more idiomatic-D that all proficient D users can easily understand and debug.

Mike

[1] https://github.com/dlang/dmd/blob/master/src/build.d
[2] https://github.com/dlang/dmd/pull/8293
[3] https://github.com/dlang/dmd/pulls?q=is%3Apr+build.d+author%3AJinShil

On Thu, Mar 14, 2019 at 12:16:26AM +0000, Mike Franklin via Digitalmars-d wrote: [...]
> There is currently an unfinished project in the DMD repository that is intended to replace all makefiles with a script written in D [1].  It was started by Seb [2] (Thanks! Seb) and I tried to move it along [3].

Finally!


> There are a number of reasons for doing this.
> 
> 1. The current build system relies on Posix-y and Nix-y tooling to work.  Much of that tooling is not available by default on Windows, so a number of 3rd party dependencies need to be installed first to get a build on Windows.  This increases the barrier to entry for those wishing to contribute to D or build one's own compiler.

Yes, that is rather annoying. (Not that I'd know, though, since I don't
use Windows.)


> 2. The build system is difficult to understand, further heightening the barrier to entry.  The one thing all contributors to D have in common is the D language itself.  It will be much easier to understand and debug a build system written in D than one written with makefiles.

That depends on how well-written the build system is. :-P  As somebody once said, "you can write assembly code in *any* language..."


> 3. Why in the world do we work so hard building extremely powerful programming languages like D to automate our lives, and then resort to inferior tools like makefiles to build our software? rdmd makes scripting in D easy an convenient.

This has come up many times before, with the usual response being Andrei saying that nothing stands out among the large crowd of makefile alternatives.  Has this stance changed since?


> build.d currently works for Linux, but it needs more work for Windows. It's almost there; it just needs a little more work.
> 
> It currently mimics the makefiles, as it was ported from them, but I believe that once build.d is working, it can be refactored to something more idiomatic-D that all proficient D users can easily understand and debug.
[...]

OK, some questions.

1) Why not use an existing build system written in D?  There's Button,
Atila's build tool (sorry forgot what it was called), Dub (ick - sorry),
etc..

2) Won't there be bootstrapping issues to consider, if building dmd requires build.d, but build.d needs to be compiled with dmd?


T

-- 
Valentine's Day: an occasion for florists to reach into the wallets of nominal lovers in dire need of being reminded to profess their hypothetical love for their long-forgotten.