The monthly meeting for March took place on March 3rd, 2023, at 14:30 UTC, and lasted about an hour and fifteen minutes. The following people attended:

• Walter Bright
• Iain Buclaw
• Ali Çehreli
• Martin Kinkelin
• Dennis Korpel
• Mathias Lang
• Átila Neves
• Razvan Nitu
• Mike Parker
• Robert Schadek

The summary

Dennis

Dennis had nothing for us this time.

Robert

Robert had nothing other than to say that the compiler is still too slow and he still likes D.

Razvan

Preview Switches

Razvan had been looking at the reported issues for -preview=nosharedaccess and had found only a handful from Átila. This suggests that either we have a good implementation without many bugs, or it isn't being used very much. Átila suggested no one is using it at all. Razvan noted that one of the bugs Átila had reported was that trying to instantiate a shared class resulted in an error. If that doesn't work, there may be other issues. He wondered what path we could take to get to where the preview becomes the default. Other than waiting for people to use it and report bugs on it, what can we do?

Átila said he had been trying to get Phobos and DRuntime to compile with it, and then once they do, try it on dub packages. He said he'd encountered so much noise, he decided to wait until the bugs he'd reported are fixed before reporting more.

Iain suggested we should have a pipeline that builds with all preview switches turned on by default. Dennis noted that would be a problem with preview switches that are broken.

Mathias asked what a good litmus test for previews would look like. He'd had problems with nosharedaccess and rvaluerefparam not working when he tried them, then he'd given up on previews. Átila didn't know a good litmus test but was working on going through the previews and deciding on either making it the default or scrapping it, especially the ones that had been there for years, so that they aren't left in limbo. I suggested that at some point on the other side of our organizational build-up, we establish a policy for periodic reviews of previews.

Razvan said his main question is whether preview switches are serving their intended purpose. They were started so that people could test new features before they are officially part of the language, but in practice, it seems they are rarely used. He thinks we should have a different mechanism to ensure new features are tested before they are merged into a compiler release, then once they are merged they're ready for use. The preview switch seems to him like a graveyard for features.

I said that seems like a communication issue. Right now, we just throw a feature behind a preview switch and do nothing beyond that. I suggested we come up with a staged process that motivates people to use the previews. Robert would go a step further. He thinks there should be a burden on anyone wanting to merge a new feature to test the feature with the list of dub packages that get tested with new compiler releases, then investigate any errors. The top 30 or whatever packages should compile both with and without the preview before it's allowed in. Átila agreed.

Mathias said the idea of the preview is fine and it can work. We just need to take it seriously. He said he agrees with Robert. When he implemented -preview=in, he did exactly what Robert suggested. He enabled it by default and then examined what it broke. Generally, previews are implemented with half-baked code and we don't test if they work. We need to ensure that people implementing previews do the work correctly.

Walter said we could submit a pull request for each preview feature to turn it on by default. We don't have to merge them. That would ensure that it's run through BuildKite (which tests several dub packages), and we'll have links to everything that breaks. Then we reach out to the maintainers of broken packages to help upgrade their code.

In the end, we decided that Razvan and Átila will gather up issues with all of the preview switches and we can decide their fates at future meetings.

@system and pure

Razvan said that once in a while there are issues with @system and code and pure. He says that trying to do system-level things, like allocating memory without initializing it, can clash with purity, whereas in @safe code purity will usually be compatible. But then you can do something like void initialize a stack variable in @safe code and return it, and now you have this clash between two concepts. Looking forward to the day when safety and DIP1000 become the default, he was wondering if there's a way we can enforce that safety and purity don't go along together.

This led to a side discussion about why people would do things like allocating memory without initializing it. Robert said that whether they should or shouldn't be doing it, the language allows it. Either the language should prevent it, or it should work, or give useful error messages. Just telling people not to do it isn't practical.

Razvan brought up std.experimental.allocator as an example. Something like Mallocator can't be pure, but allocators like that still need to be used in pure code. Walter said that fundamentally a storage allocator cannot be pure, and the language is correct in its behavior. If you want to call an allocator from pure code, then you're going to have to do something dirty in your code. And that's okay. The language shouldn't support by default making a storage allocator pure. It's doing its job, saying you can't do this because you're changing state. So you have to fake it by doing a dirty cast in there somewhere. The reason why D supports system code is so you can do dirty things when you have to. It's the same with const. D does not support logical const, because logical const is not const. It's not enforceable. That's a feature. If you want logical const, you have to do something dirty to get it, and that's appropriate.

Walter also said that whether void initialization should be allowed in pure code is a different question. Void initializing an int and returning it isn't pure, is it? So maybe we should disallow void initializations in pure code. He thinks that's a reasonable proposition. Átila agreed and added that system calls are by their very nature not pure. It's okay that they're not pure because it would be lying to say otherwise. Walter agreed. System calls usually change the system. There's no way the compiler can tell. Maybe some really are pure, and we could individually mark them as such. But in general, you just have to cheat on it. He likes that D's notions of const and pure are stringent.

Dennis had what he called a "wild proposition" that he wasn't sure he agreed with himself: what if we get rid of pure? People seem to dislike attributes, and he hasn't met anyone saying "Wow, D has purity and it saves all my code". What does it buy us? Átila said that he says that, so now Dennis has met someone who does. Dennis said he uses pure as well, but he never feels like he really needs it.

Átila said that at Symmetry, because people didn't use pure when they should, there's missing functionality that wouldn't be missing had they used it. He cited an example from the SIL project (Symmetry's internal DSL). The TL;DR: a feature to look up a function name that involves calling functions to get their outputs---you don't want to call a function that's going to format your hard drive. If a D function is marked pure, then it's also pure in SIL and therefore callable in this case.

Walter noted that pure functions also can be made multi-threaded without risk. They live on their own island that doesn't interact with anything else. Átila said you can also initialize immutable things from pure functions, which is also very handy. Robert said that in his D code at Symmetry, there are a few functions that are pure because it was simple to mark them. Nothing else is pure or immutable. For every example, he bets we can find a counterexample.

There was more discussion in this vein, then Walter said that pure can be difficult to use but should be a goal. A good principle in programming is that everything a function does should come through the front door. If you can make a function pure, then you know that's the case. You know certain things about it: it has no side effects, it's thread-safe, etc.

Dennis's motivation for bringing this up was complaints about attribute soup. This led to a short discussion that meandered through tangential topics and isn't worth summarizing here.

Iain

Iain reported that he'd released 2.102.2 on March 1st and followed that with the 2.103.3 beta which had been cut and merged into the stable branch. So for the next two weeks, upstream dmd development should be focused on stabilizing the stable branch before the release candidate, so preferably all bug fixes should target stable and any open bug fixes should be retargeted to stable. Once the RC was released (as it was on March 16), then things could go back to normal where only regression fixes target stable.

On the GDC front, a major new GCC version is released every year, typically in May or the end of April. So the speed of GCC 13 development was winding down. Iain was following the dmd stable branch, so the GDC 13 release should be around D 1.103.1. At the moment, he had a couple of issues to resolve. He'd had a few missing template symbols when running the test suite, probably due to a change in the template emission strategy. The reason it affects him is that he builds DRuntime and Phobos using separate compilation. He hadn't yet found the time to resolve that.

Martin asked which front-end version the template emission bugs showed up with and whether it was DRuntime or Phobos showing them. Iain said it was in mainline Phobos after January. They come back as unresolved because of link time failures. He's aware he's probably the only person who builds DRuntime and Phobos using separate compilation. Martin said he only uses separate compilation for unit tests now, otherwise, memory usage explodes. Iain said he compiles unit tests separately as well. It's very helpful to compile each unit test in isolation, as you can amp up the parallelism on the make side, and one failed unit test module doesn't prevent the testing of other modules.

Iain said there had been a few codegen glue changes as well, which had been introducing regressions. Because a change like this is in the backend of dmd, he can work out how it's implemented and figure out what he needs to be doing to implement it for GDC. Then once he does, he spends 15 minutes figuring out how to break it. Then the first test he writes shows that there's run-time corruption.

He'd been constrained on time for the past few months, but he did hope to be on 2.103 by the end of March, regardless.

GDC documentation was now online at gnu.org. He said there were probably still a few gaps left to fill.

He had been supporting C++ developers in GCC porting D to their platform. They'd raised some points, some of which can be easily explained, e.g., the distinction between struct and class, but others are more interesting. One example that turns them off is that we still generate virtual function tables for extern(C++) classes that only contain fields and no functions. Another is that D doesn't allow dashes for identifier symbols, so that e.g., foo-bar.d doesn't work as a module name, whereas in C++ you can #include foo-bar.h. They find it very surprising to have file name constraints like that. He's not sure there's anything we need to do about it, but it's sometimes interesting to get opinions from someone who thinks differently when trying to use our language.

Dennis noted that the dashes in file names came up in ImportC, specifically when trying to import the Wayland headers. Walter said you just write a wrapper for it. You make another header file that just #includes the problematic one, or put the problematic file on the command line and use -mv to rename it. He doesn't see it as a significant problem. Iain says that you're then just working around a compiler that's working against you. Walter said neither of the two ways to deal with it is onerous, so he doesn't see that we need a language change for that.

I noted that the whole point behind ImportC is that it "just works". That's the goal. Walter agreed. I said that the people who need that "just works" aspect aren't going to be aware of the two workarounds. Walter said it should be documented. Átila said he didn't even know -mv existed. Martin added that if you're dealing with a small set of files, the wrapper workaround is fine, but if it's a large code base with numerous problematic header file names, it doesn't work.

Walter said he has so many issues with ImportC that he considers this one low priority and not a blocker. ImportC wasn't compiling the standard header files. That one was a blocker because there was no workaround, and those should have priority. The file names problem has a workaround, so isn't a blocker. In any case, ImportC is going to need a valid identifier to use as a module name. Maybe something like pragma(mangle) could help.

Iain had a sudden thought: do we have a document on how to write D code coming from a C++ mindset? Walter said things like this should go into the page on interfacing with C++. It's an old document that's probably in desperate need of modernizing. Dennis and Mathias both linked the C++ to D page in the chat, and Mathias suggested it's a better place for the C++ mindset stuff as the interface page is more about the language features.

I mentioned a book called 'C++ for Java Programmers'. A 'D for C++ Programmers' book might be nice if someone with intimate knowledge of both had time to write it. Walter suggested that D might have the same issues Java has when interfacing with C++, so that book might serve as a guide for what the D version would need to cover.

Ali

Ali didn't have anything to report and just wanted to say hello to everyone.

Átila

Átila reported he'd had some internal discussions at Symmetry about DIP1008 because of some problems they'd encountered interacting with C#. Because D and C# each have a GC, they don't want to just hand over memory allocated from one GC to the other side, so they'd used @nogc everywhere. They ran into problems with exceptions. DIP1008 didn't work for them, and they ended up having to manually deal with the reference counts. The root of their problem is that they're interacting with C# and the D library they're using is quirky, so he's not sure if there's anything actionable for us.

Martin added that they'd be quite happy if @nogc checks were skipped for throw expressions. Especially with the recent change to not use the GC in stack trace generation. He wanted to experiment with this to see if it helped with their use case at Symmetry.

Átila had recently been talking to Iain about ImportC and needed to talk to Walter about it later. He's also been talking with Mathias about dub. Átila thinks it makes sense to take the stuff he'd done in reggae and put it in dub. Most people are going to use dub, and that's what we point them to, so it makes sense to put reggae's functionality there.

He repeated that he'd been going through the preview switches to decide their fate. If they're new, that's fine, but things like rvaluerefparam and nosharedaccess that had been there for a long time need to either be made the default or dropped. He'd started by trying to compile things with nosharedaccess and immediately encountered a bunch of bugs.

Finally, he asked what everyone thought about including Phobos in the dmd/druntime mono repository. He said he'd seen no good reason not to in the forum thread he'd started. Iain said that from his perspective, DRuntime and Phobos are the same thing anyway. Mathias voiced support for it. Robert said that if it happened, the first thing he'd do is replace some instances of filter with the Phobos version, as he thinks we should be using Phobos in DMD anyway. (Note that no final decision was made here.)

Walter

Standalone lexer

Over a series of pull requests, Walter had gotten closer to making the lexer a standalone module, meaning it should be easy for someone who wants to build something like a syntax highlighter in their editor---they'll be able to just pull in the lexer and nothing else. Part of that included having the error handler come in through the front door. He knows that caused unhappiness with some people. He was sorry about that, but the result was worthwhile. It works nicely, and he'd already done some pull requests to use error sinks in other parts of the code. So far it's working well. It makes the lexer's unit tests simpler, as it's much easier to mock the lexer when everything comes through the front door instead of global variables on the side. It wasn't there yet, but getting it there was on his list.

ImportC

ImportC was slowly grinding along. His priority was getting all the standard header files to compile on all the platforms. A lot of the system .h files on the various platforms have made a feeble attempt to be usable by different compilers but fail miserably, and each one fails in its unique fashion. They'd be easy to fix if he could edit those files, but of course, he can't. So he's left with trying to come up with creative workarounds. That's high on his list. Anyone trying to use ImportC by importing system header files is just going to quit and write off ImportC as unusable when it fails.

Template lowerings in CTFE

One problem he'd discovered was that the semantic routines take several constructs and lower them to templates. They do that even when running in CTFE. Those templates can generate a lot of bloat. In CTFE, it should not do the lowering. CTFE finds it much easier to interpret the higher-level code. Upon investigation, he found that CTFE reverses some of the lowerings so that it can interpret them. If you're doing certain things in CTFE like array concatenation, you're suffering from a lot of invisible template bloat. That needs to be fixed. He was about halfway through it. He was looking at all the lowerings and rewriting them so that the lowering isn't done in CTFE. He hopes that this is another step in reducing the problems with memory consumption and slow CTFE speeds. He also hoped to be able to remove the "lowering unwinding" that goes on in the interpreter.

Iain noted that in the PR rewriting the _d_newclass runtime hook to a template, he'd noticed the same thing Walter had just described. He got the author to change things such that rather than doing this lowering->unlowering->relowering thing, the rewrite is put into a field called lowering, leaving the original node intact. When you go through CTFE, you just work on that node specifically. Then once you're in codegen, you hit that node and go into the lowering field. Walter said that wasn't done with the other rewrites, but Teodor Dutu and Razvan plan to go back and change them (here's an example of work done toward that end). Walter said this will also be a big win for allowing more expansive use of CTFE in BetterC.

Resolving disagreements

One question that had come up recently was how to resolve disagreements between contributors. For example, what should a reviewer do if Walter wants to merge a PR over objections from Iain or Martin? Walter declared that Iain had the last word on anything involving GDC. He's in charge of it, he knows what he's doing, and Walter has full confidence in him. The same goes for Martin---he has the last word on how things happen in LDC. As far as D goes, when there's a dispute and there's no meeting of the minds, Walter has to make a decision. That's his responsibility. In that case, if he asks a team member to do something on the D side that goes against a position taken by Iain or Martin, then Iain and Martin need to direct their ire at Walter and not the team member. He's responsible for taking the heat for his decisions. No team member should be held accountable for that. (To be clear, neither Iain nor Martin has directed any ire at any team member. This came up in the context of reviewing a PR about which Walter and Iain disagreed, and the reviewer was concerned about upsetting one or the other.)

Focusing on bugs

As decided in the January meeting, Walter had continued and was continuing to focus on bugs rather than new features. Sum types had come up in a wishlist someone had sent to me. Walter said that he would like to have sum types. It's a modern feature people expect to have and we ought to do them. He posted a draft proposal back in November, but that's on the back burner for now. We've got enough on our plate.

Me

Random stuff

I reported that Robert had come into the Jitsi instance 20 minutes early, and he and I had discussed the state of his work toward the Bugzilla to GitHub transition. He'd told me the script was just about ready and he could send it to me very soon. Given that we were both planning vacations over the coming weeks, we decided to hold off until the middle of April.

I asked everyone to consider emailing me some of their issues so I could incorporate them into the gripes and wishes dataset I was collecting. We discuss several different issues in these meetings, but it would be nice for them to send me their pet peeves so they don't get lost.

I informed everyone that I had recently received an email from someone at a company using D in multiple production projects: a 360-degree photo booth, karaoke on a Ferris wheel, and a 2mx2m interactive wall. All of the projects use a telegram bot written in D that reports logs and diagnostics. The karaoke project uses ImportC.

Gripes and wishes

I gave an update on the Gripes and Wishes campaign. I'd received some positive remarks about the decision to focus on stability and robustness after I noted it in the summary of the January meeting (see the section titled 'John and Amaury'). Most people sent me two or three items, some actionable, some more general. I noted that the biggest exception was Timon, who sent me had sent me a long list of specific items (the longest by far!). I intended to split everything up into two reports: one for the actionable things and one for the general.

The biggest surprise for me here was that I had expected a handful of commonly hated or desired actionable items to float to the top, but that hadn't been the case as far as I could see so far. A few items may have been repeated once or twice, but almost everything was unique. Nothing was overwhelmingly common.

Quarterly and monthly meetings

Our next meeting, scheduled for April 7, was going to be a quarterly. I wanted to start bringing more people into the quarterlies, as shops were using D in production who weren't represented. As such, I didn't think it was feasible for us to continue to divide the quarterlies into two parts (we had always let the industry reps go first, and then they could stick around as we moved into our regular monthly meeting business after). The size of the quarterlies had already expanded to a degree that they could go quite long, and more people would just make it longer. I proposed that we split the quarterlies into two separate meetings: the quarterly on the first Friday of the month for the industry reps, and then a regular monthly meeting on the following Friday. Everyone agreed.

UPDATE: I had hoped to get the gripes and wishes mostly organized during downtime on my trip to the States at the end of March, expecting that I would find myself with plenty of it. It didn't turn out that way. I'm going to get it completed as soon as I wrap up the April meeting summaries.

Martin

Martin said he'd been busy with LDC as he had to catch up with the latest DMD releases. He thinks that's always good as he has CI test coverage for platforms that aren't tested by DMD, so as always he'd found some minor regressions. Those were fixed in DMD 2.102.2.

He'd found some regressions related to the lowering of runtime hooks to templates. For example, the new lowering for array concatenation meant that CTFE string concatenation didn't work for targets like WASM because the new lowering required some standard C bindings. Luckily, it was simple to fix. In general, these lowerings need to be carefully designed in this regard.

Átila wondered if we could add LDC to the CI somehow so that we catch stuff like this beforehand. Martin said there was no way at all.

He's quite happy about the new 'lowering' field that Iain mentioned. They'd been spinning around the idea that there could be a fourth semantic phase where the more high-level AST is lowered to the actual codegen AST, and this seems like a step toward that. Or something similar at least.

From the perspective of the glue layer for LDC, one of the bigger changes in the 2.102 release was the change regarding vector comparisons. They now yield a vector mask based on the front-end AST result. He was surprised as the change didn't make it to the changelog. This is a pain with floating-point vectors. The bitwise operators aren't available for floating-point vectors, so he had to do a lot of reinterpret casts to treat them as integer vectors for the bitwise ops. In that regard, GDC and LDC treat casts of vectors (e.g., casting a float4 vector to int4) as reinterpret casts, but with DMD that's a conversion. That's an important difference. Another thing is that identity comparisons aren't available in DMD or GDC at the moment because they're prevented in the front end, but LDC does support those and lowers them to yield a scalar boolean.

Mathias

Mathias wanted to get to a point with dub that overcomes the bug where it brings in all potential dependencies. When a dependency is part of a subconfiguration, but a different subconfiguration is enabled, that dependency shouldn't be pulled in at all. That's probably the most reported dub issue. As an example, we can't have test-only dependencies at the moment, as that will contaminate everything else. He'd been working with that as well as some speed improvements prompted by a recent discussion with Átila.

The other thing is that he wanted to get -preview=in enabled by default. He was fairly happy with the way it works. He'd been using it for years in production and it was working exactly as he expected. A few minor things had been missing to make the transition easier. One was the inference of storage class on function literals, a bug he'd fixed a few days before the meeting. He was still trying to figure out the best deprecation path for a few things, but getting closer to getting to the state where we can enable the feature.

He said he wants to see colors in his stack traces, so he had been working on the demangler so that we'll be able to colorize D symbols straight from the demangler. If it doesn't work, then it doesn't work, but it will make the demangler code better for sure.

The next meeting

The next meeting was a quarterly and took place on April 7th at 14:00 UTC. This was the first case following our decision to split out a monthly from each quarterly. April's monthly meeting is scheduled for April 14th at 14:00 UTC.

On Tuesday, 11 April 2023 at 21:53:50 UTC, Mike Parker wrote:

>

ImportC was slowly grinding along. His priority was getting all the standard header files to compile on all the platforms. A lot of the system .h files on the various platforms have made a feeble attempt to be usable by different compilers but fail miserably, and each one fails in its unique fashion. They'd be easy to fix if he could edit those files, but of course, he can't.

Can't the changes to those files by stored in the compiler? Walter obviously can't change the raw header files, but he can make changes to the files before they're used by the compiler. If that's not possible, can't a modified set of header files be available for download, and if you want to use a system .h file, you have to use the modified version instead?

On Tuesday, 11 April 2023 at 21:53:50 UTC, Mike Parker wrote:

>

The monthly meeting for March took place on March 3rd, 2023, at 14:30 UTC, and lasted about an hour and fifteen minutes. The following people attended:

As before, a great deal of interesting stuff here. Also useful: with these we aren't left guessing what the Foundation is thinking. We have a much better idea about what kind of ranting could be of value.

On Tuesday, 11 April 2023 at 21:53:50 UTC, Mike Parker wrote:

>

Walter also said that whether void initialization should be allowed in pure code is a different question. Void initializing an int and returning it isn't pure, is it? So maybe we should disallow void initializations in pure code. He thinks that's a reasonable proposition.

I disagree: it makes sense as is. Consider:

@safe pure int fun()
{ int result = void;
  return result;
}

@safe void gun()
{ int a = fun(), b = fun();

  [a, b].each!writeln;
}

What are values of a and b? They are unspecified. Since fun is pure, the compiler might cache the unspecified value of a = fun() to b, but so what? b is also unspecified, so it's expected it could happen to always be same as a.

pure can and should be able to return unspecified values, as long as they're unspecified the same way for the same argument set. About what would be unspecified in different way, and thus impure, this is an example:

int global;
@safe int impureFun()
{ int local = void;
  if (local < global) return global;
  else return local;
}

This also returns an unspecified value (unless global == int.max) but it's unspecified in different way depending on global state (never less than global at time of call) so the compiler rightfully rejects it if you try to mark it pure.

On Tuesday, 11 April 2023 at 21:53:50 UTC, Mike Parker wrote:

>

The language shouldn't support by default making a storage allocator pure. It's doing its job, saying you can't do this because you're changing state. So you have to fake it by doing a dirty cast in there somewhere. The reason why D supports system code is so you can do dirty things when you have to. It's the same with const. D does not support logical const, because logical const is not const. It's not enforceable. That's a feature. If you want logical const, you have to do something dirty to get it, and that's appropriate.

No, these are different.

pure only says the compiler is allowed to cache the result of a function and reuse it for similar calls. You are allowed to do debug logging, or return different results with subsequent calls (by casting an impure function to pure), as long as you accept that this results in unspecified behaviour. This means marking a custom allocator pure is okay (the pure factory function rule forbids the compiler from caching the memory address of the returned result).

On the other hand, mutating anything through a const reference is not only unspecific behafiour, it is undefined behaviour. The compiler may simply assume you don't do this, so if you do, anything may happen. Any D program using const as "logical" is excommunicated by the spec.

With pure hacks, there's more than one thing that may happen, but the options are still limited. With const hacks, they are not. The former are dirty hacks but legal, the latter are totally illegal.

It they are intended to be the same, the spec needs changes.

On Tuesday, 11 April 2023 at 21:53:50 UTC, Mike Parker wrote:

>

Resolving disagreements

Well that's disappointing. I guess DMD will just continue to bleed contributors whenever Walter decides to do something that the entire contributor base that is a very bad idea.