Permalink
Reply
The D Language Foundation's monthly meeting for December 2023 took place on Friday the 8th at 16:00 UTC. It lasted two hours.
The Attendees
The following people attended the meeting:
- Andrei Alexandrescu
- Paul Backus
- Walter Bright
- Iain Buclaw
- Martin Kinkelin
- Razvan Nitu
- Mike Parker
- Adam D. Ruppe
- Steven Schveighoffer
- Timon Gehr
- Adam Wilson
The Summary
Me
I opened the meeting by detailing my plans for how we run our monthlies going forward. The approach we'd been using from the beginning was to give each person a turn, allowing them to bring up multiple topics before moving on to the next person. Sometimes someone had nothing to report, sometimes one or two issues, sometimes a progress report on their work, and sometimes one or more issues that got us into long discussions. Given that our invite list had grown quite a bit in the preceding months, this had become cumbersome, leading to overly long meetings or discussions being abruptly tabled in the interest of moving on.
So, I informed everyone that starting with our January monthly meeting, I would send out the invitations earlier than usual and ask everyone to email me a prioritized list of agenda items. Then at the meeting, I would only call on the people who sent me a list, going round robin in the order I received them, starting with the first item from each list, then the second, and so on, until either there were no more items or we hit the hour-and-a-half mark. If we did hit the time limit, we could decide whether to go on to two hours or not.
I introduced and welcomed Paul Backus. This was his first time with us. I'd had a call with him a couple of weeks before and invited him to join us. He'd said, "Sure, I can come in for a meeting." I told him I wasn't asking him to come to one meeting but to join us as a regular member. I thought his contributions would be valuable. Happily, he accepted.
UPDATE: We did start the new approach in January.
Adam W.
Adam W. opened by telling us he'd received an email from someone using the crypto library he'd written a few years ago. The person reported some bugs and made the case that D needed crypto in the standard library. Without that, it was hard to know which library to trust. Adam W. said he was going to fix the bugs and, since he had now started working on Phobos v3 (which the person who emailed him had no way of knowing), wanted to talk later about getting crypto in.
Regarding Phobos v3, Adam W. said he and Walter had discussed it at the Seattle meetup in November. He'd sent Walter a document and Walter had suggested he put it up on GitHub for feedback and discussion. At the moment, he and Walter were having a debate over how to version the standard library, and he expected that would continue at the next Seattle meetup the following week.
UPDATE: You can view the Phobos v3 Design document in Adam W.'s repo, and participate in the ongoing discussion. The repo was initially private, but he made it public during this meeting.
Timon
Timon said he had some type system and language issues but didn't think they were at the forefront right now. He suggested we could discuss them in a different meeting. I asked if he wanted to get in on our planning sessions. He said he did.
Adam R.
Adam R. said that the PR for the @standalone attribute opened in August and the string interpolation thing that had been open for a while were both stalled out. He then said he was very interested in the Phobos v3 stuff and would love to see the document. Adam W. said made the repo public and posted the link in the chat.
I said I'd thought Walter had approved @standalone in a previous meeting, then asked if Razvan or Dennis had looked at the PR. Dennis said there had been a weird test failure on it, but it was passing now. He thought the bug might still be there, but just wasn't triggering. Even so, since it was green, it was going to be merged. Paul said he'd dug into it and found that the problem was a timeout on the CI test runner, so it shouldn't be a blocker.
Walter said there had been disagreement over whether the feature should be an attribute or a pragma, and that had been his main objection to it. He thought it just didn't look good as an attribute, especially given how it was an extremely specific use case. He thought pragma was more suitable for it.
Adam R. noted that pragmas weren't backward compatible. Walter said unrecognized pragmas were supposed to be ignored. That was their intended purpose. Dennis said that's not how the compiler currently worked. There was a switch to ignore pragmas. Walter hadn't been aware that had been added. He told Adam he'd take a look at the PR again.
Timon commented that although pragmas are supposed to be benign, pragma mangle could break safety guarantees. Walter said he certainly hadn't thought about that. He asked Timon to post a Bugzilla issue, and that's now Issue #24275.
UPDATE: Walter approved the PR, and @standalone was included in 2.107.0. With the same release, unrecognized pragmas are no longer an error and are instead ignored.
I next reminded everyone that Átila had volunteered to write the spec for the string interpolation implementation. He'd gone on vacation for a couple of weeks, and when he came back he'd told Adam he was too busy catching up with Symmetry stuff and would get back to it later. Adam said a problem with D in general was that everything moved at such a slow, slow pace. I told him I dreamed that we'd be in a position to make things go faster at some point.
Paul
Questions
Paul said he didn't have any specific issues to bring up, just a couple of questions. First, he asked about the status of DIP 1000 and scope, and what the plans were for moving forward. Who was responsible for it and where were they writing things down about it?
Walter said he and Dennis had been the ones working on it, and they'd only been writing things down in terms of bug reports and pull request comments. I said that my understanding was that we'd decided to defer the full implementation to an edition, and I wasn't sure if the documentation was up to date. Dennis said the spec should be mostly up to date, though it may not be complete on every detail. The main issue was that we'd halted making it default because of breakage, so we'd decided to wait until editions were fleshed out. There had been some alternative proposals at DConf, so he was unsure how to continue when we were not going for DIP 1000 by default yet. We could continue fixing bugs in the meantime, though.
Paul said he was just wondering where to go to know that these kinds of decisions had been made, and how things were progressing. Was it just here in these meetings and the summaries, or was there another written record? I told him there were no other records.
Paul's next question was whether anyone was working on extending attribute inference to all functions, and no one knew of any such effort. Walter said a big issue with that was that it would dramatically slow down the compiler. After a bit of discussion, Paul clarified that the reason he was asking these questions was because he was concerned there might be work or planning going on that interested parties would miss, and that could lead to duplicated effort.
Walter said we didn't know what people were doing outside of what we talked about in our meetings. People tended to self-select. The way to find out was to post in the newsgroups and ask if anyone was working on anything. I reminded everyone that we had a project tracker that we all should be using so that anyone could see what we were working on as we were doing it. I was the only one using it regularly. It would be great if all of us could incorporate it into our workflows.
Speaking of attribute inference...
Adam R. asked Walter if he'd approve global attribute inference if someone showed him an implementation that didn't significantly increase compile times. Or did he have other concerns? Walter said that was the main concern because it required the compiler to do semantic analysis of the function's body. You could also run into chicken and egg problems when you needed to know the attributes, so you needed to compile some other function, then that function required something else to be compiled first, and that could cause all these circular reference issues. But compile speed was the primary issue. For performance reasons, the D compiler tried to compile only what it had to.
Walter's main fear was that it would change the order of semantic analysis, which was a very fragile part of DMD. He was afraid that existing code would break in really weird ways. He suggested we could do it gradually, like turn it on for constructors or member functions, and see how that goes before enabling it generally.
Razvan said he expected we'd have more occurrences of an issue that currently happened with attribute inference on templates. If you had two templates cyclically calling each other, then the compiler just inferred them as being @system. It was more conservative in this aspect, not smart. Because templates were often used like normal functions, that problem didn't appear very often. But if you turned on attribute inference for all functions, it was going to hit more frequently. We'd need to fix this issue first.
Timon thought that recursion in template functions was actually quite common. He said in general, the issue with attribute inference was that it wasn't decideable with all the introspection capabilities D had. What should probably be done was not to compute the smallest fixed point, which was done now, but to compute the largest fixed point.
Walter agreed, saying that now if the compiler failed to infer attributes it took the most conservative approach and decided there were no attributes. We could flip it around and make it more aggressive when it hit the circular conundrum. He wasn't sure how it would work out. Timon said it should only do that if there was no reason not to do it. If something caused a safety error, then the compiler shouldn't infer safe, but otherwise it should. Right now, it just didn't.
Walter said the problem was that if you did that, you'd have to go back and all the functions that you'd already marked as being safe would have to be unmarked. He wasn't saying it wasn't implementable, but it just wouldn't work the same way. Timon said it was probably tricky. This was why it hadn't been done yet, but we could work on it. Razvan agreed, but he just wanted to point out that he thought this was going to bite us.
Adam said he just wanted a commitment that if we could solve the problems, the speed problem, etc., that it would get in. In that case, it would be worth trying, but wouldn't be otherwise. Walter said if the problems could be solved, then it was worth merging.
Walter noted his own experience with this. He kept trying to add const and other attributes to the DMD source code and found that D was completely recursive. Everything eventually wound up depending on everything else, making it hard to use const anywhere in it. It was frustrating, and he expected other attributes would have the same problem.
Steve
Steve said he'd done his new class teaching D to homeschool students, and it had gone very well. He was in the process of building a website that he could reuse for it later. He was scheduled to teach a follow-on class in January.
Iain
Iain said he'd pushed out 2.106.0 at the beginning of the month.
On the infrastructure side, he said that tour and run.dlang.io/.org had gone offline for about half a month. So he'd set up a copy inside a container on one of the build CI machines that he owned. It was a permanent place for it until we were ready to move it to a DLF server. He asked if there was anyone looking into making that happen. I told him I'd had an update from Petar Kirov. He wanted to get the Tour set up so that it was just a one-click, automatic deployment. That's what we were waiting on, but I had no idea when he would get it done. He did have access to our VPS account, though, so whenever it was ready he could get it set up without blocking on anyone else. Then any of us who had access to the account could spin it up when needed. Ideally, I thought that should generally be Iain, Mathias, Robert, and whoever else was suited for it.
Next, Iain said GDC was still tracking mainline. He was in the process of bootstrapping and regression testing the merge commit for 2.106. Once he got the results back from that, he'd push it. He'd gotten sidetracked on that a bit because the refactoring Razvan had been doing had triggered an old template regression. It didn't affect the newer releases, but some targets depended on an older version of GDC to bootstrap the latest compiler. He'd worked out that it was just the first version of GDC that got merged into GCC that was affected, but he wasn't quite ready to bump the baseline. So long as we kept GDC 9 working with the DMD mainline, he was happy.
Martin
Sucess stories
Martin said he would start with some little success stories. First, 2.106 was looking pretty good. He'd tested it on the Symmetry code base and had encountered no unit test regressions so far. Nor had he noticed any performance regressions. He hadn't checked very deeply, but it was looking good. Ditto for LDC 1.36: no unit test regressions anywhere.
He said he'd undertaken his first ImportC experiments over the past few weeks and it was looking good. He'd done it on production code, using three libraries, including Postgres. The preprocessed C file on Windows was over 200,000 lines, over 400,000 when using the Microsoft preprocessor, and ImportC was able to swallow all of it. The general bindings seemed to be sufficient.
He said the C libraries they used didn't make use of a lot of macros, so that was one reason why it worked so well. He'd also tried it on python.h, which was a monster of macros, and nothing worked there. They still needed to use DPP for that. But comparing ImportC and DPP, ImportC was easier once he figured out that he shouldn't compile the C file, just import it. That got him around the issues regarding inline assembly and missing GCC macros he'd brought up at the last meeting.
So, Martin could confirm that ImportC was usable, at least for some scenarios. He thought that was pretty great. It worked on Windows and Linux. He hadn't tested it on Mac but thought it was cool that it worked on Windows. It was also faster than DPP. For example, one DPP invocation that took 10 minutes on Windows was done in something like a second with ImportC. Very cool.
Walter thanked Martin for the highlight of this meeting. He said he loved hearing about when things worked.
Martin said one thing that did come up was missing attributes. He'd seen a Bugzilla issue about it. The C functions defined in C headers got no attributes at all. No nothrow, no @nogc, etc. The only concern in that issue was a theoretical problem that the C function might take a callback, a function pointer, which might be a D function, which might be using the GC or throwing exceptions. He thought that was such a rare case, we just shouldn't worry about it. If you wanted to do that, you could just declare that specific C function in D yourself with the attributes that you needed, because you couldn't insert those attributes in the C file.
Regression in 2.103
Martin said there was a regression about missing struct TypeInfo symbols in 2.103. He gave his analysis of the problem, saying it was related to making decisions about codegen based on the scope of a template instance, and how the compiler decided not to do codegen in speculative contexts. He thought he had a temporary fix for it, but he needed to dustmite the problematic use case.
He said he'd been thinking of automating the dustmite process. He was tired of trimming down because it took so much time. Symmetry's compiler invocations had huge command lines, and getting all of the dependencies, all of the imports, and dustmiting was a huge PITA. But in this case, once he got it minimized, he'd submit a proper fix for it.
Makefiles
Next, Martin reported he'd just started getting rid of the DMD makefiles that had been deprecated for years. The DMD release process still needed them, but he had a PR for that.
As part of this, he'd taken the original posix.mak file and switched it to a generic makefile to also work on Windows. There were some constraints, of course. We needed GNU make instead of Digital Mars make on Windows, and all the GNU tools needed to be in the path. That comes with Git for Windows nowadays. To really clean up and make the top-level generic makefile really work, he proposed getting rid of the Digital Mars makefiles, the win32.mak and win64.mak files. We had the POSIX makefile which worked everywhere except Windows, and on Windows we had win32.mak which was just for the OMF stuff, and win64.mak which was for both 32-bit and 64-bit MSCOFF builds. We could just merge that into one makefile as long as there were no objections to relying on the GNU tools on Windows. He asked if Walter was okay with that.
This took us into a long discussion about duplicated code in the makefiles, what GNU tools we'd need, whether it would be difficult to merge all three makefiles into one, runtime integration tests, and a mini-rant by Walter asking why we had makefiles in the first place. He said that compiling DMD should be simple, and these makefiles we had that used every feature of GNU make were not simple.
In the end, Walter said he could not argue that we should keep maintaining different makefiles for Digital Mars make and GNU make. Three different makefiles with build.d thrown in was an abomination anyway you looked at it. I suggested we discuss the details of this in our next planning session, and everyone agreed.
During this discussion, Dennis and Adam R. had brought up dmd -i in the text chat, and Dennis had given a command line that worked right now. Martin noted that was only for the compiler. We didn't have anything similar for DRutime or Phobos. Those were fully makefile.
Andrei said that we'd had this kind of discussion for a long time. He thought we should just let Matin run roughshod over whatever we have now and make it into something different. Because at this point, anything different was almost certain to be better. He said we were in that space now. Everyone in the room hated what was there now. So why not just make it different? Maybe at least a few of us would hate it less.
And as for depending on GNU make, we should just get over ourselves, bite the bullet, and just use that tool. He thought Walter had said very well his point that using different build systems for different OSes was an untenable position. He added that he'd been at a conference talk not long ago where he'd learned that apparently everybody and their blind dog was using CMake now. He knew we'd be groaning over our muted mics about his mentioning CMake, but maybe we should just look into getting with the program and using CMake.
UPDATE: The deprecated makefiles were removed in the 2.107.0 release. Additionally, the top-level posix.mak was renamed to Makefile and now includes the functionality of both win32.mak and win64.mak, both of which were removed. The release also included makefile cleanup for DRuntime and Phobos.
Dennis
ImportC
Dennis said he'd not had much success with ImportC yet. He'd been sending Walter bug reports about struct initializers. Walter apologized and said that was his (Walter's) fault. He found the C initializer syntax to be the most incomprehensible, awful thing, which was why D didn't use that syntax at all. He said he was continually tripped up by that. He sometimes despaired of ever making a working implementation that does the C initializer syntax. He'd seen Dennis's regressions and was going to try to fix them.
Dennis thanked him and said he did see the light at the end of the tunnel. His project was closer and closer to being compiled with ImportC. Walter thanked him for being diligent about filing the bugs in Bugzilla.
Restructuring DRuntime
Dennis said there was a proposal, in the form of some PRs, to restructure DRuntime. He'd emailed Walter and Iain, and he thought Iain had already seen it. Walter said he hadn't seen it yet, but he would look at it.
Dennis just wanted to know, without looking at the details, if Walter was open to the idea of restructuring DRuntime with separate folders for each OS, or if his initial instinct was that it was a bad idea. Walter said that had proposed the same back before Razvan and Dennis joined us, there had been a huge fight about it, and his proposal had been shot down. He'd thought things would be easier if each OS had its own tree of implementations, but other people didn't like it. He was a little surprised to see it had resurfaced.
Dennis asked if Walter remembered who those people were. Walter said the thought Iain had been one of them. Iain said that, from memory, Walter had picked a very trivial module to convert, and as the cherry on top, that looked fine. But then Iain had picked a substantially more complex module and it had all fallen apart relatively easily. He said he hadn't liked the idea of taking something that's encapsulated into a thousand-line module, and then splitting that into five potential thousand-line modules.
Razvan said he'd taken a look at those PRs and it seemed that they followed that pattern. He didn't understand the benefit and said it seemed like a matter of taste. Walter said the benefit, in his experience, was that whenever you tried to support multiple operating systems with one file, you'd wind up with an incomprehensible tangle, and then when you added something, not all the OSes would be checked to see if they still worked with that file. Then you'd end up having incorrect declarations for a particular OS. This was especially bad when you had bit flags, and the bit flags hand magic numbers varied from compiler to compiler. Ending up with the wrong magic number for an OS could lead to really, really strange problems. By separating them, when you updated the Windows file, you wouldn't accidentally break the Mac implementation.
There was also a reading comprehension problem with reading a file that was full of conditional compilation. It becomes difficult to see which OS a piece of code is for. Splitting them up means more lines of code, sure, but the compiler isn't compiling more lines of code. It just picks the path through and compiles the one for the current OS. So he didn't think the old "don't repeat yourself" mantra was useful to apply here. When you split it out and you're figuring out how it works, you've only got one OS to deal with at a time and not a rat's nest of versioning to deal with.
Iain said that then you'd just move the rat's nest of versioning in with it. For Linux in particular, you couldn't just say version(linux) and you'd be fine. You had to ask if you're working with the GNU C runtime, the MUSL runtime, etc. Walter said he was right, but with Linux and the Unix thing, you had POSIX. That took care of a lot of that. You couldn't get rid of versions completely.
There was some more discussion along these lines. Finally, Walter said that with the proposed change, he'd need input from Iain and Martin on whether it was acceptable. Iain said that his first impression of it was that he didn't like having a generic fallback. Walter agreed. He said he'd leave it up to Iain on whether to accept the proposal. He didn't want to repeat the long argument they'd had over it last time. Iain said he would need to reread the old thread from back then to see if he still agreed with everything he'd written back then.
Iain then pondered if we did go forward with dicing things by OS, would we then dice by runtime? By architecture? Where would it stop? Walter thought if we restricted it to Windows, POSIX, OSX, Linux, and FreeBSD, we should be fine.
Martin said he didn't like this idea at all. He totally got Walter's first point about breaking one implementation when modifying another, but he hoped in the future we'd be able to have some little CI checks running on these systems. So then maybe we could use ImportC to check our declarations against the ImportC declarations. But he disagreed completely with Walter's second point about legibility. He detailed some examples of how and why he found that having everything in the same module improved legibility. He conceded that in some cases, splitting things up would be an improvement, but felt that most of the time it would be the opposite.
Walter said that Martin was really saying was that we shouldn't make a hard and fast rule. We should use our best judgment on when to split things up. But one thing he did really want is that whenever there were versions for operating systems in the code, there had to be an else clause that had a static assert(0) in it. That way, when a new OS was added, you'd instantly find all the cases in the runtime that you needed to check for OS-specific dependencies. He didn't like having a generic version in this case because it would always lead to someone forgetting to update an else clause in there, causing some kind of mismatch that would be hell to debug.
Andrei said that this whole little debate was about where to draw the line between "I don't want to duplicate code" and "I want to specialize code for each OS". An untested generic version was not great to have, so the static assert(0) thing was great to have. But that's not to say that someone would figure out once it fires that one implementation works for both this and that OS, and then would simply copy and paste one implementation in place of the static assert. He would say that everyone present was a good engineer and was able to look at the code and decide if it was good engineering or bad engineering.
Andrei
Andrei said he'd been looking over the Phobos v3 document that Adam W. had linked, and he had a few comments. He understood that this was not a finished document and just a list of thoughts. He thought there were some great thoughts here. The point that the existing Phobos should continue to be supported was a great decision. And the discussions about it on GitHub involving other people were great.
However, though the document was aspirationally in the right place, he thought there was a need for a careful study of language and features and library organization as to how that's to be implemented without duplicating the entire body of Phobos. So one question would be, what steps could we take to avoid duplicating the 700,000 lines of Phobos that we already had in their entire 90%, 80%, or even 40% if we were to reuse parts of Phobos core? He thought the document should say that we wanted to support the existing Phobos, but we should also include in the document a versioning mechanism that structured multiple versions.
Then he said that from a bird's eye view, the document should not be about what should not be in Phobos v3, only what should be there. It was good to put in writing what shouldn't be there, but Phobos v3 shouldn't be defined by saying it was Phobos without X, Y, and Z. It should be more that Phobos 3 was Phobos with this great overarching awesomeness, X, Y, and Z. It should have some positive things as opposed to saying "it's not going to have that crap".
At a lower level, "no memory allocation" was a nice aspirational thought, but it was completely unfeasible. It completely locked out things like "we want to have a hash table in Phobos". You couldn't have a hash table in Phobos if you didn't allocate memory. He thought it would be good to instead say we wanted to build hash tables and we wanted to build containers and see how far we could go by deferring memory allocation.
Regarding exceptions, Andrei thought that designing without exceptions was a difficult endeavor. There should be more discussion about whether or not we want to use exceptions or optional, or that kind of stuff. C++ had this new thing, std::expected, which was either a value or it wasn't, and you could make decisions based on that. That was a possibility.
Paul noted he'd been doing some exploratory work to see what it would take to implement a safe allocator interface and containers that use it by using DIP 1000 and system variables. He said anyone interested in that should get in touch with him. Andrei said that was great, but that seemed like a separate body of work than what was in the document. The document said Phobos would do no memory allocations, but if it's calling into allocators and saying that, then that's just dishonest.
Walter said that the stuff in the document about not allocating was just aspirational. He thought several functions in Phobos allocated when they didn't have to, and they threw exceptions when they didn't have to. He'd been able to rewrite std.path so that it didn't do any memory allocation. No one else had done that or thought about it. He hadn't thought it was possible until it suddenly came to him how to do it. But he also realized that it wasn't going to work everywhere. The idea was, could we figure out a way to do error returns that don't require exceptions, or solving these other problems without redefining the problem? When we could make it work, there were various ways of approaching it and maybe one would fit.
Walter agreed with Andrei that you couldn't have hashtables without allocations. He noted that when Andrei wrote std.algorithm, only one function in it allocated. Andrei said it was the Levenshtein Distance implementation, and that shouldn't even need memory. Anyway, Andrei just wanted to say that the document should be modified to say "no unnecessary memory allocations" rather than "no allocations".
Andrei noted that in the text chat, there was mention that returning error values had problems that motivated exceptions in the first place. He agreed. He said that in C++ there seemed to be a federation of ways to address error handling and people preferred one single way. So we should standardize on just exceptions, or just expected, or just error codes. That would seem to be better than "let's do a variety of these".
Razvan
Razvan started by saying he had two students working on using dmd-as-a-library to replace other external dependencies.
dfmt
One of Razvan's students was a SAOC student who was making consistent progress on replacing libdparse with dmd-as-a-library in dfmt. He was at the point where it seemed he was going to be able to finish the project. So by the end of SAOC, he would have replaced libdparse entirely. So the problem that had arisen was, how could we integrate this work?
Razvan had initially thought it could just be integrated upstream into dfmt, but the dfmt maintainer, Jan Jurzitza (Webfreak), had pointed out that dfmt didn't necessarily need to parse the code. It was just using a lexer and only parsed a little bit in some cases when it needed to extract some information. In that case, if there was a breaking change in DMD, dfmt was still able to format your code. But with dmd-as-a-library, because the lexer was kind of a monolith, it wasn't breaking down into a lot of functions that you could overwrite. So what they were doing was creating an AST and then traversing it to output the formatted code. It was more work, but on the flip side, you were able to do more complicated passes and more complicated formatting than dfmt was currently able to do. But if there was a breaking change in DMD, then dfmt might crash with a compilation error in old code.
So Jan had suggested they start a separate project, dmd-fmt, and then use that. Razvan didn't like that approach, because if dfmt still had a small frontend that could be modified, then you'd always have to backport things. It was doable and wasn't complicated, but he felt it missed the point. People would just use dfmt and he didn't know if anyone would even use dmd-fmt. He felt the best approach was to integrate dmd-as-a-library upstream into dfmt and then specify a version with the build system. Or maybe editions could help solve it.
Martin said he'd thought along similar lines and thought we should bundle dfmt with the compiler. That way, the version of dfmt that ships with a compiler is compatible with and on the path with that compiler. Razvan said he thought the same and asked if anyone had any other opinions or if we should move forward with shipping dfmt with the compiler. No one had any objections.
dscanner
Razvan's second student was doing his Bachelor's degree on finishing the work of getting dmd-as-a-library into dscanner. The project was around 70% complete, and Razvan expected they'd get it finished by the end of the university year.
The problem here was the dscanner had a larger frontend than dfmt, and by "frontend" he meant code that is common for both the dmd-as-a-library and libdparse implementations. So two or three months before, Jan, who also maintained dscanner, had done some upstream refactoring where he modified the constructor of a base class which is inherited by everyone. So then trying to rebase with the upstream had resulted in hundreds of conflicts that took a couple of days to resolve. They were trivial fixes, but they had to be done by hand, one by one.
The question was, should they continue on the path they'd been on, making all the changes with the ultimate goal of pushing the final result upstream, or should they go ahead and get what's done upstream now, and keep pushing changes up incrementally? He felt the latter was the best approach, as then they'd no longer suffer from breaking changes and if the student disappeared, the work that was done would still be there. It would mean in upstream dfmt, some of the work would be done by dmd-as-a-library, and some by libdparse, but the ultimate goal was to replace libdparse, so he didn't see that as a problem.
No one had any opinions on this, so Razvan said he'd discuss it further with Jan.
AST refactoring
For his final issue, Razvan said he'd been working on pulling all of the semantic routines out of the AST nodes, and that had been annoying Iain very much. He'd made consistent progress. One issue that he didn't know how to solve was that some AST nodes had to store their backend symbols. He imagined this was probably annoying for GDC and LDC because he didn't think they were using those fields for anything. He wanted to pull them out somehow, but he didn't know of an efficient way to do it. He said the standard way would be to use a hash table, but he worried that would have an impact on performance.
Walter said this had come up before, and he knew that Iain objected to having DMD backend symbols in the AST nodes. Walter thought GDC and LDC could just ignore them. Martin said that's what LDC does. He didn't care about those symbols and provided some details about what the implementation does. He had no objections and said from his side, Razvan was free to do what he needed to with this.
Razvan said the main reason he was doing this was to break the dependency between the AST nodes and the backend. Otherwise, this refactoring of the semantic routines would be impossible. Since this stored data was typically a pointer, one solution would be to just use a void pointer there and whenever it is used in the front end, cast it to the appropriate type, and then the back end can take it.
Walter said that sounded like a workable solution. Razvan said that Iain had objected to it a month before by saying he didn't think it was as small a change for the C++ code bases, so Razvan should hold fire.
Iain said he still hadn't had the time to look into it. Martin said that from his perspective, there were no issues. He actually utilized void* in D files for certain internal C++ structures in LDC for which he lacked the definitions on the D side.
Iain said that on his side, this DMD backend symbol is actually an alias to a GCC anonymous union. They do use this symbol as a cache pointer. He said he should probably be using a hash table instead. And GCC's hash table was probably more efficient than DRuntime's. He didn't have the time to determine how much of a pain it would be to insert casts everywhere on the C++ side.
Razvan linked proof-of-concept PR. He said that if we could agree this was a workable path forward, then he would need to do this for other symbols as well. It was a small change for this particular symbol for DMD, but for other symbols, he would have to insert probably thousands of casts.
There was more discussion about GDC and LDC internals, and finally, it was agreed to just make it void*.
Walter
Walter said someone had published a forum post about how they'd used D to glue C++ and Rust together. Walter had posted that to Hacker News, and no one had come over to help out.
One of the complaints Walter often heard about D was that nobody was using it. So what would help was when he or anyone shared things like this somewhere was for people from the D community to come by, leave a comment, do some upvoting, do some downloading, and just stamp their feet and let people outside of the forums know they existed. He thought it would greatly help D.
He'd hoped to get enough upvotes on that to make it on the front page of HN but had failed miserably. But that's what other language communities did. Rust had a constant stream of people going around looking for any place they could interject some comment about Rust. And the Zig people and the Nim people had done it. The D people just didn't do it. It wasn't a good thing for us to be modest and reticent talking about D.
Walter would just ask people to drop into places like that to be heard and be seen. That kind of thing was important to build social proof for D that there is a community, we are active, and all of that. He encouraged everyone to create a Hacker News account and just drop by once in a while to see if there's anything worthwhile to respond to.
The next meeting
Our next meeting was a quarterly meeting with industry reps on January 5, 2024. Our next monthly meeting was held the following Friday, January 12.
If you have anything you'd like to bring up for discussion at one of our monthly meetings, please reach out to me (social@dlang.org), and I'll discuss with you how best to go about it.
