Permalink
Reply
The February meeting took place on February 3rd, 2023, at 15:00 UTC. The following people attended:
- Walter Bright
- Iain Buclaw
- Ali Çehreli
- Martin Kinkelin
- Dennis Korpel
- Mathias Lang
- Átila Neves
- Razvan Nitu
- Mike Parker
- Robert Schadek
The meeting lasted just under 50 minutes. We don't normally constrain the duration of our meetings, but this time we had to keep it under an hour because we had another one scheduled at 16:00 UTC. We've been having weekly meetings every Friday since late January. I'll be able to talk about those at some point, but not just yet.
The summary
Me
I opened the meeting by letting everyone know that Ikey Doherty of the SerpentOS project was scheduled to join us for our next meeting. We'll still be having our weekly meetings in March, so to make sure Ikey has enough time to address us, we should start our March meeting at 14:30.
Later, I talked about videos. The DConf Online 2020 & 2021 videos I've rendered have all used the H.264 encoding, as that's what my editor supported at the time. I have a different editor now, but I still rendered the DConf Online '22 videos in the same encoding. I believe videos from the last three DConf editions are also encoded in H.264, and I don't know yet about the older ones.
Now that we're storing our videos on Backblaze, I've begun rerendering them as H.265 to cut down the file sizes. Our monthly Backblaze bill for storing all the videos and the compiler downloads archive is less than $3.00, so it's not like we're paying a lot. But every penny saved counts.
Mathias
Mathias had recently been using D for scripting a web interface. He's missing the ease of use of having a lot of things in the standard library. We have an okay JSON package. That makes a world of difference, but if you have to do something with, e.g., YAML, you have to take the extra step of searching for a library. He thinks we should discuss bringing more things into the standard library.
Robert agreed. Specifically, regarding text file formats, he thinks we should have a package that supports multiple formats with a somewhat standardized API. YAML, SDLang, XML, and JSON 5 would be a good start.
Razvan
Razvan informed us that the deadline for GSoC applications was coming up. He still needed a list of potential projects and potential mentors. Our projects repository contains several ideas, but not all of them are suitable for GSoC. He suggested selecting four or five for the application. In 2021, we submitted a link to the project ideas repository. He felt that might have harmed our prospects, and thinks they were expecting a distilled page of more organized project ideas. He wanted to make sure Walter and Átila are okay with whatever projects we add to that list.
Robert suggested an email discussion to settle on projects. He proposed the text format package as a potential project, and Átila gave this thumbs up. The final list of projects Razvan submitted is at https://github.com/dlang/projects/pulls. Ultimately, our application was rejected.
Dennis
Dennis reported on his pull request adding named arguments for functions. Walter hadn't yet responded to a ping, so Dennis asked if he could look at it. Walter subsequently did so, Dennis addressed his feedback, and the PR was merged.
Robert
Robert had nothing to report other than his usual complaint that the compiler is too slow.
Walter asked if he was using fullyQualifiedName. Robert said yes, but that's not the problem. It's templates in general. It's not that template usage in Phobos is the problem, it's that the compiler is too slow in compiling them. Walter noted fullyQualifiedName triggered a chain of instantiations. He had a PR to replace it with a new __traits. He suggested Robert see how that impacted his compile times. Robert said he had looked at the instantiations in his compiles and expected he wouldn't even notice it. That said, he thinks __traits and std.traits are critical for metaprogramming and he thinks the PR is a good idea.
Walter doesn't think there's one single solution to speeding up template processing. He's implemented several optimizations in the past that have sped up things like, e.g., the AliasSequence template, and he's always looking for other opportunities to enhance template processing performance. Robert noted how we had discussed the topic in our December meeting. He thinks the main issue is separate compilation units. He emphasized that compiling source should no longer be seen as batch processing. It's an interactive game today.
Other than that, he says D is still the best language.
Razvan asked him about the status of the Bugzilla->GitHub migration. Robert said he still needed to make time to finish the script. Currently, the script has no proper UI. Manipulating it requires editing the source. He wants to set up a proper user interface so that whoever ends up running it (probably me) can have an easier time of it. And he still needed to account for corner cases, as when the GitHub API sometimes times out.
I asked Robert to email me as soon as the script is ready. Then I can ask Walter to contact Brad Roberts so we can set up a time to put the Bugzilla database into read-only mode. Robert suggested we don't need to contact Brad. All we need is the GH auth token and then everything is handled by the REST API. I argued that if Bugzilla is updated during or after the migration, we'll miss the new comments or issues. Robert said the script can remind the user to check for new issues immediately after the migration, and that a comment can be left at the bottom of each existing issue indicating that it has been migrated and any future comments should go to the GitHub issue.
Mathias asked if the script would close the Bugzilla issues after migrating them, and Robert recalled we had disagreed on that in a previous discussion. Here, we agreed the issues should be closed after migration. Given the GH API rate limit, the migration will likely end up taking place over a few days. At the end of the migration, we can rerun the script if any new issues came in during that time and migrate those.
I didn't bring this up at the meeting, but we'll still need to get in touch with Brad to put our Bugzilla into read-only mode so that issues aren't reported there post-migration.
Martin
Martin reported that he had started the beta of the next LDC release. At the time, he was hunting down some LDC-specific regressions (compiler crashes) that had shown up with the Symmetry code base.
He had discussed with Átila expanding the BuildKite projects being tested. Overall, the Symmetry code base has at least 100 third-party dependencies on code.dlang.org. It's normal to hit at least some problems in those packages with a compiler bump. Some of the 2.101 problems had yet to be resolved. These might have been prevented if the problematic projects had been added to BuildKite.
An annoying bug he had run into before and recently encountered again is that Phobos on Windows doesn't properly support long paths. Windows has a special syntax for denoting long file paths that go beyond its old MAX_PATH limit of 260 characters (strings prefixed with \\?\ are treated as long paths). This prefix is not required by the MS linker, by LLD, or by DMD. They handle it internally. Phobos doesn't. Where this is showing up is with dub. It uses a hash for paths to compiler output, and the hash takes up 64 characters of the 260-character limit. It's a real PITA. In his case, the LDC beta releases contain a -beta-1 in the name, and with dub's hashes, the length ends up just over the limit. So to test things on Windows he has to fake the version just to make it work.
Martin filed an issue for this last year, so if anyone wants to tackle it, you'll be plugging a Phobos hole that shouldn't exist in 2023.
Ali suggested that in the meantime, Martin might see if hard links can help him out if Windows supports them. He had discovered them recently and they solved a problem he'd had with file paths that contained problematic characters.
Iain
Iain reported on a GDC regression someone had filed as a dub issue: GDC was failing to compile dub when warnings are configured as errors. He needed to track that down. The issue is still open.
He also noted that DMD 2.102 had been released two days before the meeting.
Ali
Ali had nothing to report.
Átila
Átila had nothing to report.
Walter
Walter was having an annoying issue with build.d. It's required to build the compiler and build and run the test suite. Unfortunately, it requires a working Phobos to function. As it's currently set up, testing means compiling build.d with the compiler under test. This has caused him no end of frustration. build.d should be built with the D compiler and not the compiler under test. And other programs (like checkWhiteSpace) are being run before the test suite is executed which also require a working compiler. What should happen is that you build the compiler and then run the test suite before all those programs are run. When he builds a faulty compiler, checkwhitespace has crashed. It should never crash.
Dennis said that the test suite had grown into a mess over the years, making it difficult to change. We have several different CI services, each with its own scripts and configuration files, sometimes in a different repository. They have subtle differences in exactly what they do. The build script itself is over 2000 lines. He finds that baffling. It's just feeding source files to the compiler to generate an executable. Why is it so complex to describe? He hopes someone can take the time to rework the build system. (We previously discussed this in our August meeting at DConf, where Iain brought it up.)
Walter asked again why the build compiler isn't used to compile build.d when testing. Razvan said it depends on which CI is running. Some use the build compiler, some use the compiler under test. Dennis said it's so hard to keep track of all of it. Walter said he runs the test locally and the problem crops up there. I noted that the build systems and tests were cobbled together by multiple people over time, and getting them straightened out is on our TODO list. Walter said he sometimes is ready to delete build.d and go back to using makefiles.
This took us into further discussion on problems with the build system and test suite, the proper way to build everything (e.g., test the compiler before building Phobos), how LDC handles it, etc. Everyone agreed we need to fix this. Mathias suggested the gist of the problem is that DMDlinks with -lphobos instead of -ldruntime. It should probably separate them as LDC does. Martin explained why he thinks using a D script to build the compiler is problematic in terms of cross-compilation and bootstrapping, and suggested we should get rid of build.d. We were running out of time, so we deferred this discussion to a later date.
Aside from that, Walter had been working on the top issues in Bugzilla that are causing people problems. One of the issues he felt he needed to fix was Rikki Cattermole's problems with ModuleInfo. Part of that means removing things from ModuleInfo that require ModuleInfo to be generated. One of those was a list of local classes that is built so that Object.factory could instantiate classes.
Martin agrees that this is work that should be done. The local classes thing just to support Object.factory is very ugly and prevents lots of otherwise unused symbols from being stripped from the final executable. He's also not very happy about the ModuleInfo dependencies on each other, which drags in lots of stuff. Especially object.d drags in so many modules with their own ModuleInfo dependencies, and those modules drag in more modules with their ModuleInfo dependencies... Whenever you use a tiny bit of DRuntime you're dragging everything in. He's not happy about that at all, but he doesn't believe it's a pressing issue.
Walter said that everyone has their own top issues with D and they're all different. He has to decide which ones to fix. (This is a good opportunity to remind everyone that I'm collecting the gripes and wishes at social@dlang.org precisely so we can figure out what emerges as the top issues. I have to say that so far, there is a very, very wide range of "top" issues. I can't know for sure until I sit down and collate all the input, but I haven't seen any specific problems mentioned by more than a couple of different people. Everyone really does have their own top issues.)
Martin said it was good that Walter had recently tackled a template emission problem. He had submitted a PR with a one-line change that made on template emission bug go away. Walter's initial comment was, "Can it really be this simple?" Martin said it wasn't that simple, as there were some cycle issues in trying to determine whether a template instantiation should be emitted. Martin says template emission is an interesting topic that needs to be tackled by people who really know the language, as opposed to the ModuleInfo stuff which only affects a tiny, tiny fraction of people.
This got Iain and Martin into a discussion about whether that one-line change is the right way to go, in which they went off into the weeds of the template emission implementation, particularly the problem with cyclic references. Walter said that we weren't going to be able to solve that problem in this meeting because we were short on time. That one-line change might not be ideal or the perfectly correct situation. But there are two things in favor of it: it passes the test suite and it's a small test case. By merging the PR, we get the test case into the test suite and a future solution is going to have to account for that test case. He'd had discussions with Átila where they agreed that one of the problems with solving template emissions issues is that no one had been able to produce small test cases. Here we have one, and we need to get as many small cases for template emission cyclic references into the test suite as we can, even if the PRs that bring them in aren't ideal solutions.
Martin's main concern is performance. Iain had done some refactoring that will likely have mitigated any performance loss the PR might have caused, but we don't have any way to measure it in the test suite. What he would like to do but had no time for was to test Walter's change on Symmetry's code base. They have a very interesting project that consists only of templates. It's always a challenge for each compiler bump. If Walter's PR passes that project as well, Martin's confidence in it would be much higher. Iain said it should also be tested against Weka's code base, so someone should ping Johan Englen in the PR thread. If it passes Symmetry and Weka, then Iain has no quarrels. Walter said that because it's a one-line change, it's very easy to back out of if things go wrong.
I interjected here to note that the ModuleInfo issue might not be a high priority problem, but it's a problem that I've seen people complain about in Discord. We have some tough problems that will require teams of people who understand the compiler and the language, as Martin said, but anything Walter can fix on his own will make someone happy. Besides, we don't have teams of people to solve the tough problems right now. We need to organize them somehow. I suggested that one step we can take to get there is to start bringing more people into our monthly meetings (not the quarterlies; we have too many people there already) in the same way we brought Mathias and Robert in. (They were previously just D users working in their own interests, but now they also work on tasks that arise in our meetings, and they enhance the range of perspectives on the issues we discuss.) So we should consider other regular contributors who we can bring in.
The next meeting
As noted above, the next meeting is scheduled for March 3rd at 14:30 UTC. As always, if you have any specific concerns, ideas, or prospects to discuss with us, please reach out and let me know. I'm happy to bring you into one of our monthly meetings to hear you out.