Permalink
Reply
Summary
The D Language Foundation's monthly meeting for November 2023 took place on Friday, the 10th, at 16:00 UTC. It lasted around one hour and ten minutes.
The Attendees
The following people attended the meeting:
- Walter Bright
- Martin Kinkelin
- Mathias Lang
- Átila Neves
- Mike Parker
- Adam D. Ruppe
- Robert Schadek
- Steven Schveighoffer
- Adam Wilson
The Summary
Steve and Me
I have to apologize to Steve. I managed to botch the initial recording, so whatever he and I said at the top of the meeting is lost. I'm pretty sure I talked about preliminary planning for DConf '24, but beyond that, I don't recall. I also know that whatever Steve reported, it wasn't anything that sparked debate. The first audible words I have from him are, "That's about all I have, really."
Adam Ruppe (AR)
@standalone
The first item on AR's list was the @standalone attribute that he had proposed in a previous meeting. He said he'd like to see something happen with that. Dennis had a PR for it. AR had pinged him about it, and Dennis had made a few updates. He just wanted to make sure it wasn't forgotten. Walter asked for a link to the PR, and Adam said he'd get it to him later.
(UPDATE: The PR was eventually merged and released in DMD 2.107.0.)
String interpolation
Next, AR brought up the string interpolation implementation. He said he had a quite a full implementation with a lot of examples down. He said that several people had tried it and even those who weren't convinced before had come around. He wanted to get that merged as well. Walter asked if there was a specification for it. AR said it was basically the same as DIP 1036, but slightly different. Walter said that's what he means; it needs to be a complete and correct specification for what the proposal is. AR said Walter could go read the code. Walter said we don't document things by telling people to go read the code.
This sparked a discussion regarding the requirement for a spec and the history of string interpolation proposals. In the end, Átila offered to try out AR's implementation and write out the spec as he understood it, and asked if Steve or AR would be willing to correct it. Steve said it was just a simplified version of 1036, so it shouldn't take much time to write the proposal as long as it wasn't something that's going to be dead on arrival. Átila said he was volunteering to write the spec, but he was going to be on holiday for a couple of weeks and wouldn't get started on it until sometime after he got back.
Walter said the last time he had reviewed the proposal, he'd been excoriated by people telling him "that's not what the actual implementation is." Steve noted that the proposal Walter had reviewed was an incomplete one by John and Andrei that they hadn't touched in years and said that Walter had misunderstood the point of some of the examples.
Átila confirmed he would write the spec. Walter and Steve both said "all right." AR said he'd follow up on it because he really wanted to see it done. He'd used it in production and it was very useful. If Átila had any questions about what it did, he should try it. Átila reiterated that he was going to try it first and document it after, and would ask AR to correct his understanding if it was wrong.
(UPDATE: Ultimately, Átila finished the proposal. After a long discussion in the forums, Walter approved the feature and it was merged. Steven Schveighoffer is working on a spec PR.)
Bugzilla to GitHub transition
Next, AR said people had been asking about the timeline for the Bugzilla to GitHub transition. Robert said he had tested the migration API with the Visual D issues and that worked. But in a recent planning session, potential issues with the dlang bot had come up. He detailed his understanding of what the bot was doing and what he thought he needed to change to ensure everything keeps working post-migration. It was just a matter of making the time to investigate and implement everything. AR said it's fair to say we're probably still looking at several months. Robert said he'd do what he could to not make it that long.
Status of editions and the future of Phobos
Next, AR asked about the status of the editions proposal and plans for the future of Phobos. He figured they would be related. Átila said the editions proposal wasn't finished yet, and he was now going to kick it further down the road to work on the string interpolation spec. Átila said AR was probably right that the future of Phobos would be tied to editions. Walter had sent him an email a few days before about the new Phobos, but Átila thought we needed to figure out a few details about editions first.
(UPDATE: Adam Wilson has been pushing things forward on the Phobos 3 front, most recently announcing that Phobos 3 development is open. Átila finished a rough draft of the editions proposal and solicited internal feedback. There's been some debate about the details, and Átila is going to apply some revisions. Once we're all (or mostly) in alignment with the revised draft, we'll publish it for feedback.)
.init as a member
AR wanted us to be aware of an old, closed PR regarding .init and member functions that would have deprecated allowing init as a member. It didn't work according to the spec, and when you did have an init member, it caused trouble with other things. Paul Backus ran into trouble with it when working on an allocator proposal.
The is operator and static arrays
Using is on static arrays was deprecated in 2016. In the current behavior, the static arrays decayed to a slice, meaning the pointers were compared. AR said that was useless on static arrays. Since it had been deprecated for many years, he thought it was time to switch over to a mem compare.
Walter noted we'd had a problem with deprecations breaking code and upsetting users. Mathias said the current behavior makes writing generic code annoying. He had the problem in one of his libraries. He agreed we should finally change it. We wouldn't be preventing users from doing anything, but making the behavior make sense.
Walter said it wasn't a matter of making sense. It was a matter of breaking code. And this is the problem we're looking to solve with editions. Steve suggested that this change wouldn't break code because no one would ever do this. No one cares if two static arrays point to the exact same location, they care if the elements inside the array are the same. He brought up comparing two structs as an example of the same thing.
Mathias gave a little detail about the problem he encountered. Steve noted that if anyone was relying on the current behavior, it was still going to work anyway. Because static arrays usually live on the stack, then two static arrays with the same elements are equivalent to two slices with the same address. AR noted that might not be true when there's a float involved.
This took us into a discussion of the details of float comparisons. Walter reiterated that we needed to avoid breakage as much as possible and that editions were the way out of it. After AR noted that we'd be removing a deprecation in this case, Átila said that tilted him in favor of it. Walter said he'd need to look into it more offline. AR said he'd follow up on it.
Amaury's GC
AR reported that Amaury Séchet had been working on a new GC for his SDC compiler and hoped to upstream it once it was stable. AR wanted to make sure we were aware of it because it was going to be a large PR representing months of work. Walter noted that the GC is meant to be pluggable and asked if Amaury's implementation was intended to fit into it. AR said he'd been having some trouble fitting it into the interface, but that was the goal.
Walter said the interface could be improved if need be. Being able to switch between GC implementations was a big deal. Steve said he'd talked with Amaury about the GC at DConf and thought it was going to be a huge benefit to D once it worked. It probably would require changing the GC API. Walter said that was fine. Changing it in a user-visible way would probably require an edition.
Server downtime
Someone had asked AR to bring up the recent extended run.dlang.io downtime. This was something that happened sometimes, but we needed a way to resolve it more quickly. I told him this was something I'd tried to put together. My goal had been to gather important ecosystem services onto servers managed by the DLF. The maintainers of those services had agreed in principle to make it happen, but so far none of them had found the time to get it done, and chasing them down about it had fallen farther down my priority list. Particularly regarding run.dlang.io and the D Tour (it's the same app), the maintainer wanted to make it easier to deploy and maintain before migrating it so that anyone on our list of admins could handle it without much effort. We'd gotten some of Vladimir's stuff migrated (like the doc tester), but still had a list of services to move. So yes, getting that stuff under control of multiple admins was something we wanted to get done at some point.
(NOTE: run.dlang.io and the D Tour are currently running on temporary servers until Petar is ready to do the full migration. I haven't had and don't have the bandwidth to coordinate migrating all the services that need it. Under our reorganization, this isn't my area of responsibility anyway. I need to have a discussion with Robert and Mathias---our ecosystem guys---about this. At the moment, they've got their hands full with some higher priority stuff. But we're working our way toward it.)
Adam Wilson (AW)
AW gave some feedback on the earlier discussions about string interpolation and the GC. He then reported that since people had heard he was regularly meeting up with Walter, they'd been sending him topics on Discord to bring up.
He said he'd been talking with Rikki Cattermole about the export attribute. Rikki had said that export was automatic on POSIX and opt-in on Windows. He was working on a DIP to make it opt-in everywhere. AW and Walter had talked about problems with the Windows symbol table at one of the D meetups, which was the reason export was opt-in on Windows, and he thought this DIP lined up with Walter's thinking. AW said he'd really like to see this get in.
Walter said that shared libraries on Windows just did not work like they did on Linux. Trying to pretend otherwise would just lead to madness. Any fix for export had to acknowledge that you couldn't write the same code and have it work successfully with both Linux and Windows shared libraries. It didn't work with C or C++. You had to account for the differences.
He said he was no expert on shared libraries, so he appreciated AW's expertise here. But he did know very well how the C implementation works on Windows since he'd implemented it. It was the same implementation used by Microsoft C. Windows programmers would have expectations that it worked like that.
AW said they probably would, but he didn't think it would be a big deal. The big thing for us was the GC. The C and C++ guys never had to deal with the GC. We're always going to have to think about it. So we should just accept that it was going to be different and make it work the way we want it to work.
Walter noted that DLL usage on Windows had significantly changed over the years. He wasn't up to date on all the issues and was glad AW was on the job since he had a better understanding. AW said he didn't know that he understood it well enough, either. Walter said that was always a worry with Microsoft stuff as they made things so unnecessarily complicated.
AW said a lot of the DLL changes had to do with what MS were doing with .NET and the really small DLL assemblies they were using there, where they were assigning a 10MB space to an 8kb DLL, then running out of address space to load DLLs and stuff like that.
Walter asked if there was a model we could follow to do things like .NET. AW said .NET was JITed, so they were doing all kinds of magic under the hood to get the JIT to line stuff up. He didn't know or even understand everything they'd been doing, but he knew they loved their shared libraries. He thought Rikki's DIP was an important first step for us to enhance our DLL support.
(UPDATE: Now that the new DIP process is open, Rikki has submitted his DIP to the DIP Development forum for community feedback.)
Dennis
The is operator and interfaces
Dennis also had an issue related to the is operator, but for interfaces rather than static arrays. When an interface is cast to Object, the pointer might be shifted 8 or 16 bits higher or lower because the interface has a slight offset. The is operator does a bit comparison, so if you compare an interface to the same interface, cast it to an Object, it won't bit compare. The spec says it should return true since it's referring to the same object. So there'd been some discussion about what was really intended and what was the most useful behavior.
Walter said he'd never thought of that. Steve said it sounded like a bug. Dennis said the spec and the implementation differed, so which was right? Steve thought the implementation was correct. You have two pointers, they point at different things, they shouldn't compare equal.
Átila said he didn't know. Steve said you could always cast the interface to an object and then compare it. Walter thought an interface wasn't the same thing as an object that supports that interface. Steve reiterated that the way to handle this was just to cast the interface to an object. Because what you're really asking in that case isn't if they're the same interface, but if they're the same object.
After some more discussion about whether it should be deprecated or we should just tell people not to do it, and the confusion that would cause, Dennis noted that Petar Kirov had highlighted some object functions in C# and Java that handle this case and had suggested we could change the behavior of is to match those or add a new function. Everyone agreed a new function would be the way to go. In the meantime, the spec should be updated to match the behavior.
Booleans and bit patterns other than 0 and 1
Next, Dennis reminded us there'd not yet been any resolution on whether Booleans should allow bit patterns other than 0 and 1. He said Paul Backus had observed that C had a trap representation for Booleans outside of 0 and 1 that optimizers used and it was undefined behavior if a Boolean wasn't 0 or 1. So if we wanted to be competitive or compatible with that, perhaps we should do that as well. So the proposal was to make it unsafe to reinterpret cast something to a Boolean in a new edition or a preview switch.
Martin thought reinterpret casts were always unsafe. Dennis said no. For example, going from float to int was safe. Martin said that for a reinterpret cast we needed to take the address, otherwise it was not a reinterpret cast. Then he acknowledged that we did have some special cases. Dennis mentioned int array to float array, or a union overlapping an int and float. Those were currently safe.
Walter said that if one of them was a pointer, you couldn't do it in safe code. But when you were just dealing with integer bit patterns, the compiler allowed it. Dennis said it was better not to allow casting bytes to Booleans without truncating because that would either be undefined behavior or a performance penalty on every read.
Átila asked if this was a problem in practice. Dennis said there were some people who were adamant about it, and the related issue had come up on our list through the Gripes and Wishes campaign.
This discussion got down into the weeds of implementation details, but ultimately it came down to what happened when a safe function had a Boolean parameter, which Steve brought up. Walter said it was up to the user to ensure that they were actually passing safe data to a function. Steve agreed, saying that yes, if you passed a Boolean to a safe function, it should be either 0 or 1.
At this point, Walter said he agreed with Dennis that safe code should disallow operations that may result in a bit pattern other than 0 or 1 in a bool. Dennis said that would be really nice. Robert said this sounded like something that should be in an edition. All the things we inherited from C that we'd like to get rid of should go in editions, and this was one of them. Everyone agreed.
Dennis already had a spec PR open to specify safe values for bool. Walter gave the green light to merge it.
Robert
Robert talked a little about what he thought he needed to do for the Bugzilla to GH migration, just expanding a little on what he'd said when AR had asked about it earlier and in our planning session the week before. He then noted that given all the niche stuff everyone was bringing up, his code must be the most boring code ever as he never encountered these issues.
Mathias
Mathias had nothing to report.
Martin
The main thing Martin had been working on was bumping the front end and library version to a recent stable, and that had been extremely painless. It had been quite a while since 2.105, so he'd expected some trouble. Although there'd been a lot of refactoring and stuff moving around, it hadn't been a problem so far.
He said CI was looking very good even on the rather exotic platforms they tested. So 2.106 seemed to be in quite good shape. He also noted that he almost couldn't believe how trivial it had been to enable static associative array initialization. It was a two-liner. He said, "Very nice job on that front." So he was looking forward to 2.106.
Átila
Átila had already given us an update on editions during Adam Ruppe's turn. So now he just wanted to comment on DLLs.
They had to use DLLs at Symmetry and had run into many, many, many problems that Martin could describe better than he could. For instance, they had problems with too many symbols that wouldn't fit into a DLL. Adam Wilson said that was the sort of thing Rikki was hoping to solve. Átila said it would be good to get it all solved.
Walter
Phobos v3
Walter said he'd realized there were a lot of motivating factors for the next version of Phobos that he didn't think everyone knew about, so he'd written them down and emailed them to Átila. He offered to send the list to anyone else present who wanted to provide feedback. Átila said the email hadn't contained anything new. It had all been discussed in DLF meetings. Walter agreed, but it hadn't been written down anywhere that he was aware of. He'd been a bit concerned that everyone had their own idea of what the next Phobos should look like, so he wanted to get something written down that we could debate and argue about.
ImportC
Next, he said he'd been struggling with ImportC because of all the endless terrible things that showed up in standard C headers. Things that were non-standard, things that GCC did which weren't documented anywhere. Even clang was exhibiting the same behavior, but he didn't know how they'd figured it out since it didn't appear to be part of the standard or anything.
So trying to get ImportC to behave the same way had been causing him a lot of grief. Átila brought up a quote he thought was from Coverity: there was no such thing as a C language in practice. Walter thought that was absolutely correct. There were things that made him stop and think "it can't behave this way". So he'd go back and look at his old C compiler and find it matched how the specs said it should work, not the way the other compilers did it. He wondered why this never came up in those days. It was maddening constantly running into this stuff. He'd been fixing them one by one.
Steve said that the main reason he'd wanted to use ImportC for his Raylib port was so that he could say to users that they only needed dub. They could add the library, build it, and we could still use C code in there. But he'd realized that a C compiler was still needed anyway in order to run the preprocessor with all of its predefined macros and such. At that point, it might be better to just add features to dub where we could tell it to go call the C compiler and build this stuff. And ImportC might be better off as just a C importer. Just read the C files and spit out the interfaces rather than compiling them.
Walter said it couldn't be that way because of inline functions. There was just no point in having half a C compiler because it would be too limited to be useful. You had inline C functions, macro definitions, and stuff like that which were C code. And anyway, the latest problems he was having were with declarations, not statements. So if we couldn't compile the C code, we'd suffer all kinds of problems.
Martin noted that we already needed the system C compiler installed anyway to drive the linker and for the C libraries. So dragging in the extra dependency for the preprocessor shouldn't be a big deal. Although on Windows, we did ship the linker and the MinGW-based import libraries so it was self-contained there.
He also said he'd given ImportC a test drive with the Postgres headers. He was surprised to find that we were almost there. He'd encountered 5 or 6 problems related to missing GCC intrinsics and one double-underscore define that wasn't guarded with an ifdef and was related to how it was compiled on that system.
He'd also hit an issue with LDC related to inline assembly. DMD just skipped it because it was GCC syntax, but LDC and probably GDC parsed it. It was a tiny issue related to string literals. When he encountered that, he'd thought like Steven did that it would be nice to be able to at least try to suppress function generation. Even if it were an inline function in the headers, we might not need it. So if there were a switch to work around problems like this, he was pretty sure that there were probably just a few dozen functions in the C universe that we needed, and we most likely didn't need all the inline assembly stuff.
Walter asked if the issues Martin had mentioned were in Bugzilla. Martin said he hadn't filed them because these were things that couldn't be fixed from the outside. The headers had to be modified to really fix them. Walter said that one of the rules he was trying to follow with ImportC was that he didn't want to force the user to modify the files. They were not going to go modify things in /usr/include. So he had to find some kind of hack to make a thing work. And he had found a surprising number of hacks to do that.
So Walter said when encountering an ImportC problem like this, don't assume he couldn't fix it. Put it on Bugzilla anyway, tag it with ImportC, and he'd have a look at it. Maybe he'd have a brainwave on it.
Meetups
Walter noted that he and AW and Bruce had been meeting up for beers and discussion at the Red Robin in Seattle. He said if anyone was in the area they were welcome to join. He then realized he should announce it on the newsgroups.
(UPDATE: Walter has since started posting in the announce forum in advance of every Seattle meetup.)
The Next Meeting
The December monthly meeting took place on December 8 at 16:00 UTC. I've allotted some time to get caught up on all the meeting summaries and planning updates, so I'll have that one ready soon.
As always, I'd like to remind you that if you have anything you want to bring to us in one of our monthly meetings, please let me know and I'll set something up.
