February 03

On Saturday, 3 February 2024 at 03:36:08 UTC, Adam Wilson wrote:

>

On Friday, 2 February 2024 at 19:39:34 UTC, Lance Bachmeier wrote:

>

On Friday, 2 February 2024 at 18:57:48 UTC, Adam Wilson wrote:

>

I cannot argue strenuously enough against putting the packaging system in to the compiler directly. Not only will it balloon the build times by orders of magnitude, it is a serious violation of separation-of-concerns. The compiler should concern itself with compiling code and nothing else. The compiler is the wrong place to be handling third-party packages, that is properly the venue of the build system.

I responded to the other comment, but I'll give the same answer. What I'm proposing doesn't have anything to do with the packaging system, Dub, or any of that. The only thing the compiler would have to know is how to download the source files of a package if they haven't already been downloaded.

But that's not true is it? How does it know which versions of the package are acceptable?

It doesn't need to if only the most recent version as of the compiler release is supported.

>

Which of the acceptable versions does it pull?

Answered above.

>

How does it handle the packages' dependencies? You simply cannot have the compiler pull a package without going through the entire dependency resolution process.

This is actually a non-issue for many packages on code.dlang.org. I've been downloading the D source files and putting them in my repos for years. I specified that this would be for some of the popular packages, not arbitrary packages.

> > >

People always say there are good reasons, but are light on actual reasons. Saying that Phobos has a high bar is not itself a reason so much as an excuse to not make the effort.

A sufficient reason is that I know how things get done around here, and an updated Phobos with these additions will optimistically be released in 2032. Most of the work has already been done. Why not reuse it?

That's not a reason. You're saying that you know what the reason is, that you're not going to state the reason,

I actually listed multiple good reasons in my original comment, all of which you ignored, so I gave another.

> >

It's been eight months since I proposed a simple solution to the problem that library writers have to support too many compiler releases. Label some compiler releases as not being supported by library writers. It would have solved the problem fully and it could have been done by the end of the day. Obviously we couldn't do that. Instead we got editions. Eight months later, we still don't have an initial proposal, and the problem for library writers continues.

That is pretty much where this is going to end. Walter and I discussed tying Phobos releases to Compiler Editions, and he was amenable to the idea, his only sticking point was to not use "Editions" to describe Phobos, and "Versions" or "Releases" works just fine for our purposes. I would expect to see Phobos 3 land with the first Edition. Both of which I would expect in 2025.

I am sorry to hear that it's not getting there as fast as you want it to, but that is the nature of volunteer projects, so unless you're willing to step up and help out, it's going to happen at pretty much whatever pace the rest of us can afford. But don't expect much sympathy when you're whining about a problem you're not doing anything to help solve.

This...doesn't make sense. There was a complaint that library writers have to support too many compiler releases. Well, that's an easy problem to solve, which the open source world has used for decades. Just say "we only support releases X and Y". I even gave examples. Wouldn't take any work at all, just a simple management decision, done.

Now you're telling me I should just shut up because I'm not the one doing any of the work that doesn't need to be done?

I give up.

February 04

On Saturday, 3 February 2024 at 12:53:34 UTC, Paul Backus wrote:

>

Obviously we are not going to get all of Phobos to work with BetterC, but significant chunks of it already do, and there's a good deal more that could be considered low-hanging fruit.

I think the right approach here is to try to make code BetterC-compatible when we can, and to at least keep BetterC compatibility in mind as a "nice-to-have" goal (but not a requirement) when doing API design. For example, if the only thing keeping a function from being BetterC-compatible is that it uses the GC to allocate memory for an error message, it's probably worth a little bit of extra effort to either get rid of that GC dependency, or provide a version (D_BetterC) alternative.

This is the path that we're taking. We want to make Phobos as widely usable as possible, but there is a balancing act we have to perform. If we restrict the language features we allow ourselves to much, there are entire classes of capabilities we cannot deliver. BetterC is too much, for example, we wouldn't be able to use classes/interfaces or the GC where it's appropriate.

JMD and I were talking about sockets and I think he came up with an interesting approach to this problem. He built a low-level API that is either BetterC compliant or very close to it, and then built a higher level API on top of that. The low-level API is painful to use but it allowed him to build the high-level API he wanted in a way that made sense. You can use the low-level API if you absolutely need that level of performance/control/etc, but most projects will find the high-level API to be sufficient for their needs.

Perhaps this is a concept that we should investigate for other areas of Phobos?

February 04

On Saturday, 3 February 2024 at 11:54:48 UTC, ryuukk_ wrote:

>

Oh, and Exception Handling?

Get in native tuple and tagged union in the language, and you'll be able to build nice error handling code

I hope no EH in Phobos 3

The EH problem has two potential answers.

The first is the sumtype answer. That would be zero runtime cost and would achieve a similar result as the current EH scheme.

The second is that, absent sumtype exceptions, Phobos will be designed to not use exceptions wherever possible. There will always be cases where exceptions are needed so we can't entirely rule them out, but any time one is added it will require some serious justification.

This goes back to not wanting to box ourselves into any tight corners or inferior designs in the name of slavishly attempting to a too-restrictive standard. Consider this potential problem, we completely eradicate EH in Phobos 3, create a number of less-than-ideal implementations, and then the next Edition of D drops with sumtype exceptions and now we have to go back and rebuild all those known inferior designs because EH became acceptable to the crowd again.

It is better to build the best design up-front, pay a small penalty for it, then work on minimizing the penalty.

February 04

On Saturday, 3 February 2024 at 19:08:34 UTC, Lance Bachmeier wrote:

>

It doesn't need to if only the most recent version as of the compiler release is supported.

This makes a number of assumptions about both the package itself and is inherently limiting. Problems:

  1. Updating a package for security/bugs/etc. now requires a compiler update.
  2. Chicken-egg problems. Compiler makes breaking change that breaks the package, but the package maintainer cannot update the package until after the compiler has shipped. Therefore the package, and all of it's users, are broken for an entire release cycle. This could theoretically be solved with extremely tight coordination with package maintainers, but nothing in D's history suggests that there is even a remote possibility of this occurring on even a semi-repeatable basis. The package maintainers are volunteers too, if they miss the deadline, broken code ships and people yell at us for shipping broken code.
  3. More chicken-egg problems. The syntax you propose implies the execution of DUB in the background. OK, but DUB is built with DMD, so by this design, you need DUB to pull a Phobos package to build DUB. Yes, there are bootstrapping designs, but ask Iain or Kinke about the bootstrapping rules for GCC or LLVM some time, they are highly restrictive.
  4. Enforces the use of DUB, no other tool/package system can be built or substituted without changing the compiler syntax. D has a relatively diverse build tool ecosystem, are we telling them to just go pound sand?
  5. Not all packages conform to Phobos' standards. Many gc-allocate heavily, or use exceptions, etc. By doing this we are explicitly telling entire groups of the D userspace that chunks of Phobos aren't for them, but that we can't easily tell them which chunks those are because we don't control them, so now they have to go lobby individual maintainers to support them. The screaming about Phobos' problems are loud enough without inviting even more trouble by pulling in packages that were never intended or designed to be held to Phobos' standards.

And this doesn't address the fact that the first thing the community will ask for is the ability to specify package versions.

I can hear Atila (Mr. Build-Speed himself) warming up his fingers from here for an explanation as to why any one of the above four problems make this idea untenable. But all five make it a complete non-starter with everybody who maintains the compiler. I submit that there are more useful ways to expend energy than pining for something that is simply never going to happen.

>

This is actually a non-issue for many packages on code.dlang.org. I've been downloading the D source files and putting them in my repos for years. I specified that this would be for some of the popular packages, not arbitrary packages.

This implies that there are design constraints on the package then (no dependencies?), correct?

>

I actually listed multiple good reasons in my original comment, all of which you ignored, so I gave another.

You have correctly pointed out that I did not do a point-by-point answer to your claims so here you go. (Relevant snippet below with responses interspersed)

>

There are good reasons to not add that stuff to Phobos.

I disagree, expanding Phobos expands the range of software that can be built without needing DUB, thus expanding the capabilities of software that can be built where DUB is unavailable or untenable, while avoiding the problems listed above.

>

The standard to get something in Phobos is high enough that it leads to a lot of wasted time for many people,

It is high, and there is a good reason for that. Phobos has to work everywhere, it has to meet a broad variety of use case, and user environments.

I do agree that there are adjustments that could be made. For example, we tend to spend a lot of time on unnecessary bikeshedding and in some cases bikeshedding is used to intentionally stall out PRs. So one change than can be made is to focus on more qualitative reviews and not allow bikeshedding to stall a PR. Another, more controversial, change that I've been considering proposing, but know is going to receive considerable resistance, is allowing PR's that do not fulfill all the environmental rules (like not using the GC or using exceptions) to pass and then inviting others to modify the code to fit those rules. This is a deeply controversial idea and the debates I've had that only nibble at the edges of it have been ... emotional.

>

it doesn't bloat the compiler download size,

Just how much does the library code add to the download though? Most of the download size in the binaries, not the library code. Assuming an atrocious 5:1 text-compression ratio we would have to add 10MB of code to add just 2MB of download size. For reference, Phobos V 2.106 is 12.6MB on my NTFS partition. We could triple the capability of the standard library for the cost of an additional 4MB on a 32MB installer. In this modern era of Cable/DSL/Fiber I fail to see how an additional megabyte or two of download size is a serious concern, especially since 10-20MB of library code is a serious amount of additional utility that would be available OOTB.

>

it takes advantage of work that's already been done,

This is true, but is the work that's been done of the quality we want to ship as a top-line package? Does it follow the standards and idioms that the users of Phobos are used too?

>

and it could be done quickly, rather than the years it takes to get stuff done otherwise.

This is likely untrue. First, even using DUB as an external executable, you still need to teach DMD how to execute DUB and parse the dub -v output. That is a non-trivial amount of work. And we would have to figure out how this interacts with traditional dub project files (if two different versions are specified, what happens? The interactions are non-trivial and will require significant amount of code and testing of the compiler. Then you have to teach the library which packages are acceptable and manually maintain lists of versions match which compilers.

>

This...doesn't make sense. There was a complaint that library writers have to support too many compiler releases. Well, that's an easy problem to solve, which the open source world has used for decades. Just say "we only support releases X and Y". I even gave examples. Wouldn't take any work at all, just a simple management decision, done.

Now you're telling me I should just shut up because I'm not the one doing any of the work that doesn't need to be done?

I give up.

As I pointed out above, there is in fact significant work that would need to be done.

February 04

On Friday, 2 February 2024 at 09:09:37 UTC, Adam Wilson wrote:

>

Fix std.traits

Can you expand on this a bit more? I've contributed to std.traits in the past and would like to help improve it for Phobos v2.

February 04
On Sunday, February 4, 2024 2:07:44 AM MST Meta via Digitalmars-d wrote:
> On Friday, 2 February 2024 at 09:09:37 UTC, Adam Wilson wrote:
> > Fix std.traits
>
> Can you expand on this a bit more? I've contributed to std.traits in the past and would like to help improve it for Phobos v2.

The main thing that it needs IMHO is going through it and making sure that it's doing the right thing for each trait and that the exact set of traits that it has is really what we want, since right now, several of them are doing the wrong thing (e.g. treating enum as if it were its base type with stuff like isIntegral, isNumeric, etc., which is a source of bugs). The names also potentially need to be cleaned up, and it just needs to be looked over in general to make sure that it's really what we want. It's grown organically over the years, the current situation with D is not the same as when std.traits was started, and we've learned more since then, so the exact set of traits that we want to have in there likely should be adjusted now that we have an opportunity to essentially start from scratch with it.

That being said, as I understand it, what initiated the idea to rework it as one of Phobos v3's priorities is that Walter doesn't understand traits like isSomeChar and finds them confusing, though I suspect that that's probably because he hasn't done much with them, if nothing else, because dmd doesn't use Phobos, and I would expect that it does a lot less with templated code than a lot of D code does, though I haven't dug through its code lately.

I also get the impression that some folks just plain don't like what the module does, but given how D's template constraints work, we fundamentally need traits like it has even if the exact set that is in there needs work.

There's also likely some debate to be had over std.traits vs __traits. On the whole, std.traits adds stuff on top of __traits, so plenty of folks use both, but part of the idea behind std.traits was to make it so that most code wouldn't need to use __traits. The advantage of that is that the result is cleaner code (since __traits tends to be pretty ugly), but it also means more template instantiations. So, I expect that there is going to be some disagreement on what we should be doing there (though there's no reason why we can't have traits in std.traits which simply wrap __traits stuff even if some folks want to use __traits directly as much as possible). But regardless, there are a number of things that std.traits tests that aren't in __traits.

In any case, as things stand, I don't think there really is agreement on what should be done with std.traits, and deciding what exactly needs to happen with it is one of the things that we're going to need to figure out.

- Jonathan M Davis



February 04

On Sunday, 4 February 2024 at 09:30:58 UTC, Jonathan M Davis wrote:

>

So, I expect that there is going to be some disagreement on what we should be doing there (though there's no reason why we can't have traits in std.traits which simply wrap __traits stuff even if some folks want to use __traits directly as much as possible). But regardless, there are a number of things that std.traits tests that aren't in __traits.

In any case, as things stand, I don't think there really is agreement on what should be done with std.traits, and deciding what exactly needs to happen with it is one of the things that we're going to need to figure out.

  • Jonathan M Davis

On Sunday, 4 February 2024 at 02:25:25 UTC, Adam Wilson wrote:

> >

This goes back to not wanting to box ourselves into any tight
corners or inferior designs in the name of slavishly attempting to a too-restrictive standard. Consider this potential problem, we completely eradicate EH in Phobos 3, create a number of less-than-ideal implementations, and then the next Edition of D drops with sumtype exceptions and now we have to go back and rebuild all those known inferior designs because EH became acceptable to the crowd again.

D's feature has problem, and is annoying to use? Don't be like C++, don't bloat the std, fix the language

STD should not be the place to fix language's shortcoming

EH is evil, some platforms has banned it completly (apple), and microsoft is all in rust, they are rewritting their C# application in Rust too, not just C++, it is an inferior design

https://www.theregister.com/2024/01/31/microsoft_seeks_rust_developers/

February 04
On Sunday, February 4, 2024 3:13:39 AM MST ryuukk_ via Digitalmars-d wrote:
> EH is evil, some platforms has banned it completly (apple), and microsoft is all in rust, they are rewritting their C# application in Rust too, not just C++, it is an inferior design
>
>
> https://www.theregister.com/2024/01/31/microsoft_seeks_rust_developers/

A number of us do not agree with you, and I for one never will. For many situations, exceptions are by far the best error handling mechanism that there is. D's implementation of it could certainly use some improvements (e.g. while being able to have a hierarchy for exceptions is valuable, the fact that they're classes and thus have to be heap allocated is a problem for code that needs to avoid the GC), so it would be great if D's implementation for exceptions could be improved, but the basic concept is solid and makes a lot of code cleaner as a result (e.g. I'd never want to write a parser that didn't use exceptions; being able to separate the error handling code from the actual parsing makes the code _far_ cleaner and far less error-prone than any other error handling mechanism that I've ever encountered).

Obviously, there are situations where exceptions are not appropriate (e.g. it was definitely a mistake to have Phobos decoding Unicode all of the place and potentially throw a UTFException from practically anywhere as a result), and any properly written library is going to have to be intelligent about when any particular error handling mechanism is used, but exceptions are far, far too useful to not use - especially when the main alternative is checking return values all over the place. So, while there are places that Phobos currently uses exceptions where it shouldn't, there are others where using them is very much the right thing to do IMHO, and I would use them again in the next version of Phobos (and argue strongly against anyone trying to not use them there), because if I didn't, the result would be worse code that was harder to use.

In general, exceptions are best when

1. It's cleaner to assume that an operation will succeed, and it's reasonable to assume that it will succeed. Parsing is a great example of this. You're not going to do something like validate that an XML document is going to parse correctly and then parse it, since that would basically mean parsing it twice. You're just going to parse it and then report when that fails so that the caller can handle it. And exceptions make that very clean, because then you only have to have the one place at the top that catches the exception, and the rest of the code can just do its thing. The way that exceptions bubble up, allowing code along the way to completely ignore the error handling and then have somewhere higher up the stack that is actually in a position to handle the error condition catch the exception and handle it is an amazing feature.

2. It's also best to use exceptions when you can't guarantee that an operation will succeed but where it's reasonable to assume that it will - particularly when it's actually impossible to check that an operation will succeed beforehand. A great example of this would be reading a file. It's good practice to check that the file exists first, but even then, you can't guarantee that opening it will succeed, because the file could be removed between the time you check and the time you go to open it (or a variety of other errors could occur which cause opening or reading the file to fail). The same goes for most file operations really. Any of them could fail, and there needs to be a way to report that, and it clutters up the code considerably if you have to deal with checking return values all over the place - on top of the fact that it makes it impossible to simply return an object or buffer from a function and use it if you also have to check an error condition. You're stuck either returning the object via one of the parameters or using a compound type where you have to check if the operation succeeded and then extract the actual return value from it, which absolutely destroys your ability to chain function calls - on top of making it far more likely that code will fail to check the return value like it should, resulting in it assuming that the call succeeded when it didn't.

3. For constructors, exceptions are really your only option unless you want to get into doing two-part initialization, which is known to be error-prone, and the advice I've almost always seen is to avoid it like the plague. So, if a constructor needs to validate its arguments - or report any other error condition that might occur while it's being constructed - an exception is going to be the way to do it unless you want to do something like add a member function to the type just so that you can check whether it was constructed correctly, and at that point, you might as well just be doing two-part initialization.

4. And exceptions are an excellent choice in any situation where you need to be sure that an error condition is not ignored - or at least put the code in a position where either the exception is caught and handled in whatever manner is appropriate, or the program is killed, because the error condition wasn't handled. Obviously, not all error conditions fall into that category, but there are plenty of situations where error conditions must be handled for the application to operate properly, and that's much more likely to happen with an exception than with any other error reporting mechanism that D has. Some other languages force that by forcing you to check return values, but that can be incredibly annoying, and it results in far more verbose code, because you have to put error-handling code everywhere instead of just letting the exception bubble up to the code that can actually handle it properly.

Now, obviously, there are situations where exceptions aren't appropriate. An obvious one is where an error is likely, because using an exception in that kind of situation is going to be inefficient in comparison to other options, and if the error path is that likely, then trying to separate out the error handling code like you do with try-catch statements really doesn't make sense anyway.

And of course there are situations where validating stuff ahead of time is perfectly reasonable and allows the code beyond that to just assume that everything works without needing to report error conditions at all, because the validation was already done. Unicode decoding is one such case where that often makes sense. If you validate the Unicode when you read in a file, then the rest of the code doesn't have to. Phobos, unfortunately, currently picks the worst of both worlds, because readText checks the Unicode, and then you have auto-decoding all over the place which validates the text again and throws an exception if it's invalid. So, we end up with a bunch of code that can't be nothrow or @nogc just because of a potential UTFException which can't possibly be thrown if the text was already validated.

And of course there are also situations where it's actually reasonable to ignore an error, in which case, throwing an exception wouldn't make sense - but no other reporting mechanism would either. Unicode can actually be handled in such a fashion in some cases, because it has the replacement character so that you can use it anywhere that you encounter an invalid code point, which gives text processing a way to process text without worrying about invalid Unicode. That's obviously not a good choice in all cases, but without auto-decoding, that choice is left up to the programmer as it should be.

And with D, there are situations where exceptions can't be used where they could be in a language like C++, because D's exceptions are allocated on the heap, which not only means allocating memory, but it typically means using the GC, which is fine for most code but not fine for all code. So, anyone in that kind of situation is going to need to avoid exceptions even if they would otherwise be the best choice.

And naturally, there are idiots who use exceptions for stupid things (e.g. throwing an exception when you're checking a condition instead of just returning bool, or having a type implement an interface and then throw an exception if you call a function that it doesn't actualy support), which I expect is part of the reason that some folks dislike exceptions. So, there's no question that exceptions can be a problem, but overall, in the hands of someone who knows what they're doing, they can be an excellent tool and result in much cleaner, less error-prone APIs.

Obviously, you're free to hate exceptions for whatever reasons you may have, and we're obviously not going to use them for everything in Phobos, because that would be dumb, but given that there are cases where code is clearly cleaner and easier to use correctly when exceptions are used, you're not going to find general agreement with the idea that exceptions are inherently bad. I expect that for most people, the disagreement is going to be on which cases exactly they make sense for, and which they don't, not whether they're a bad idea in general.

And personally, I wouldn't use D if it didn't have exceptions. It would be far too miserable to not have them. So, while D's exceptions could certainly use some improvement, I never would have started using D if it hadn't supported exceptions, and if Walter seriously tried to get rid of them, I'd go find another language to use instead. The exact form exceptions in D take may change at some point in the future, but their basic functionality is IMHO critical to writing good APIs, and part of the reason that languages like C are miserable to use is that they don't have a comparable feature.

- Jonathan M Davis



February 04
On Sun, Feb 04, 2024 at 05:35:02AM -0700, Jonathan M Davis via Digitalmars-d wrote:
> On Sunday, February 4, 2024 3:13:39 AM MST ryuukk_ via Digitalmars-d wrote:
> > EH is evil, some platforms has banned it completly (apple), and microsoft is all in rust, they are rewritting their C# application in Rust too, not just C++, it is an inferior design
[...]
> A number of us do not agree with you, and I for one never will. For many situations, exceptions are by far the best error handling mechanism that there is.

Exceptions are like the GC: they free your APIs from being littered with error-handling paraphrenalia that crowd out the actual business logic. Just like not having the GC forces your code and APIs to be crowded with memory-management paraphrenalia. Both memory management and error-handling are secondary concerns; the primary concern is actually making progress in your problem domain: doing what the code set out to achieve in the first place. As such, code should be focused as much as possible on the business logic rather than have secondary concerns occupy most of the space. Ideally, the main code should *only* express the business logic; memory management and error-handling should be done in a footnote rather than in the main body of the code.

Try/catch blocks let you do this to a large extent; whereas having to manually check error codes or unwrap error sumtypes just adds excessive verbosity to the primary logic, which makes code more tedious to write, harder to read, and less maintainable as a result.

I suspect, though, that most objections to EH are directed primarily at the specific implementation of it using libunwind like in C++, rather than the concept itself of EH. That part I agree with: the way EH is implemented could be done better. But as far as the concept is concerned it's fine, and trying to get rid of it is just silly.


T

-- 
The easy way is the wrong way, and the hard way is the stupid way. Pick one.
February 04
On Sunday, 4 February 2024 at 09:30:58 UTC, Jonathan M Davis wrote:
> The main thing that it needs IMHO is going through it and making sure that it's doing the right thing for each trait and that the exact set of traits that it has is really what we want, since right now, several of them are doing the wrong thing (e.g. treating enum as if it were its base type with stuff like isIntegral, isNumeric, etc., which is a source of bugs).

I have bad news for you. This behavior is not the fault of std.traits, but is actually baked into the language itself. For example, the spec for __traits(isIntegral) says:

> The integral types are: byte, ubyte, short, ushort, int, uint, long, ulong, cent, ucent, bool, char, wchar, dchar, vectors of integral types, and enums with an integral base type.

https://dlang.org/spec/traits.html#isIntegral

Maybe we can clean this up in a new language edition, but as long as this is how the underlying __traits work, std.traits should work the same way.