Phobos sometimes gets a bad name around these parts, but actually it's pretty cool (in spite of well-known warts).  One of the cool things about it is the batteries-included philosophy that used to be its guiding principle.  Over the course of my usage of D, the following have stood out as being particularly convenient when I needed it:

1) std.bigint: one time, I was working on a quadratic irrational arithmetic module, and pretty quickly discovered that numerical overflow was a problem that cropped up pretty often.  Or rather, I was seeing what I *thought* was caused by arithmetic overflow, and wanted to be sure.  Solution?  Easy: templatize the coefficient type, import std.bigint, and within several minutes, I have a working implementation that can support arbitrarily large coefficients -- with which I not only confirmed that the observed problems were caused by arithmetic overflow, but also obtained a working solution simultaneously.  Win!

2) std.numeric.poly: another time, I had to work with evaluating a bunch of polynomials, and specifically, with evaluating their roots quickly and accurately.  I could roll my own, poorly, and get results that may or may not be highly-skewed by accumulated errors... or I could use std.numeric.poly to get a reliable evaluation of the polynomial, and, better yet, use std.numeric.findRoot to compute roots with the confidence that the value I get will have as small an error as is possible within the constraints of built-in hardware floats. Plus, this saved me tons of development and debugging time to roll my own solution. Win!

3) Just today, I needed to implement a fast Fourier transform. Not expecting this to be in Phobos, I was glancing over various algorithms to decide which one suited my use case best and is easy enough to implement quickly. And then I discovered std.numeric.Fft, already done and nicely-packaged and ready to use.  Total win!!

Seriously, D + Phobos is cool beans.  Yes it's not perfect -- the language has its fair share of dark corners, WATs, and Phobos has its share of poorly-designed APIs and outdated modules.  But seriously, a lot of what's currently there is pretty darned cool, and we shouldn't let all the bad stuff cloud our appreciation of just how cool it already is.


T

-- 
If the comments and the code disagree, it's likely that *both* are wrong. -- Christopher

On Wednesday, 24 June 2020 at 21:37:37 UTC, H. S. Teoh wrote:
> Phobos sometimes gets a bad name around these parts, but actually it's pretty cool (in spite of well-known warts).  One of the cool things about it is the batteries-included philosophy that used to be its guiding principle.  Over the course of my usage of D, the following have stood out as being particularly convenient when I needed it:
>
> 1) std.bigint: one time, I was working on a quadratic irrational arithmetic module, and pretty quickly discovered that numerical overflow was a problem that cropped up pretty often.  Or rather, I was seeing what I *thought* was caused by arithmetic overflow, and wanted to be sure.  Solution?  Easy: templatize the coefficient type, import std.bigint, and within several minutes, I have a working implementation that can support arbitrarily large coefficients -- with which I not only confirmed that the observed problems were caused by arithmetic overflow, but also obtained a working solution simultaneously.  Win!
>
> 2) std.numeric.poly: another time, I had to work with evaluating a bunch of polynomials, and specifically, with evaluating their roots quickly and accurately.  I could roll my own, poorly, and get results that may or may not be highly-skewed by accumulated errors... or I could use std.numeric.poly to get a reliable evaluation of the polynomial, and, better yet, use std.numeric.findRoot to compute roots with the confidence that the value I get will have as small an error as is possible within the constraints of built-in hardware floats. Plus, this saved me tons of development and debugging time to roll my own solution. Win!
>
> 3) Just today, I needed to implement a fast Fourier transform. Not expecting this to be in Phobos, I was glancing over various algorithms to decide which one suited my use case best and is easy enough to implement quickly. And then I discovered std.numeric.Fft, already done and nicely-packaged and ready to use.  Total win!!
>
> Seriously, D + Phobos is cool beans.  Yes it's not perfect -- the language has its fair share of dark corners, WATs, and Phobos has its share of poorly-designed APIs and outdated modules.  But seriously, a lot of what's currently there is pretty darned cool, and we shouldn't let all the bad stuff cloud our appreciation of just how cool it already is.
>
>
> T

I suggest checking http://mir-algorithm.libmir.org/.
Ilya has been doing a truly amazing job at both developing, maintaining and optimizing this library.
Also he has put a hard focus on memory allocators, @nogc and betterC (for exposing libmir to C and C++ users and also for scripting languages like Python or Julia).
AFAIK his work has been sponsored by Symmetry, so I suppose libmir gets a lot of internal use there.

Meanwhile, if you look at std.numeric, for past several years, basically all contributions but a few have been just around basic stuff like code style, imports and fixing a few compile errors in generic code.

Phobos is dead. There haven't been any new modules since ~2016. Actually the last one (IIRC) was std.experimental.ndslice, which Ilya contributed and then removed (1-2 releases after) from Phobos to start libmir.

Since std.experimental.allocator was added to Phobos circa 2015, there hasn't been a single Phobos or Druntime module using it (ndslice and checkedint don't count). Many packages on code.dlang.org enthusiastically embraced std.experimental.allocator only to get burned a few dmd releases after as Phobos doesn't follow SemVer. The solution? Many of the most popular OSS D apps libraries (I can only comment on those) transitioned to using the Dub fork: https://github.com/dlang-community/stdx-allocator. How does it help? Well, OSS D libraries can't control the compiler that their users are using. For example, vibe.d tries to support the last 5 major versions of dmd/ldc, and Dub the last 15+ major versions. This means they can't depend on std.experimental.* as there's no telling what kind changes can happen for such a large timespan. Instead they can stick a particular version of https://code.dlang.org/packages/stdx-allocator in their dub.{json,sdl} file and be sure that it won't suddenly break their code.

The sooner we realize this and start collectively working on improving Dub and code.dlang.org, the better. Yes, Dub is not a great build tool. And so what? Most great build tools were not so great in their infancy. I doubt Scons was much better in 1999. If people continue avoiding Dub for whatever reason it won't get better by itself.
There's no reason why Dub can't expose a lower-level DSL for precise target dependency graph building. It's just a matter of people making a sustained effort in contributing one.

The Rust community has realized that a slim standard library + a solid package manager is the way to go. Rust is also backed by Mozilla. And so what? I seriously doubt that more than 5-10% of all of the 42k packages on https://crates.io/ were created by Mozilla employees. Instead the core Rust team realized that they can't build everything by themselves, and focused on helping the community scale by putting a concentrated effort on Cargo (their version of Dub).

On Thursday, 25 June 2020 at 06:52:05 UTC, Petar Kirov [ZombineDev] wrote:
>
> The sooner we realize this and start collectively working on improving Dub and code.dlang.org, the better.

FYI for anyone interested, this has been a topic of discussion at the past two quarterly foundation meetings. We have outlined a path forward, but we just need to secure funding. We're currently awaiting news on that front. Whether the news is good or bad, I'll be able to announce the plans shortly after I hear it. If it's bad news, we'll likely have to seed the initiative with the HR fund and depend on the community to keep it going long-term until we get an alternative solution. I'm hopeful it won't come to that.

On Wednesday, 24 June 2020 at 21:37:37 UTC, H. S. Teoh wrote:
>
> 3) Just today, I needed to implement a fast Fourier transform. Not expecting this to be in Phobos, I was glancing over various algorithms to decide which one suited my use case best and is easy enough to implement quickly. And then I discovered std.numeric.Fft, already done and nicely-packaged and ready to use.  Total win!!
>

What if the language you're using doesn't have FFT in its standard lib? Hm. Maybe this would be a good starting point:

http://rosettacode.org/wiki/Fast_Fourier_transform

Maybe in other languages there are third party libraries that handle things like FFT, well, so I have heard. Never mind.

On Thursday, 25 June 2020 at 06:52:05 UTC, Petar Kirov [ZombineDev] wrote:
> Since std.experimental.allocator was added to Phobos circa 2015, there hasn't been a single Phobos or Druntime module using it (ndslice and checkedint don't count). Many packages on code.dlang.org enthusiastically embraced std.experimental.allocator only to get burned a few dmd releases after as Phobos doesn't follow SemVer. The solution? Many of the most popular OSS D apps libraries (I can only comment on those) transitioned to using the Dub fork: https://github.com/dlang-community/stdx-allocator. How does it help? Well, OSS D libraries can't control the compiler that their users are using. For example, vibe.d tries to support the last 5 major versions of dmd/ldc, and Dub the last 15+ major versions. This means they can't depend on std.experimental.* as there's no telling what kind changes can happen for such a large timespan. Instead they can stick a particular version of https://code.dlang.org/packages/stdx-allocator in their dub.{json,sdl} file and be sure that it won't suddenly break their code.

This gets brought up quite often. Breaking changes doesn't seem to be a concern. I'm not even talking about the DIP changes that are going to cause breaking changes. There's just compiler changes that break codes for almost every new release. There doesn't even need to be a big change, just slowing down the releases would help. So there's less versions you have to deal with to support a longer term of time.

On Thursday, 25 June 2020 at 06:52:05 UTC, Petar Kirov [ZombineDev] wrote:
> On Wednesday, 24 June 2020 at 21:37:37 UTC, H. S. Teoh wrote:
>> [...]
>
> I suggest checking http://mir-algorithm.libmir.org/.
> Ilya has been doing a truly amazing job at both developing, maintaining and optimizing this library.
> Also he has put a hard focus on memory allocators, @nogc and betterC (for exposing libmir to C and C++ users and also for scripting languages like Python or Julia).
> AFAIK his work has been sponsored by Symmetry, so I suppose libmir gets a lot of internal use there.
>
> [...]

I know that we are quite few on this side, but IMHO a well shaped standard library has advantages over a sparse ecosystem of independent library ...

Anyway, if we can, we try to stick with Phobos, as long as we don't have a particular problem to solve that needs an external library: recent example, Martin std.io or SumType instead of std.variant ...

On Thursday, 25 June 2020 at 06:52:05 UTC, Petar Kirov [ZombineDev] wrote:
>
> The Rust community has realized that a slim standard library + a solid package manager is the way to go.

The practice seems to indicate that the theory is false.

The very first Rust program I built with Rust was an _example_ program for the Amethyst game engine (granted, a big endeavour).

It downloaded _146_ packages and then proceeded to build them at a glacial pace. This was a really "npm" moment.

In D my _full_ products have 6 packages (24 when counting sub-packages).

It seems slim stdlib has enormous community implications and require inordinate amount of coordination and agreements ; it changes the whole culture.

On Thursday, 25 June 2020 at 09:08:37 UTC, Mike Parker wrote:
> On Thursday, 25 June 2020 at 06:52:05 UTC, Petar Kirov [ZombineDev] wrote:
>>
>> The sooner we realize this and start collectively working on improving Dub and code.dlang.org, the better.
>
> FYI for anyone interested, this has been a topic of discussion at the past two quarterly foundation meetings. We have outlined a path forward, but we just need to secure funding. We're currently awaiting news on that front. Whether the news is good or bad, I'll be able to announce the plans shortly after I hear it. If it's bad news, we'll likely have to seed the initiative with the HR fund and depend on the community to keep it going long-term until we get an alternative solution. I'm hopeful it won't come to that.

Thank you for the update Mike, I appreciate it.

I think even the fact that this has been the topic of discussion for the those meetings is a good news in itself!

I haven't been able to contribute to D in the past 1-2 years as I would like to, but before that, at some point, I was in the top 3 people in terms of number of pull request code reviews, so back then I had very good visibility of the how things were going.

I'm currently the CTO of a small startup, and needless to say, I'd very much like our team to use D for more things, but the biggest blockers for us is that the competition (Rust, Go, TypeScript) has much better developed ecosystem. Strangely for many, IDE support is not important for us. (And especially given D's metaprogramming capabilities, I haven't had high expectations anyway :D)
Most of the libraries that we need will never be part of Phobos. If the third-party library & package ecossytem doesn't catch up with Rust or Nim, using D is just not going to be practical for the core of our business.

On Thursday, 25 June 2020 at 11:09:04 UTC, Avrina wrote:
>
> This gets brought up quite often. Breaking changes doesn't seem to be a concern. I'm not even talking about the DIP changes that are going to cause breaking changes. There's just compiler changes that break codes for almost every new release. There doesn't even need to be a big change, just slowing down the releases would help. So there's less versions you have to deal with to support a longer term of time.

Breaking changes are not a problem. As someone who was very involved in pull request reviews in the past, I can definitely say that outside of std.experimental, for the past few years we have been relatively rigorous (compared to say 5 years ago), and almost every breaking change that we were aware has first gone through a deprecation process.

So breaking changes are not a problem by themselves. The problem is how they're delivered. The problem is that Phobos is distributed as part of each compiler release. You can't keep using an older version of Phobos if you need the new platform support of say LDC 1.22.0. And vice versa, if you need a new Phobos function, but there's a compiler bug preventing you from upgrading you're again stuck between a rock and a hard place.

And what frustrates me the most is that this is an already solved problem technically, it's just that the community doesn't want to reach consensus on using and investing in improving Dub. Today, in all of our projects at work, the version of TypeScript that we used is managed by SemVer. The version of all our dependencies is also managed by SemVer. If I want to use the latest language features of TypeScript, I'm not afraid to upgrade it, as each of my dependencies can use whatever TS version they want (again following SemVer), and so on.

On Thursday, 25 June 2020 at 12:13:21 UTC, Paolo Invernizzi wrote:
>
> I know that we are quite few on this side, but IMHO a well shaped standard library has advantages over a sparse ecosystem of independent library ...

Who says that this is a binary choice? And also, the way you fraise it it's obvious that say Flutter is better choice than React as everything is very tightly integrated and you could build your whole app without reaching for third-party code. Also, it's obvious that in terms of trust, it's better to rely on the diligence of the upsteam development team, than random strangers from the internet.

But what if 90% of the code my team needs has no place in Phobos? We already have to rely heavily on third-party packages. Yes there are a lot of risks and we try to be careful, but we have no other choice. We could spend years reiventing the wheel (high-quality code that already exists in other languages), or we could focus on delivering products to our users based on the already existing ecosystem in other languages. Well, a choice would exist if we had an unlimited budget, but given that we're not as lucky, we can only use other languages for our projects currently.

Technically, phobos is actually already on Dub (https://code.dlang.org/packages/phobos), so in the future nothing could prevent you to continue using it, but you would just get it from there and not from the compiler distribution archive. It's the same code, made by the same people, just with a different distribution model.

The way I see things is that we as a community need to focus on a vetted, well shaped collection of libraries.
We need to have process where third-party libraries are able to gain broader support and join dlang-community. Also we need dlang-community to have a healthy number of people who actually maintain the code.
We need the leadership to realize that investing just in dmd, druntime and phobos is necessary, but very insufficient.
W&A need to stop pretending that Dub doesn't exist. I feel that unless we embrace using code.dlang.org as method of distribution of everything that's currently part of the compiler archive, Dub's limitations will never be addressed and from that the broader community will suffer.

> Anyway, if we can, we try to stick with Phobos, as long as we don't have a particular problem to solve that needs an external library: recent example, Martin std.io or SumType instead of std.variant ...

See, some of the problems in Phobos already have a good solution. The problem that we need to address is the trust.
The more high-quality and well-maintained libraries there are, the less cautious one would need to be before reaching to code.dlang.org.