On Sunday, 14 April 2019 at 11:21:09 UTC, Andre Pany wrote:
>
> Dub has a working eco system and a lot of developers put effort into it. It will take years to have another solution with the same functionality like Dub and please do not forget we still have to maintain Dub until the new solution can replace Dub.
> Could you please write the details why a correction of Dub isn't possible and a complete rewrite is the only solution?

I think on the ecosystem side, dub is the #1 thing that ever came to D. I remember what life was before dub. First of all, there was no central repository, so it was hard to find a package you're interested in to begin with. Secondly, every package required additional steps to download dependencies, usually through a bash script. Then you had to roll a dice which build tool it expects, will it be bud, dsss, maybe makefiles, maybe something else? Oh, and if you were on Windows, you were screwed 99% of time, because the author loves bash and never tested it on non-POSIX platforms.

Right now the situation is soo much better. I can download a random project from the internet, type dub build and it just works. It will download and build necessary dependencies and then build the project, usually without any manual steps involved.

Personally I haven't really encountered these pain points with D, so it's hard for me to say if it's a cause I'd donate to, but I think anything related to improving Dub is crucial for D ecosystem to flourish.

On Sun, 2019-04-14 at 10:53 +0000, Seb via Digitalmars-d wrote: […]
> 
> I had to use Cargo the other day and compiling 80 (!)
> dependencies was a
> miracle (parallel fetching and builds).
[…]

And herein lies the problem. Cargo is properly supported with actual resource that isn't the occasional volunteer effort. Also Cargo has a significantly better way of working with dependencies and build than Dub. Perhaps the D community as a whole should stop working on DMD/LDC/GDC etc. and start working on Dub?

Much as I prefer D over Rust for the GTK+ and GStreamer stuff I am doing, there is always the Rustward drive because of Cargo and because of the Rust plugin in CLion (*).

It is clear that Dub was and is a good idea per se, but that lots of decisions made along the way have meant it not as good for D as Cargo is for Rust. :-(


(*) I know I had promised to help out on the CLion D plugin and have failed. The Big Problem™ is that it is easier to write code now using Rust and CLion with the Rust plugin than it is to get the CLion D plugin to a state where it is usable in production. I chat with the JetBrains/CLion folks from time to time, especially at ACCU conferences. Clearly they cannot take over management of the D plugin for IntelliJ IDEA and CLion as there is no perceived traction of D compared to Rust, but neither can they justify providing any resource to support the D plugin for the exact same reason. Rust has perceived traction in the market, D has none. :-(

-- 
Russel.
===========================================
Dr Russel Winder      t: +44 20 7585 2200
41 Buckmaster Road    m: +44 7770 465 077
London SW11 1EN, UK   w: www.russel.org.uk

On Sun, 2019-04-14 at 13:40 +0000, Adam D. Ruppe via Digitalmars-d wrote:
> 
[…]
> No and no. Dub delivers negative value for my use-case; it is a cost to me to maintain these definition files for other people (and it is a support pain too, which people dropping in having dub problems that don't exist in the underlying compiler), even though I don't use it.

So that is fine, you do not use it. But for those who have experience Cargo with Rust, Go, Conan with C++, etc. D needs something like Dub.

> I don't use it because it offers nothing of value to me; it solves a problem I don't even have (and does so in a way that is mostly incompatible with my existing code). It is all negative with nothing in the positive column to balance it out.

See above. :-)

> 
> I'd say we split dub up into three concerns:
> 
> 1) code.dlang.org. I do see some value in this, and think it is salvageable in its current form.
> 
> 2) dub, the package manager. Maybe useful, though I don't personally believe in dependencies, I can see some value, though I'd want to change it so it has more compatibility with other workflows (like mine) and be sure to limit its scope to developer use-cases; end users should never know.
> 
> 3) dub, the build tool. I'd rather have it either do absolutely nothing here, or just delegate to something else.

I disagree. The lessons, particularly from Cargo/Rust, and Go is that "small language, small standard library, large pool of trivially accessible dependencies" is the way things are going just now. Having a dependency and build manager is what Maven and (far better) Gradle have been doing for yonks in the JVM-based milieu, this way of working is finally penetrating into the native code arena.

Make, CMake, SCons, Meson, etc. remain tied to the "I have all the source I need locally, everything else is provided via precompiled static or dynamic libraries that are present locally". If this works for a given use case fine. But increasingly the JVM world, Cargo/Rust, Go are working with a distributed source code approach. An approach that can use a central publishing place, local file locations, or Git/Mercurial/Breezy repositories.

Having worked with Gradle, Cargo, and Go, returning to the approach of Make, CMake, SCons, and Meson really has no appeal to me at all.

A consequence of this for me is that for D almost all of Phobos should not be in Phobos but should be in the Dub repository as packages, and Dub should work with Git/Mercurial/Breezy repositories as easily as it does with the Dub repostory and local filestores.

-- 
Russel.
===========================================
Dr Russel Winder      t: +44 20 7585 2200
41 Buckmaster Road    m: +44 7770 465 077
London SW11 1EN, UK   w: www.russel.org.uk

On Sunday, 14 April 2019 at 11:38:23 UTC, Guillaume Piolat wrote:
> On Sunday, 14 April 2019 at 11:21:09 UTC, Andre Pany wrote:
>>
>> Dub has a working eco system and a lot of developers put effort into it. It will take years to have another solution with the same functionality like Dub and please do not forget we still have to maintain Dub until the new solution can replace Dub.
>> Could you please write the details why a correction of Dub isn't possible and a complete rewrite is the only solution?
>>
>
> Don't mistake me, DUB is a good software that is very useful and that's a feat for a build system - something very easy to hate.
>
> A lot of things are possible incrementally, but I believe in the DUB design some things will not be possible.
>
> - dub.selections.json is supposed to encode dependency resolution, but fundamentally it does that for a particular optional dependencies selections set and configuration.
>
> - I personally think it has way to many features, and in such situations it's way harder to find those to remove because agreement is difficult
>
> - work is piling on for DUB because it is designed in a way that say yes to everything, features have piled on with no regards for who would maintain them
>
> Withing DUB there is a much simpler subset to be discovered I believe. For the record I was an early contributor, I did what I could.

+1 to all your points, including your previous post on this thread.

FWIW, I'm currently writing code to use dub as a library and bypass most of it.

Rewriting from scratch isn't feasible - it has to be bug compatible. But reusing the parts that get information from packages and ignoring everything else...

I don't have much experience contributing to dub but I have contributed to dub-registry in the past and must I say, it wasn't a pleasant experience for me. Everything I'm writing in this post is from a year ago, I am not sure what's the state of the ecosystem now.

Contributing starts with setting up a local development environment. For dub-registry it was awful to say the least. I had to fix my local vibe-d source and some setup code while making sure that I don't commit it to one of my frontend-related branches.

Compilation of dub-registry on a laptop with just 4Gb RAM takes ages and hangs up the whole system. A tiny edit in any diet template caused them all to be parsed over and over again. I'd like to remind you that I was focusing on frontend and user integration therefore I had to recompile registry *many times*. Imagine how much it slowed me down.

Other thing is, of course, technology that registry used. It was ancient. I couldn't push the idea of adding a css preprocessor like SASS to sort out the CSS anarchy we had through maintainers who wanted to avoid any third-party dependencies for the sake of simpler development. Yes, 1000-line CSS file which had styles for the whole registry and had 0 useful documentation was kept. (I wonder if it is still there lol)

After all the struggles I opened a PR. Usually it took a week or two to get merged, bigger changes were hanging for an entire month. This was so frustrating. I appreciate all of the maintainers who provided their feedback and pointed out at my obvious mistakes but we could have saved so much of each other's time if dub-registry used somewhat modern technologies, had documentation and maybe some guidelines.

Later dub-registry has been split into the registry itself and dub documentation. Yes, now simple PRs to dub had to have a documentation PR to this new repository. They are slowing each other down for very little benefit.

In conclusion, I will not support dub's further development. It is a piece of software I saw so much potential in, but it failed all of my expectations. It is not up to modern world's standards. It slows down the whole community.

I totally agree with people who see the only solution to this problem in a complete rewrite of dub. My only addition is that registry has to be rewritten as well, now using modern technologies. There is no shame if it will be written in JavaScript or whatever. D is not the right tool for that particular job.

Best regards,
Anton

On Monday, 15 April 2019 at 09:34:00 UTC, Anton Fediushin wrote:
> I don't have much experience contributing to dub but I have contributed to dub-registry in the past and must I say, it wasn't a pleasant experience for me. Everything I'm writing in this post is from a year ago, I am not sure what's the state of the ecosystem now.
>
> [...]

What issues do you have specific to Dub?

You wrote: it slows down the whole community. This statement is not true. It does not slow me down, actually it is working like a charme for me now (I did a few dub pull requests).

Kind regards
Andre

On Monday, 15 April 2019 at 11:06:56 UTC, Andre Pany wrote:
> On Monday, 15 April 2019 at 09:34:00 UTC, Anton Fediushin wrote:
>> I don't have much experience contributing to dub but I have contributed to dub-registry in the past and must I say, it wasn't a pleasant experience for me. Everything I'm writing in this post is from a year ago, I am not sure what's the state of the ecosystem now.
>>
>> [...]
>
> What issues do you have specific to Dub?
>
> You wrote: it slows down the whole community. This statement is not true. It does not slow me down, actually it is working like a charme for me now (I did a few dub pull requests).
>
> Kind regards
> Andre

It is slowing down the d community because it's not what a package manager and a build system of a modern programming language should look like. For example, LDC is able to compile code for quite a few architectures, even GPUs, yet you cannot painlessly integrate that with dub.

As soon as you want to do something slightly unusual, your dub.json/dub.sdl becomes a spaghetti of preBuildCommands/postBuildCommands. In my personal opinion, package file of a high-level build system (which is what dub is trying to be) should never contain any shell commands.

And yes, dub.json/dub.sdl is another problem of dub. Having two package formats is nothing but a bad decision. Sure, sdl can contain comments that are very useful when you are trying to make sense of pre/postBuildCommands-spaghetti. This is how one bad design decision depends on another. Thing is, it's impossible to fix without breaking changes

Best regards,
Anton

On Monday, 15 April 2019 at 13:34:21 UTC, Anton Fediushin wrote:
> On Monday, 15 April 2019 at 11:06:56 UTC, Andre Pany wrote:
>> On Monday, 15 April 2019 at 09:34:00 UTC, Anton Fediushin wrote:
>>> I don't have much experience contributing to dub but I have contributed to dub-registry in the past and must I say, it wasn't a pleasant experience for me. Everything I'm writing in this post is from a year ago, I am not sure what's the state of the ecosystem now.
>>>
>>> [...]
>>
>> What issues do you have specific to Dub?
>>
>> You wrote: it slows down the whole community. This statement is not true. It does not slow me down, actually it is working like a charme for me now (I did a few dub pull requests).
>>
>> Kind regards
>> Andre
>
> It is slowing down the d community because it's not what a package manager and a build system of a modern programming language should look like. For example, LDC is able to compile code for quite a few architectures, even GPUs, yet you cannot painlessly integrate that with dub.
>
> As soon as you want to do something slightly unusual, your dub.json/dub.sdl becomes a spaghetti of preBuildCommands/postBuildCommands. In my personal opinion, package file of a high-level build system (which is what dub is trying to be) should never contain any shell commands.
>
> And yes, dub.json/dub.sdl is another problem of dub. Having two package formats is nothing but a bad decision. Sure, sdl can contain comments that are very useful when you are trying to make sense of pre/postBuildCommands-spaghetti. This is how one bad design decision depends on another. Thing is, it's impossible to fix without breaking changes
>
> Best regards,
> Anton

Integration of dub with LDC is working fine. I created a tutorial (german) here:

http://d-land.sepany.de/tutorials/einplatinenrechner/einstieg-in-die-raspberry-pi-entwicklung-mit-ldc/

But yes, integration could be better.

I use the shell commands to compile a git commit ID into the executable. It works like a charme. There are cases where shell commands are highly valuable and does not lead to spaghetti code.

Without specific examples it is hard to discuss wheter something works or not.

Kind regards
Andre

I've personally put a lot of effort into trying to fix DUB's problems in the past. Though I would be happy to be proven wrong, at this point, I'm all but convinced the problems with DUB are far too fundamental.

I know this is the sort of proposal that makes people cringe, and often for good reason, but in this case, I really do think it would be quicker, easier, and produce a better result to simply re-design it from the ground up (while making sure to leverage the existing code.dlang.org ecosystem in some say), than to try to wrangle this 600lb gorilla into something it was never designed to be.

I've long been hoping to take a stab at this myself, and I often find myself thinking through details of how it would work. I would, however, need help with the dependency-resolution algorithm (perhaps somebody could go into DUB and work to decouple its dep resolution algoritms from the rest DUB as much as possible - or at least document how to interface with it as a lib).

On Monday, 15 April 2019 at 07:27:54 UTC, Russel Winder wrote:
> I disagree. The lessons, particularly from Cargo/Rust, and Go is that "small language, small standard library, large pool of trivially accessible dependencies" is the way things are going just now.

You can still have that (and more of it, as we can get more library authors on board) if the systems are more compatible.

You can keep your `dub build` command if you must, but it should just forward to some user-defined system, which is able to get metadata like build target back out of dub.


So, the dub executable does four (actually more!), conceptually independent tasks:

1) get metadata about package
2) download package for use
3) full dependency resolution
4) build

And I want to decouple those as much as we can.