Greetings folks,

In my project, I have a parent package with several sub-packages, each of which builds into either a library or an executable.

I first started observing odd problems when I was running dub build, it would complain about different versions of vibe-d present, and it suggested running dub upgrade. After doing this, I noticed that most subPackages were not actually being upgraded.

The only thing I have found thus far is to manually run each subPackage one at a time, e.g. dub :proto; dub :common; ....

Is it possible to get dub upgrade to recursively work on all sub-packages?

My parent package dub.sdl file:

name "funnel"
description "An in-memory queryable database for processing extreme loads of current data."
authors "Vijay Nayar"
copyright "Copyright © 2019, Vijay Nayar"
license "proprietary"
targetType "none"
targetPath "target"
dependency "funnel:proto" version=">=0.0.0"
dependency "funnel:spout" version=">=0.0.0"
dependency "funnel:stem" version=">=0.0.0"
dependency "funnel:mouth" version=">=0.0.0"
dependency "funnel:common" version=">=0.0.0"
subPackage "./common"
subPackage "./proto"
subPackage "./mouth"
subPackage "./stem"
subPackage "./spout"

Each subPackage is structured in the same way, for example, the common subPackage:

authors "Vijay Nayar"
copyright "Copyright © 2019, Vijay Nayar"
description "Common logic between the mouth and spout components."
license "proprietary"
name "common"
targetType "library"
targetPath "target"
dependency "funnel:proto" version="*"
dependency "poodinis" version="~>8.0.3"
dependency "vibe-d" version="~>0.9.4"

I mostly followed the dub documentation in setting up my project. https://dub.pm/package-format-sdl.html

On Wednesday, 2 February 2022 at 10:14:25 UTC, Vijay Nayar wrote:

I should point out there is 1 exception to subPackages not being recursive. The dub clean command does recurse, and there is source code for it here: https://github.com/dlang/dub/blob/3abaa4d5b7c3b2c21ab75370cd5330e9ae7bbd12/source/dub/dub.d#L880

On 2/2/22 5:14 AM, Vijay Nayar wrote:

name "funnel"
description "An in-memory queryable database for processing extreme loads of current data."
authors "Vijay Nayar"
copyright "Copyright © 2019, Vijay Nayar"
license "proprietary"
targetType "none"
targetPath "target"
dependency "funnel:proto" version=">=0.0.0"
dependency "funnel:spout" version=">=0.0.0"
dependency "funnel:stem" version=">=0.0.0"
dependency "funnel:mouth" version=">=0.0.0"
dependency "funnel:common" version=">=0.0.0"
subPackage "./common"
subPackage "./proto"
subPackage "./mouth"
subPackage "./stem"
subPackage "./spout"

authors "Vijay Nayar"
copyright "Copyright © 2019, Vijay Nayar"
description "Common logic between the mouth and spout components."
license "proprietary"
name "common"
targetType "library"
targetPath "target"
dependency "funnel:proto" version="*"
dependency "poodinis" version="~>8.0.3"
dependency "vibe-d" version="~>0.9.4"

If you have them in the same repository, my recommendation is to use path dependencies instead of versions.

So, e.g.:

dependency "funnel:proto" path="./proto" // in main dub.sdl
dependency "funnel:proto" path="../proto" // in sibling package

Because otherwise, dub is going to try and fetch the appropriate version from an online repository and not use your local files.

Honestly, for packages in the same repository, I'm not sure why you would version them separately from the main package. I don't even know if that works.

-Steve

On Wednesday, 2 February 2022 at 14:07:08 UTC, Steven Schveighoffer wrote:

dependency "funnel:proto" path="./proto" // in main dub.sdl
dependency "funnel:proto" path="../proto" // in sibling package

I made this change and it did indeed work correctly, thank you for that! Truthfully, it was not entirely clear to me how dub was deciding where to go to build. I had assumed that this was being done via the subPackage lines.

The examples given in the offical documentation were also using versions, and I was following that: https://dub.pm/package-format-sdl.html#sub-packages

On 2/2/22 1:42 PM, Vijay Nayar wrote:

Like many things in dub, there are multiple ways to do things. I've done some of the other things in some projects, and I found this works the best for intra-repository dependencies.

If you are depending on a subproject from outside the repository, then I think you have to use the version of the parent repository as a specified version, since the whole repo is tagged the same.

Yeah, that example shows a repository which exists purely to combine multiple subprojects into one repo (note the targetType of none). Also note the version="*", which is basically "I don't care, whatever is there". I think you could also do a path as well, but I think even if you use the * version, you might have dub trying to fetch the dependency from the internet. I'm more comfortable with the path directive, since I know it can't possibly think that it has to go elsewhere to resolve that dependency.

Dub is kind of a hot mess in terms of the dependency resolution and ways to specify projects. I would love to see it cleaned up/reimplemented.

-Steve

On Wednesday, 2 February 2022 at 19:03:35 UTC, Steven Schveighoffer wrote:

For your larger more complex projects, what build system do you use? So far I've been having good luck with dub, but I haven't done any mixed-language projects lately either.

On 2/2/22 4:42 PM, Vijay Nayar wrote:

I only use dub for my D projects.

I'm not saying dub is completely bad. Just that the dependency resolution needs a lot of work, and probably a rewrite.

-Steve