Hi D

So one of the projects I've been working on is moving closer to production. Currently, the pull/build/test/install cycle is handled by git, dub and GNU make.

Currently I let dub fetch dependencies off the Internet, but for mission reliability I would like to be able to handle the process without Internet access. Dub looks like it supports local repositories, but before just start "Doing something" are there any practices the community would recommend for D supply-chain management?

Thanks for any links and tips,

On Saturday, 4 January 2025 at 20:33:55 UTC, Chris Piker wrote:

I don't think there is D specific rules on dependency management, as most people use it for small scale personal/hobby projects only.

If your project can't take the risk of losing online dependencies you might just want to put and commit them under your project's version control system - this is sometimes done in Go, people there justify it that unlike JS and some other languages with tons of generated stuff Go packages are relatively small so it is just a natural choice to place them next to your code.

And in Git for example submodules (other git repos linked to your repo) are also a thing, however by default it won't clone them without explicit recursive flag, and even you have git experience it is somewhat confusing to upgrade them (at least for me).

As for the dub itself, it has very scarce info on that, it has local overrides and stuff, but I'd say this is mostly for quick fixing the build issues, not a production solution.
So just having dub.selections.json in your repository pointing to a local (committed dependencies in that same repo) packages is viable option, even if something goes wrong you can always change it in dev environment to fix problems and commit back.
Unless you have license issues with dependencies this is probably the most secure one.

On Sunday, 5 January 2025 at 06:21:15 UTC, evilrat wrote:

That's nice and simple, so maybe a good idea, but it does make it more difficult to get upstream changes when desired. For example one of my dependencies dpq2 actually reduced it's dependency count over the last year, so and I definitely wanted those changes.

This is the route I'll probably take, i.e. make git submodules for my dependencies, (there's only 7 of them). Even though submodules can be a pain, we use them a lot around here so I'm committed to dealing with their idiosyncrasies.

This is very useful advice, thanks! I didn't know about dub.selections.json, especially since it's manual page is blank. That could work well with sub modules.

...now to find out how to use dub.selections.json.

Here's what I ended up doing in case it's either useful for others, or a really bad idea and serves as an anti-example.

0. For context, the working directory has roughly this setup (after git submodule calls):

git-root
 |-- main_project
 |-- deps/
 |    |-- vibe-serialization
 |    |-- ... more here
 |
 |-- makefile

1. Add dependencies as git submodules, for example:

   git submodule add https://github.com/vibe-d/vibe-serialization deps/vibe-serialization
   # more here

2. In the top level build file (makefile in my case) the following happens:

   dub add-local vibe-serialization 1.0.7
   # More locals added here

   cd main_project && dub --skip-registry=all build

   dub remove-local vibe-serialization
   # more locals removed here

3. On a regular basis:

   rm -r $HOME/.dub

You never know what could be hiding in there.

A person can just cd main_project && dub build and standard things will happen, i.e. dependencies will be downloaded from the internet. However in a production setting make is run from the top level and only the dependencies specified in the sub modules are used.

This should work unless two builds are happening at the same time in the same account since add-local and remove-local are user-wide and affect an entire user's home directory at a time. Not too bad if the user is a real person, terrible if it's the account used by a build host.

It works, but I'm sure there's a better way.

On Monday, 6 January 2025 at 03:04:40 UTC, Chris Piker wrote:

git-root
 |-- main_project
 |-- deps/
 |    |-- vibe-serialization
 |    |-- ... more here
 |
 |-- makefile

   git submodule add https://github.com/vibe-d/vibe-serialization deps/vibe-serialization
   # more here

   dub add-local vibe-serialization 1.0.7
   # More locals added here

   cd main_project && dub --skip-registry=all build

   dub remove-local vibe-serialization
   # more locals removed here

   rm -r $HOME/.dub

There is nothing wrong with your example, having two ways to build artifacts maybe confusing but this is just fine, except maybe that you should clearly state your intentions and name scripts/makefile accordingly (e.g. build_prod.sh/build_dev.sh or something, meaning that is it only going to work in specific environment because this is a part of specific procedure)

There is a problem with dub add-local though, it is only useful for developer on that same machine while tinkering with problematic packages. But this is an extra step that is easily can be forgotten leading to confusion.
I think it can be avoided entirely by simply setting explicit paths to dependencies in dub.selections.json (not suitable for library projects as you can't freeze version for your users), not sure if it will actually work with relative paths but I guess it should.

dub.selection.json:

{
	"fileVersion": 1,
	"versions": {
		"godot-dlang": {"path":"C:/godot/bin/godot-dlang"},
		"intel-intrinsics": "1.11.18",
		"pyd": "~master"
	}
}