My offline D project with locally downloaded subpackage directory trees has built flawlessly with nix tools for previous versions of dub from dub-v1.23.0 through dub-v1.30.0 which is the version currently in nixpkgs.

However, a change to dub introduced in 1.31.0 results in my project failing to build using dub with nix tools.  The build attempt results in the nix not finding build directory paths as before (the build directory within nixpkgs has been affected as now it attempts to build in a write protected area (it ends up in what nix calls a "homeless shelter"!)

I am hoping that it might be easy to guess what the change that would
have such an effect is likely to be... and that it can be fixed or
reverted.
https://github.com/dlang/dub/releases/tag/v1.31.0

To give an idea of my current dub build dependencies etc. I include a
copy of my dub.json (below)

Incidentally, within nix I have flakes with nixpkgs overlays which
make it easy to play with various version changes, and modify the
build process for nixpkgs for various things D (ldc2 dub dtools and
the like).
Dub builds fine and passes most if not all of its tests in nix (in the
current nixpkgs build several of  the dub tests remain removed for
having failed in the past).
I would be happy to share these, as they may reduce the hurdle for
anyone using D who is interested in experimenting with nix.

{
  "authors": [
                               "Ralph Amissah"
  ],
  "copyright":                 "2015 - 2023 Ralph Amissah",
  "name":                      "spine",
  "version":                   "0.12.0",
  "description":               "an object-centric sisu-like document parser",
  "homepage":                  "https://sisudoc.org",
  "license":                   "AGPL-3.0+",
  "targetPath":                "./bin",
  "sourcePaths":               [ "./src/doc_reform" ],
  "stringImportPaths":         [ "./views" ],
  "buildRequirements":         [ "allowWarnings" ],
  "targetType":                "executable",
  "buildTypes": {
    "dmd": {
      "dflags":                [ "-J=./views", "-I=./src/doc_reform" ],
      "buildOptions":          [ "inline", "verbose"  ],
      "buildRequirements":     [ "allowWarnings" ]
    },
    "ldc2": {
      "dflags":                [ "-O2", "-J=./views", "-I=./src/doc_reform" ],
      "buildOptions":          [ "optimize", "verbose" ],
      "buildRequirements":     [ "allowWarnings" ]
    },
    "ldmd2": {
      "dflags":                [ "-O2", "-boundscheck=on",
"-J=./views", "-I=./src/doc_reform" ],
      "buildOptions":          [ "optimize", "verbose"  ],
      "buildRequirements":     [ "allowWarnings" ]
    }
  },
  "dependencies": {
    "spine:d2sqlite3":         "*",
    "spine:imageformats":      "*",
    "spine:dyaml":             "*"
  },
  "subPackages": [
    {
      "name":        "d2sqlite3",
      "description": "A thin wrapper around SQLite 3",
      "homepage":    "https://github.com/dlang-community/d2sqlite3",
      "authors":     [ "Nicolas Sicard", "Other contributors: see
Github repo" ],
      "copyright":   "Copyright 2011-18 Nicolas Sicard",
      "license":     "BSL-1.0",
      "sourcePaths": [ "./src/ext_depends/d2sqlite3/source" ],
      "importPaths": [ "./src/ext_depends/d2sqlite3/source" ],
      "configurations": [
        {
          "name":                "d2sqlite3",
          "targetType":          "library",
          "systemDependencies":  "SQLite version >= 3.8.7",
          "libs":                [ "sqlite3" ],
          "excludedSourceFiles": [ "source/tests.d" ]
        }
      ]
    },
    {
      "name":        "imageformats",
      "description": "Decoders for PNG, TGA, BMP, JPEG and encoders
for PNG, TGA, BMP.",
      "authors":     [ "Tero Hänninen" ],
      "license":     "BSL-1.0",
      "sourcePaths": [ "./src/ext_depends/imageformats" ],
      "importPaths": [ "./src/ext_depends/imageformats" ]
    },
    {
      "name":        "dyaml",
      "description": "YAML parser and emitter",
      "authors": [
        "Ferdinand Majerech",
        "Cameron \"Herringway\" Ross"
      ],
      "license":     "BSL-1.0",
      "homepage":    "https://github.com/dlang-community/D-YAML",
      "copyright":   "Copyright © 2011-2018, Ferdinand Majerech",
      "sourcePaths": [
        "./src/ext_depends/D-YAML/source",
        "./src/ext_depends/tinyendian/source"
      ],
      "importPaths": [
        "./src/ext_depends/D-YAML/source",
        "./src/ext_depends/tinyendian/source"
      ],
      "preGenerateCommands": [
        "rm -rf ./src/ext_depends/D-YAML/examples",
        "rm -rf ./src/ext_depends/D-YAML/testsuite"
      ]
    }
  ],
  "configurations": [
    {
      "name":                  "default",
      "targetName":            "spine"
    },
    {
      "name":                  "ldc2",
      "targetName":            "spine-ldc"
    },
    {
      "name":                  "ldmd2",
      "targetName":            "spine-ldc"
    },
    {
      "name":                  "dmd",
      "targetName":            "spine-dmd"
    }
  ]
}

The most likely change to result in this is: https://github.com/dlang/dub/pull/2542

This changed the location for the cache of builds, which is likely the problem you are facing.

You could try ``--cache=local`` when building, that should limit it to the directory of the root package you are building.

Rikki, thanks for your response. Something is changed from v1.31.0 I hope for a fix or workaround.

Unfortunately ``--cache=local`` did not solve the problem.

using ``--cache=local`` :

dub >= v1.31.0 build breaks (trying to build a subpackage, happens to
be dyaml i believe as the first sub-package it tries to build)
   > Error /homeless-shelter: Permission denied

dub <= v1.30.0 build is fine

So, more generally:

On Fri, May 26, 2023 at 11:28 AM Richard (Rikki) Andrew Cattermole via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>
> The most likely change to result in this is: https://github.com/dlang/dub/pull/2542
>
> This changed the location for the cache of builds, which is likely the problem you are facing.

- Something changed from v1.31.0 to affect building with nix (and that
may well be it).
- I hope for a fix or workaround (that allows me to continue to use
dub with nix).

- On IVY and the future direction of D, I like the sound of this, who
wouldn't it sounds like a great promise: "Our enthusiasm is high, and
we're ready to get going. I think you'll like where we're headed."
- On the other hand, I have been extremely content with what I have
been able to do with the D and nix ecosystems as they have been, so, I
am 100% with you provided I can continue to do what I have been doing.
- Otherwise, belatedly, please note my first MAJOR "Gripe and Wish";
if we find an elegant way around it, and no problem.

On Friday, 26 May 2023 at 18:53:36 UTC, Ralph Amissah wrote:
> Rikki, thanks for your response. Something is changed from v1.31.0 I hope for a fix or workaround.
>
> Unfortunately ``--cache=local`` did not solve the problem.
>
> using ``--cache=local`` :
>
> dub >= v1.31.0 build breaks (trying to build a subpackage, happens to
> be dyaml i believe as the first sub-package it tries to build)
>    > Error /homeless-shelter: Permission denied
>
> [...]

thanks for the response, this seems to be a bug. I'll investigate this in dub and ensure --cache=local works, as well as giving the option to always use cache local by default, so that users with read-only filesystems can continue to use dub, even though filesystem usage will be much higher.

On Fri, May 26, 2023 at 4:51 PM WebFreak001 via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>
> thanks for the response, this seems to be a bug. I'll investigate this in dub and ensure --cache=local works, as well as giving the option to always use cache local by default, so that users with read-only filesystems can continue to use dub, even though filesystem usage will be much higher.

Thank you. I am grateful for your work and that of others in the D
(and nix and wider FOSS) community(ies)

I look forward to being able to report that (build options are preserved and) everything is once again as it should be.

On Friday, 26 May 2023 at 20:48:41 UTC, WebFreak001 wrote:
> On Friday, 26 May 2023 at 18:53:36 UTC, Ralph Amissah wrote:
>> Rikki, thanks for your response. Something is changed from v1.31.0 I hope for a fix or workaround.
>>
>> Unfortunately ``--cache=local`` did not solve the problem.
>>
>> using ``--cache=local`` :
>>
>> dub >= v1.31.0 build breaks (trying to build a subpackage, happens to
>> be dyaml i believe as the first sub-package it tries to build)
>>    > Error /homeless-shelter: Permission denied
>>
>> [...]
>
> thanks for the response, this seems to be a bug. I'll investigate this in dub and ensure --cache=local works, as well as giving the option to always use cache local by default, so that users with read-only filesystems can continue to use dub, even though filesystem usage will be much higher.

Grateful for any updates on this problem.

I have been following and testing subsequent dub releases with Nix flakes (nix build) (on NixOS).

Dub 1.33.0 was released too soon after this report to take it into account;
Dub 1.33.1 released July 2nd still does not address the issue.

I would just like to be re-assured that the issue is not forgotten.

https://github.com/NixOS/nixpkgs/pull/226813

It would be wonderful to know that it has been sorted.
thanks,
Ralph

On Thursday, 6 July 2023 at 13:28:38 UTC, Ralph wrote:
> On Friday, 26 May 2023 at 20:48:41 UTC, WebFreak001 wrote:
>> [...]
>
> Grateful for any updates on this problem.
>
> I have been following and testing subsequent dub releases with Nix flakes (nix build) (on NixOS).
>
> Dub 1.33.0 was released too soon after this report to take it into account;
> Dub 1.33.1 released July 2nd still does not address the issue.
>
> I would just like to be re-assured that the issue is not forgotten.
>
> https://github.com/NixOS/nixpkgs/pull/226813
>
> It would be wonderful to know that it has been sorted.
> thanks,
> Ralph

ah sorry no progress on it yet, but the upstream issue is here: https://github.com/dlang/dub/issues/2654

mea culpa (I have had another look)

Yes there was a change in the behavior of dub between v1.30.0 and v1.31.0 onward which (on NixOS) resulted in a build problem with attempting to write to a non-writable /homeless-shelter directory (on NixOS, possibly using nix generally I have not checked).

However, the change does not really matter as there is a simple solution within nix which is in the buildPhase to make sure that the directory HOME exists and/or is set to a writable directory.

export HOME=$(pwd)

which I just found here:
- https://github.com/NixOS/nix/issues/670

(It is not for want of having searched for the like (and not found such) earlier though).

I would not bother further digging into this matter of what changed within dub, at least on NixOS it does not seem to matter provided you know what to do if caught by it.

On the positive side I can confirm that all versions of dub including latest dub 1.33.1, build in nix and pass almost all tests (several were removed to make dub work with earlier nixpkg builds).

will update:
On Thursday, 6 July 2023 at 15:17:49 UTC, WebFreak001 wrote:
>>
>> https://github.com/NixOS/nixpkgs/pull/226813

Thank you for your feedback,
Ralph