Jump to page: 1 2
Thread overview
How hard would it be to create a dub2deb tool?
Feb 17, 2021
deadalnix
Feb 18, 2021
Jacob Carlborg
Feb 23, 2021
James Lu
Feb 18, 2021
Dukc
Feb 18, 2021
deadalnix
Feb 18, 2021
H. S. Teoh
Feb 22, 2021
Denis Feklushkin
Feb 19, 2021
Dukc
Feb 24, 2021
FaithChurchill
Feb 20, 2021
deadalnix
Feb 18, 2021
Andre Pany
February 17, 2021
For reference, this is what I'm referring to: https://github.com/LeoIannacone/npm2deb

I don't think that would be too hard, but, at the same time, I'm not super familiar with the dub ecosystem to be honest so maybe there is something I'm overlooking.

I think such a tool would be super valuable as it would increase options to access d libs. This seems to be a low hanging fruit.
February 18, 2021
On Wednesday, 17 February 2021 at 17:14:35 UTC, deadalnix wrote:
> For reference, this is what I'm referring to: https://github.com/LeoIannacone/npm2deb
>
> I don't think that would be too hard, but, at the same time, I'm not super familiar with the dub ecosystem to be honest so maybe there is something I'm overlooking.
>
> I think such a tool would be super valuable as it would increase options to access d libs. This seems to be a low hanging fruit.

There's a tool called FPM [1], which is somewhat of a Swiss army knife when it comes to building and converting packages of various kinds. If support for Dub is added to that, then it would be possible to output Deb, RPM, tar, macOS .pkg and a couple of other packages as well.

[1] https://github.com/jordansissel/fpm

--
/Jacob Carlborg
February 18, 2021
On Wednesday, 17 February 2021 at 17:14:35 UTC, deadalnix wrote:
> For reference, this is what I'm referring to: https://github.com/LeoIannacone/npm2deb
>
> I don't think that would be too hard, but, at the same time, I'm not super familiar with the dub ecosystem to be honest so maybe there is something I'm overlooking.
>
> I think such a tool would be super valuable as it would increase options to access d libs. This seems to be a low hanging fruit.

Not .deb, but converting DUB to a general package format nonetheless:

https://github.com/lionello/dub2nix
February 18, 2021
On Wednesday, 17 February 2021 at 17:14:35 UTC, deadalnix wrote:
> For reference, this is what I'm referring to: https://github.com/LeoIannacone/npm2deb
>
> I don't think that would be too hard, but, at the same time, I'm not super familiar with the dub ecosystem to be honest so maybe there is something I'm overlooking.
>
> I think such a tool would be super valuable as it would increase options to access d libs. This seems to be a low hanging fruit.

My gut feeling is, the real problem we have to solve here is to provide something similar like python wheels [1].
Many python packages like e.g. pyarrow are wrappers for C libraries. The python wheel has pre build (shared objects) for various operation systems. Python package manager decides based on the host OS which python wheel to download.
No need to install additional system packages via apt-get and getting mad while solving build issues.

If we could add s.th. similar to the dub infrastructure (registry and client), this would remove a lot work and headaches for dub package users.

[1] https://realpython.com/python-wheels/

Kind regards
Andre

February 18, 2021
On Thursday, 18 February 2021 at 14:15:15 UTC, Dukc wrote:
> Not .deb, but converting DUB to a general package format nonetheless:
>
> https://github.com/lionello/dub2nix

That's an interesting project, and certainly contains the seeds of what I'm looking for, but some of the design decision are IMO misguided. Nothign against the author per se, these error seems to be pervasive in the npm, pip, ruby gem ecosystems and many more.

You simply can't download a bunch of crap from the internet and deploy it this way. First, this is very insecure (see https://medium.com/@alex.birsan/dependency-confusion-4a5d60fec610 for the latest iteration of the madness) but it also a reproducibility problems (the source may change from under your feets) and availability (someone pulling leftpad can bring down your whole deployment capability).

This is why you want to be able to package things and deploy them as deb/rpm/dmg/whatever
February 18, 2021
On Thu, Feb 18, 2021 at 07:31:10PM +0000, deadalnix via Digitalmars-d wrote: [...]
> Nothign against the author per se, these error seems to be pervasive in the npm, pip, ruby gem ecosystems and many more.
> 
> You simply can't download a bunch of crap from the internet and deploy it this way. First, this is very insecure (see https://medium.com/@alex.birsan/dependency-confusion-4a5d60fec610 for the latest iteration of the madness) but it also a reproducibility problems (the source may change from under your feets) and availability (someone pulling leftpad can bring down your whole deployment capability).
> 
> This is why you want to be able to package things and deploy them as deb/rpm/dmg/whatever

+1.  I have always been skeptical of having my ability to build/deploy depend on some random 3rd party provider somewhere out there in the wild internet whose availability/continued existence is not under my control.

But it seems nobody wants to acknowledge that the king has no clothes.

Maybe the latest dependency confusion madness will finally pull away the wool.  But I'm not holding my breath.


T

-- 
Жил-был король когда-то, при нём блоха жила.
February 19, 2021
On Thursday, 18 February 2021 at 19:31:10 UTC, deadalnix wrote:
> You simply can't download a bunch of crap from the internet and deploy it this way. First, this is very insecure (see https://medium.com/@alex.birsan/dependency-confusion-4a5d60fec610 for the latest iteration of the madness) but it also a reproducibility problems (the source may change from under your feets) and availability (someone pulling leftpad can bring down your whole deployment capability).
>
> This is why you want to be able to package things and deploy them as deb/rpm/dmg/whatever

Good news - none of these are problems for dub2nix. The final Nix derivation (Nix install script) won't use dub2nix program directly, it uses `dub.selections.nix` file the package maintainer has pregenerated with the tool. `dub selections.nix` links directly to github projects, and to specific versions of them - newer version of the DUB package won't be used unless the maintainer regenerates `dub.selections.nix`. Nix forces this design - Internet downloads are verified with a sha256 provided in the dub derivation precisely because of the issues you mentioned.

As for the availibility issue, Nix caches Internet downloads done by the derivations, and if you get a package published at Nixpkgs repository, it'll be cached in their servers too.
February 19, 2021
On Friday, 19 February 2021 at 08:07:29 UTC, Dukc wrote:
> On Thursday, 18 February 2021 at 19:31:10 UTC, deadalnix wrote:
>> You simply can't download a bunch of crap from the internet and deploy it this way. First, this is very insecure (see https://medium.com/@alex.birsan/dependency-confusion-4a5d60fec610 for the latest iteration of the madness) but it also a reproducibility problems (the source may change from under your feets) and availability (someone pulling leftpad can bring down your whole deployment capability).
>>
>> This is why you want to be able to package things and deploy them as deb/rpm/dmg/whatever
>
> Good news - none of these are problems for dub2nix. The final Nix derivation (Nix install script) won't use dub2nix program directly, it uses `dub.selections.nix` file the package maintainer has pregenerated with the tool. `dub selections.nix` links directly to github projects, and to specific versions of them - newer version of the DUB package won't be used unless the maintainer regenerates `dub.selections.nix`. Nix forces this design - Internet downloads are verified with a sha256 provided in the dub derivation precisely because of the issues you mentioned.
>
> As for the availibility issue, Nix caches Internet downloads done by the derivations, and if you get a package published at Nixpkgs repository, it'll be cached in their servers too.

@deadalnix Required reading: https://edolstra.github.io/pubs/phd-thesis.pdf
:P
February 19, 2021
On Friday, 19 February 2021 at 08:43:21 UTC, Petar Kirov [ZombineDev] wrote:
>
> @deadalnix Required reading: https://edolstra.github.io/pubs/phd-thesis.pdf
> :P

Here's a more approachable and up to date introduction to Nix:
https://www.youtube.com/watch?v=D5Gq2wkRXpU
February 20, 2021
On Friday, 19 February 2021 at 08:07:29 UTC, Dukc wrote:
> Good news - none of these are problems for dub2nix. The final Nix derivation (Nix install script) won't use dub2nix program directly, it uses `dub.selections.nix` file the package maintainer has pregenerated with the tool. `dub selections.nix` links directly to github projects, and to specific versions of them - newer version of the DUB package won't be used unless the maintainer regenerates `dub.selections.nix`. Nix forces this design - Internet downloads are verified with a sha256 provided in the dub derivation precisely because of the issues you mentioned.
>
> As for the availibility issue, Nix caches Internet downloads done by the derivations, and if you get a package published at Nixpkgs repository, it'll be cached in their servers too.

I'm kind of dumb because I was actually aware of NixOS, and didn't made the link at all.

Still not quite what I need, but indeed much better than what I assumed in my initial assessment.
« First   ‹ Prev
1 2