February 16, 2013 The DUB package manager | ||||
---|---|---|---|---|
| ||||
With the recent talk about Orbit, I thought it is time to also announce the package manager that we have been working out based on the simple VPM system that has always been in vibe.d. I don't really like stepping into competition with Jacob here (*), but the approach is different enough that I think it should be put on the table. Some may already have noticed it as it's mentioned already on the vibe.d website and is currently hosted on the same domain as the old VPM registry: http://registry.vibed.org/ DUB has two important development goals: - Simplicity: Making a DUB package, as well as using one as a dependency should be as simple as possible to facilitate broad usage, also and especially among language newcomers. Procedural build scripts often scare away people, although their added complexity doesn't matter for bigger projects. I think they should be left as an option rather than the default. Turning a library/application into a DUB package can be as simple as adding a package.json file with the following content (mysql-native is automatically made available during the build in this example): { "name": "my-library", "dependencies": {"mysql-native": ">=0.0.7"} } If the project is hosted on GitHub, it can be directly registered on the registry site and is then available for anyone to use as a dependency. Alternatively, it is also possible to use a local directory as the source for a particular package (e.g. for closed source projects or when working on both the main project and the dependency at the same time). - Full IDE support: Rather than focusing on performing the build by itself or tying a package to a particular build tool, DUB translates a general build receipt to any supported project format (it can also build by itself). Right now VisualD and MonoD are supported as targets and rdmd is used for simple command line builds. Especially the IDE support is really important to not simply lock out people who prefer them. Apart from that we have tried to be as flexible as possible regarding the way people can organize their projects (although by default it assumes source code to be in "source/" and string imports in "views/", if those folders exist). There are still a number of missing features, but apart from those it is fully usable and tested on Windows, Linux, and Mac OS. GitHub repository: https://github.com/rejectedsoftware/dub https://github.com/rejectedsoftware/dub-registry Preliminary package format documentation: http://registry.vibed.org/package-format (*) Originally I looked into using Orbit as the package manager for vibe.d packages, but it just wasn't far enough at the time and had some traits that I wasn't fully comfortable with. |
February 16, 2013 Re: The DUB package manager | ||||
---|---|---|---|---|
| ||||
Posted in reply to Sönke Ludwig | On 2013-02-16 18:10, Sönke Ludwig wrote: > With the recent talk about Orbit, I thought it is time to also announce > the package manager that we have been working out based on the simple > VPM system that has always been in vibe.d. I don't really like stepping > into competition with Jacob here (*), but the approach is different > enough that I think it should be put on the table. > > Some may already have noticed it as it's mentioned already on the vibe.d > website and is currently hosted on the same domain as the old VPM registry: > > http://registry.vibed.org/ > > > DUB has two important development goals: > > - Simplicity: > > Making a DUB package, as well as using one as a dependency should be > as simple as possible to facilitate broad usage, also and especially > among language newcomers. Procedural build scripts often scare away > people, although their added complexity doesn't matter for bigger > projects. I think they should be left as an option rather than the > default. > > Turning a library/application into a DUB package can be as simple as > adding a package.json file with the following content (mysql-native > is automatically made available during the build in this example): > > { > "name": "my-library", > "dependencies": {"mysql-native": ">=0.0.7"} > } Using a full blow language can look pretty declarative as well: name = "my-library"; dependencies = ["mysql-native": ">=0.0.7"]; I don't see why something like the above would scare away people. It's even less code than the JSON above. > If the project is hosted on GitHub, it can be directly registered on > the registry site and is then available for anyone to use as a > dependency. Alternatively, it is also possible to use a local > directory as the source for a particular package (e.g. for closed > source projects or when working on both the main project and the > dependency at the same time). > > - Full IDE support: > > Rather than focusing on performing the build by itself or tying a > package to a particular build tool, DUB translates a general > build receipt to any supported project format (it can also build > by itself). Right now VisualD and MonoD are supported as targets and > rdmd is used for simple command line builds. Especially the IDE > support is really important to not simply lock out people who prefer > them. I think it looks like you're tying the user to a particular build tool. I don't think it's the business of the package manager to build software. That's the job of a build tool. The package manager should just invoke the build tool. You just need to support a few build tools, like rdmd, make and then also support shell script. Things like setting compiler flags does really not belong in a package manger. I have not looked at the source code for DUB at all. In general how is it organized. Can it be used as a library to easily build, say a GUI or directly integrate into an IDE? -- /Jacob Carlborg |
February 16, 2013 Re: The DUB package manager | ||||
---|---|---|---|---|
| ||||
Posted in reply to Sönke Ludwig | On 2013-02-16 18:10, Sönke Ludwig wrote: > Preliminary package format documentation: > http://registry.vibed.org/package-format BTW, "x64" is not a predefined version identifier. -- /Jacob Carlborg |
February 16, 2013 Re: The DUB package manager | ||||
---|---|---|---|---|
| ||||
Posted in reply to Jacob Carlborg | Am 16.02.2013 19:10, schrieb Jacob Carlborg: > > I think it looks like you're tying the user to a particular build tool. I don't think it's the business of the package manager to build software. That's the job of a build tool. The package manager should just invoke the build tool. You just need to support a few build tools, like rdmd, make and then also support shell script. Not at all, the idea is to have a number of "build script" generators so everyone can choose whatever fits best, otherwise a generic rdmd/dmd based build works out of the box with no need to install an additional tool. Invoking a generic external tool is easy enough to add and already planned, so this should not be a limiting factor in any way. > > Things like setting compiler flags does really not belong in a package manger. What makes you think so? Just because of your definition of "package manager" or for a concrete reason? There are things like setting up import paths to dependent projects that only the package manager can do, because it knows their locations (and it can be desirable for many reasons to not install dependencies into a predefined place). Bringing package management and build receipt together is convenient and natural here. You could also say that it is a meta-build tool (along the lines of cmake) with package support if you want. > > I have not looked at the source code for DUB at all. In general how is it organized. Can it be used as a library to easily build, say a GUI or directly integrate into an IDE? > There should be no problem with that. The API will need to be cleaned up a bit, but generally it's a shallow "app.d" file that does command line processing and a number of modules that do the actual work (in a package "dub"). |
February 16, 2013 Re: The DUB package manager | ||||
---|---|---|---|---|
| ||||
Posted in reply to Jacob Carlborg | Am 16.02.2013 19:11, schrieb Jacob Carlborg:
> On 2013-02-16 18:10, Sönke Ludwig wrote:
>
>> Preliminary package format documentation: http://registry.vibed.org/package-format
>
> BTW, "x64" is not a predefined version identifier.
>
Thanks, you are right, that was a typo.
|
February 16, 2013 Re: The DUB package manager | ||||
---|---|---|---|---|
| ||||
Posted in reply to Sönke Ludwig | On Saturday, 16 February 2013 at 17:10:33 UTC, Sönke Ludwig wrote:
> Apart from that we have tried to be as flexible as possible regarding
> the way people can organize their projects (although by default it
> assumes source code to be in "source/" and string imports in "views/",
> if those folders exist).
Many projects put their source code in src/ instead of source/ .
Can DUB support this ?
|
February 16, 2013 Re: The DUB package manager | ||||
---|---|---|---|---|
| ||||
Posted in reply to Jacob Carlborg | Am 16.02.2013 19:10, schrieb Jacob Carlborg:
>
> Using a full blow language can look pretty declarative as well:
>
> name = "my-library";
> dependencies = ["mysql-native": ">=0.0.7"];
>
> I don't see why something like the above would scare away people. It's even less code than the JSON above.
>
I was thinking more along the lines of make, cmake, D based build scripts in the form as proposed in the past here and others. Such procedural build files often tend to get bloated over time and hard to work with. While this is not necessarily a problem with the tools itself, it can still make people look for alternatives.
That said, if a scripting language is used (almost) purely to provide a declarative system as in your example, it doesn't have to be bad at all. The question that arguably remains is just if the added flexibility is actually useful or necessary and if such a thing "pulls its own weight" (WRT implementation and API complexity).
|
February 16, 2013 Re: The DUB package manager | ||||
---|---|---|---|---|
| ||||
Posted in reply to SomeDude | Am 16.02.2013 20:07, schrieb SomeDude:
> On Saturday, 16 February 2013 at 17:10:33 UTC, Sönke Ludwig wrote:
>> Apart from that we have tried to be as flexible as possible regarding the way people can organize their projects (although by default it assumes source code to be in "source/" and string imports in "views/", if those folders exist).
>
> Many projects put their source code in src/ instead of source/ . Can DUB support this ?
Yes, it supports a "sourcePath" field to customize that.
|
February 16, 2013 Re: The DUB package manager | ||||
---|---|---|---|---|
| ||||
Posted in reply to Sönke Ludwig | On 2013-02-16 19:39, Sönke Ludwig wrote: > Not at all, the idea is to have a number of "build script" generators so > everyone can choose whatever fits best, otherwise a generic rdmd/dmd > based build works out of the box with no need to install an additional > tool. Invoking a generic external tool is easy enough to add and already > planned, so this should not be a limiting factor in any way. Ok, I see. But it feels wrong to me that it should generate a build script. I think the package should already contain a build script. > What makes you think so? Just because of your definition of "package > manager" or for a concrete reason? There are things like setting up > import paths to dependent projects that only the package manager can do, > because it knows their locations (and it can be desirable for many > reasons to not install dependencies into a predefined place). Bringing > package management and build receipt together is convenient and natural > here. I'm thinking that there should be a build tool that drives everything. The build script contains package dependencies. The build tool will ask the package manager to get linker/import paths and libraries for the dependencies. But I guess that basically the same as how DUB is working. But the package manager drives everything instead of the build tool. > You could also say that it is a meta-build tool (along the lines of > cmake) with package support if you want. > There should be no problem with that. The API will need to be cleaned up > a bit, but generally it's a shallow "app.d" file that does command line > processing and a number of modules that do the actual work (in a package > "dub"). Good, that's how it's supposed to be organized. -- /Jacob Carlborg |
February 16, 2013 Re: The DUB package manager | ||||
---|---|---|---|---|
| ||||
Posted in reply to Jacob Carlborg Attachments:
| On Sat, 2013-02-16 at 20:15 +0100, Jacob Carlborg wrote: […] > I'm thinking that there should be a build tool that drives everything. The build script contains package dependencies. The build tool will ask the package manager to get linker/import paths and libraries for the dependencies. […] First a plug for people to think of SCons and Waf, not to mention Gradle, when starting to rattle of build tools. The SBT folks are using Scala in quite interesting ways to enable Scala to define project specifications with all the non-inferable dependencies. The Gradle folks have already done almost everything the SBT folks are doing, but using Groovy rather than Scala. The Go folk have done something interesting in that they have merged the whole concept of configuration and build by doing everything over DVCS. You put your sources in Git, Mercurial, Bazaar (they really should include Fossil as well but…) and these can be Got and the modules created within the standard structure local hierarchy. The have a single command that performs all activity. D has rdmd but compared to Go's go command it cannot do a lot. I would suggest now is the time to think outside the box, analogous to the ways the Gradle and SBT folk have on the JVM and the way the Go folk have for native builds of statically linked code. Instead of thinking in terms of compile, link, modules, dependencies, what is the workflow that makes D the compelling language for building Fortran/C/C++/D systems. This is suggesting that the milieu to attack is the one currently being won by Python in the computationally intensive areas of bioinformatics and scientific computing. Can D be the project specification language using convention over configuration. Project directories are in a standard form (with exceptions describable) with dependencies either inferred by scanning the project sources or specified in a trivial way in the project specification file. Thus I suggest that it is not that the build tool is embedded in the package manager but that package and dependency management is part of the build system. -- Russel. ============================================================================= Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder@ekiga.net 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel@winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder |
Copyright © 1999-2021 by the D Language Foundation