View mode: basic / threaded / horizontal-split · Log in · Help
February 16, 2013
Re: The DUB package manager
On Sat, Feb 16, 2013 at 08:15:13PM +0100, Jacob Carlborg wrote:
> On 2013-02-16 19:39, Sönke Ludwig wrote:
> 
> >Not at all, the idea is to have a number of "build script" generators
> >so everyone can choose whatever fits best, otherwise a generic
> >rdmd/dmd based build works out of the box with no need to install an
> >additional tool. Invoking a generic external tool is easy enough to
> >add and already planned, so this should not be a limiting factor in
> >any way.
> 
> Ok, I see. But it feels wrong to me that it should generate a build
> script. I think the package should already contain a build script.
[...]

I think Sönke's idea is actually very good. I know we all have our own
preferences for build systems (I know I do -- for example, I abhor
anything to do with makefiles), but having a standardized way to specify
a build has many advantages. Imagine the user-unfriendliness of
downloading a bunch of packages from the D package manager, only to
discover that one requires make, another requires cmake, another
requires SCons, another requires Ant, pretty soon, what should be just a
simple automatic download turns into a nightmare of installing 20
different build systems just so you can use a bunch of packages from the
standard D package manager.

Having a standardized way of generating build scripts is good, because
then the D package manager can target the *end user*'s preferred build
system, rather than whatever build system the package writers chose. The
package writers can just specify how to build the stuff, then let the D
packager generate makefiles for one user, Ant files for another user,
etc.. This makes it much more friendly to use, and therefore, more
likely people will actually use it.


T

-- 
Guns don't kill people. Bullets do.
February 16, 2013
Re: The DUB package manager
Am Sat, 16 Feb 2013 11:37:00 -0800
schrieb "H. S. Teoh" <hsteoh@quickfur.ath.cx>:

> 
> Having a standardized way of generating build scripts is good, because
> then the D package manager can target the *end user*'s preferred build
> system, rather than whatever build system the package writers chose.
> The package writers can just specify how to build the stuff, then let
> the D packager generate makefiles for one user, Ant files for another
> user, etc.. This makes it much more friendly to use, and therefore,
> more likely people will actually use it.
> 

Having a common standard build tool is always good. But some kind of
projects require custom build scripts (calling swig, generating
interface files for ipc stuff, executing the C compiler to check if a
function is available to disable / enable additional features, calling
pkgconfig for some reason, compiling additional c/c++ files, assembling
additional files, ...).


I think splitting DUB into a package manger and build tool would be a
good idea. Ship them as one package, so that everyone using the package
manager also has the build tool installed. And integrate them as much
as possible, the default setup can still work exactly the same as if
they were integrated.

The benefit of splitting them in this way: You're forced to provide
interfaces for the build tool to communicate with the package manager
and every other build tool can use those interfaces as well. This way
there are no second class build systems.

As an example:

package.json
{
	"name": "myproject",
	"description": "A little web service of mine.",
	"authors": ["Peter Parker"],
	"homepage": "http://myproject.com",
	"license": "GPL v2",
	"dependencies": {
		"vibe-d": ">=0.7.11"
	},
       "build": "DUB"
}

build.json
{
	"configurations": {
		"metro-app": {
			"versions": ["MetroApp"],
			"libs": ["d3d11"]
		},
		"desktop-app": {
			"versions": ["DesktopApp"],
			"libs": ["d3d9"]
		}
	}
}

doing a "dub-pkg install myproject" should fetch the sources, then call
"dub-build build.json". dub-build will have to ask the package manager
for some information: "dub-pkg package.json --query dependencies",
"dub-pkg package.json --query --package=vibe.d --link-path". Or it
might require some additional actions: "dup-pkg --install-dependency
d3d9"
February 16, 2013
Re: The DUB package manager
Am 16.02.2013 21:02, schrieb Johannes Pfau:
> This way there are no second class build systems.

I actually think it is indeed _good_ to have a first class build system
for exactly the reason that H. S. Teoh gave. If other build systems
really are on the same level as the standard one, it poses the risk of
fragmentation among different packages and users would possibly have to
install a number of different build tools to build all dependencies.

That said, providing an interface to the dependency information to
better support other build tools definitely is a good thing no matter
which way is taken.

My idea for the things you mentioned (swig, c, etc.) was to have a set
of hooks that can be used to run external tools (invoked before
build/project file generation, before build or after build). That
together with your proposed interface should provide all the necessary
flexibility while putting an emphasis on a standard way to describe the
build process.
February 16, 2013
Re: The DUB package manager
Am 16.02.2013, 22:21 Uhr, schrieb Sönke Ludwig:
> I actually think it is indeed _good_ to have a first class build system
> for exactly the reason that H. S. Teoh gave. If other build systems
> really are on the same level as the standard one, it poses the risk of
> fragmentation among different packages and users would possibly have to
> install a number of different build tools to build all dependencies.

++1

Another issue: I understand why you are using json but it is
not the best suited format IMHO. D put some restriction on
module names, thus the format can be simplified. Compare:

{
  "name": "myproject",

  "description": "A little web service of mine.",

  "authors": ["Peter Parker"],

  "homepage": "http://myproject.com",

  "license": "GPL v2",

  "dependencies": {
		"vibe-d": ">=0.7.11"
   }
}


 name: myproject;

 description: A little web service of mine.;

 authors: [Peter Parker, Fritz Walter];

 homepage: "http://myproject.com";

 license": GPL v2;

 dependencies: [
    vibe: >= 0.7.11  # a comment;
 ];

Using ';' as end-char and omit all these double quotes
makes it much cleaner and less error-prone  for the user
as well as automatic generation. Double quotes are needed
only for strings with none alphanumeric characters.

Adding '#' or '//' for comments would also be a good idea.

I also think that something like json objects, which are
key-value pairs anyway, are dispensable. Hence everything
can be key-value pairs.

Exporting to other formats will probably needed anyway.

Peter
February 16, 2013
Re: The DUB package manager
On Saturday, 16 February 2013 at 17:10:33 UTC, Sönke Ludwig wrote:
>  - Full IDE support:
>
>    Rather than focusing on performing the build by itself or 
> tying a
>    package to a particular build tool, DUB translates a general
>    build receipt to any supported project format (it can also 
> build
>    by itself). Right now VisualD and MonoD are supported as 
> targets and
>    rdmd is used for simple command line builds. Especially the 
> IDE
>    support is really important to not simply lock out people 
> who prefer
>    them.
>

Cool, I guess creating a built-in import/export functionality for 
DUB scripts in Mono-D should be amazing. Will note this down on 
my todo list.
February 16, 2013
Re: The DUB package manager
On Sat, 16 Feb 2013 22:21:55 +0100
Sönke Ludwig <sludwig@outerproduct.org> wrote:
> 
> My idea for the things you mentioned (swig, c, etc.) was to have a set
> of hooks that can be used to run external tools (invoked before
> build/project file generation, before build or after build). That
> together with your proposed interface should provide all the necessary
> flexibility while putting an emphasis on a standard way to describe
> the build process.

I like a lot of what you've said, but my concern about this part is,
does it support things like:

- Dependencies between custom build steps (so that custom build steps
aren't needlessly re-run when they're not out-of-date, and can
possibly be parallelized)

- Multiple custom build targets

- Multiple custom build configurations

I think those are essential for a real general-purpose build tool.
February 16, 2013
Re: The DUB package manager
On 2/16/13, Sönke Ludwig <sludwig@outerproduct.org> wrote:
> Some may already have noticed it as it's mentioned already on the vibe.d
> website and is currently hosted on the same domain as the old VPM registry:
>
> http://registry.vibed.org/

So can we start using this already or is this just a preview? It says
here[1] it's precompiled for win32, but where do we get the executable
from? I thought a --recursive clone would fetch all dependencies, but
app.d won't compile without vibe.

[1] https://github.com/rejectedsoftware/dub

Anyway it looks interesting!
February 16, 2013
Re: The DUB package manager
On 2/16/13, Andrej Mitrovic <andrej.mitrovich@gmail.com> wrote:
> So can we start using this already or is this just a preview? It says
> here[1] it's precompiled for win32, but where do we get the executable
> from? I thought a --recursive clone would fetch all dependencies, but
> app.d won't compile without vibe.
>
> [1] https://github.com/rejectedsoftware/dub
>
> Anyway it looks interesting!
>

Ah I didn't spot the download link http://registry.vibed.org/download

I guess this could be made more visible by adding a link to the
download page from the github repository, and maybe putting the { *
Using DUB * Download * Publishing packages * Helping developme }
section at the top instead of the bottom.

Well, the latter is a stylistic issue so I don't mind it being this
way. On typical websites I find important stuff at the top, and
copyright and privacy policies at the bottom.
February 16, 2013
Re: The DUB package manager
On 2/16/13, Andrej Mitrovic <andrej.mitrovich@gmail.com> wrote:
> Ah I didn't spot the download link http://registry.vibed.org/download

I get an error running dub.exe:

---------------------------
dub.exe - Unable To Locate Component
---------------------------
This application has failed to start because libevent.dll was not
found. Re-installing the application may fix this problem.
---------------------------

You might want to list all the dependencies needed for dub or
distribute them in a zip.
February 16, 2013
Re: The DUB package manager
On Saturday, 16 February 2013 at 19:35:47 UTC, Russel Winder 
wrote:
> On Sat, 2013-02-16 at 20:15 +0100, Jacob Carlborg wrote:
> […]
>> I'm thinking that there should be a build tool that drives 
>> everything. The build script contains package dependencies. 
>> The build tool will ask the package manager to get 
>> linker/import paths and libraries for the dependencies.
> […]
>
> First a plug for people to think of SCons and Waf, not to 
> mention
> Gradle, when starting to rattle of build tools.
>
> The SBT folks are using Scala in quite interesting ways to 
> enable Scala
> to define project specifications with all the non-inferable
> dependencies.
>
> The Gradle folks have already done almost everything the SBT 
> folks are
> doing, but using Groovy rather than Scala.
>
> The Go folk have done something interesting in that they have 
> merged the
> whole concept of configuration and build by doing everything 
> over DVCS.
> You put your sources in Git, Mercurial, Bazaar (they really 
> should
> include Fossil as well but…) and these can be Got and the 
> modules
> created within the standard structure local hierarchy.  The 
> have a
> single command that performs all activity. D has rdmd but 
> compared to
> Go's go command it cannot do a lot.
>
> I would suggest now is the time to think outside the box, 
> analogous to
> the ways the Gradle and SBT folk have on the JVM and the way 
> the Go folk
> have for native builds of statically linked code.
>
> Instead of thinking in terms of compile, link, modules, 
> dependencies,
> what is the workflow that makes D the compelling language for 
> building
> Fortran/C/C++/D systems. This is suggesting that the milieu to 
> attack is
> the one currently being won by Python in the computationally 
> intensive
> areas of bioinformatics and scientific computing.
>
> Can D be the project specification language using convention 
> over
> configuration. Project directories are in a standard form (with
> exceptions describable) with dependencies either inferred by 
> scanning
> the project sources or specified in a trivial way in the project
> specification file.
>
> Thus I suggest that it is not that the build tool is embedded 
> in the
> package manager but that package and dependency management is 
> part of
> the build system.

I'm having good success using D itself as the build tool 
language, and I'm at the point now where I'm getting much better 
results than what I was getting out of using external build 
tools, so for me there's no looking back.

I run rdmd on a .d "script" that I wrote that's setup to do 
exactly what I want.

What is missing from D is a good library of functions that are 
generally useful for writing build scripts. Phobos already 
supplies a great deal of what is needed that can be used to 
construct what is missing.

The benefit of using D is that I do not have to install and learn 
a secondary language or conform to a specific build format, or 
follow any weird restrictions. What I want to do is build my D 
code, not learn something new and perform acrobatics to get the 
job done. I can even use the same D process to build C/C++ code 
if I wanted to expand on it.

What I'm saying here is that I see no reason to use a language 
other than D itself as the build tool. What D can use is an 
addition to Phobos that supplies the necessary generalized 
functions that all build tools should supply, and I don't think 
there's all that much that's missing.

For a package manager, some standards may be required, but it too 
can be done completely with D.

Why use json (which is a subset of javascript), or ruby, or 
python, etc? Is there something fundamentally wrong with D that 
makes it unsuitable for this role?

--rt
1 2 3 4 5 6
Top | Discussion index | About this forum | D home