February 16, 2013
On Sat, Feb 16, 2013 at 08:15:13PM +0100, Jacob Carlborg wrote:
> On 2013-02-16 19:39, Sönke Ludwig wrote:
> 
> >Not at all, the idea is to have a number of "build script" generators so everyone can choose whatever fits best, otherwise a generic rdmd/dmd based build works out of the box with no need to install an additional tool. Invoking a generic external tool is easy enough to add and already planned, so this should not be a limiting factor in any way.
> 
> Ok, I see. But it feels wrong to me that it should generate a build script. I think the package should already contain a build script.
[...]

I think Sönke's idea is actually very good. I know we all have our own preferences for build systems (I know I do -- for example, I abhor anything to do with makefiles), but having a standardized way to specify a build has many advantages. Imagine the user-unfriendliness of downloading a bunch of packages from the D package manager, only to discover that one requires make, another requires cmake, another requires SCons, another requires Ant, pretty soon, what should be just a simple automatic download turns into a nightmare of installing 20 different build systems just so you can use a bunch of packages from the standard D package manager.

Having a standardized way of generating build scripts is good, because then the D package manager can target the *end user*'s preferred build system, rather than whatever build system the package writers chose. The package writers can just specify how to build the stuff, then let the D packager generate makefiles for one user, Ant files for another user, etc.. This makes it much more friendly to use, and therefore, more likely people will actually use it.


T

-- 
Guns don't kill people. Bullets do.
February 16, 2013
Am Sat, 16 Feb 2013 11:37:00 -0800
schrieb "H. S. Teoh" <hsteoh@quickfur.ath.cx>:

> 
> Having a standardized way of generating build scripts is good, because then the D package manager can target the *end user*'s preferred build system, rather than whatever build system the package writers chose. The package writers can just specify how to build the stuff, then let the D packager generate makefiles for one user, Ant files for another user, etc.. This makes it much more friendly to use, and therefore, more likely people will actually use it.
> 

Having a common standard build tool is always good. But some kind of projects require custom build scripts (calling swig, generating interface files for ipc stuff, executing the C compiler to check if a function is available to disable / enable additional features, calling pkgconfig for some reason, compiling additional c/c++ files, assembling additional files, ...).


I think splitting DUB into a package manger and build tool would be a good idea. Ship them as one package, so that everyone using the package manager also has the build tool installed. And integrate them as much as possible, the default setup can still work exactly the same as if they were integrated.

The benefit of splitting them in this way: You're forced to provide interfaces for the build tool to communicate with the package manager and every other build tool can use those interfaces as well. This way there are no second class build systems.

As an example:

package.json
{
	"name": "myproject",
	"description": "A little web service of mine.",
	"authors": ["Peter Parker"],
	"homepage": "http://myproject.com",
	"license": "GPL v2",
	"dependencies": {
		"vibe-d": ">=0.7.11"
	},
        "build": "DUB"
}

build.json
{
	"configurations": {
		"metro-app": {
			"versions": ["MetroApp"],
			"libs": ["d3d11"]
		},
		"desktop-app": {
			"versions": ["DesktopApp"],
			"libs": ["d3d9"]
		}
	}
}

doing a "dub-pkg install myproject" should fetch the sources, then call "dub-build build.json". dub-build will have to ask the package manager for some information: "dub-pkg package.json --query dependencies", "dub-pkg package.json --query --package=vibe.d --link-path". Or it might require some additional actions: "dup-pkg --install-dependency d3d9"
February 16, 2013
Am 16.02.2013 21:02, schrieb Johannes Pfau:
> This way there are no second class build systems.

I actually think it is indeed _good_ to have a first class build system for exactly the reason that H. S. Teoh gave. If other build systems really are on the same level as the standard one, it poses the risk of fragmentation among different packages and users would possibly have to install a number of different build tools to build all dependencies.

That said, providing an interface to the dependency information to better support other build tools definitely is a good thing no matter which way is taken.

My idea for the things you mentioned (swig, c, etc.) was to have a set of hooks that can be used to run external tools (invoked before build/project file generation, before build or after build). That together with your proposed interface should provide all the necessary flexibility while putting an emphasis on a standard way to describe the build process.
February 16, 2013
Am 16.02.2013, 22:21 Uhr, schrieb Sönke Ludwig:
> I actually think it is indeed _good_ to have a first class build system
> for exactly the reason that H. S. Teoh gave. If other build systems
> really are on the same level as the standard one, it poses the risk of
> fragmentation among different packages and users would possibly have to
> install a number of different build tools to build all dependencies.

++1

Another issue: I understand why you are using json but it is
not the best suited format IMHO. D put some restriction on
module names, thus the format can be simplified. Compare:

{
  "name": "myproject",

  "description": "A little web service of mine.",

  "authors": ["Peter Parker"],

  "homepage": "http://myproject.com",

  "license": "GPL v2",

  "dependencies": {
		"vibe-d": ">=0.7.11"
   }
}


 name: myproject;

 description: A little web service of mine.;

 authors: [Peter Parker, Fritz Walter];

 homepage: "http://myproject.com";

 license": GPL v2;

 dependencies: [
    vibe: >= 0.7.11  # a comment;
 ];

Using ';' as end-char and omit all these double quotes
makes it much cleaner and less error-prone  for the user
as well as automatic generation. Double quotes are needed
only for strings with none alphanumeric characters.

Adding '#' or '//' for comments would also be a good idea.

I also think that something like json objects, which are
key-value pairs anyway, are dispensable. Hence everything
can be key-value pairs.

Exporting to other formats will probably needed anyway.

Peter
February 16, 2013
On Saturday, 16 February 2013 at 17:10:33 UTC, Sönke Ludwig wrote:
>  - Full IDE support:
>
>    Rather than focusing on performing the build by itself or tying a
>    package to a particular build tool, DUB translates a general
>    build receipt to any supported project format (it can also build
>    by itself). Right now VisualD and MonoD are supported as targets and
>    rdmd is used for simple command line builds. Especially the IDE
>    support is really important to not simply lock out people who prefer
>    them.
>

Cool, I guess creating a built-in import/export functionality for DUB scripts in Mono-D should be amazing. Will note this down on my todo list.
February 16, 2013
On Sat, 16 Feb 2013 22:21:55 +0100
Sönke Ludwig <sludwig@outerproduct.org> wrote:
> 
> My idea for the things you mentioned (swig, c, etc.) was to have a set of hooks that can be used to run external tools (invoked before build/project file generation, before build or after build). That together with your proposed interface should provide all the necessary flexibility while putting an emphasis on a standard way to describe the build process.

I like a lot of what you've said, but my concern about this part is, does it support things like:

- Dependencies between custom build steps (so that custom build steps aren't needlessly re-run when they're not out-of-date, and can possibly be parallelized)

- Multiple custom build targets

- Multiple custom build configurations

I think those are essential for a real general-purpose build tool.

February 16, 2013
On 2/16/13, Sönke Ludwig <sludwig@outerproduct.org> wrote:
> Some may already have noticed it as it's mentioned already on the vibe.d website and is currently hosted on the same domain as the old VPM registry:
>
> http://registry.vibed.org/

So can we start using this already or is this just a preview? It says here[1] it's precompiled for win32, but where do we get the executable from? I thought a --recursive clone would fetch all dependencies, but app.d won't compile without vibe.

[1] https://github.com/rejectedsoftware/dub

Anyway it looks interesting!
February 16, 2013
On 2/16/13, Andrej Mitrovic <andrej.mitrovich@gmail.com> wrote:
> So can we start using this already or is this just a preview? It says here[1] it's precompiled for win32, but where do we get the executable from? I thought a --recursive clone would fetch all dependencies, but app.d won't compile without vibe.
>
> [1] https://github.com/rejectedsoftware/dub
>
> Anyway it looks interesting!
>

Ah I didn't spot the download link http://registry.vibed.org/download

I guess this could be made more visible by adding a link to the download page from the github repository, and maybe putting the { * Using DUB * Download * Publishing packages * Helping developme } section at the top instead of the bottom.

Well, the latter is a stylistic issue so I don't mind it being this way. On typical websites I find important stuff at the top, and copyright and privacy policies at the bottom.
February 16, 2013
On 2/16/13, Andrej Mitrovic <andrej.mitrovich@gmail.com> wrote:
> Ah I didn't spot the download link http://registry.vibed.org/download

I get an error running dub.exe:

---------------------------
dub.exe - Unable To Locate Component
---------------------------
This application has failed to start because libevent.dll was not found. Re-installing the application may fix this problem.
---------------------------

You might want to list all the dependencies needed for dub or distribute them in a zip.
February 16, 2013
On Saturday, 16 February 2013 at 19:35:47 UTC, Russel Winder wrote:
> On Sat, 2013-02-16 at 20:15 +0100, Jacob Carlborg wrote:
> […]
>> I'm thinking that there should be a build tool that drives everything. The build script contains package dependencies. The build tool will ask the package manager to get linker/import paths and libraries for the dependencies.
> […]
>
> First a plug for people to think of SCons and Waf, not to mention
> Gradle, when starting to rattle of build tools.
>
> The SBT folks are using Scala in quite interesting ways to enable Scala
> to define project specifications with all the non-inferable
> dependencies.
>
> The Gradle folks have already done almost everything the SBT folks are
> doing, but using Groovy rather than Scala.
>
> The Go folk have done something interesting in that they have merged the
> whole concept of configuration and build by doing everything over DVCS.
> You put your sources in Git, Mercurial, Bazaar (they really should
> include Fossil as well but…) and these can be Got and the modules
> created within the standard structure local hierarchy.  The have a
> single command that performs all activity. D has rdmd but compared to
> Go's go command it cannot do a lot.
>
> I would suggest now is the time to think outside the box, analogous to
> the ways the Gradle and SBT folk have on the JVM and the way the Go folk
> have for native builds of statically linked code.
>
> Instead of thinking in terms of compile, link, modules, dependencies,
> what is the workflow that makes D the compelling language for building
> Fortran/C/C++/D systems. This is suggesting that the milieu to attack is
> the one currently being won by Python in the computationally intensive
> areas of bioinformatics and scientific computing.
>
> Can D be the project specification language using convention over
> configuration. Project directories are in a standard form (with
> exceptions describable) with dependencies either inferred by scanning
> the project sources or specified in a trivial way in the project
> specification file.
>
> Thus I suggest that it is not that the build tool is embedded in the
> package manager but that package and dependency management is part of
> the build system.

I'm having good success using D itself as the build tool language, and I'm at the point now where I'm getting much better results than what I was getting out of using external build tools, so for me there's no looking back.

I run rdmd on a .d "script" that I wrote that's setup to do exactly what I want.

What is missing from D is a good library of functions that are generally useful for writing build scripts. Phobos already supplies a great deal of what is needed that can be used to construct what is missing.

The benefit of using D is that I do not have to install and learn a secondary language or conform to a specific build format, or follow any weird restrictions. What I want to do is build my D code, not learn something new and perform acrobatics to get the job done. I can even use the same D process to build C/C++ code if I wanted to expand on it.

What I'm saying here is that I see no reason to use a language other than D itself as the build tool. What D can use is an addition to Phobos that supplies the necessary generalized functions that all build tools should supply, and I don't think there's all that much that's missing.

For a package manager, some standards may be required, but it too can be done completely with D.

Why use json (which is a subset of javascript), or ruby, or python, etc? Is there something fundamentally wrong with D that makes it unsuitable for this role?

--rt