View mode: basic / threaded / horizontal-split · Log in · Help
March 19, 2011
Re: Dream package management system (Was: a cabal for D ?)
On 2011-03-18 18:04, Chris Manning wrote:
> On 17/03/2011 22:49, Jason E. Aten wrote:
>> Somewhat tongue in cheek, we could call it dabal.
>>
>> As in, "get on dabal!" :-)
>
> If D gets accepted for Google Summer of Code, I think this would be a
> great idea for a project and I would be interested in implementing it as
> a student. Although, it does seem overly ambitious so maybe only some of
> this could be for the gsoc (and if I do this It'd be great to carry on
> working on it anyway).
>
> What does everybody think about this? Should I draw up a proposal of
> some kind?
>
> Chris

I've been thinking for quite some time to build a package management 
system for D, lets call it dpac as an example. This is the ideas that I 
have:

Basically copy how RubyGems works.
Use Ruby as a DSL for dpacsepc files which is used to create to create 
the dpac file. This is an example for how a file used to build a package 
could look like:

name "Foo Bar"
summary "This is the Foo Bar package"
version "1.0.0"
type :lib
author "Jacob Carlborg"
files ["lib.d"] # list of the files in the package
build :make # other options could be :dsss :cmake and so on
dversion 2 # D1 or D2

Build a dpac package out of the dpacspec file:

dpac foobar.dpacspec

Publish the package:

$ dpac publish foobar

Install the package:

$ dpac install foobar

A dpac package would just be a zip file (or some other type of archive) 
containing all the necessary files to build the package and a file with 
meta data.

All packages would be manged on a basic RESTful web server. Using GET to 
download a package and POST to publish a package.

I'm working on a build system for D that I was thinking about to 
integrate with the package management system. Then the build system 
could track the files needed to build the package, making the "files" 
attribute optional.

I also has a tool called DVM, https://bitbucket.org/doob/dvm , used for 
installing and managing different versions of D compilers. I was 
thinking about integrating DVM with the package management system to be 
able to install different packages for different compilers.

-- 
/Jacob Carlborg
March 19, 2011
Re: Dream package management system (Was: a cabal for D ?)
On 19/03/2011 14:36, Jacob Carlborg wrote:
> On 2011-03-18 18:04, Chris Manning wrote:
>> On 17/03/2011 22:49, Jason E. Aten wrote:
>>> Somewhat tongue in cheek, we could call it dabal.
>>>
>>> As in, "get on dabal!" :-)
>>
>> If D gets accepted for Google Summer of Code, I think this would be a
>> great idea for a project and I would be interested in implementing it as
>> a student. Although, it does seem overly ambitious so maybe only some of
>> this could be for the gsoc (and if I do this It'd be great to carry on
>> working on it anyway).
>>
>> What does everybody think about this? Should I draw up a proposal of
>> some kind?
>>
>> Chris
>
> I've been thinking for quite some time to build a package management
> system for D, lets call it dpac as an example. This is the ideas that I
> have:
>
> Basically copy how RubyGems works.
> Use Ruby as a DSL for dpacsepc files which is used to create to create
> the dpac file. This is an example for how a file used to build a package
> could look like:
>
> name "Foo Bar"
> summary "This is the Foo Bar package"
> version "1.0.0"
> type :lib
> author "Jacob Carlborg"
> files ["lib.d"] # list of the files in the package
> build :make # other options could be :dsss :cmake and so on
> dversion 2 # D1 or D2
>
> Build a dpac package out of the dpacspec file:
>
> dpac foobar.dpacspec
>
> Publish the package:
>
> $ dpac publish foobar
>
> Install the package:
>
> $ dpac install foobar
>
> A dpac package would just be a zip file (or some other type of archive)
> containing all the necessary files to build the package and a file with
> meta data.
>
> All packages would be manged on a basic RESTful web server. Using GET to
> download a package and POST to publish a package.
>
> I'm working on a build system for D that I was thinking about to
> integrate with the package management system. Then the build system
> could track the files needed to build the package, making the "files"
> attribute optional.
>
> I also has a tool called DVM, https://bitbucket.org/doob/dvm , used for
> installing and managing different versions of D compilers. I was
> thinking about integrating DVM with the package management system to be
> able to install different packages for different compilers.
>
I was thinking about something more similar to portage's ebuild system 
or arch's AUR. This would mean that the sources could be stored anywhere 
and just the info to build a package would be stored in a centralised 
location.

For publishing these build "scripts", again, AUR's system comes to mind, 
although perhaps something more controll(ed/able). Also, gentoo's 
sunrise overlay.

Of course, this comes with the downside of the user having to compile 
the packages on their end.

Chris
March 19, 2011
Re: Dream package management system (Was: a cabal for D ?)
On 2011-03-19 17:05, Chris Manning wrote:
> On 19/03/2011 14:36, Jacob Carlborg wrote:
>> On 2011-03-18 18:04, Chris Manning wrote:
>>> On 17/03/2011 22:49, Jason E. Aten wrote:
>>>> Somewhat tongue in cheek, we could call it dabal.
>>>>
>>>> As in, "get on dabal!" :-)
>>>
>>> If D gets accepted for Google Summer of Code, I think this would be a
>>> great idea for a project and I would be interested in implementing it as
>>> a student. Although, it does seem overly ambitious so maybe only some of
>>> this could be for the gsoc (and if I do this It'd be great to carry on
>>> working on it anyway).
>>>
>>> What does everybody think about this? Should I draw up a proposal of
>>> some kind?
>>>
>>> Chris
>>
>> I've been thinking for quite some time to build a package management
>> system for D, lets call it dpac as an example. This is the ideas that I
>> have:
>>
>> Basically copy how RubyGems works.
>> Use Ruby as a DSL for dpacsepc files which is used to create to create
>> the dpac file. This is an example for how a file used to build a package
>> could look like:
>>
>> name "Foo Bar"
>> summary "This is the Foo Bar package"
>> version "1.0.0"
>> type :lib
>> author "Jacob Carlborg"
>> files ["lib.d"] # list of the files in the package
>> build :make # other options could be :dsss :cmake and so on
>> dversion 2 # D1 or D2
>>
>> Build a dpac package out of the dpacspec file:
>>
>> dpac foobar.dpacspec
>>
>> Publish the package:
>>
>> $ dpac publish foobar
>>
>> Install the package:
>>
>> $ dpac install foobar
>>
>> A dpac package would just be a zip file (or some other type of archive)
>> containing all the necessary files to build the package and a file with
>> meta data.
>>
>> All packages would be manged on a basic RESTful web server. Using GET to
>> download a package and POST to publish a package.
>>
>> I'm working on a build system for D that I was thinking about to
>> integrate with the package management system. Then the build system
>> could track the files needed to build the package, making the "files"
>> attribute optional.
>>
>> I also has a tool called DVM, https://bitbucket.org/doob/dvm , used for
>> installing and managing different versions of D compilers. I was
>> thinking about integrating DVM with the package management system to be
>> able to install different packages for different compilers.
>>
> I was thinking about something more similar to portage's ebuild system
> or arch's AUR. This would mean that the sources could be stored anywhere
> and just the info to build a package would be stored in a centralised
> location.
>
> For publishing these build "scripts", again, AUR's system comes to mind,
> although perhaps something more controll(ed/able). Also, gentoo's
> sunrise overlay.
>
> Of course, this comes with the downside of the user having to compile
> the packages on their end.
>
> Chris

Oh, I forgot a part. Just as with RubyGems you would be able to install 
a package from a repository (git, mercurial or something else) as long 
as it contains the file containing the package specification file 
(dpacspec).

Of course you could manually downloading a package and installing it.

-- 
/Jacob Carlborg
March 19, 2011
incremental builds for D projects... challenging or close at hand?
> On Sat, 19 Mar 2011 12:19:58 +0100, Jacob Carlborg wrote:
>> What are people's experiences with the various options for build
>> systems with D?
> 
> It's not very easy to make an incremental build system for D because of
> several reasons. Some are due to how the language works and some are due
> to how DMD works:
> 
> * DMD doesn't output all data in all the object files - This can perhaps
> be solved by compiling with the -lib switch
> 
> * When you change one D file you need to recompile ALL files that depend
> on the changed file. To compare with C/C++ which has source and header
> files you only need to recompile the source file if you change it
> 
> * DMD doesn't keep the fully qualified module name when naming object
> files resulting in foo.bar will conflict with bar.bar. Issue 3541.

[The above is from the packaging system discussion in the "a cabal for 
D?" thread; here I am branching this to a new topic because I'd like 
anyone interested in incremental build processes to notice and contribute 
if they have input.]

That is an interesting observation, Jacob. Thank you for pointing that 
out. 

Is there anything else (open question anyone) that would prevent D 
projects from doing incremental builds?  Lack of support for incremental 
builds is a show stopper.  Or in this case, the show would never get 
funded to begin with.

Or to ask it another way, what would it take to get incremental builds?
March 19, 2011
Re: Dream package management system (Was: a cabal for D ?)
Even AutoHotkey has a system of finding and installing libraries. It
can run example code without installing (well, it is a scripting
language), show the source, documentation and some other things.

Snapshot: http://i.imgur.com/Uv1Gr.jpg
March 22, 2011
Re: a cabal for D ?
On 17/03/11 21.44, Jason E. Aten wrote:
> Please correct me if I'm wrong, but I observe that there doesn't appear
> to be a package management system / standard repository for D libraries.
> Or is there?
>
> I'm talking about something as easy to use as R's CRAN,
>> install.packages("rforest")
>
> or cpan for perl, ctan for latex, dpgk/apt for debian, cabal for Haskell/
> Hackage, etc.
>
> If there's not a commonly utilized one currently, perhaps we could
> "borrow" cabal, with a trivial port.  cabal is Haskell's package manager.
>
> Not only does having a standard package install system facilitate
> adoption, it greatly facilitates code sharing and library maturation.

It seems quite a few people are interested in such a tool. I suggest 
that those who are interested in the subject form some kind of SIG and 
creates a page on http://prowiki.org/wiki4d with ideas, insights, 
requirements that the rest of the community can follow. I think 
discussions still need to be done in this newsgroup though.

This means that someone will actually need to implement something or 
modify an existing solution at some point for it to make sense. But 
initially we need to settle on something that "everybody" would be happy 
with and would support as the standard tool if possible.

I'm willing to throw in some time implementing it when the curl stuff 
that I'm currently working on is done. Hopefully others can put some 
work in this tool as well to get things rolling.

Anyone think this is a good idea?

Should I create a SIG page?

/Jonas
March 22, 2011
Re: a cabal for D ?
On 2011-03-22 12:03, Jonas Drewsen wrote:
> On 17/03/11 21.44, Jason E. Aten wrote:
>> Please correct me if I'm wrong, but I observe that there doesn't appear
>> to be a package management system / standard repository for D libraries.
>> Or is there?
>>
>> I'm talking about something as easy to use as R's CRAN,
>>> install.packages("rforest")
>>
>> or cpan for perl, ctan for latex, dpgk/apt for debian, cabal for Haskell/
>> Hackage, etc.
>>
>> If there's not a commonly utilized one currently, perhaps we could
>> "borrow" cabal, with a trivial port. cabal is Haskell's package manager.
>>
>> Not only does having a standard package install system facilitate
>> adoption, it greatly facilitates code sharing and library maturation.
>
> It seems quite a few people are interested in such a tool. I suggest
> that those who are interested in the subject form some kind of SIG and
> creates a page on http://prowiki.org/wiki4d with ideas, insights,
> requirements that the rest of the community can follow. I think
> discussions still need to be done in this newsgroup though.
>
> This means that someone will actually need to implement something or
> modify an existing solution at some point for it to make sense. But
> initially we need to settle on something that "everybody" would be happy
> with and would support as the standard tool if possible.
>
> I'm willing to throw in some time implementing it when the curl stuff
> that I'm currently working on is done. Hopefully others can put some
> work in this tool as well to get things rolling.
>
> Anyone think this is a good idea?
>
> Should I create a SIG page?
>
> /Jonas

I think it's a good idea. I'm planning to start on a tool like this when 
I finished the std.net.isemail module.

-- 
/Jacob Carlborg
April 01, 2011
Re: a cabal for D ?
Hi Russel. (BTW, I was the guy a few weeks ago at the London Java 
Community Meetup pub meet that Barry Cranford mentioned in an SMS. Sorry 
you couldn't make it, maybe next time. :p )

On 18/03/2011 08:52, Russel Winder wrote:
>
> Note that every language-specific package manager conflicts directly
> with every operating system package manager.  Thus RubyGems, CPAN,
> Cabal, Maven, Go, etc. conflicts with the package management of Debian,
> Fedora, SUSE, FreeBSD, MacPorts, etc.

Which is the way it should be. (Note that "conflicts" is likely not the 
best word, rather "ignores" or "does not make use or interact with" is a 
better term.) I'm well convinced of this because these two - OS and 
language package manager - /usually/ serve very distinct purposes. The 
OS PMs are usually focused on installing applications, not building them 
(yes, there are exceptions, like Gentoo). In particular the OS PMs most 
likely won't support functionality you would want from a language PM. 
(say for example that you would want to specify that your library has a 
dependency on any other library that exports a D package named org.foo.bar?)
But we don't even need to go there, just the fact that OS PMs are not 
cross-platform is reason enough for them to be unsuitable to be used (on 
their own) as language PM. There needs to be a language PM abstracted 
from OS/platform.
Perhaps some additional (and optional) integration with the OS PM could 
be available in the language PM. But the language PM would still be "on 
top". And even so I'm not convinced this integrations makes much sense. 
Maybe for dynamic and interpreted languages like Ruby and Python, where 
built application and sources are often the same... here perhaps the 
term "conflict" as mentioned above is more appropriate, the distinction 
between end-user and developer-user is less clear. But none of this 
applies to D, being a compiled language.

-- 
Bruno Medeiros - Software Engineer
April 01, 2011
Re: a cabal for D ?
On 18/03/2011 18:42, Russel Winder wrote:
> On Fri, 2011-03-18 at 10:41 -0400, Jesse Phillips wrote:
> [ . . . ]
>> Otherwise I would be for using the native packaging system, but it
>> isn't easy for the contributor, the maintainer of the server, or the
>> one building a system that works with all of them.
>
> The problem is that the system administrator wants to use just the OS
> packaging system to make things easy for them to maintain the system.
> The user wants to be able to install stuff easily and may not have
> sufficient permissions to actually use the OS packaging system.  Sys
> admins don't want to have to learn N packages to deal with N languages,
> the use probably only works in a couple of languages and so doesn't care
> if they have to learn two different packagin systems so as to get stuff
> done.
>

I agree sys admins should not have to learn language specific package 
managers just because the user wants to use one. Rather what should 
happen is that language PMs should be able to be "installed" on any 
user-specified location, and installed multiple times. Just like a 
compiler should. They should not be designed to work only as an OS 
"singleton", so to speak.

> This is not an easy issue.  I just find the knee-jerk reaction of "we
> have this new language therefore we must have a brand new (build system|
> packaging system|shell|implementation of every comms protocol|new user
> interface library|operating system)" leads to too many distractions from
> getting stuff done using the good tools that are already available.
>

What I think makes sense for a D package manager is to integrate with 
existing *build* systems (make, Cmake, Ant, Maven, SCons, etc., etc., 
whatever might be appropriate). But definitely not with OS package managers.

-- 
Bruno Medeiros - Software Engineer
April 01, 2011
Re: Dream package management system (Was: a cabal for D ?)
On 19/03/2011 14:36, Jacob Carlborg wrote:
> On 2011-03-18 18:04, Chris Manning wrote:
>> On 17/03/2011 22:49, Jason E. Aten wrote:
>>> Somewhat tongue in cheek, we could call it dabal.
>>>
>>> As in, "get on dabal!" :-)
>>
>> If D gets accepted for Google Summer of Code, I think this would be a
>> great idea for a project and I would be interested in implementing it as
>> a student. Although, it does seem overly ambitious so maybe only some of
>> this could be for the gsoc (and if I do this It'd be great to carry on
>> working on it anyway).
>>
>> What does everybody think about this? Should I draw up a proposal of
>> some kind?
>>
>> Chris
>
> I've been thinking for quite some time to build a package management
> system for D, lets call it dpac as an example. This is the ideas that I
> have:
>
> Basically copy how RubyGems works.
> Use Ruby as a DSL for dpacsepc files which is used to create to create
> the dpac file. This is an example for how a file used to build a package
> could look like:
>
> name "Foo Bar"
> summary "This is the Foo Bar package"
> version "1.0.0"
> type :lib
> author "Jacob Carlborg"
> files ["lib.d"] # list of the files in the package
> build :make # other options could be :dsss :cmake and so on
> dversion 2 # D1 or D2
>

Copying Rubygems would be a bad idea, or at least a very incomplete one. 
And the example code above shows why, doesn't anyone else see it?
Just look at the line:
build :make # other options could be :dsss :cmake and so on
Basicly, this would be a D package manager that requires another package 
manager to build D artifacts... kinda not very useful, right? If we skip 
the issue of building D artifacts (executables, dynamic libraries, 
static libraries, etc.) we skip one essential aspect of a D 
package/build management system (if not the main aspect, but that is 
debatable...). This is why I think just copying/cloning a PM from 
interpreted languages is not helping us much.

-- 
Bruno Medeiros - Software Engineer
1 2 3 4
Top | Discussion index | About this forum | D home