BCS wrote:
> My program imports lib A and B. Lib A imports lib C and asks for version "X". Lib B imports lib C and asks for version "!X". Who wins?

Compilation error.

Hello Walter,

> BCS wrote:
> 
>> My program imports lib A and B. Lib A imports lib C and asks for
>> version "X". Lib B imports lib C and asks for version "!X". Who wins?
>> 
> Compilation error.
> 

Exactly. If there were a way for A to ask for X or Y and B to ask for Y or Z than the solution is easy: Y.

-- 
... <IXOYE><

Hello BCS,

> Hello Walter,
> 
>> BCS wrote:
>> 
>>> My program imports lib A and B. Lib A imports lib C and asks for
>>> version "X". Lib B imports lib C and asks for version "!X". Who
>>> wins?
>>> 
>> Compilation error.
>> 
> Exactly. If there were a way for A to ask for X or Y and B to ask for
> Y or Z than the solution is easy: Y.
> 

And to finish the thought; a system that only allows a program to ask for a single version is worse than one that doesn't allow any or allows many.

-- 
... <IXOYE><

On Thu, 06 May 2010 16:01:49 -0700, Walter Bright wrote:

> Graham Fawcett wrote:
>> Simple and well conceived! I would like this for D please (and am
>> willing to help :)).
> 
> Any help in this direction will be most appreciated.

I'm slowly working on a proposal, trying to incorporate the feedback we've received on this thread. I've got a few specific questions for the group.

(For clarity's sake, I'll refer to downloadable, third-party packages as "subprojects" here, so as not to overload terms like library, module and package. It's not a great name, but it's unambiguous.)

Let's say I want to use a subproject in my application: let's take sybrandy's new logging module as an example. Assume I have tool support for declaring my app's dependency on his subproject, and for downloading and storing his source files.

It seems to me that a decent tool should not only download the subproject's source, but also compile it into a static library. It should also inform my app's build process, stating what compiler and linker flags I need in order to build and link my app against the subproject.

Do you agree that precompiling a subproject is a desirable feature?

If so, what's the minimum harnessing we can impose upon a subproject writer, to make their library compilable by an automated tool?

We could require, e.g., that the subproject author maintain a Makefile with a 'd-preinstall target, which compiles the package into an installable form (perhaps copying the libfile, along with a link to the root of the sources, into an $INSTALL_TARGET directory). The installer would then finish the installation process.

An alternative to 'make d-preinstall' would be a more D-specific package descriptor, like Haskell's ".cabal" descriptors, which tell the installer what the package contains, and how to build and install it. The 'descriptor' approach puts a bigger burden on the design of the install-tool, and loses some flexibility compared with the 'make' approach; but lost flexibility isn't necessarily bad.

Given the host of ways that people organize their own code, I don't think we can expect an automated tool to guess the layout and compilation instructions for any arbitrary subproject. I think we either need source layout conventions, or we need a demarcation point like 'make d-preinstall'.

Or, what else? Any thoughts?

Thanks,
Graham

Hello Graham,

> If so, what's the minimum harnessing we can impose upon a subproject
> writer, to make their library compilable by an automated tool?

For many cases, there is a zero config solution that will work much of the time: Compile all files and pack them into a .lib/.a

Until that doesn't work, the writer shouldn't have to do anything but supply a name for the lib and a list of files.

-- 
... <IXOYE><

Hi BCS,

On Thu, 13 May 2010 01:53:34 +0000, BCS wrote:
> Hello Graham,
> 
>> If so, what's the minimum harnessing we can impose upon a subproject writer, to make their library compilable by an automated tool?
> 
> For many cases, there is a zero config solution that will work much of the time: Compile all files and pack them into a .lib/.a

Right. Last night I reviewed a set of dsource.org projects, and I see how a zero-config solution would work for many of them.

> Until that doesn't work, the writer shouldn't have to do anything but supply a name for the lib and a list of files.

It won't take long before more options are needed (I would still vote to have recommended linker flags be declarable, to support wrappers to C libraries), but this is a good start.

Any preferences re: how to declare the list of files? (Would you want glob patterns, exclusions, etc.?)

I'm tempted to support a truly zero-config version that

(a) derives the library name from the parent directory name of the
    source (skipping intermediary directories like 'branches/foo/',
    'trunk', etc.);

(b) derives the list of files by including all *.d files, except those
    in directories named 'test' and except those which contain a
    main() function.

Graham

Hello Graham,

> (b) derives the list of files by including all *.d files, except those
> in directories named 'test' and except those which contain a
> main() function.

I'm thinking of how to make this work for the simple publication model of "Put it all out via HTTP/SVN/CVS/GIT/etc.". Anything that works for that should generalize well.

How about: Download files as needed to do imports from any local code but don't build them. Once you have copies of everything you need, build one lib per mapping rule that includes everything retrieved via that rule. At that point the name doesn't hardly matter at all. You can make it the FQN of the mapped package if you want it easy to interpret.

As for linker flags... pragams or header comments? or how about a default flags file at the root of the mapped source.

-- 
... <IXOYE><

Hi BCS,

On Thu, 13 May 2010 17:31:34 +0000, BCS wrote:

>> (b) derives the list of files by including all *.d files, except those
>> in directories named 'test' and except those which contain a main()
>> function.
> 
> I'm thinking of how to make this work for the simple publication model of "Put it all out via HTTP/SVN/CVS/GIT/etc.". Anything that works for that should generalize well.

Yes, agreed, 'simple publication' is what I hope to target. I'm confident that issues like versioning, dependencies, and cataloguing can be addressed once 'simple publication' is stable.

> How about: Download files as needed to do imports from any local code but don't build them. Once you have copies of everything you need, build one lib per mapping rule that includes everything retrieved via that rule. At that point the name doesn't hardly matter at all. You can make it the FQN of the mapped package if you want it easy to interpret.

If I follow you, then the naming of libraries (or pre-compilation of libraries at all) is a non-issue: we download the necessary source files, cherry-pick the ones needed for our project, and compile them together. Or let the compiler do the cherry-picking for us: just make sure everything is "included" properly when we invoke the compiler, and it can discover all the necessary modules.

I have a prototype that does roughly that already, but I was concerned others might think the lack of a pre-compiled library was too hackish (though I quite like it!). I think I'll proceed, and come back with some sample code.

> As for linker flags... pragams or header comments? or how about a default flags file at the root of the mapped source.

Thanks to your suggestion, I just discovered 'pragma("lib", ...)' which does exactly what I wanted, and in a canoncial way. (The only linker flag that's especially important IMHO is '-l', and this pragma addresses that.) So, we ask C-wrapper authors to toss in a quick pragma, and we're off to the races.

Best,
Graham

Hello Graham,

> I was concerned
> others might think the lack of a pre-compiled library was too hackish
> (though I quite like it!). I think I'll proceed, and come back with
> some sample code.
> 
[...]
> Thanks to your suggestion, I just discovered 'pragma("lib", ...)'
> which does exactly what I wanted, and in a canoncial way. 

put those together and an idea crops up: allow more than just local lib files in that pragma, URL's come to mind but I'd think that some way to say a path is relative to the root of the current package (local, mapped or whatnot) might be every useful.


pragma(lib, "foo"); // as today
pragma(lib, "@foo"); // derived from package root, downloaded if needed, might need to ask user "y/n" for that.

(BTW: hard coding the extension looks like a bad idea to me.)



> Best,
> Graham
-- 
... <IXOYE><

Hey again,

On Thu, 13 May 2010 18:26:07 +0000, Graham Fawcett wrote:

> Hi BCS,
> 
> On Thu, 13 May 2010 17:31:34 +0000, BCS wrote:
> 
>>> (b) derives the list of files by including all *.d files, except those
>>> in directories named 'test' and except those which contain a main()
>>> function.
>> 
>> I'm thinking of how to make this work for the simple publication model of "Put it all out via HTTP/SVN/CVS/GIT/etc.". Anything that works for that should generalize well.
> 
> Yes, agreed, 'simple publication' is what I hope to target. I'm confident that issues like versioning, dependencies, and cataloguing can be addressed once 'simple publication' is stable.
> 
>> How about: Download files as needed to do imports from any local code but don't build them. Once you have copies of everything you need, build one lib per mapping rule that includes everything retrieved via that rule. At that point the name doesn't hardly matter at all. You can make it the FQN of the mapped package if you want it easy to interpret.
> 
> If I follow you, then the naming of libraries (or pre-compilation of libraries at all) is a non-issue: we download the necessary source files, cherry-pick the ones needed for our project, and compile them together. Or let the compiler do the cherry-picking for us: just make sure everything is "included" properly when we invoke the compiler, and it can discover all the necessary modules.
> 
> I have a prototype that does roughly that already, but I was concerned others might think the lack of a pre-compiled library was too hackish (though I quite like it!). I think I'll proceed, and come back with some sample code.

OK, I'm back. :) I've put a copy of my prototype, here:

http://github.com/gmfawcett/d-build

The name 'd-build' is a bit misleading, but I have nothing better to call it.

Currently it's a Python script -- sorry, I can still prototype a lot faster in Python -- but a D version will follow if there is interest.

If you're interested, please visit the github link and read the README to get an idea of what it does (and doesn't do). If you look at the DEPS file, you'll see the amazing way that I specify a dependency on a specific verision of an external project.

At this point, it's so simple that it's almost a "so what" -- but I think it is a promising start. Once the mechanics are down pat, the fun begins.

Best,
Graham