September 23, 2006
Gregor Richards wrote:
>         o The only rule is that any module must be in the most specifically-named library corresponding to that module. That is, if you have libD.a.b.so.0 and libD.a.so.0, the module a.b.c should be in libD.a.b.so.0, not libD.a.so.0. Doing otherwise is fairly ridiculous anyway.
> 
> 
> 

Why so? Why should a given library be splited across multiple so/dll files according to package, instead of being compiled in just one file?


-- 
Bruno Medeiros - MSc in CS/E student
http://www.prowiki.org/wiki4d/wiki.cgi?BrunoMedeiros#D
September 23, 2006
I like the idea of having this naming schema.

I want to add the suggestion of using the unique package naming with reversed domain names. Like e.g. mango from dsource.org

Package: org.dsource.mango in Version 1.2.3

and the resulting lib should be in /usr/lib/

lib: libD.org.dsource.mango.so.1.2.3
lnk: libD.org.dsource.mango.so.1.2
lnk: libD.org.dsource.mango.so.1
lnk: libD.org.dsource.mango.so

I think the include directory hierarchy is close related to this. I would suggest to generate a header tree like this:

in /usr/include/d/
dir: org.dsource.mango.1.2.3
lnk: org.dsource.mango.1.2
lnk: org.dsource.mango.1
lnk: org.dsource.mango
src: org.dsource.mango.1.2.3/org/dsource/mango/....

Perhaps, in future, it is possible to pack and deploy the headers in some kind of jar file.

Frank




September 23, 2006
Bruno Medeiros wrote:

> Gregor Richards wrote:
>>         o The only rule is that any module must be in the most
>> specifically-named library corresponding to that module. That is, if you
>> have libD.a.b.so.0 and libD.a.so.0, the module a.b.c should be in
>> libD.a.b.so.0, not libD.a.so.0. Doing otherwise is fairly ridiculous
>> anyway.
>> 
>> 
>> 
> 
> Why so? Why should a given library be splited across multiple so/dll files according to package, instead of being compiled in just one file?
> 
> 

You totally missed the point, if the library _is_ split, then there should be a standardized way to decide which part a module is in.

-- 
Lars Ivar Igesund
blog at http://larsivi.net
DSource & #D: larsivi
September 24, 2006
On Mon, 18 Sep 2006 21:10:46 -0700, Gregor Richards wrote:

> Since there aren't many libraries for D (that is, builds that produce .so files or the ilk)

In fact, are there any? I vaguely remember reading that D does not yet support the creation of .so files. Is that still the case?

Anyhow, let's break Gregor's proposal into two parts:
(a) creating shared libraries (DLL in Windows, .so in Linux, ? in other
*nix)
(b) using shared libraries in the build process

I will defer (a) until I know how to do it :-)

With (b) I'm willing to add this feature to Build but with idea that failure to locate a library from a package name is not a failure to build. I'm not convinced that Gregor's idea that every (top-level) package must also be a library is a valid idea. I can imagine people still wanting to do static linking of object files.

-- 
Derek Parnell
Melbourne, Australia
"Down with mediocrity!"
September 24, 2006
Derek Parnell wrote:
> On Mon, 18 Sep 2006 21:10:46 -0700, Gregor Richards wrote:
> 
> 
>>Since there aren't many libraries for D (that is, builds that produce .so files or the ilk)
> 
> 
> In fact, are there any? I vaguely remember reading that D does not yet
> support the creation of .so files. Is that still the case?
> 
> Anyhow, let's break Gregor's proposal into two parts:
> (a) creating shared libraries (DLL in Windows, .so in Linux, ? in other
> *nix)
> (b) using shared libraries in the build process
> 
> I will defer (a) until I know how to do it :-)
> 
> With (b) I'm willing to add this feature to Build but with idea that
> failure to locate a library from a package name is not a failure to build.
> I'm not convinced that Gregor's idea that every (top-level) package must
> also be a library is a valid idea. I can imagine people still wanting to do
> static linking of object files.
> 

This last paragraph makes me think that I've miscommunicated my concept *urk*

1) Failure to locate an appropriately-named library is /not/ a failure, just a failure in that subsystem (if indeed the symbol can't be found anywhere, that'd be a failure).  This would probably be a report-but-continue type warning.
2) My creation of libraries is slightly more flexible than what you're suggesting, but less flexible than what you'd like I think.  Any /package/ can be a library (whether top-level or not), only making libraries of top-level packages is just a /default/.  The only reason for making this restriction is, to not place a reasonable restriction on what modules are contained in a library makes the naming exponentially complex or impossible.  Urk.

As per making .so files, DMD can't do it but GDC can.  For DMD to do it, it would need the equivalent of an -fPIC option, to produce position-independent code - the linking part is easy.  Of course, since either can create .a files, its implementable so long as DMD users are willing to surrender the ability to make .so files until it's capable. If build did this all in its automatic processing, the source maintainer wouldn't need to care whether the compiler being used is capable of producing libraries of any sort.

(Actually, DMD may be producing position independent code already for all I know, in which case it's perfectly capable, just need -L-shared - unfortunately, this is hard to test, since much software will "just work" even if it's not made position independent)

Furthermore, this proposal is not shared-library specific, it just shows its greatest benefit there (since the reliability in naming is carried through every conceivable stage of making/running a program).

 - Gregor Richards
September 24, 2006
Gregor Richards wrote:
> Derek Parnell wrote:
> 
>> On Mon, 18 Sep 2006 21:10:46 -0700, Gregor Richards wrote:
>>
>>
>>> Since there aren't many libraries for D (that is, builds that produce .so files or the ilk)
>>
>>
>>
>> In fact, are there any? I vaguely remember reading that D does not yet
>> support the creation of .so files. Is that still the case?
>>
>> Anyhow, let's break Gregor's proposal into two parts:
>> (a) creating shared libraries (DLL in Windows, .so in Linux, ? in other
>> *nix)
>> (b) using shared libraries in the build process
>>
>> I will defer (a) until I know how to do it :-)
>>
>> With (b) I'm willing to add this feature to Build but with idea that
>> failure to locate a library from a package name is not a failure to build.
>> I'm not convinced that Gregor's idea that every (top-level) package must
>> also be a library is a valid idea. I can imagine people still wanting to do
>> static linking of object files.
>>
> 
> This last paragraph makes me think that I've miscommunicated my concept *urk*
> 
> 1) Failure to locate an appropriately-named library is /not/ a failure, just a failure in that subsystem (if indeed the symbol can't be found anywhere, that'd be a failure).  This would probably be a report-but-continue type warning.
> 2) My creation of libraries is slightly more flexible than what you're suggesting, but less flexible than what you'd like I think.  Any /package/ can be a library (whether top-level or not), only making libraries of top-level packages is just a /default/.  The only reason for making this restriction is, to not place a reasonable restriction on what modules are contained in a library makes the naming exponentially complex or impossible.  Urk.
> 
> As per making .so files, DMD can't do it but GDC can.  For DMD to do it, it would need the equivalent of an -fPIC option, to produce position-independent code - the linking part is easy.  Of course, since either can create .a files, its implementable so long as DMD users are willing to surrender the ability to make .so files until it's capable. If build did this all in its automatic processing, the source maintainer wouldn't need to care whether the compiler being used is capable of producing libraries of any sort.
> 
> (Actually, DMD may be producing position independent code already for all I know, in which case it's perfectly capable, just need -L-shared - unfortunately, this is hard to test, since much software will "just work" even if it's not made position independent)
> 
> Furthermore, this proposal is not shared-library specific, it just shows its greatest benefit there (since the reliability in naming is carried through every conceivable stage of making/running a program).
> 
>  - Gregor Richards

According to DMD's docs, it has an -fPIC flag.  I see no compelling reason why it shouldn't be able to create .so files, given that.

To make a .so file (just a guess):

dmd <source files> -fPIC -L-shared -L-Wl-soname=libfoo.so.0 -oflibfoo.so.0.0.0

(In case you don't get the discrepency in naming, it's all about how ELF .so files are versioned: the first number after .so is essentially the ABI version, so it should only change when non-backwards-compatible changes are made.  The name that programs keep in them has one number. The next is generally used for backwards-compatible ABI changes, the next for other changes)

If you are willing to implement this in build, I'd be glad to extend my help, especially on getting all the UNIX stuff named/installed as would be traditional for a UNIX system.  (And of course I've also offered to make the changes myself, but I'm under the distinctive impression that you'd prefer to make big changes like that yourself - perfectly understandable)

 - Gregor Richards
September 25, 2006
On Sun, 24 Sep 2006 02:24:06 -0700, Gregor Richards wrote:

> Derek Parnell wrote:
>> On Mon, 18 Sep 2006 21:10:46 -0700, Gregor Richards wrote:
>> 
>>>Since there aren't many libraries for D (that is, builds that produce .so files or the ilk)
>> 
>> In fact, are there any? I vaguely remember reading that D does not yet support the creation of .so files. Is that still the case?
>> 
>> Anyhow, let's break Gregor's proposal into two parts:
>> (a) creating shared libraries (DLL in Windows, .so in Linux, ? in other
>> *nix)
>> (b) using shared libraries in the build process
>> 
>> I will defer (a) until I know how to do it :-)
>> 
>> With (b) I'm willing to add this feature to Build but with idea that failure to locate a library from a package name is not a failure to build. I'm not convinced that Gregor's idea that every (top-level) package must also be a library is a valid idea. I can imagine people still wanting to do static linking of object files.
>> 
> 
> This last paragraph makes me think that I've miscommunicated my concept *urk*

Oh, I wouldn't blame my failure to understand on your ability to communicate ;-)

> 1) Failure to locate an appropriately-named library is /not/ a failure, just a failure in that subsystem (if indeed the symbol can't be found anywhere, that'd be a failure).  This would probably be a report-but-continue type warning.

You still seem to be saying that the *only* place that module ought to reside in is in libraries. Whereas I maintain that they *could* also reside in object files (.o/.obj) files. So if Build doesn't find a library that matches the import name, why can't it use the object file that maps to the import name - and if the object file doesn't exist then why can't I have Build compile the module's source to create it? The fact that the library doesn't exist is not a failure then - or must there be a switch to say that only libraries are acceptable for this build process?

Example:
  import a.b.c;

Looks for
   libD.a.b.so
   libD.a.b.a
   libD.a.so
   libD.a.a
   a.b.c.o

and places the first one it comes across onto the linker command line. If it finds none of these, it will compile a.b.c.d to form an object file.

And then looks for
   a.b.c.di
   a.b.c.d

to see if the source file is more recent than the 'object' file found earlier.

So a further complication is what should Build do if the module source files are more recent than the matching library? Currently, it recompiles the source files and uses the resulting object files in the linkage.

-- 
Derek
(skype: derek.j.parnell)
Melbourne, Australia
"Down with mediocrity!"
25/09/2006 9:41:49 AM
September 25, 2006
Derek Parnell wrote:
> On Sun, 24 Sep 2006 02:24:06 -0700, Gregor Richards wrote:
> 
> 
>>Derek Parnell wrote:
>>
>>>On Mon, 18 Sep 2006 21:10:46 -0700, Gregor Richards wrote:
>>>
>>>
>>>>Since there aren't many libraries for D (that is, builds that produce .so files or the ilk)
>>>
>>>In fact, are there any? I vaguely remember reading that D does not yet
>>>support the creation of .so files. Is that still the case?
>>>
>>>Anyhow, let's break Gregor's proposal into two parts:
>>>(a) creating shared libraries (DLL in Windows, .so in Linux, ? in other
>>>*nix)
>>>(b) using shared libraries in the build process
>>>
>>>I will defer (a) until I know how to do it :-)
>>>
>>>With (b) I'm willing to add this feature to Build but with idea that
>>>failure to locate a library from a package name is not a failure to build.
>>>I'm not convinced that Gregor's idea that every (top-level) package must
>>>also be a library is a valid idea. I can imagine people still wanting to do
>>>static linking of object files.
>>>
>>
>>This last paragraph makes me think that I've miscommunicated my concept *urk*
> 
> 
> Oh, I wouldn't blame my failure to understand on your ability to
> communicate ;-)
> 
> 
>>1) Failure to locate an appropriately-named library is /not/ a failure, just a failure in that subsystem (if indeed the symbol can't be found anywhere, that'd be a failure).  This would probably be a report-but-continue type warning.
> 
> 
> You still seem to be saying that the *only* place that module ought to
> reside in is in libraries.

Aha!  This is a core misunderstanding.  All of this library stuff is supposed to come well after failing to find it in an object file - the order in my head is: source files, object files, specified libraries, automatically-discovered libraries.  That is, it should FIRST look for source, then objects, and only if failing in both of those should it look at libraries - libraries are solely for things that are NOT included with the source!

> Whereas I maintain that they *could* also reside
> in object files (.o/.obj) files. So if Build doesn't find a library that
> matches the import name, why can't it use the object file that maps to the
> import name - and if the object file doesn't exist then why can't I have
> Build compile the module's source to create it?  The fact that the library
> doesn't exist is not a failure then - or must there be a switch to say that
> only libraries are acceptable for this build process?
> 
> Example:
>   import a.b.c;
> 
> Looks for    libD.a.b.so
>    libD.a.b.a
>    libD.a.so
>    libD.a.a
>    a.b.c.o
> 
> and places the first one it comes across onto the linker command line. If
> it finds none of these, it will compile a.b.c.d to form an object file.
> 
> And then looks for
>    a.b.c.di
>    a.b.c.d
> 
> to see if the source file is more recent than the 'object' file found
> earlier.
> 
> So a further complication is what should Build do if the module source
> files are more recent than the matching library? Currently, it recompiles
> the source files and uses the resulting object files in the linkage.
> 

The timing on source/object files has no reason to change - the only issue is with conflicts, and in my opinion source should /always/ trump libraries on conflicts.

 - Gregor Richards
September 25, 2006
On Sun, 24 Sep 2006 17:14:06 -0700, Gregor Richards wrote:

> Derek Parnell wrote:
>> You still seem to be saying that the *only* place that module ought to reside in is in libraries.
> 
> Aha!  This is a core misunderstanding.  All of this library stuff is supposed to come well after failing to find it in an object file - the order in my head is: source files, object files, specified libraries, automatically-discovered libraries.  That is, it should FIRST look for source, then objects, and only if failing in both of those should it look at libraries - libraries are solely for things that are NOT included with the source!
> 

Ok! So the example should be more like ...

Example:
   import a.b.c;

Looks for
    a.b.c.o
    libD.a.b.so
    libD.a.b.a
    libD.a.so
    libD.a.a

and places the first one it comes across onto the linker command line. If
it finds none of these, or finds that the source is more recent, it will
compile a.b.c.d to form an object file.

-- 
Derek
(skype: derek.j.parnell)
Melbourne, Australia
"Down with mediocrity!"
25/09/2006 10:17:17 AM
September 25, 2006
Derek Parnell wrote:
> On Sun, 24 Sep 2006 17:14:06 -0700, Gregor Richards wrote:
> 
> 
>>Derek Parnell wrote:
>>
>>>You still seem to be saying that the *only* place that module ought to
>>>reside in is in libraries.
>>
>>Aha!  This is a core misunderstanding.  All of this library stuff is supposed to come well after failing to find it in an object file - the order in my head is: source files, object files, specified libraries, automatically-discovered libraries.  That is, it should FIRST look for source, then objects, and only if failing in both of those should it look at libraries - libraries are solely for things that are NOT included with the source!
>>
> 
> 
> Ok! So the example should be more like ...
> 
> Example:
>    import a.b.c;
>  Looks for     a.b.c.o
>     libD.a.b.so
>     libD.a.b.a
>     libD.a.so
>     libD.a.a
>  and places the first one it comes across onto the linker command line. If
> it finds none of these, or finds that the source is more recent, it will
> compile a.b.c.d to form an object file.
> 

Exactly!  Using libraries isn't supposed to /replace/ using source or already-compiled object files, it's supposed to be an adjunct - in many cases, it's just inconvenient to use object files or source, and of course there's the possibility of proprietary software in which a library is clearly the superior method.  Obviously, if you included the source by some means, you intend to use them - otherwise, libraries are probably more useful.

 - Gregor Richards