View mode: basic / threaded / horizontal-split · Log in · Help
February 03, 2012
libphobos.so libdruntime.so
As time goes by the D runtime will have it's place on Unix systems next to  
the C runtime. The upcoming support for PIC is the next step. Now I just  
want to quickly raise awareness for "sonames". For any library, there may  
be several incompatible versions installed, since new features are added  
and old features a deprecated. This happens in Phobos as well, no doubt  
:). So the solution is to put the version in the library name as well, so  
they can coexist in the same directory (/usr/lib).

Here is an example for the two versions of libunique I have installed:
	/usr/lib64/libunique-1.0.so         (link to  
/usr/lib64/libunique-1.0.so.0.100.6)
	/usr/lib64/libunique-1.0.so.0       (link to  
/usr/lib64/libunique-1.0.so.0.100.6)
	/usr/lib64/libunique-1.0.so.0.100.6
	/usr/lib64/libunique-3.0.so         (link to  
/usr/lib64/libunique-3.0.so.0.0.2)
	/usr/lib64/libunique-3.0.so.0       (link to  
/usr/lib64/libunique-3.0.so.0.0.2)
	/usr/lib64/libunique-3.0.so.0.0.2
As you can see there is actually the full version down to the tiniest  
minor version appended to the file name and several layers of coarser  
versioning. An application can now link against libunique-1.0.so to get  
the old API and /usr/lib64/libunique-3.0.so to get the new API.

The same has to happen with druntime and Phobos2 or otherwise our programs  
will break with every new release that deprecates or changes non-template  
functions. That would probably be *every* release at the moment, so it  
could look like this:
/usr/lib64/libphobos2.so     (link to /usr/lib64/libphobos2.so.060)
/usr/lib64/libphobos2.so.058
/usr/lib64/libphobos2.so.059
/usr/lib64/libphobos2.so.060
/usr/lib64/libdruntime.so     (link to /usr/lib64/libdruntime.so.060)
/usr/lib64/libdruntime.so.058
/usr/lib64/libdruntime.so.059
/usr/lib64/libdruntime.so.060

There are two steps involved in getting this out of the door now:
1) I'm not an expert with these things, but from the looks of it, I think  
the Makefile should handle appending the version string
2) The runtime should be downloadable as a separate package (like the  
famous MSVC Runtime Redistributables)
Developers have three choices then:
- static linking
- packaging the so/dll with their application (always using the  
tested-works version)
- use the system installation of druntime/Phobos2 (benefit from patches  
(as far as WinSxS doesn't intervene))

-- Marco
February 03, 2012
Re: libphobos.so libdruntime.so
On Friday, February 03, 2012 02:29:20 Marco Leise wrote:
> As time goes by the D runtime will have it's place on Unix systems next to
> the C runtime. The upcoming support for PIC is the next step. Now I just
> want to quickly raise awareness for "sonames". For any library, there may
> be several incompatible versions installed, since new features are added
> and old features a deprecated. This happens in Phobos as well, no doubt
> 
> :). So the solution is to put the version in the library name as well, so
> 
> they can coexist in the same directory (/usr/lib).
> 
> Here is an example for the two versions of libunique I have installed:
> 	/usr/lib64/libunique-1.0.so         (link to
> /usr/lib64/libunique-1.0.so.0.100.6)
> 	/usr/lib64/libunique-1.0.so.0       (link to
> /usr/lib64/libunique-1.0.so.0.100.6)
> 	/usr/lib64/libunique-1.0.so.0.100.6
> 	/usr/lib64/libunique-3.0.so         (link to
> /usr/lib64/libunique-3.0.so.0.0.2)
> 	/usr/lib64/libunique-3.0.so.0       (link to
> /usr/lib64/libunique-3.0.so.0.0.2)
> 	/usr/lib64/libunique-3.0.so.0.0.2
> As you can see there is actually the full version down to the tiniest
> minor version appended to the file name and several layers of coarser
> versioning. An application can now link against libunique-1.0.so to get
> the old API and /usr/lib64/libunique-3.0.so to get the new API.
> 
> The same has to happen with druntime and Phobos2 or otherwise our programs
> will break with every new release that deprecates or changes non-template
> functions. That would probably be *every* release at the moment, so it
> could look like this:
> /usr/lib64/libphobos2.so     (link to /usr/lib64/libphobos2.so.060)
> /usr/lib64/libphobos2.so.058
> /usr/lib64/libphobos2.so.059
> /usr/lib64/libphobos2.so.060
> /usr/lib64/libdruntime.so     (link to /usr/lib64/libdruntime.so.060)
> /usr/lib64/libdruntime.so.058
> /usr/lib64/libdruntime.so.059
> /usr/lib64/libdruntime.so.060
> 
> There are two steps involved in getting this out of the door now:
> 1) I'm not an expert with these things, but from the looks of it, I think
> the Makefile should handle appending the version string
> 2) The runtime should be downloadable as a separate package (like the
> famous MSVC Runtime Redistributables)
> Developers have three choices then:
> - static linking
> - packaging the so/dll with their application (always using the
> tested-works version)
> - use the system installation of druntime/Phobos2 (benefit from patches
> (as far as WinSxS doesn't intervene))

I would point out that druntime is bundled with libphobos.a. I wouldn't expect 
libdruntime to be on the system.

- Jonathan M Davis
February 03, 2012
Re: libphobos.so libdruntime.so
On Thu, Feb 02, 2012 at 05:33:49PM -0800, Jonathan M Davis wrote:
[...]
> I would point out that druntime is bundled with libphobos.a. I
> wouldn't expect libdruntime to be on the system.
[...]

Are there any *good* reasons why druntime and libphobos are not
dynamically linked? In the long run, we need to support that, since
otherwise D binaries will be unnecessarily large and the OS won't be
able to optimize memory usage by sharing library images with multiple
processes.


T

-- 
BREAKFAST.COM halted...Cereal Port Not Responding. -- YHL
February 03, 2012
Re: libphobos.so libdruntime.so
On Thursday, February 02, 2012 18:34:34 H. S. Teoh wrote:
> On Thu, Feb 02, 2012 at 05:33:49PM -0800, Jonathan M Davis wrote:
> [...]
> 
> > I would point out that druntime is bundled with libphobos.a. I
> > wouldn't expect libdruntime to be on the system.
> 
> [...]
> 
> Are there any *good* reasons why druntime and libphobos are not
> dynamically linked? In the long run, we need to support that, since
> otherwise D binaries will be unnecessarily large and the OS won't be
> able to optimize memory usage by sharing library images with multiple
> processes.

Because dmd does not yet support shared libraries on Linux.

- Jonathan M Davis
February 03, 2012
Re: libphobos.so libdruntime.so
Am 03.02.2012, 03:34 Uhr, schrieb H. S. Teoh <hsteoh@quickfur.ath.cx>:

> Are there any *good* reasons why druntime and libphobos are not
> dynamically linked? In the long run, we need to support that, since
> otherwise D binaries will be unnecessarily large and the OS won't be
> able to optimize memory usage by sharing library images with multiple
> processes.
>
>
> T

No fear, the people in charge know about all that, it was technical  
reasons that held back the support. That said, there are people who prefer  
static linking. May they speak for themselves...
February 03, 2012
Re: libphobos.so libdruntime.so
On Friday, February 03, 2012 04:27:37 Marco Leise wrote:
> Am 03.02.2012, 03:34 Uhr, schrieb H. S. Teoh <hsteoh@quickfur.ath.cx>:
> > Are there any *good* reasons why druntime and libphobos are not
> > dynamically linked? In the long run, we need to support that, since
> > otherwise D binaries will be unnecessarily large and the OS won't be
> > able to optimize memory usage by sharing library images with multiple
> > processes.
> > 
> > 
> > T
> 
> No fear, the people in charge know about all that, it was technical
> reasons that held back the support. That said, there are people who prefer
> static linking. May they speak for themselves...

Dynamic linking is evil. Static linking is _way_ better when you can do it. 
The problem is, of course, that you often need dynamic linking for a variety 
of reasons (saving memory being one of them).

- Jonathan M Davis
February 03, 2012
Re: libphobos.so libdruntime.so
On Friday, 3 February 2012 at 03:40:07 UTC, Jonathan M Davis 
wrote:
> Dynamic linking is evil.

Amen!

> dynamic linking for a variety of reasons (saving memory being 
> one of them).

Smart operating systems don't even need it for this;
they can do de-duplication of identical pages in both
memory and on the drive.
February 03, 2012
Re: libphobos.so libdruntime.so
Am 03.02.2012, 04:42 Uhr, schrieb Adam D. Ruppe  
<destructionator@gmail.com>:

> On Friday, 3 February 2012 at 03:40:07 UTC, Jonathan M Davis wrote:
>> Dynamic linking is evil.
>
> Amen!
>
>> dynamic linking for a variety of reasons (saving memory being one of  
>> them).
>
> Smart operating systems don't even need it for this;
> they can do de-duplication of identical pages in both
> memory and on the drive.

Are you serious? I have seen a flag in Linux for hosting virtual machines,  
but it must be an enormous overhead to check every executable page or  
every executable file on the file system for duplicates. If this were for  
free, it would be great, but the most spread OSes today and in the  
foreseeable future wont have filesystems that do something like automatic  
hardlinks of duplicate pages in executables. Both the file system and more  
importantly executable formats aren't ready for this. Let's imagine the  
library to link against against was Gtk. Would you want every binary  
download on the internet to include libraries of that size?
The best we have for the job is dynamic linking. Personally I think it is  
quite good, but you have to be careful as a library author to properly  
version API differences.
February 03, 2012
Re: libphobos.so libdruntime.so
P.S.: Another good example for the benefit of size reduction are filter  
plugins for multimedia application. Those are usually a dozen small  
libraries with a few lines of code, where the ratio of own code vs runtime  
can quickly run out of proportion. I think someone here mentioned writing  
plugins for some VST program.
February 03, 2012
Re: libphobos.so libdruntime.so
On Friday, 3 February 2012 at 04:23:34 UTC, Marco Leise wrote:
> I have seen a flag in Linux for hosting virtual machines, but 
> it must be an enormous overhead to check every executable page 
> or every executable file on the file system for duplicates.

tbh I haven't benchmarked it, but one of the cloud
providers I worked on used it on Solaris, and we had
a good experience.

> great, but the most spread OSes today and in the foreseeable 
> future wont have filesystems that do something like automatic 
> hardlinks of duplicate pages in executables.

Regardless, most computers today aren't exactly
counting their kilobytes either. (And those that are
don't multitask much anyway.)

Phobos, in a typical D program, consumes a few hundred
kb. Maybe if you're running 1,000 D processes at once will
it become a problem... but that's not a realistic scenario.

Distributing standalone D programs, or having D programs that
use different versions of Phobos *is* something we face. With
static linking, this isn't a problem. It's mindlessly easy.

> Let's imagine the library to link against against was Gtk. 
> Would you want every binary download on the internet to include 
> libraries of that size?

Yes, actually. I'd rather download a 10 MB executable that
*actually works* than have to download a separate library,
and hope I get the right version.

(And 10 MB is probably an overestimate! I've used statically
linked Qt apps that come in at only about 6 MB.)


There's times when dynamic linking is a good call. Core
operating system libraries, plugins, stuff like that.

But, Phobos isn't one of them, and I doubt it ever will be.
« First   ‹ Prev
1 2 3
Top | Discussion index | About this forum | D home