Thread overview | ||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
February 03, 2012 libphobos.so libdruntime.so | ||||
---|---|---|---|---|
| ||||
As time goes by the D runtime will have it's place on Unix systems next to the C runtime. The upcoming support for PIC is the next step. Now I just want to quickly raise awareness for "sonames". For any library, there may be several incompatible versions installed, since new features are added and old features a deprecated. This happens in Phobos as well, no doubt :). So the solution is to put the version in the library name as well, so they can coexist in the same directory (/usr/lib). Here is an example for the two versions of libunique I have installed: /usr/lib64/libunique-1.0.so (link to /usr/lib64/libunique-1.0.so.0.100.6) /usr/lib64/libunique-1.0.so.0 (link to /usr/lib64/libunique-1.0.so.0.100.6) /usr/lib64/libunique-1.0.so.0.100.6 /usr/lib64/libunique-3.0.so (link to /usr/lib64/libunique-3.0.so.0.0.2) /usr/lib64/libunique-3.0.so.0 (link to /usr/lib64/libunique-3.0.so.0.0.2) /usr/lib64/libunique-3.0.so.0.0.2 As you can see there is actually the full version down to the tiniest minor version appended to the file name and several layers of coarser versioning. An application can now link against libunique-1.0.so to get the old API and /usr/lib64/libunique-3.0.so to get the new API. The same has to happen with druntime and Phobos2 or otherwise our programs will break with every new release that deprecates or changes non-template functions. That would probably be *every* release at the moment, so it could look like this: /usr/lib64/libphobos2.so (link to /usr/lib64/libphobos2.so.060) /usr/lib64/libphobos2.so.058 /usr/lib64/libphobos2.so.059 /usr/lib64/libphobos2.so.060 /usr/lib64/libdruntime.so (link to /usr/lib64/libdruntime.so.060) /usr/lib64/libdruntime.so.058 /usr/lib64/libdruntime.so.059 /usr/lib64/libdruntime.so.060 There are two steps involved in getting this out of the door now: 1) I'm not an expert with these things, but from the looks of it, I think the Makefile should handle appending the version string 2) The runtime should be downloadable as a separate package (like the famous MSVC Runtime Redistributables) Developers have three choices then: - static linking - packaging the so/dll with their application (always using the tested-works version) - use the system installation of druntime/Phobos2 (benefit from patches (as far as WinSxS doesn't intervene)) -- Marco |
February 03, 2012 Re: libphobos.so libdruntime.so | ||||
---|---|---|---|---|
| ||||
Posted in reply to Marco Leise | On Friday, February 03, 2012 02:29:20 Marco Leise wrote:
> As time goes by the D runtime will have it's place on Unix systems next to the C runtime. The upcoming support for PIC is the next step. Now I just want to quickly raise awareness for "sonames". For any library, there may be several incompatible versions installed, since new features are added and old features a deprecated. This happens in Phobos as well, no doubt
>
> :). So the solution is to put the version in the library name as well, so
>
> they can coexist in the same directory (/usr/lib).
>
> Here is an example for the two versions of libunique I have installed:
> /usr/lib64/libunique-1.0.so (link to
> /usr/lib64/libunique-1.0.so.0.100.6)
> /usr/lib64/libunique-1.0.so.0 (link to
> /usr/lib64/libunique-1.0.so.0.100.6)
> /usr/lib64/libunique-1.0.so.0.100.6
> /usr/lib64/libunique-3.0.so (link to
> /usr/lib64/libunique-3.0.so.0.0.2)
> /usr/lib64/libunique-3.0.so.0 (link to
> /usr/lib64/libunique-3.0.so.0.0.2)
> /usr/lib64/libunique-3.0.so.0.0.2
> As you can see there is actually the full version down to the tiniest
> minor version appended to the file name and several layers of coarser
> versioning. An application can now link against libunique-1.0.so to get
> the old API and /usr/lib64/libunique-3.0.so to get the new API.
>
> The same has to happen with druntime and Phobos2 or otherwise our programs
> will break with every new release that deprecates or changes non-template
> functions. That would probably be *every* release at the moment, so it
> could look like this:
> /usr/lib64/libphobos2.so (link to /usr/lib64/libphobos2.so.060)
> /usr/lib64/libphobos2.so.058
> /usr/lib64/libphobos2.so.059
> /usr/lib64/libphobos2.so.060
> /usr/lib64/libdruntime.so (link to /usr/lib64/libdruntime.so.060)
> /usr/lib64/libdruntime.so.058
> /usr/lib64/libdruntime.so.059
> /usr/lib64/libdruntime.so.060
>
> There are two steps involved in getting this out of the door now:
> 1) I'm not an expert with these things, but from the looks of it, I think
> the Makefile should handle appending the version string
> 2) The runtime should be downloadable as a separate package (like the
> famous MSVC Runtime Redistributables)
> Developers have three choices then:
> - static linking
> - packaging the so/dll with their application (always using the
> tested-works version)
> - use the system installation of druntime/Phobos2 (benefit from patches
> (as far as WinSxS doesn't intervene))
I would point out that druntime is bundled with libphobos.a. I wouldn't expect libdruntime to be on the system.
- Jonathan M Davis
|
February 03, 2012 Re: libphobos.so libdruntime.so | ||||
---|---|---|---|---|
| ||||
On Thu, Feb 02, 2012 at 05:33:49PM -0800, Jonathan M Davis wrote: [...] > I would point out that druntime is bundled with libphobos.a. I wouldn't expect libdruntime to be on the system. [...] Are there any *good* reasons why druntime and libphobos are not dynamically linked? In the long run, we need to support that, since otherwise D binaries will be unnecessarily large and the OS won't be able to optimize memory usage by sharing library images with multiple processes. T -- BREAKFAST.COM halted...Cereal Port Not Responding. -- YHL |
February 03, 2012 Re: libphobos.so libdruntime.so | ||||
---|---|---|---|---|
| ||||
On Thursday, February 02, 2012 18:34:34 H. S. Teoh wrote:
> On Thu, Feb 02, 2012 at 05:33:49PM -0800, Jonathan M Davis wrote: [...]
>
> > I would point out that druntime is bundled with libphobos.a. I wouldn't expect libdruntime to be on the system.
>
> [...]
>
> Are there any *good* reasons why druntime and libphobos are not dynamically linked? In the long run, we need to support that, since otherwise D binaries will be unnecessarily large and the OS won't be able to optimize memory usage by sharing library images with multiple processes.
Because dmd does not yet support shared libraries on Linux.
- Jonathan M Davis
|
February 03, 2012 Re: libphobos.so libdruntime.so | ||||
---|---|---|---|---|
| ||||
Posted in reply to H. S. Teoh | Am 03.02.2012, 03:34 Uhr, schrieb H. S. Teoh <hsteoh@quickfur.ath.cx>:
> Are there any *good* reasons why druntime and libphobos are not
> dynamically linked? In the long run, we need to support that, since
> otherwise D binaries will be unnecessarily large and the OS won't be
> able to optimize memory usage by sharing library images with multiple
> processes.
>
>
> T
No fear, the people in charge know about all that, it was technical reasons that held back the support. That said, there are people who prefer static linking. May they speak for themselves...
|
February 03, 2012 Re: libphobos.so libdruntime.so | ||||
---|---|---|---|---|
| ||||
Posted in reply to Marco Leise | On Friday, February 03, 2012 04:27:37 Marco Leise wrote:
> Am 03.02.2012, 03:34 Uhr, schrieb H. S. Teoh <hsteoh@quickfur.ath.cx>:
> > Are there any *good* reasons why druntime and libphobos are not dynamically linked? In the long run, we need to support that, since otherwise D binaries will be unnecessarily large and the OS won't be able to optimize memory usage by sharing library images with multiple processes.
> >
> >
> > T
>
> No fear, the people in charge know about all that, it was technical reasons that held back the support. That said, there are people who prefer static linking. May they speak for themselves...
Dynamic linking is evil. Static linking is _way_ better when you can do it. The problem is, of course, that you often need dynamic linking for a variety of reasons (saving memory being one of them).
- Jonathan M Davis
|
February 03, 2012 Re: libphobos.so libdruntime.so | ||||
---|---|---|---|---|
| ||||
Posted in reply to Jonathan M Davis | On Friday, 3 February 2012 at 03:40:07 UTC, Jonathan M Davis wrote: > Dynamic linking is evil. Amen! > dynamic linking for a variety of reasons (saving memory being one of them). Smart operating systems don't even need it for this; they can do de-duplication of identical pages in both memory and on the drive. |
February 03, 2012 Re: libphobos.so libdruntime.so | ||||
---|---|---|---|---|
| ||||
Posted in reply to Adam D. Ruppe | Am 03.02.2012, 04:42 Uhr, schrieb Adam D. Ruppe <destructionator@gmail.com>:
> On Friday, 3 February 2012 at 03:40:07 UTC, Jonathan M Davis wrote:
>> Dynamic linking is evil.
>
> Amen!
>
>> dynamic linking for a variety of reasons (saving memory being one of them).
>
> Smart operating systems don't even need it for this;
> they can do de-duplication of identical pages in both
> memory and on the drive.
Are you serious? I have seen a flag in Linux for hosting virtual machines, but it must be an enormous overhead to check every executable page or every executable file on the file system for duplicates. If this were for free, it would be great, but the most spread OSes today and in the foreseeable future wont have filesystems that do something like automatic hardlinks of duplicate pages in executables. Both the file system and more importantly executable formats aren't ready for this. Let's imagine the library to link against against was Gtk. Would you want every binary download on the internet to include libraries of that size?
The best we have for the job is dynamic linking. Personally I think it is quite good, but you have to be careful as a library author to properly version API differences.
|
February 03, 2012 Re: libphobos.so libdruntime.so | ||||
---|---|---|---|---|
| ||||
Posted in reply to Marco Leise | P.S.: Another good example for the benefit of size reduction are filter plugins for multimedia application. Those are usually a dozen small libraries with a few lines of code, where the ratio of own code vs runtime can quickly run out of proportion. I think someone here mentioned writing plugins for some VST program. |
February 03, 2012 Re: libphobos.so libdruntime.so | ||||
---|---|---|---|---|
| ||||
Posted in reply to Marco Leise | On Friday, 3 February 2012 at 04:23:34 UTC, Marco Leise wrote: > I have seen a flag in Linux for hosting virtual machines, but it must be an enormous overhead to check every executable page or every executable file on the file system for duplicates. tbh I haven't benchmarked it, but one of the cloud providers I worked on used it on Solaris, and we had a good experience. > great, but the most spread OSes today and in the foreseeable future wont have filesystems that do something like automatic hardlinks of duplicate pages in executables. Regardless, most computers today aren't exactly counting their kilobytes either. (And those that are don't multitask much anyway.) Phobos, in a typical D program, consumes a few hundred kb. Maybe if you're running 1,000 D processes at once will it become a problem... but that's not a realistic scenario. Distributing standalone D programs, or having D programs that use different versions of Phobos *is* something we face. With static linking, this isn't a problem. It's mindlessly easy. > Let's imagine the library to link against against was Gtk. Would you want every binary download on the internet to include libraries of that size? Yes, actually. I'd rather download a 10 MB executable that *actually works* than have to download a separate library, and hope I get the right version. (And 10 MB is probably an overestimate! I've used statically linked Qt apps that come in at only about 6 MB.) There's times when dynamic linking is a good call. Core operating system libraries, plugins, stuff like that. But, Phobos isn't one of them, and I doubt it ever will be. |
Copyright © 1999-2021 by the D Language Foundation