February 03, 2012
On Thu, Feb 02, 2012 at 07:19:37PM -0800, Jonathan M Davis wrote:
> On Thursday, February 02, 2012 18:34:34 H. S. Teoh wrote:
[...]
> > Are there any *good* reasons why druntime and libphobos are not dynamically linked? In the long run, we need to support that, since otherwise D binaries will be unnecessarily large and the OS won't be able to optimize memory usage by sharing library images with multiple processes.
> 
> Because dmd does not yet support shared libraries on Linux.
[...]

Hmph. Since gdc *does* support shared libraries (I think?), it should be possible, at least in principle, no?


T

-- 
I think Debian's doing something wrong, `apt-get install pesticide', doesn't seem to remove the bugs on my system! -- Mike Dresser
February 03, 2012
On Fri, 03 Feb 2012 05:47:28 +0100, H. S. Teoh <hsteoh@quickfur.ath.cx> wrote:

> On Thu, Feb 02, 2012 at 07:19:37PM -0800, Jonathan M Davis wrote:
>> On Thursday, February 02, 2012 18:34:34 H. S. Teoh wrote:
> [...]
>> > Are there any *good* reasons why druntime and libphobos are not
>> > dynamically linked? In the long run, we need to support that, since
>> > otherwise D binaries will be unnecessarily large and the OS won't be
>> > able to optimize memory usage by sharing library images with
>> > multiple processes.
>>
>> Because dmd does not yet support shared libraries on Linux.
> [...]
>
> Hmph. Since gdc *does* support shared libraries (I think?), it should be
> possible, at least in principle, no?
>
It does as much as dmd. The issues are with missing druntime support.
I have implemented that but we are still merging.
February 03, 2012
On 2012-02-03 02:33, Jonathan M Davis wrote:
> On Friday, February 03, 2012 02:29:20 Marco Leise wrote:
>> As time goes by the D runtime will have it's place on Unix systems next to
>> the C runtime. The upcoming support for PIC is the next step. Now I just
>> want to quickly raise awareness for "sonames". For any library, there may
>> be several incompatible versions installed, since new features are added
>> and old features a deprecated. This happens in Phobos as well, no doubt
>>
>> :). So the solution is to put the version in the library name as well, so
>>
>> they can coexist in the same directory (/usr/lib).
>>
>> Here is an example for the two versions of libunique I have installed:
>> 	/usr/lib64/libunique-1.0.so         (link to
>> /usr/lib64/libunique-1.0.so.0.100.6)
>> 	/usr/lib64/libunique-1.0.so.0       (link to
>> /usr/lib64/libunique-1.0.so.0.100.6)
>> 	/usr/lib64/libunique-1.0.so.0.100.6
>> 	/usr/lib64/libunique-3.0.so         (link to
>> /usr/lib64/libunique-3.0.so.0.0.2)
>> 	/usr/lib64/libunique-3.0.so.0       (link to
>> /usr/lib64/libunique-3.0.so.0.0.2)
>> 	/usr/lib64/libunique-3.0.so.0.0.2
>> As you can see there is actually the full version down to the tiniest
>> minor version appended to the file name and several layers of coarser
>> versioning. An application can now link against libunique-1.0.so to get
>> the old API and /usr/lib64/libunique-3.0.so to get the new API.
>>
>> The same has to happen with druntime and Phobos2 or otherwise our programs
>> will break with every new release that deprecates or changes non-template
>> functions. That would probably be *every* release at the moment, so it
>> could look like this:
>> /usr/lib64/libphobos2.so     (link to /usr/lib64/libphobos2.so.060)
>> /usr/lib64/libphobos2.so.058
>> /usr/lib64/libphobos2.so.059
>> /usr/lib64/libphobos2.so.060
>> /usr/lib64/libdruntime.so     (link to /usr/lib64/libdruntime.so.060)
>> /usr/lib64/libdruntime.so.058
>> /usr/lib64/libdruntime.so.059
>> /usr/lib64/libdruntime.so.060
>>
>> There are two steps involved in getting this out of the door now:
>> 1) I'm not an expert with these things, but from the looks of it, I think
>> the Makefile should handle appending the version string
>> 2) The runtime should be downloadable as a separate package (like the
>> famous MSVC Runtime Redistributables)
>> Developers have three choices then:
>> - static linking
>> - packaging the so/dll with their application (always using the
>> tested-works version)
>> - use the system installation of druntime/Phobos2 (benefit from patches
>> (as far as WinSxS doesn't intervene))
>
> I would point out that druntime is bundled with libphobos.a. I wouldn't expect
> libdruntime to be on the system.
>
> - Jonathan M Davis

If Phobos and druntime would be compiled as dynamic libraries both libphobos and libdruntime would be needed to be shipped.

-- 
/Jacob Carlborg
February 03, 2012
Marco Leise wrote:
> As time goes by the D runtime will have it's place on Unix systems next to the C runtime. The upcoming support for PIC is the next step. Now I just want to quickly raise awareness for "sonames". For any library, there may be several incompatible versions installed, since new features are added and old features a deprecated. This happens in Phobos as well, no doubt :). So the solution is to put the version in the library name as well, so they can coexist in the same directory (/usr/lib).
> 
> Here is an example for the two versions of libunique I have installed:
>     /usr/lib64/libunique-1.0.so         (link to
> /usr/lib64/libunique-1.0.so.0.100.6)
>     /usr/lib64/libunique-1.0.so.0       (link to
> /usr/lib64/libunique-1.0.so.0.100.6)
>     /usr/lib64/libunique-1.0.so.0.100.6
>     /usr/lib64/libunique-3.0.so         (link to
> /usr/lib64/libunique-3.0.so.0.0.2)
>     /usr/lib64/libunique-3.0.so.0       (link to
> /usr/lib64/libunique-3.0.so.0.0.2)
>     /usr/lib64/libunique-3.0.so.0.0.2
> As you can see there is actually the full version down to the tiniest
> minor version appended to the file name and several layers of coarser
> versioning. An application can now link against libunique-1.0.so to get
> the old API and /usr/lib64/libunique-3.0.so to get the new API.
> 
> The same has to happen with druntime and Phobos2 or otherwise our
> programs will break with every new release that deprecates or changes
> non-template functions. That would probably be *every* release at the
> moment, so it could look like this:
> /usr/lib64/libphobos2.so     (link to /usr/lib64/libphobos2.so.060)
> /usr/lib64/libphobos2.so.058
> /usr/lib64/libphobos2.so.059
> /usr/lib64/libphobos2.so.060
> /usr/lib64/libdruntime.so     (link to /usr/lib64/libdruntime.so.060)
> /usr/lib64/libdruntime.so.058
> /usr/lib64/libdruntime.so.059
> /usr/lib64/libdruntime.so.060
> 
> There are two steps involved in getting this out of the door now:
> 1) I'm not an expert with these things, but from the looks of it, I
> think the Makefile should handle appending the version string
> 2) The runtime should be downloadable as a separate package (like the
> famous MSVC Runtime Redistributables)
> Developers have three choices then:
> - static linking
> - packaging the so/dll with their application (always using the
> tested-works version)
> - use the system installation of druntime/Phobos2 (benefit from patches
> (as far as WinSxS doesn't intervene))
> 
	Do not forget to add the proper soname information *inside* the
files. With GNU ld you need these options: "-soname
libphobos2.so.060"...

		Jerome
-- 
mailto:jeberger@free.fr
http://jeberger.free.fr
Jabber: jeberger@jabber.fr



February 03, 2012
> The same has to happen with druntime and Phobos2 or otherwise our programs will break with every new release that deprecates or changes non-template functions. That would probably be *every* release at the moment, so it could look like this:
> /usr/lib64/libphobos2.so     (link to /usr/lib64/libphobos2.so.060)
> /usr/lib64/libphobos2.so.058
> /usr/lib64/libphobos2.so.059
> /usr/lib64/libphobos2.so.060
> /usr/lib64/libdruntime.so     (link to /usr/lib64/libdruntime.so.060)
> /usr/lib64/libdruntime.so.058
> /usr/lib64/libdruntime.so.059
> /usr/lib64/libdruntime.so.060

And you don't consider this insane?
The 'shared' approach is fine if a library has settled and is used pervasively like C runtime.
Also the library needs to have an appropriate development approach with major (feature) revisions and smaller non-breaking versions.
Phobos is a fast-moving target and doesn't fit in this model neither will it in the foreseeable future.

Why do people always treat D like a mainstream language? It isn't.
The chance that one has more than a few real D apps on one's machine is quite low. The chance that they use the very same version of phobos/druntime is even lower.
And usually the only ones you actually use are developed by yourself anyway.

I rather have a slightly bigger executable than having my system cluttered with hundreds of phobos versions I don't need.
And you should keep in mind that dmd's phobos is currently 17MB, gdc's is 25+5.5. Plus most apps only use a small share of that.


Making druntime shared sometime is ok I think, but it's just not ready yet. See the recent associative arrays dilemma. And the crappy GC.
February 03, 2012
On Fri, Feb 03, 2012 at 08:28:04PM +0100, Trass3r wrote: [...]
> Why do people always treat D like a mainstream language? It isn't.

But things like shared libraries that will become necessary once it becomes mainstream. Lack of shared library support is one of the barriers to it becoming mainstream (among many other things).


> The chance that one has more than a few real D apps on one's machine
> is quite low. The chance that they use the very same version of
> phobos/druntime is even lower.
> And usually the only ones you actually use are developed by yourself
> anyway.
> 
> I rather have a slightly bigger executable than having my system cluttered with hundreds of phobos versions I don't need.

Um, that's what you use a package manager for...


> And you should keep in mind that dmd's phobos is currently 17MB, gdc's is 25+5.5. Plus most apps only use a small share of that.
> 
> Making druntime shared sometime is ok I think, but it's just not ready yet. See the recent associative arrays dilemma. And the crappy GC.

It's strange, I noticed one thing about the D forums, and that is people keep complaining about this problem and that problem, this lack and that lack, and it seems very few are willing to actually do something about it. Like write some code and submit patches. I mean, this isn't some closed, proprietary system where you pretty much have to hope that the developers will implement what you want; the phobos/druntime source code and dmd frontend are open-source for a reason. And hopefully that reason isn't just so you feel all nice and warm inside. The source code is there so that people can look at it and more importantly improve it.

Same goes for lack of support for certain features that some regard as mainstream or critical. If library X doesn't exist yet, nothing stops anyone from writing it and submitting it for review. And even if it's not approved, nobody stops you from posting your code online somewhere and make it available as a 3rd party library. I find it utterly strange that people would just keep complaining about the lack of feature X yet it seems they don't want to write it either. If that's the case, then we're going nowhere. Nobody wants to implement feature X yet people are clamoring for feature X. Strange.


T

-- 
Never trust an operating system you don't have source for! -- Martin Schulze
February 03, 2012
> But things like shared libraries that will become necessary once it
> becomes mainstream. Lack of shared library support is one of the
> barriers to it becoming mainstream (among many other things).

Support for that is almost ready even in dmd.
You were talking about making phobos shared and that's a different thing.


>> I rather have a slightly bigger executable than having my system
>> cluttered with hundreds of phobos versions I don't need.
>
> Um, that's what you use a package manager for...

I didn't say anything about management.


>> And you should keep in mind that dmd's phobos is currently 17MB,
>> gdc's is 25+5.5. Plus most apps only use a small share of that.
>>
>> Making druntime shared sometime is ok I think, but it's just not
>> ready yet. See the recent associative arrays dilemma. And the crappy
>> GC.
>
> It's strange, I noticed one thing about the D forums, and that is people
> keep complaining about this problem and that problem, this lack and that
> lack, and it seems very few are willing to actually do something about
> it.

I wasn't complaining, I was stating that even druntime is far from being stable.
February 03, 2012
Am Fri, 03 Feb 2012 21:46:49 +0100
schrieb Trass3r <un@known.com>:

> > But things like shared libraries that will become necessary once it becomes mainstream. Lack of shared library support is one of the barriers to it becoming mainstream (among many other things).
> 
> Support for that is almost ready even in dmd.
> You were talking about making phobos shared and that's a different
> thing.
> 

There's probably no need to make phobos shared by default, but it should be supported. IIRC a shared runtime is necessary for dynamically loaded D plugins, plugins written in D to be used in C apps and similar stuff.

> >> I rather have a slightly bigger executable than having my system cluttered with hundreds of phobos versions I don't need.
> >
> > Um, that's what you use a package manager for...
> 
> I didn't say anything about management.
> 
> 
> >> And you should keep in mind that dmd's phobos is currently 17MB, gdc's is 25+5.5. Plus most apps only use a small share of that.
> >>
> >> Making druntime shared sometime is ok I think, but it's just not ready yet. See the recent associative arrays dilemma. And the crappy GC.
> >
> > It's strange, I noticed one thing about the D forums, and that is people keep complaining about this problem and that problem, this lack and that lack, and it seems very few are willing to actually do something about it.
> 
> I wasn't complaining, I was stating that even druntime is far from being stable.


February 04, 2012
Am 03.02.2012, 20:28 Uhr, schrieb Trass3r <un@known.com>:

>> The same has to happen with druntime and Phobos2 or otherwise our programs will break with every new release that deprecates or changes non-template functions. That would probably be *every* release at the moment, so it could look like this:
>> /usr/lib64/libphobos2.so     (link to /usr/lib64/libphobos2.so.060)
>> /usr/lib64/libphobos2.so.058
>> /usr/lib64/libphobos2.so.059
>> /usr/lib64/libphobos2.so.060
>> /usr/lib64/libdruntime.so     (link to /usr/lib64/libdruntime.so.060)
>> /usr/lib64/libdruntime.so.058
>> /usr/lib64/libdruntime.so.059
>> /usr/lib64/libdruntime.so.060
>
> And you don't consider this insane?

More with the wink of an eye, that in the event of shared libraries, deprecations of functions in every release will have this effect. :)

> The 'shared' approach is fine if a library has settled and is used pervasively like C runtime.
> Also the library needs to have an appropriate development approach with major (feature) revisions and smaller non-breaking versions.
> Phobos is a fast-moving target and doesn't fit in this model neither will it in the foreseeable future.
>
> Why do people always treat D like a mainstream language? It isn't.

How about having the option? Handling this correctly from the start on will reduce the likelyhood that someone considers D a toy language.

> The chance that one has more than a few real D apps on one's machine is quite low. The chance that they use the very same version of phobos/druntime is even lower.
> And usually the only ones you actually use are developed by yourself anyway.

I'm using software written in OCaml, Python, Java, C++ and Delphi and it is possible that there will be D apps I install through the package manager, once there are standard packages in Linux distributions. And I think that is more likely if D can come with the runtime as a shared library. And like Johannes said earlier, there may be situations where you depend on the availability of shared libraries.

> Making druntime shared sometime is ok I think, but it's just not ready yet. See the recent associative arrays dilemma. And the crappy GC.

Fair enough. I guess, I just don't want to say "I knew this would happen" later. :)
February 04, 2012
On 2012-02-03 20:55, H. S. Teoh wrote:
> On Fri, Feb 03, 2012 at 08:28:04PM +0100, Trass3r wrote:
> [...]
>> Why do people always treat D like a mainstream language? It isn't.
>
> But things like shared libraries that will become necessary once it
> becomes mainstream. Lack of shared library support is one of the
> barriers to it becoming mainstream (among many other things).
>
>
>> The chance that one has more than a few real D apps on one's machine
>> is quite low. The chance that they use the very same version of
>> phobos/druntime is even lower.
>> And usually the only ones you actually use are developed by yourself
>> anyway.
>>
>> I rather have a slightly bigger executable than having my system
>> cluttered with hundreds of phobos versions I don't need.
>
> Um, that's what you use a package manager for...

Even if you would use a package manager to install the applications/libraries it would still be cluttered with different versions of Phobos.

-- 
/Jacob Carlborg