--- The Problem ---

I was using Derelict to play around with SDL/OpenGL in D for a bit, and I was initially unable to do so because DerelictSDL2's search paths for .so files could not find libSDL2_ttf.so on my Linux distribution (Gentoo).

This seems quite solvable these days, so I entered an issue on DerelictSDL2's issue tracker.  As I typed away, it became apparent that this is probably something that the entire D community should care about: intelligent lookup of a system's .so files (or lack thereof) could have a profound impact on user experience for anyone on Linux using programs written in D.  Mike (aldacron) responded with a request to gather input from the D community at large, so here it is ;)

The original issue tracker is here:
https://github.com/DerelictOrg/DerelictSDL2/issues/28

--- One Solution ---

As an initial proposal, how about this:
1. Look in ./
2. Look in $LD_LIBRARY_PATH
3. Look in all directories listed in /etc/ld.so.conf
4. Look in a baked-in list of last-ditch fallback options. Although I'm sure better could be done, my current suggestion would look like this:
4a. ~/lib
4b. /usr/local/lib
4c. /usr/lib

This logic should probably end up in Phobos or somewhere suitably common, and revised as needed, so that all D programs can benefit from the community's research and experience on the subject.

As a data point: on my Gentoo box, the truly important step (usually) is to look in '/etc/ld.so.conf'.  My machine has a fairly comprehensive list in that file, and LD_LIBRARY_PATH doesn't have squat.

It may be desirable to first search folders or specific paths that are explicitly provided by the caller, and possibly skip the default search paths entirely if the caller insists.  I'm not sure how that would look, since the explicit path would probably be passed to a (non-Phobos) library-specific loading function (ex: DerelictSDL2.load(...)) before making its way to any community ordained logic.

In retrospect, I also have some reservations about looking in './'.  I am not sure what the security implications are.  If there are no security concerns, then I, as both a developer and a user, would intuitively want D programs to look in ./ first (unless, perhaps, the program has an explicit search path defined, such as a packaged folder full of .so files, then /maybe/ those should go first).  As a developer, I might want to drop my program's .so/.dll files into the same directory as the executable, and I'd expect it to "just work". This is important whenever package managers can't be relied upon to deliver a correct version of the .so (ex: for obscure libraries), or can't be relied upon to release the program at all.  As a user, I sometimes want to replace .so/.dll files with different versions (ex: if the developer's version links to symbols or is compiled with features that don't exist on my system and are not strictly needed by the program).



Is this a reasonable approach, or is there a more appropriate way to handle this search?

El 17/09/14 a les 07:52, Chad Joan via Digitalmars-d ha escrit:
> I was using Derelict to play around with SDL/OpenGL in D for a bit, and I was initially unable to do so because DerelictSDL2's search paths for .so files could not find libSDL2_ttf.so on my Linux distribution (Gentoo).

.so files are shared libraries on Linux. These libraries should be available on specific directories listed in "/etc/ld.so.conf" and "/etc/ld.so.conf.d/*" files. Place your shared libraries in some of these paths and then run "ldconfig" command to create symlinks to the latest libraries version and to update the "/etc/ld.so.cache" cache file. This is not specific for D but for all Linux programs.

Regards,
-- 
Jordi Sayol

On Wednesday, 17 September 2014 at 05:52:29 UTC, Chad Joan wrote:
> --- The Problem ---
>
> I was using Derelict to play around with SDL/OpenGL in D for a bit, and I was initially unable to do so because DerelictSDL2's search paths for .so files could not find libSDL2_ttf.so on my Linux distribution (Gentoo).
>
If you're not using the slotted ebuild from:
https://bugs.gentoo.org/show_bug.cgi?id=481788
...you won't have that so file; only the version that links with SDL 1.2.  And if we look:

wyatt@Yue ~ $ ls -F -w66 /usr/lib/libSDL* |grep ttf
/usr/lib/libSDL_ttf-2.0.so.0@
/usr/lib/libSDL_ttf-2.0.so.0.10.1*
/usr/lib/libSDL_ttf.so@

Sure enough!  I'd suggest pinging that bug because SDL2 isn't going away any time soon.

> intelligent lookup of a system's .so files (or lack thereof) could have a profound impact on user experience for anyone on Linux using programs written in D.  Mike (aldacron) responded with a request to gather input from the D community at large, so here it is ;)
>
While I certainly couldn't blame you for harshing on the dynamic linker most days, this (surprisingly) isn't one of them.  It's innocent.  Shocking, I know.

> As an initial proposal, how about this:
> 1. Look in ./
> 2. Look in $LD_LIBRARY_PATH
> 3. Look in all directories listed in /etc/ld.so.conf
> 4. Look in a baked-in list of last-ditch fallback options. Although I'm sure better could be done, my current suggestion would look like this:
> 4a. ~/lib
> 4b. /usr/local/lib
> 4c. /usr/lib
>
As it happens, this is almost exactly what actually happens:
http://unix.stackexchange.com/a/22999

> This logic should probably end up in Phobos or somewhere suitably common, and revised as needed, so that all D programs can benefit from the community's research and experience on the subject.
>
On Linux (and OSX?), at least, E_NOTOURPROBLEM.  I possibly recall there was some amount of hilarity related to this on Windows, but that's an area where I Just Don't Care™.

-Wyatt

I presume you mean dynamically loaded .so libraries and not just dynamically linked? Because latter use the same library path resolution rules as in C world (ld takes care of that)

As for dynamically loaded ones - it is somewhat tough question. Loading non-local plugin .so libraries is quite unusual, Derelict is the only software I know that does it. I am not sure it is even a good idea in general.

Are there any examples how this is solved in other languages / platforms?

On Wed, Sep 17, 2014 at 04:43:04PM +0000, Dicebot via Digitalmars-d wrote:
> I presume you mean dynamically loaded .so libraries and not just dynamically linked? Because latter use the same library path resolution rules as in C world (ld takes care of that)
> 
> As for dynamically loaded ones - it is somewhat tough question. Loading non-local plugin .so libraries is quite unusual, Derelict is the only software I know that does it. I am not sure it is even a good idea in general.
> 
> Are there any examples how this is solved in other languages / platforms?

In C / Posix, you can load any library dynamically using dlopen(), dlsym(), et al, manually "link" your program by extracting function pointers and data pointers. Some applications use this to implement a plugin mechanism -- some Linux media players do this, I believe, and browsers probably also use the same mechanism for browser plugins.

In any case, dlopen() uses the OS's dynamic linker to find libraries, so it's really a matter of setting up your OS environment to be able to find the libraries. It's not really within the scope of the D runtime to attempt to configure this. Furthermore, some application plugin frameworks may do things like fiddle with LD_LIBRARY_PATH to get it to look for libraries where the dynamic linker wouldn't usually look; in that case, you're really at the mercy of the plugin framework author(s) as to how to configure such lookups. Again, it's not really within the D runtime's mandate to control such things.

As for how it works on Windows, I have no idea at all. It's probably completely different from Posix, which is more reason to leave it up to plugin framework implementors to implement, rather than hard-coding an incomplete / inconsistent implementation in druntime.


T

-- 
One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie. -- The Silicon Valley Tarot

On Wednesday, 17 September 2014 at 17:13:07 UTC, H. S. Teoh via
Digitalmars-d wrote:

> As for how it works on Windows, I have no idea at all. It's probably
> completely different from Posix, which is more reason to leave it up to
> plugin framework implementors to implement, rather than hard-coding an
> incomplete / inconsistent implementation in druntime.
>
>
> T

FYI: Windows rules are here:
http://msdn.microsoft.com/en-us/library/windows/desktop/ms682586(v=vs.85).aspx