Guys (and gals, if there are any yet on the D ng)

I've dug out the extent of my work on the D dynamic library loader, and I'm keen to get some input wrt the overall design before implementing it for Linux. I encountered these issues several years ago while designing on a platform-independent component-based parser architecture  - <modesty-break>which went on to become the world standard for the (>50% global market share) client</modesty-break>.

Basically, on Win32 one can effect reference counting on dynamic libraries via the LoadLibrary, FreeLibrary and GetModuleFileName functions.

 - you load the library via LoadLibrary
 - you "release" the library via FreeLibrary
 - you "increment" a reference via calls to GetModuleFileName and then
LoadLibrary

When working with dynamic architectures, it's sometimes really useful to be able to have client-side (i.e. in your code, rather than just relying on the system) reference-counted dynamic loading. Unfortunately, UNIX systems don't provide the analogous functionality. The way I've worked around this in the past (in C++ land) is to have a centralising API, which is the only port of call for library loading and unloading, e.g.

  HXModule ExeModule_Load(char const *name);
  HXModule ExeModule_AddRef(HXModule hxmod);
  HXModule ExeModule_Release(HXModule hxmod);
  void            *ExeModule_GetSymbol(HXModule hxmod, char const *symbol);

In non-Win32 systems, the library maintains internal records that associate the module handle with the underlying operating system handle, and either implements an internal reference-count, or associates the module handle with the module name. Naturally, we'd use associative arrays for speed, but even linear management is not going to be troublesome given the reasonable dynamic library limits of a given process, and the relative costs of the underlying OS loading call. In fact, if we internalise the reference-count and only do a single load/free, this'll probably be faster.

This is what I'd like to do for D, so that there's a single API for all concerned. Naturally, being as how we're all supposed to be heart-and-soul Object Obsessed Desparados these days, I will also provide a wrapping auto class along with the function-based API.

I'd appreciate any feedback you may wish to give.

Matthew

P.S. Happy New Year to all, btw.

Too late chaps and chappettes - I've finished it! (It shows how much writing a book can adle the mind, as this was my way of taking a morning off in celebration of it being a new year. <g> )

It's fully platform-independent, and works with both the Win32 LoadLibrary and Linux dlopen APIs (albeit that I tested it on Win32 with my UNIX Emulation lib).

The module's std.loader, and contains an API, and a thin class over the top of that.

    // Functions
    typedef void    *HXModule;

    public int ExeModule_Init(); // Called internally, in static
initialisation
    public void ExeModule_Uninit(); // Called internally, in static
un-initialisation

    public HXModule ExeModule_Load(in char[] moduleName);
    public HXModule ExeModule_AddRef(in HXModule hModule);
    public void ExeModule_Release(inout HXModule hModule);

    public void *ExeModule_GetSymbol(inout HXModule hModule, in char[]
symbolName);

    public char[] ExeModule_Error();


    // Class(es)

    class ExeModuleException;

    auto class ExeModule
    {
        this(inout HXModule hModule, boolean bTakeOwnership);
        this(char[] moduleName); // Throws ExeModuleException
        ~this();
        void Close();

        void *GetSymbol(in char[] symbolName); // Throws ExeModuleException
        void *FindSymbol(in char[] symbolName); // Returns NULL if not there

        HXModule Handle();
        char[] Path();
    }


I've just sent it to Walter, so hopefully it might make it into Phobos for 0.77. It's even got comments!

The one caveat is that it's not thread-safe in the Linux version, as there are several threading aspects I'm hoping to get Walter to include in D in the near future that will make most of the issues moot.

Now back to that damn book ...

Cheers

Matthew



"Matthew" <matthew.hat@stlsoft.dot.org> wrote in message news:bsvmh0$2gap$1@digitaldaemon.com...
> Guys (and gals, if there are any yet on the D ng)
>
> I've dug out the extent of my work on the D dynamic library loader, and
I'm
> keen to get some input wrt the overall design before implementing it for Linux. I encountered these issues several years ago while designing on a platform-independent component-based parser architecture  - <modesty-break>which went on to become the world standard for the (>50% global market share) client</modesty-break>.
>
> Basically, on Win32 one can effect reference counting on dynamic libraries via the LoadLibrary, FreeLibrary and GetModuleFileName functions.
>
>  - you load the library via LoadLibrary
>  - you "release" the library via FreeLibrary
>  - you "increment" a reference via calls to GetModuleFileName and then
> LoadLibrary
>
> When working with dynamic architectures, it's sometimes really useful to
be
> able to have client-side (i.e. in your code, rather than just relying on
the
> system) reference-counted dynamic loading. Unfortunately, UNIX systems
don't
> provide the analogous functionality. The way I've worked around this in
the
> past (in C++ land) is to have a centralising API, which is the only port
of
> call for library loading and unloading, e.g.
>
>   HXModule ExeModule_Load(char const *name);
>   HXModule ExeModule_AddRef(HXModule hxmod);
>   HXModule ExeModule_Release(HXModule hxmod);
>   void            *ExeModule_GetSymbol(HXModule hxmod, char const
*symbol);
>
> In non-Win32 systems, the library maintains internal records that
associate
> the module handle with the underlying operating system handle, and either implements an internal reference-count, or associates the module handle
with
> the module name. Naturally, we'd use associative arrays for speed, but
even
> linear management is not going to be troublesome given the reasonable dynamic library limits of a given process, and the relative costs of the underlying OS loading call. In fact, if we internalise the reference-count and only do a single load/free, this'll probably be faster.
>
> This is what I'd like to do for D, so that there's a single API for all concerned. Naturally, being as how we're all supposed to be heart-and-soul Object Obsessed Desparados these days, I will also provide a wrapping auto class along with the function-based API.
>
> I'd appreciate any feedback you may wish to give.
>
> Matthew
>
> P.S. Happy New Year to all, btw.
>
>