Thread overview | |||||||
---|---|---|---|---|---|---|---|
|
September 07, 2013 Merge pull request #593 from dawgfoto/dynamicLoading | ||||
---|---|---|---|---|
| ||||
We just merged the low-level implementation for full shared library support. There should be an article or some tutorial but I simply lack the time for that. So if you want to experiment with it you should read this and have a look at the test cases (https://github.com/D-Programming-Language/druntime/tree/master/test/shared). I'd be thankful for every bug report, please add the keyword dll (http://d.puremagic.com/issues/buglist.cgi?keywords=dll&resolution=---). Just a repost of the github comment (https://github.com/D-Programming-Language/druntime/pull/593#issuecomment-23913789). I updated and finished the low-level implementation. This now supports almost all shared library scenarios merely automatically. It is also tested pretty thoroughly. Here are the main gotchas for experimenting. There is an extra layer of thread local reference accounting over the global one in the runtime linker which has some implications on how to use shared libraries. Loading libraries should only be done through Runtime.loadLibrary/Runtime.unloadLibrary (or rt_loadLibrary/rt_unloadLibrary for C), so that the thread local accounting doesn't miss anything. The recommended pattern to load a D library from a pure C program is to dlopen phobos/druntime first and to use rt_loadLibrary/rt_unloadLibrary hereafter. Loading a library only initializes it and it's linked dependencies for the current thread. Threads where a library is not initialize will not see the library's modules, nor the EH tables and the GC will miss to scan the library's TLS memory. A spawned thread (core.thread.Thread) will initialize the same libraries as it's parent thread. There is an implicit dlopen call for every dynamically loaded library so to prevent the parent thread from unloading. When a thread finishes it will drop/unload all remaining references to loaded libraries. The high-level implementation of the shared libraries support will provide a method to iterate over all loaded libraries, thus it will be possible to explicitly unload unused inherited libraries. Finding the dependencies of a library is somewhat expensive so it is only performed for D libraries and non-recursively. Because all dependencies are loaded before the library itself we do know the recursive dependencies. In case of loading a C library that depends on D libraries or for D->C->D dependency chains we will miss the recursive dependency. This might possibly lead to uninitialized threads when loading such a library in more than one thread. I will try to fix any arising bugs ASAP but I don't have much time left currently. Please tag bugzilla entries with the dll keyword. Work on the high-level interface will start around the end of next week. |
September 07, 2013 Re: Merge pull request #593 from dawgfoto/dynamicLoading | ||||
---|---|---|---|---|
| ||||
Posted in reply to Martin Nowak | Am 07.09.2013 19:09, schrieb Martin Nowak:
> We just merged the low-level implementation for full shared library
> support. There should be an article or some tutorial but I simply lack
> the time for that. So if you want to experiment with it you should read
> this and have a look at the test cases
> (https://github.com/D-Programming-Language/druntime/tree/master/test/shared).
>
> I'd be thankful for every bug report, please add the keyword dll
> (http://d.puremagic.com/issues/buglist.cgi?keywords=dll&resolution=---).
>
>
> Just a repost of the github comment
> (https://github.com/D-Programming-Language/druntime/pull/593#issuecomment-23913789).
>
>
> I updated and finished the low-level implementation.
> This now supports almost all shared library scenarios merely automatically.
> It is also tested pretty thoroughly.
> Here are the main gotchas for experimenting.
>
> There is an extra layer of thread local reference accounting over the
> global one in the runtime linker which has some implications on how to
> use shared libraries.
>
> Loading libraries should only be done through
> Runtime.loadLibrary/Runtime.unloadLibrary
> (or rt_loadLibrary/rt_unloadLibrary for C), so that the thread local
> accounting doesn't
> miss anything.
> The recommended pattern to load a D library from a pure C program is to
> dlopen
> phobos/druntime first and to use rt_loadLibrary/rt_unloadLibrary hereafter.
>
> Loading a library only initializes it and it's linked dependencies for
> the current
> thread.
>
> Threads where a library is not initialize will not see the library's
> modules, nor the EH
> tables and the GC will miss to scan the library's TLS memory.
>
> A spawned thread (core.thread.Thread) will initialize the same libraries
> as it's parent
> thread. There is an implicit dlopen call for every dynamically loaded
> library so to
> prevent the parent thread from unloading. When a thread finishes it will
> drop/unload all
> remaining references to loaded libraries.
> The high-level implementation of the shared libraries support will
> provide a method to
> iterate over all loaded libraries, thus it will be possible to
> explicitly unload unused
> inherited libraries.
>
> Finding the dependencies of a library is somewhat expensive so it is
> only performed for D libraries and non-recursively.
>
> Because all dependencies are loaded before the library itself we do know
> the recursive dependencies. In case of loading a C library that depends
> on D libraries or for D->C->D dependency chains we will miss the
> recursive dependency.
> This might possibly lead to uninitialized threads when loading such a
> library in more than one thread.
> I will try to fix any arising bugs ASAP but I don't have much time left
> currently.
> Please tag bugzilla entries with the dll keyword.
>
> Work on the high-level interface will start around the end of next week.
Which plattforms does this work on?
|
September 07, 2013 Re: Merge pull request #593 from dawgfoto/dynamicLoading | ||||
---|---|---|---|---|
| ||||
Posted in reply to Benjamin Thaut | On 09/07/2013 07:15 PM, Benjamin Thaut wrote:
>
> Which plattforms does this work on?
Linux only for now
|
September 07, 2013 Re: Merge pull request #593 from dawgfoto/dynamicLoading | ||||
---|---|---|---|---|
| ||||
Posted in reply to Martin Nowak | On Saturday, 7 September 2013 at 17:09:17 UTC, Martin Nowak wrote:
> We just merged the low-level implementation for full shared library support. There should be an article or some tutorial but I simply lack the time for that. So if you want to experiment with it you should read this and have a look at the test cases (https://github.com/D-Programming-Language/druntime/tree/master/test/shared).
> I'd be thankful for every bug report, please add the keyword dll (http://d.puremagic.com/issues/buglist.cgi?keywords=dll&resolution=---).
>
>
> Just a repost of the github comment (https://github.com/D-Programming-Language/druntime/pull/593#issuecomment-23913789).
>
> I updated and finished the low-level implementation.
> This now supports almost all shared library scenarios merely automatically.
> It is also tested pretty thoroughly.
> Here are the main gotchas for experimenting.
>
> There is an extra layer of thread local reference accounting over the global one in the runtime linker which has some implications on how to use shared libraries.
>
> Loading libraries should only be done through Runtime.loadLibrary/Runtime.unloadLibrary
> (or rt_loadLibrary/rt_unloadLibrary for C), so that the thread local accounting doesn't
> miss anything.
> The recommended pattern to load a D library from a pure C program is to dlopen
> phobos/druntime first and to use rt_loadLibrary/rt_unloadLibrary hereafter.
>
> Loading a library only initializes it and it's linked dependencies for the current
> thread.
>
> Threads where a library is not initialize will not see the library's modules, nor the EH
> tables and the GC will miss to scan the library's TLS memory.
>
> A spawned thread (core.thread.Thread) will initialize the same libraries as it's parent
> thread. There is an implicit dlopen call for every dynamically loaded library so to
> prevent the parent thread from unloading. When a thread finishes it will drop/unload all
> remaining references to loaded libraries.
> The high-level implementation of the shared libraries support will provide a method to
> iterate over all loaded libraries, thus it will be possible to explicitly unload unused
> inherited libraries.
>
> Finding the dependencies of a library is somewhat expensive so it is only performed for D libraries and non-recursively.
>
> Because all dependencies are loaded before the library itself we do know the recursive dependencies. In case of loading a C library that depends on D libraries or for D->C->D dependency chains we will miss the recursive dependency.
> This might possibly lead to uninitialized threads when loading such a library in more than one thread.
> I will try to fix any arising bugs ASAP but I don't have much time left currently.
> Please tag bugzilla entries with the dll keyword.
>
> Work on the high-level interface will start around the end of next week.
Good job, Congrats!!
|
September 08, 2013 Re: Merge pull request #593 from dawgfoto/dynamicLoading | ||||
---|---|---|---|---|
| ||||
Posted in reply to Martin Nowak | Most awesome |
Copyright © 1999-2021 by the D Language Foundation