I apologize if this has been asked before but I couldn't find help here or anywhere else.

I'm trying to load my code as a dynamic library in a C++ program. It is working, but I have a problem upon program exit with thread local storage.

What I'm trying to do:

The c++ loads my dynamic library from a thread (not the main thread) and when the application exits from the main thread (without properly unloading the library), the finalizers run in the main thread and do not know about the loaded library (because it's kept in TLS _loadedDSOs in the thread that loaded the library) but the handle to the library is recorded in __global _handleToDSOs. When the main thread exits the loaded libraries is compared to the open handles. Since the main thread thinks nothing is loaded (_loadedDSOs is empty) the code aborts because the handle _handleToDSOs is not empty. This causes the d runtime to abort. I've reduced the problem to this example code:

#include <dlfcn.h>
#include <pthread.h>
#include <stdbool.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>

void* loadlib(void*);

void main()
{
    int counter = 0;
    pthread_t thread1;
    pthread_attr_t attrs;

    pthread_attr_init(&attrs);

    pthread_create(&thread1, &attrs, &loadlib, NULL);

    while (true) {
        printf("Main %d\n", counter++);
        usleep(300000);
        if (counter == 5)
            break;
    }
}

void* loadlib(void* args)
{
    int counter = 100;

    void* dlib = dlopen("libdlibrary.so", RTLD_LAZY | RTLD_GLOBAL);
    if (dlib == NULL) {
        printf("Can't open dlib\n");
        exit(0);
    }

    void (*dfunc)() = dlsym(dlib, "dfunc");

    dfunc();

    while (true) {
        printf("Thread %d\n", counter++);
        usleep(700000);
    }
}

Compile with cc -o app app.c

And the D code as a library

import std.stdio;
import core.runtime;

extern (C) void dfunc() {
    writeln("Hello from D");
// rt_init();
}

Compile with: ldc2 -shared dlibrary.d

And run the app with: LD_LIBRARY_PATH=. ./app

This causes the crash.

I realize that this is not the correct way to do things, but the problem is, I can't change the way the C++ program works which is: It loads my library from a thread, but when it exits, it just quits the main thread. Things I can't do on the C++ program:

1. load the lib from the main thread. (My code is a plugin which only becomes active when it is being loaded by the thread)
2. gracefully stop the thread that loaded the library and unload the library
3. the c++ program never unloads the lib so pragma(crt_destructor) doesn't work either
4. rt_init() doesn't help either.

Is there anything I can do to fix this issue on the D side?

Help would be greatly appreciated!

On Wednesday, 15 November 2023 at 22:41:19 UTC, Jaap Geurts wrote:

I do not understand what you want to achieve, but maybe (ab)using the atexitfrom the D shared object to register a function in the D shared object at app exit?
Be-aware when dlclose is used on the D shared object, the app will crash due to the function-pointer pointing to already unmapped lib function is called by atexit.

I however never tried this, might not work.

On Thursday, 16 November 2023 at 17:59:56 UTC, Arjan wrote:

I want to load a plugin into QML. This works with the help of dqml. It crashes upon exit of the QML app but I must have a clean exit. What I think happens is the following:

Qt loads the DSO (my plugin written in D). In an init routine I start the D runtime with rt_init(). This happens in a thread created by Qt which means that D will setup TLS or reuse existing TLS for that thread. Further the runtime keeps track of libraries which have been loaded. So when my plugin is loaded, the D runtime scans all segments looking for loaded libraries which were dynamically linked-in into my plugin. It then stores those pointers to the loaded libraries in a __gshared collection so that whenever another thread tries to open an already stored library, the runtime will return that pointer. The runtime also stores which libraries are available for each thread and it stores that for each thread individually in TLS space. What happens is this:

• At startup the __gshared library collection gets populated with pointers of already loaded libraries. Second the TLS collection of open libraries also gets setup for the current thread.
• At exit, when Qt quits the main thread(without properly unloading libraries or quitting other threads nicely), the D exit handler runs. It sees no library pointers in the collection in the current TLS space (because now it's looking in the TLS of main() but the __gshared collection still has references. Then it bugs out with a safeAbort saying that loading and unloading isn't synchronized. This is technically correct but not helpful.

The best solution for me would be that the D runtime would still report an error, but not call abort().

I tried this, but doesn't work either, because when I register my function I'm late to the party and my function gets called after D's runtime exit handler has been called. Even if I could intercept the call it still wouldn't work, because atexit() runs in the main thread and the current TLS library collection is empty and I can't unload the library because the __gshared collection is internal to D's runtime.

My code example above is a good explanation of what happens in my setup. It crashes with the exact same error message.

Thanks for taking the time to answer me.

On Monday, 20 November 2023 at 23:03:43 UTC, Jaap Geurts wrote:

The order of calling registered function by atexit should be in LIFO, IIRC, that would imply the D's runtime exit handler is registered later? Or not using the atexit at all?

Still there would be the TLS thread issue to solve..

Isn't the module destructor static ~this or shared static ~this called in the correct thread and correct order?

I see. Will play around with it if time permits..