I have a multi-threaded application, whose threads normally run forever. But I need to profile this program, so I compile the code with -profile, send a SIGTERM and call exit(0) from my signal handler to exit the program. The problem is that I get the profiling information only from the main thread, but not from the other ones.

Is there a way to get the profiling information from all threads before terminating the program? Maybe some way to finish the threads gracefully? or manully call "write trace.log"-function for a thread?

Here is a small example that demonstrates the problem:

import core.thread;
import core.stdc.stdlib;

shared bool done = false;

void run()
{
    while (!done)
    {
        foo;
    }
}

void foo()
{
    new Object;
}

void main()
{
    auto thread = new Thread(&run);
    thread.start;
    Thread.sleep(3.seconds);

    exit(0); // Replace with "done = true;" to get the expected behaviour.
}

There is already an issue: https://issues.dlang.org/show_bug.cgi?id=971
The hack was to call trace_term() in internal/trace. call_term() doesn't exist anymore, I tried to export the static destructor from druntime/src/rt/trace.d with:

extern (C) void _staticDtor449() @nogc nothrow;

(on my system) and call it manually. I get some more information this way, but the numbers in the profiling report are still wrong.

Exit is not "normal exit" for D programs so, do not use it.
Your threads should stop at some point to make able the app exit successfully.
There's a "join" method. You can use it with your "done" variable.

Also there was an issue that profiling doesn't work with multi-threaded apps and leads to a crash.
Don't know if it is fixed.

On Thursday, 27 July 2017 at 14:30:33 UTC, Eugene Wissner wrote:
> I have a multi-threaded application, whose threads normally run forever. But I need to profile this program, so I compile the code with -profile, send a SIGTERM and call exit(0) from my signal handler to exit the program. The problem is that I get the profiling information only from the main thread, but not from the other ones.
>
> [...]

You will need to run it single threaded.
If you want to use the builtin-profiler.

On Thursday, 27 July 2017 at 14:52:18 UTC, Stefan Koch wrote:
> On Thursday, 27 July 2017 at 14:30:33 UTC, Eugene Wissner wrote:
>> I have a multi-threaded application, whose threads normally run forever. But I need to profile this program, so I compile the code with -profile, send a SIGTERM and call exit(0) from my signal handler to exit the program. The problem is that I get the profiling information only from the main thread, but not from the other ones.
>>
>> [...]
>
> You will need to run it single threaded.
> If you want to use the builtin-profiler.

Are there profilers that work well with dmd? valgrind? OProfile?

On Thursday, 27 July 2017 at 14:44:31 UTC, Temtaime wrote:
> Also there was an issue that profiling doesn't work with multi-threaded apps and leads to a crash.
> Don't know if it is fixed.

Was fixed two years ago:
http://forum.dlang.org/post/mia2kf$djb$1@digitalmars.com

On 2017-07-27 16:30, Eugene Wissner wrote:
> I have a multi-threaded application, whose threads normally run forever. But I need to profile this program, so I compile the code with -profile, send a SIGTERM and call exit(0) from my signal handler to exit the program. The problem is that I get the profiling information only from the main thread, but not from the other ones.
> 
> Is there a way to get the profiling information from all threads before terminating the program? Maybe some way to finish the threads gracefully? or manully call "write trace.log"-function for a thread?

As others have mentioned, you should in general avoid calling "exit" in a D program. There's a C function called "atexit" that allows to register a callback that is called after calling "exit". You could perhaps join the threads there. I don't know if that helps with the profiling though.

-- 
/Jacob Carlborg

On Friday, 28 July 2017 at 06:32:59 UTC, Jacob Carlborg wrote:
> On 2017-07-27 16:30, Eugene Wissner wrote:
>> I have a multi-threaded application, whose threads normally run forever. But I need to profile this program, so I compile the code with -profile, send a SIGTERM and call exit(0) from my signal handler to exit the program. The problem is that I get the profiling information only from the main thread, but not from the other ones.
>> 
>> Is there a way to get the profiling information from all threads before terminating the program? Maybe some way to finish the threads gracefully? or manully call "write trace.log"-function for a thread?
>
> As others have mentioned, you should in general avoid calling "exit" in a D program. There's a C function called "atexit" that allows to register a callback that is called after calling "exit". You could perhaps join the threads there. I don't know if that helps with the profiling though.

Unfortunately I can't join threads, because the program wouldn't exit then, the threads run forever normally. I thought maybe there is some way to kill a thread gracefully in linux, so it can write its profiling information; or another way to get profiling.
Thanks anyway.

On Friday, 28 July 2017 at 08:06:33 UTC, Eugene Wissner wrote:
> On Friday, 28 July 2017 at 06:32:59 UTC, Jacob Carlborg wrote:
>> On 2017-07-27 16:30, Eugene Wissner wrote:
>>> I have a multi-threaded application, whose threads normally run forever. But I need to profile this program, so I compile the code with -profile, send a SIGTERM and call exit(0) from my signal handler to exit the program. The problem is that I get the profiling information only from the main thread, but not from the other ones.
>>> 
>>> Is there a way to get the profiling information from all threads before terminating the program? Maybe some way to finish the threads gracefully? or manully call "write trace.log"-function for a thread?
>>
>> As others have mentioned, you should in general avoid calling "exit" in a D program. There's a C function called "atexit" that allows to register a callback that is called after calling "exit". You could perhaps join the threads there. I don't know if that helps with the profiling though.
>
> Unfortunately I can't join threads, because the program wouldn't exit then, the threads run forever normally. I thought maybe there is some way to kill a thread gracefully in linux, so it can write its profiling information; or another way to get profiling.
> Thanks anyway.

There's no "gracefully" way to kill a thread.
If your thread cannot join, then you're doing something wrong

On Friday, 28 July 2017 at 09:02:10 UTC, Temtaime wrote:
> There's no "gracefully" way to kill a thread.
> If your thread cannot join, then you're doing something wrong

Our programs are intended to run "forever". 24/7 servers.