Thread overview
Got some problemmes with new version of compiler 2.090
Jan 19, 2020
uranuz
Jan 19, 2020
uranuz
Jan 19, 2020
berni44
Jan 19, 2020
uranuz
Jan 19, 2020
user1234
Jan 19, 2020
Mathias Lang
Jan 19, 2020
uranuz
Jan 19, 2020
uranuz
Jan 27, 2020
uranuz
January 19, 2020
Hello! I have got some problemmes with new compiler version that I am trying to debug already for several days, because I cannot locate the source of bug using GDB, because there are no useful backtrace.
The first problemme was `Memory allocation failed`.
I was executing my code step by step. And it was failing with such error near where I was using std.exception: enforce.
I need to say that I running server in threaded mode using TaskPool class. But when running in `plain` mode without threads all goes just fine.
What changes are made in enforce, TaskPool or memory allocation that can be a source of such bug?

And another error that I have encountered just recently:
//---
Signal        Stop	Print	Pass to program	Description
SIGUSR1       No	No	Yes		User defined signal 1
Signal        Stop	Print	Pass to program	Description
SIGUSR2       No	No	Yes		User defined signal 2
Source directories searched: /usr/include/dmd/druntime/import:/home/uranuz/sources/yar_mkk:$cdir:$cwd
Source directories searched: /usr/include/dmd/phobos:/usr/include/dmd/druntime/import:/home/uranuz/sources/yar_mkk:$cdir:$cwd
Source directories searched: /home/uranuz/sources/webtank/src:/usr/include/dmd/phobos:/usr/include/dmd/druntime/import:/home/uranuz/sources/yar_mkk:$cdir:$cwd
Source directories searched: /home/uranuz/sources/ivy/src:/home/uranuz/sources/webtank/src:/usr/include/dmd/phobos:/usr/include/dmd/druntime/import:/home/uranuz/sources/yar_mkk:$cdir:$cwd
Source directories searched: /home/uranuz/sources/yar_mkk/src:/home/uranuz/sources/ivy/src:/home/uranuz/sources/webtank/src:/usr/include/dmd/phobos:/usr/include/dmd/druntime/import:/home/uranuz/sources/yar_mkk:$cdir:$cwd
Running executable
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x7ffff7ff6700 (LWP 14697)]
[New Thread 0x7ffff7ff1700 (LWP 14698)]
[New Thread 0x7ffff7fec700 (LWP 14699)]
[New Thread 0x7ffff7fe7700 (LWP 14700)]
[New Thread 0x7ffff7fe2700 (LWP 14701)]
[New Thread 0x7ffff7fdd700 (LWP 14702)]
[New Thread 0x7ffff7fd8700 (LWP 14703)]
[New Thread 0x7ffff7fbc700 (LWP 14704)]
[New Thread 0x7ffff7fb7700 (LWP 14705)]
[New Thread 0x7ffff7fb2700 (LWP 14706)]
[New Thread 0x7ffff7fad700 (LWP 14707)]
[New Thread 0x7ffff7fa8700 (LWP 14708)]
[New Thread 0x7ffff7fa3700 (LWP 14709)]
[New Thread 0x7ffff7f9e700 (LWP 14710)]
[New Thread 0x7ffff7f99700 (LWP 14711)]
warning: Error removing breakpoint 1
warning: Error removing breakpoint 2
warning: Error removing breakpoint 3
warning: Error removing breakpoint 4
warning: Error removing breakpoint 5
warning: Error removing breakpoint 6
warning: Error removing breakpoint 7
[New Thread 0x7ffff177a700 (LWP 14790)]

Thread
1 "mkk_main_servic" received signal SIGTRAP, Trace/breakpoint trap.
0x0000555555ce9cd8 in webtank.net.server.thread_pool.ThreadPoolServer._runLoop() (this=0x7ffff7e9c660) at ../webtank/src/webtank/net/server/thread_pool.d:61
61					if( client is null )
[Switching to thread 17 (Thread 0x7ffff177a700 (LWP 14790))](running)

Thread
17 "mkk_main_servic" received signal SIGTRAP, Trace/breakpoint trap.
[Switching to Thread 0x7ffff177a700 (LWP 14790)]
0x0000555555d3b79b in _D7webtank3net6server6common14processRequestFC3std6socket6SocketCQClQCgQCf5iface10IWebServerZv (server=0x7ffff7e9c670, sock=0x7ffff7f044c0) at ../webtank/src/webtank/net/server/common.d:95
95		HTTPOutput response = new HTTPOutput();

Thread
17 "mkk_main_servic" received signal SIGSEGV, Segmentation fault.
0x0000555555deda1e in _d_newclass ()
bt
#0  0x0000555555deda1e in _d_newclass ()
#1  0x0000555555d3b7a6 in _D7webtank3net6server6common14processRequestFC3std6socket6SocketCQClQCgQCf5iface10IWebServerZv (server=0x7ffff7e9c670, sock=0x7ffff7f044c0) at ../webtank/src/webtank/net/server/common.d:95
#2  0x0000555555cea5ef in _D3std11parallelism__T3runTPFCQBc6socket6SocketC7webtank3net6server5iface10IWebServerZvTQChTCQBtQBoQBn11thread_pool16ThreadPoolServerZQEiFQEhKQEjKQCcZv (_param_2=@0x7ffff7f005e8: 0x7ffff7e9c660, _param_1=@0x7ffff7f005e0: 0x7ffff7f044c0, fpOrDelegate=0x555555d3b780 <_D7webtank3net6server6common14processRequestFC3std6socket6SocketCQClQCgQCf5iface10IWebServerZv>) at /usr/include/dmd/phobos/std/parallelism.d:761
#3  0x0000555555ce9f91 in _D3std11parallelism__T4TaskSQBaQz3runTPFCQBn6socket6SocketC7webtank3net6server5iface10IWebServerZvTQChTCQBtQBoQBn11thread_pool16ThreadPoolServerZQEt4implFPvZv (myTask=0x7ffff7f005a0) at /usr/include/dmd/phobos/std/parallelism.d:444
#4  0x0000555555dfe363 in std.parallelism.AbstractTask.job() ()
#5  0x0000555555dfe4b4 in _D3std11parallelism8TaskPool5doJobMFPSQBkQBj12AbstractTaskZv ()
#6  0x0000555555dfe62a in std.parallelism.TaskPool.executeWorkLoop() ()
#7  0x0000555555dfe5d0 in std.parallelism.TaskPool.startWorkLoop() ()
#8  0x0000555555dea41e in core.thread.osthread.Thread.run() ()
#9  0x0000555555e2bf56 in thread_entryPoint ()
#10 0x00007ffff720b6db in start_thread (arg=0x7ffff177a700) at pthread_create.c:463
#11 0x00007ffff657288f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
//-----

Seems that some error occured when allocating new class instance. It is very weird
January 19, 2020
And also when all `enfroce`ments pass and condition is true then there are no `Memory allocation failed` errors. So it the most certaily a bug in enforce or with the way it allocates memory for exceptions, maybe.
January 19, 2020
On Sunday, 19 January 2020 at 08:54:43 UTC, uranuz wrote:
> The first problemme was `Memory allocation failed`.

Maybe it could be helpful if you could provide a minimum example, e.g. using dustmite [1] to create one.

[1] https://github.com/CyberShadow/DustMite/wiki
January 19, 2020
Seems that problemme with memory allocation repeats even using regular `throw new Exception`. But I don't understand why it is only occur in threaded code.
I don't know how to use dustmite unfortunately. Seems that I need to formulate some condition to check stdout or stderror. But my programme doesn't output anything there. Even if error occurs it is being sent by TCPSocket
January 19, 2020
On Sunday, 19 January 2020 at 09:57:09 UTC, uranuz wrote:
> Seems that problemme with memory allocation repeats even using regular `throw new Exception`. But I don't understand why it is only occur in threaded code.
> I don't know how to use dustmite unfortunately.

You can also use https://github.com/CyberShadow/Digger on the non minimized code.
The goal is to point to a specific compiler change from which your server start failing.
If this works (*) this will already give a better overview of the issue, although we already know it's certainly related to druntime and more especially the GC.

> Seems that I need to formulate some condition to check stdout or stderror. But my programme doesn't output anything there. Even if error occurs it is being sent by TCPSocket

(*): often it works but points to merge commits from stable master, which is not so helpful.
January 19, 2020
I think this would be better suited for `D.learn` or stackoverflow. But the thread is here, so next time :)

On Sunday, 19 January 2020 at 08:54:43 UTC, uranuz wrote:
> The first problemme was `Memory allocation failed`.
> I was executing my code step by step. And it was failing with such error near where I was using std.exception: enforce.
> I need to say that I running server in threaded mode using TaskPool class. But when running in `plain` mode without threads all goes just fine.
> What changes are made in enforce, TaskPool or memory allocation that can be a source of such bug?

This kind of issue is really platform specific, so it would be helpful to know which OS you are running, what's the env like (How much memory ? Is it virtualized ?), etc...

> And another error that I have encountered just recently:
> [...]

That looks related. Can you start your program with `--DRT-gcopts=parallel:0` ?
Alternatively you can use this in your code in one of the module that is compiled in:
```
static if (__VERSION__ >= 2087)
    extern(C) __gshared string[] rt_options = [ "gcopt=parallel:0" ];
```

January 19, 2020
> This kind of issue is really platform specific, so it would be helpful to know which OS you are running, what's the env like (How much memory ? Is it virtualized ?), etc...

I am using Ubuntu 18.04 with latest updates. It is not virtualized (AMD Ryzen 3700 processor). I have 32GB of memory, so I am very doubt that there could be any memory lack issue.
Nothing special or specific about it, I believe, because Ubuntu is enough popular and wide-spread distributive of Linux. And one of the distributives listed on dlang "Downloads" page. So I believe that DMD should be tested on this OS.

When I find enough time I shall try manually reduce problemme, because I don't have any idea how to apply Dustmite, etc. to this.
But for now seems that I need to revert to older compiler version, because I have already spend so much time for things that are not related to main goals of my project.

I was just interested if someone of developers remembers what changes were made to compiler/ library that may cause this problemme?

Thanks

January 19, 2020
Seems that problemme was introduced between 2.089.1 and 2.090 exactly.

January 27, 2020
I have rewritten bunch of code and now it doesn't crash anymore. But still I failed to figure out why it was failing...