September 19, 2021

On Sunday, 19 September 2021 at 20:12:45 UTC, eugene wrote:

>

On Monday, 13 September 2021 at 17:54:43 UTC, eugene wrote:

>

full src is here
http://zed.karelia.ru/0/e/edsm-in-d-2021-09-10.tar.gz

I've also made two simple examples, just in case

Now, let's put some pressure on garbage collector

I rearranged the code of main() like this:

void main(string[] args) {

    auto Main = new Main();
    auto stopper = new Stopper();

    Main.run();
    stopper.run();

    writeln(" === Hello, world! === ");
    auto md = new MessageDispatcher();
    md.loop();
    writeln(" === Goodbye, world! === ");
}

And it works correctly! Miracles... :)

September 20, 2021

On Sunday, 19 September 2021 at 21:10:16 UTC, eugene wrote:

>

I rearranged the code of main() like this:

Similar rearrangement fixed the echo-client as well.
(I moved creation of Stopper to the very beginning of main())

September 21, 2021

On Monday, 13 September 2021 at 17:18:30 UTC, eugene wrote:

>

I do not understand at all why GC considers those sg0 and sg1 as unreferenced.
And why old gdc (without -Os) and old ldc do not.

Conclusion:

There's nothing special about sg0 and sg1, except that they're part of Stopper. The Stopper in main() is collected before the end of main() because it's not used later in the function and because there are apparently no other references to it that the GC can find (because the only reference is hidden inside the Linux epoll API).

More discussion:

https://forum.dlang.org/thread/siajpj$3p2$1@digitalmars.com
http://dpldocs.info/this-week-in-d/Blog.Posted_2021_09_20.html

Misaligned pointers are one way to hide objects from the GC but in this case they really weren't relevant. I just had a confused idea of the epoll API, because I'd only ever used it with a single static array that all epoll functions referenced, similarly to poll(). But actually epoll copies the event structures that you give it, and returns them on epoll_wait. That's wild.

September 21, 2021

On Tuesday, 21 September 2021 at 19:42:48 UTC, jfondren wrote:

>

On Monday, 13 September 2021 at 17:18:30 UTC, eugene wrote:
There's nothing special about sg0 and sg1, except that they're part of Stopper. The Stopper in main() is collected before the end of main() because it's not used later in the function

Okay, but how could you explain this then

void main(string[] args) {

    auto Main = new Main();
    Main.run();

    auto stopper = new Stopper();
    stopper.run();
d-lang/edsm-in-d-simple-example-2 $ ./test | grep STOPPER
'STOPPER' registered 5 (esrc.Signal)
'STOPPER' registered 6 (esrc.Signal)
'STOPPER @ INIT' got 'M0' from 'SELF'
'STOPPER' enabled 5 (esrc.Signal)
'STOPPER' enabled 6 (esrc.Signal)
___!!!___edsm.StageMachine.~this(): STOPPER destroyed...
   !!! esrc.EventSource.~this() : esrc.Signal (owner STOPPER, fd = 5) this @ 0x7fc9ab1a9150
   !!! esrc.EventSource.~this() : esrc.Signal (owner STOPPER, fd = 6) this @ 0x7fc9ab1a9180

Now, change operation order in the main like this:

void main(string[] args) {

    auto Main = new Main();
    auto stopper = new Stopper();

    Main.run();
    stopper.run();
d-lang/edsm-in-d-simple-example-2 $ ./test | grep STOPPER
'STOPPER' registered 5 (esrc.Signal)
'STOPPER' registered 6 (esrc.Signal)
'STOPPER @ INIT' got 'M0' from 'SELF'
'STOPPER' enabled 5 (esrc.Signal)
'STOPPER' enabled 6 (esrc.Signal)

Everything is Ok now, stopper is not collected soon after start.
So the question is how this innocent looking change
can affect GC behavior so much?...

>

Misaligned pointers are one way to hide objects from the GC but in this case they really weren't relevant.

For sure.

September 21, 2021

On Tuesday, 21 September 2021 at 20:17:15 UTC, eugene wrote:

>

Now, change operation order in the main like this:

void main(string[] args) {

    auto Main = new Main();
    auto stopper = new Stopper();

    Main.run();
    stopper.run();
d-lang/edsm-in-d-simple-example-2 $ ./test | grep STOPPER
'STOPPER' registered 5 (esrc.Signal)
'STOPPER' registered 6 (esrc.Signal)
'STOPPER @ INIT' got 'M0' from 'SELF'
'STOPPER' enabled 5 (esrc.Signal)
'STOPPER' enabled 6 (esrc.Signal)

Everything is Ok now,

I don't think this is reliably OK. If you're not using Stopper later in the function, and if there are no other references to it, then the GC can collect it. It just has no obligation to collect it, so minor differences like this might prevent that from happening for particular compilers/options/versions.

C# and Go have 'keepalive' functions to avoid similar behavior, and Java's just as aggressive about potential collection. It's just something that mostly doesn't matter until it becomes an incredibly weird bug with code like yours.

September 21, 2021
On Tue, Sep 21, 2021 at 07:42:48PM +0000, jfondren via Digitalmars-d-learn wrote:
> On Monday, 13 September 2021 at 17:18:30 UTC, eugene wrote:
> > I do not understand at all why GC considers those sg0 and sg1 as
> > unreferenced.
> > And why old gdc (without -Os) and old ldc do not.
> 
> Conclusion:
> 
> There's nothing special about sg0 and sg1, except that they're part of Stopper. The Stopper in main() is collected before the end of main() because it's not used later in the function and because there are apparently no other references to it that the GC can find (because the only reference is hidden inside the Linux epoll API).

Quick and dirty workaround: keep references to those objects in static variables to prevent GC collection:

	auto myFunc(...) {
		static MyType* dontCollect = null;

		MyType* obj = new MyObject(...);
		dontCollect = obj;
		scope(exit) dontCollect = null; // may collect after function exits

		... // function body goes here
	}


T

-- 
Verbing weirds language. -- Calvin (& Hobbes)
September 21, 2021

On Tuesday, 21 September 2021 at 20:17:15 UTC, eugene wrote:

>

Now, change operation order in the main like this:

Actually, all proposed 'fixes'

  • use stopper somehow in the end (writeln(stopper.sg0.number))
  • change operation order
  • etc

are strange. I mean it's strange (for me) that these
fixes make garbage collector behave as needed.

September 21, 2021
On Tue, Sep 21, 2021 at 08:17:15PM +0000, eugene via Digitalmars-d-learn wrote: [...]
> ```d
> void main(string[] args) {
> 
>     auto Main = new Main();
>     Main.run();
> 
>     auto stopper = new Stopper();
>     stopper.run();
> ```
[...]
> ```d
> void main(string[] args) {
> 
>     auto Main = new Main();
>     auto stopper = new Stopper();
> 
>     Main.run();
>     stopper.run();
> ```
[...]
> Everything is Ok now, stopper is not collected soon after start. So the question is how this innocent looking change can affect GC behavior so much?...

In the first example, the compiler sees that the lifetime of Main is
disjoint from the lifetime of stopper, so it's free to reuse the same
stack space (or register(s)) to store both variables. (This is a pretty
standard optimization FYI.) So the line `auto stopper = new Stopper();`
would overwrite the reference to Main, and the GC would see Main as an
unreferenced object and may collect it at any point after the line
`Main.run();`.

In the second case, since the lifetimes of Main and stopper overlap, the compiler (probably) conservatively assumes that their lifetimes last until the end of the function, and so reserves disjoint places for them on the stack.  This does not mean you're 100% safe, however. A sufficiently optimizing compiler may determine that since Main and stopper are independent, it is free to reorder the code such that the two lifetimes are independent, and therefore end up with the same situation as the first example.

If Main really depends on the existence of stopper, I'd argue that it really should store a reference to stopper somewhere, so that as long as Main is not unreferenced the GC would not collect stopper.


T

-- 
What's an anagram of "BANACH-TARSKI"?  BANACH-TARSKI BANACH-TARSKI.
September 21, 2021
On Tue, Sep 21, 2021 at 08:36:49PM +0000, eugene via Digitalmars-d-learn wrote:
> On Tuesday, 21 September 2021 at 20:17:15 UTC, eugene wrote:
> 
> > Now, change operation order in the main like this:
> 
> Actually, all proposed 'fixes'
> 
> - use stopper somehow in the end (writeln(stopper.sg0.number))
> - change operation order
> - etc
> 
> are strange. I mean it's strange (for me) that these
> fixes make garbage collector behave as needed.

It's not strange.  You're seeing these problems because you failed to inform the GC about the dependency between Main and stopper. So it's free to assume that these are two independent, unrelated objects, and therefore it can collect either one as soon as there are no more references to it.

And since stopper isn't used anymore after declaration, an optimizing compiler is free to assume that it's not needed afterwards, so it's not obligated to keep the reference alive until the end of the function.

Since in actually there *is* a dependency between these objects, the most "correct" solution is to include a reference to stopper somewhere in Main. Then the GC would be guaranteed never to collect stopper before Main becomes unreferenced.


T

-- 
Век живи - век учись. А дураком помрёшь.
September 21, 2021
On Tuesday, 21 September 2021 at 20:47:41 UTC, H. S. Teoh wrote:
> And since stopper isn't used anymore after declaration, an optimizing compiler is free to assume that it's not needed afterwards, so it's not obligated to keep the reference alive until the end of the function.

It was not obvious for me,
I thought lifetimes always
lasts until the end of a scope
(main in this case).