Menu
Search

Forums

Forum Index

Thread overview
What is safe to do in an extern (C) function and how can I test this?
Jan 25, 2022
Jaime
Jan 25, 2022
H. S. Teoh
Jan 25, 2022
Jaime
January 25, 2022
Gravatar of JaimePosted by JaimeReply
Jaime
Gravatar of Jaime


Reply

The lede:

Can I, for instance, safely call Fiber.yield in a C callback that I know will be run in a Fiber?

The stack will look like:
Thread
|- Fiber in D runtime
| |- Call into a C API (stays on same thread)
| | |- Extern (C) callback (stays on same thread)
| | | |- Fiber.yield <-- Is this OK?

Also, in general, is there a convenient way to know or test if particular C callbacks are safe? Or is it just case by case?

The context I was gonna bury the lede in but thought better of it:

So, I am a big lame novice. Don't worry, I know this already.

I'm trying to use gtk-d, but I don't even know how to use gtk, so I'm learning them both at once. Probably not a good idea, I know.

From what I can tell so far, it seems like the way gtk works is you write your whole application in terms of gtk's event loop. I really don't want to do that, so -- and I realize this is probably an even worse idea, but I don't know the right thing to do -- I decided I'd try writing a worker thread class -- "Rope" -- and run gtk's event loop in a Rope.

A Rope runs its given task in a new Fiber in a new thread, and that thread repeatedly resumes the fiber and terminates when the fiber terminates, but, in the meantime, takes some time to act as an executor each time the fiber suspends, accepting work in std.concurrency messages and running it on the same thread as its main fiber. My thought was that this way, I can call gtk-d's API from any thread, and still have all the calls happen on the same thread that the event loop is running on, as the API requires.

All I needed, then, was some way to make gtk-d suspend the fiber it's running on, to give its rope some time to service external work. My first thought was to use glib.Timeout.Timeout.add and put Fiber.yield in the timeout callback. The potential problem: the timeout callback, of course, has to be extern (C).

Now, I could just drop the more general concept of Rope and rewrite it specifically for gtk, and drop the whole Fiber part, and instead directly put the calls to std.concurrency.receiveTimeout in the glib.Timeout.Timeout.add callback. Assuming, of course, receiveTimeout would be any safer to call from extern (C) than Fiber.yield would. But that's just the trouble, I can't guess which would be safer, and even a minimal test for this question seems daunting.

January 24, 2022
Gravatar of Steven SchveighofferPosted by Steven Schveighoffer
in reply to Jaime		Reply
Steven Schveighoffer
Gravatar of Steven Schveighoffer
Posted in reply to Jaime


Reply

On 1/24/22 8:31 PM, Jaime wrote:

>

The lede:

Can I, for instance, safely call Fiber.yield in a C callback that I know will be run in a Fiber?

The stack will look like:
Thread
|- Fiber in D runtime
| |- Call into a C API (stays on same thread)
| | |- Extern (C) callback (stays on same thread)
| | | |- Fiber.yield <-- Is this OK?

Also, in general, is there a convenient way to know or test if particular C callbacks are safe? Or is it just case by case?

The context I was gonna bury the lede in but thought better of it:

So, I am a big lame novice. Don't worry, I know this already.

I'm trying to use gtk-d, but I don't even know how to use gtk, so I'm learning them both at once. Probably not a good idea, I know.

From what I can tell so far, it seems like the way gtk works is you write your whole application in terms of gtk's event loop. I really don't want to do that, so -- and I realize this is probably an even worse idea, but I don't know the right thing to do -- I decided I'd try writing a worker thread class -- "Rope" -- and run gtk's event loop in a Rope.

A Rope runs its given task in a new Fiber in a new thread, and that thread repeatedly resumes the fiber and terminates when the fiber terminates, but, in the meantime, takes some time to act as an executor each time the fiber suspends, accepting work in std.concurrency messages and running it on the same thread as its main fiber. My thought was that this way, I can call gtk-d's API from any thread, and still have all the calls happen on the same thread that the event loop is running on, as the API requires.

All I needed, then, was some way to make gtk-d suspend the fiber it's running on, to give its rope some time to service external work. My first thought was to use glib.Timeout.Timeout.add and put Fiber.yield in the timeout callback. The potential problem: the timeout callback, of course, has to be extern (C).

Now, I could just drop the more general concept of Rope and rewrite it specifically for gtk, and drop the whole Fiber part, and instead directly put the calls to std.concurrency.receiveTimeout in the glib.Timeout.Timeout.add callback. Assuming, of course, receiveTimeout would be any safer to call from extern (C) than Fiber.yield would. But that's just the trouble, I can't guess which would be safer, and even a minimal test for this question seems daunting.

I would imagine it's fine, all the fiber context switch is doing (WRT the stack) is swapping out the fiber portion of the stack (i.e. back to the Fiber.call invokation), which should include the C stack portions as well.

There's a lot of low-level details in the Fiber.d file itself:

https://github.com/dlang/druntime/blob/e390ba7e0a1f80f15e72ca773fca7252057ba4c5/src/core/thread/fiber.d#L387

-Steve

January 24, 2022
Gravatar of H. S. TeohPosted by H. S. Teoh
in reply to Jaime		Reply
H. S. Teoh
Gravatar of H. S. Teoh
Posted in reply to Jaime


Reply
On Tue, Jan 25, 2022 at 01:31:29AM +0000, Jaime via Digitalmars-d-learn wrote:
> **The lede**:
> 
> Can I, for instance, safely call Fiber.yield in a C callback that I know will be run in a Fiber?
> 
> The stack will look like:
> Thread
> |- Fiber in D runtime
> | |- Call into a C API (stays on same thread)
> | | |- Extern (C) callback (stays on same thread)
> | | | |- Fiber.yield <-- Is this OK?

I haven't tested this myself, but I *think* it should be OK.

One thing to watch out for, though, is the size of the Fiber's stack (this can be specified when you first create the Fiber).  If the C part of the code uses up too much stack space (e.g. if somewhere in the C code it tries to allocate a large object on the stack), you may inadvertently overflow the Fiber's stack and cause a crash or abort. Increasing the stack size of the Fiber when it is created should fix this problem.


T

-- 
Those who don't understand D are condemned to reinvent it, poorly. -- Daniel N
January 25, 2022
Gravatar of JaimePosted by Jaime
in reply to H. S. Teoh		Reply
Jaime
Gravatar of Jaime
Posted in reply to H. S. Teoh


Reply
On Tuesday, 25 January 2022 at 01:41:03 UTC, Steven Schveighoffer wrote:
> On 1/24/22 8:31 PM, Jaime wrote:
>> Can I, for instance, safely call Fiber.yield in a C callback that I know will be run in a Fiber?
>
> I would *imagine* it's fine, all the fiber context switch is doing (WRT the stack) is swapping out the fiber portion of the stack (i.e. back to the `Fiber.call` invokation), which should include the C stack portions as well.
>
> There's a lot of low-level details in the Fiber.d file itself:
>
> https://github.com/dlang/druntime/blob/e390ba7e0a1f80f15e72ca773fca7252057ba4c5/src/core/thread/fiber.d#L387
>
> -Steve

On Tuesday, 25 January 2022 at 02:02:27 UTC, H. S. Teoh wrote:
> I haven't tested this myself, but I *think* it should be OK.
>
> One thing to watch out for, though, is the size of the Fiber's stack (this can be specified when you first create the Fiber).  If the C part of the code uses up too much stack space (e.g. if somewhere in the C code it tries to allocate a large object on the stack), you may inadvertently overflow the Fiber's stack and cause a crash or abort. Increasing the stack size of the Fiber when it is created should fix this problem.
>
>
> T

The insight and vote of confidence are much appreciated, you two. Steven, good call to check out the implementation of Fiber, I'll do that.