Hello, I finally write on the forum for the first time to finally find the answer to a problem I am having:

I am calling native GTK with extern(C) function declarations instead of using gtkD, mainly because I just need very few functions and I don't want a big library that would take about 20x time to compile than my project with very few binds.

And I either found a D bug or a lack in documentation about extern(C) or this is a very specific case that doesn't happen often.

I believe this is the same "bug" as this:
https://forum.dlang.org/thread/fjfftrruedmzdcqmrbci@forum.dlang.org

So the problem I am noticing is very simple: as you may know GTK callbacks have a void* user_data argument, what happens, is that if the flow of code is this: D => C => D the void* pointer when reaching the third step and finally emerging to the D language will change its value and get corrupted.

A stupid example can be passing a D object using the void* user_data GTK argument, passing a extern(C) callback function to GTK, and then inside that callback calling a extern(D) function that accepts a void*.

I already tried using Variant, but pointers to Variant get corrupted too, so it's useless.

I tested this with Valgrind and there are no stack corruptions, GDB confirms that any void* pointer doing the D => C => D route gets corrupted (changes its value to a random(?) one) in the final step.

The TL;DR is: when the flow of code is inside an extern(C) function with a D object passed as a void* pointer it seems there is no way to pass that object again to another D function accepting a void* pointer

Additional Notes:
Tried LDC LLVM and it still happens, so it's not a DMD bug

On Sunday, 28 July 2019 at 18:32:24 UTC, Federico Santamorena wrote:
> So the problem I am noticing is very simple: as you may know GTK callbacks have a void* user_data argument, what happens, is that if the flow of code is this: D => C => D the void* pointer when reaching the third step and finally emerging to the D language will change its value and get corrupted.

Can you give your exact callback function definition? I once had my context pointer corrupted because I marked my callback extern(C) instead of extern(Windows). Maybe you also have an error in your definition somewhere.

On Sunday, 28 July 2019 at 18:37:26 UTC, Dennis wrote:
> Can you give your exact callback function definition?

Yes, indeed. D's `long` and C's `long` are incompatible too, so using the wrong there is a potential for this kind of problem as well (the argument before the pointer is the wrong size, so then the pointer value is pulled from the wrong location).

On Sunday, 28 July 2019 at 18:37:26 UTC, Dennis wrote:
> On Sunday, 28 July 2019 at 18:32:24 UTC, Federico Santamorena wrote:
>> So the problem I am noticing is very simple: as you may know GTK callbacks have a void* user_data argument, what happens, is that if the flow of code is this: D => C => D the void* pointer when reaching the third step and finally emerging to the D language will change its value and get corrupted.
>
> Can you give your exact callback function definition? I once had my context pointer corrupted because I marked my callback extern(C) instead of extern(Windows). Maybe you also have an error in your definition somewhere.

It's a bit complex but:


context is a pointer to a D struct
extern(C) void gtk_search_changed(GtkEditable* widget, void* data) { //body }
context.search_input.g_signal_connect("changed", &gtk_search_changed, context);

Then inside gtk_search_changed, startCrawling gets called and context is just passed to it as the last argument:

DrillContext* startCrawling(in const(DrillConfig) config,
                                   in immutable(string) searchValue,
                                   in immutable(void function(immutable(FileInfo) result, void* userObject)) resultCallback,
                                   in void* userObject) { //body }

And gets called like this:
startCrawling(drillConfig, searchString, &resultFound, context);


Then when the resultCallback gets called its userObject (that now should be a pointer to context) is now corrupt:

void resultFound(immutable(FileInfo) result, void* userObject) { //body}

And now inside resultFound the void* is completely corrupt

On Sunday, 28 July 2019 at 18:54:48 UTC, Federico Santamorena wrote:
> void resultFound(immutable(FileInfo) result, void* userObject)

Is FileInfo the class from gtkd or is it something else?

On Sunday, 28 July 2019 at 18:54:48 UTC, Federico Santamorena wrote:
> On Sunday, 28 July 2019 at 18:37:26 UTC, Dennis wrote:
>> On Sunday, 28 July 2019 at 18:32:24 UTC, Federico Santamorena wrote:
>>> So the problem I am noticing is very simple: as you may know GTK callbacks have a void* user_data argument, what happens, is that if the flow of code is this: D => C => D the void* pointer when reaching the third step and finally emerging to the D language will change its value and get corrupted.
>>
>> Can you give your exact callback function definition? I once had my context pointer corrupted because I marked my callback extern(C) instead of extern(Windows). Maybe you also have an error in your definition somewhere.
>
> It's a bit complex but:
>
>
> context is a pointer to a D struct
> extern(C) void gtk_search_changed(GtkEditable* widget, void* data) { //body }
> context.search_input.g_signal_connect("changed", &gtk_search_changed, context);
>
> Then inside gtk_search_changed, startCrawling gets called and context is just passed to it as the last argument:
>
> DrillContext* startCrawling(in const(DrillConfig) config,
>                                    in immutable(string) searchValue,
>                                    in immutable(void function(immutable(FileInfo) result, void* userObject)) resultCallback,
>                                    in void* userObject) { //body }
>
> And gets called like this:
> startCrawling(drillConfig, searchString, &resultFound, context);
>
>
> Then when the resultCallback gets called its userObject (that now should be a pointer to context) is now corrupt:
>
> void resultFound(immutable(FileInfo) result, void* userObject) { //body}
>
> And now inside resultFound the void* is completely corrupt

Here, the GitHub file:

https://github.com/yatima1460/Drill/blob/576b4b691ea5357e5271115433eb11d4c0beeae7/Source/Frontend/GTK/Main.d

It's an experimental test using GTK bindings.

Here the flow:

D:
https://github.com/yatima1460/Drill/blob/576b4b691ea5357e5271115433eb11d4c0beeae7/Source/Frontend/GTK/Main.d#L586

extern(C):
https://github.com/yatima1460/Drill/blob/576b4b691ea5357e5271115433eb11d4c0beeae7/Source/Frontend/GTK/Main.d#L323

https://github.com/yatima1460/Drill/blob/576b4b691ea5357e5271115433eb11d4c0beeae7/Source/Frontend/GTK/Main.d#L384

D again:
https://github.com/yatima1460/Drill/blob/576b4b691ea5357e5271115433eb11d4c0beeae7/Source/Frontend/GTK/Main.d#L299

Now userObject inside resultFound is corrupt

On Sunday, 28 July 2019 at 18:59:59 UTC, Adam D. Ruppe wrote:
> On Sunday, 28 July 2019 at 18:54:48 UTC, Federico Santamorena wrote:
>> void resultFound(immutable(FileInfo) result, void* userObject)
>
> Is FileInfo the class from gtkd or is it something else?

https://github.com/yatima1460/Drill/blob/576b4b691ea5357e5271115433eb11d4c0beeae7/Source/Backend/FileInfo.d

On Sunday, 28 July 2019 at 18:32:24 UTC, Federico Santamorena wrote:
> A stupid example can be passing a D object using the void* user_data GTK argument, passing a extern(C) callback function to GTK, and then inside that callback calling a extern(D) function that accepts a void*.
>
> I already tried using Variant, but pointers to Variant get corrupted too, so it's useless.

Where is the "D object" allocated? Haven't seen anyone mention it, but the way the GC works, it has to know about the memory it needs to scan to look for an object.

So if you pass an object allocated with the GC then pass it to GTK, the GC isn't going to know about the memory GTK has allocate. The kind of problem can happen that you mention, it deallocates the object because it doesn't think it is used anymore.

On 07/28/2019 06:46 PM, Exil wrote:
> On Sunday, 28 July 2019 at 18:32:24 UTC, Federico Santamorena wrote:
>> A stupid example can be passing a D object using the void* user_data GTK argument, passing a extern(C) callback function to GTK, and then inside that callback calling a extern(D) function that accepts a void*.
>>
>> I already tried using Variant, but pointers to Variant get corrupted too, so it's useless.
> 
> Where is the "D object" allocated? Haven't seen anyone mention it, but the way the GC works, it has to know about the memory it needs to scan to look for an object.
> 
> So if you pass an object allocated with the GC then pass it to GTK, the GC isn't going to know about the memory GTK has allocate. The kind of problem can happen that you mention, it deallocates the object because it doesn't think it is used anymore.

For completeness, more other valuable information is here:

  https://dlang.org/spec/interfaceToC.html

Ali

On Monday, 29 July 2019 at 01:46:27 UTC, Exil wrote:
> On Sunday, 28 July 2019 at 18:32:24 UTC, Federico Santamorena wrote:
>> [...]
>
> Where is the "D object" allocated? Haven't seen anyone mention it, but the way the GC works, it has to know about the memory it needs to scan to look for an object.
>
> So if you pass an object allocated with the GC then pass it to GTK, the GC isn't going to know about the memory GTK has allocate. The kind of problem can happen that you mention, it deallocates the object because it doesn't think it is used anymore.

I even tried GC.disable().

The void* pointer still gets corrupted