December 13, 2021

On Monday, 13 December 2021 at 15:21:19 UTC, Jan wrote:

>

On Monday, 13 December 2021 at 13:02:50 UTC, Ola Fosheim Grøstad wrote:

>

Yes, I wouldn't want to use it, maybe manual mangling is better, but still painful. const A& is so common in C++ API's that it really should be supported out-of-the-box. All it takes is adding a deref-type-constructor to the D language spec, e.g. ref const(@deref(A))

I fully agree. This pattern is so common in C++, that I am surprised D doesn't have a way to do this already. The whole idea of linking against C++ is to interop easily, with little friction and high performance. Needing to build any shims or redesign the C++ side is very much contrary to this goal.

Does anyone know whether such issues have been discussed before? I can't imagine I'm the first one to run into this.

A similar issue about tail const classes has already been discussed:
https://digitalmars.com/d/archives/digitalmars/D/const_Class_is_mangled_as_Class_const_const_299139.html

I made a pull request, which changes the mangling to tail const for classes passed directly as parameter or return type, but now think this would break too much code: https://github.com/dlang/dmd/pull/13369
The proposal to add a deref-type-constructor, would also allow to have tail const classes.

December 13, 2021

On Monday, 13 December 2021 at 16:29:12 UTC, Tim wrote:

>

I made a pull request, which changes the mangling to tail const for classes passed directly as parameter or return type, but now think this would break too much code: https://github.com/dlang/dmd/pull/13369
The proposal to add a deref-type-constructor, would also allow to have tail const classes.

A 'deref' keyword sounds interesting. I'm not an expert on compilers, but thinking about this a bit more, to me it looks like the fundamental problem is, that D tries to apply its class pointer/reference semantics to C++, even though it could do this differently. 'Deref' would only solve one (common) issue. However, in C++ it is also very common to treat classes as value types. Maybe one could give such a hint to the D compiler instead.

If I have this C++ code:

class A { ... };

void AsValue(A value);
void AsPtr(A* value);
void AsConstPtr(const A* value);
void AsRef(A& value);
void AsConstRef(const A& value);

Afaik today I can really only bind to functions of the form 'AsPtr' and 'AsConstPtr'. And if I declare A in D as a struct, I could also bind to all the others, but as mentioned above that's not always possible.

How about in D I could declare that A should be used like a value type to pass it to a function, using the 'struct' keyword:

extern(C++) class A { ... }
extern(C++) void AsValue(struct A value);
extern(C++) void AsPtr(A value); // could stay as previously, OR
extern(C++) void AsPtr(struct A* value); // same as above
extern(C++) void AsConstPtr(const(A) value);
extern(C++) void AsConstPtr(const(struct A*) value); // same as above
extern(C++) void AsRef(ref struct A value);
extern(C++) void AsConstRef(const(ref struct A) value); // same as above

So here the 'struct' keyword would tell the compiler to treat A like a value type, just as in C++ and thus apply pointer, const and reference semantics like in C++. Additionally, if a pure 'struct A' is encountered, the compiler would need to create a copy of the object on the stack, just as it would do for structs, to pass it to C++ (which might modify the temporary). I guess this would be trickier to implement but then you would be able to pass classes to C++ under all circumstances.

The added benefit would be, that this shouldn't change existing behavior and thus not break anything.

Unfortunately I have neither the time nor expertise to change DMD myself.

December 13, 2021

On Monday, 13 December 2021 at 12:16:03 UTC, Ola Fosheim Grøstad wrote:

>

On Monday, 13 December 2021 at 12:08:30 UTC, evilrat wrote:

>

Yeah but it sucks to have making C++ wrapper just for this. I think either pragma mangle to hammer it in place or helper dummy struct with class layout that mimics this shim logic is a better solution in such cases.
Literally anything but building C++ code twice for a project.

Does something like this work?

class _A {}
struct A {}

extern(C++) void CppFunc(ref const(A) arg);

void func(_A a){
    CppFunc(*cast(A*)a);
}

Unfortunately no. Maybe the cast would even make it work, but I can't have "A" and "_A" in D as separate types, because the class is supposed to link to C++ functions and thus renaming it from A to _A breaks that. On the other hand I can't give the struct another name either, because that's how it is linked to in "CppFunc".

December 13, 2021

On Monday, 13 December 2021 at 21:17:49 UTC, Jan wrote:

>

Unfortunately no. Maybe the cast would even make it work, but I can't have "A" and "_A" in D as separate types, because the class is supposed to link to C++ functions and thus renaming it from A to _A breaks that. On the other hand I can't give the struct another name either, because that's how it is linked to in "CppFunc".

The easiest workaround is probably to set the mangling manually:

pragma(mangle, "_Z7CppFuncRK1A")
extern(C++) void CppFunc(const A arg);

This mangling is for the Itanium ABI, which is used by Linux and other operating systems. It needs to be different for Windows.

December 14, 2021

On Monday, 13 December 2021 at 12:08:30 UTC, evilrat wrote:

>

On Monday, 13 December 2021 at 11:13:12 UTC, Tejas wrote:

>

On Monday, 13 December 2021 at 09:21:26 UTC, Jan wrote:

>

[...]

You'll have to use something called a shim, it seems.

For example:

main.d :


extern(C++) class A{}

extern(C++) void cppFunc_shim(A arg);

void main(){
	A a = new A();
	cppFunc_shim(a);
}

cppShim.cpp :


class A{};

extern void cppFunc(A const &arg);

void cppFunc_shim(A *param){
	const A forwardingVar = A(*param);
	cppFunc(forwardingVar);
}

cppFunc.cpp :


#include "iostream"
class A{};

void cppFunc(A const &arg){
	//std::cout << arg << std::endl;
	std::cout << "Called cppFunc :D" << std::endl;
}	

Then pass the following on the command line(assuming all files are in the same directory):

ldmd2 main.d cppFunc.o cppShim.o -L-lstdc++

That's what it took to make it work for me, dunno if more convenient methods exist.

Hope it helps :D

Yeah but it sucks to have making C++ wrapper just for this. I think either pragma mangle to hammer it in place or helper dummy struct with class layout that mimics this shim logic is a better solution in such cases.
Literally anything but building C++ code twice for a project.

Hey, evilrat, I've seen people make claims that our C++ interop has reached phenomenal levels and that going any further would basically require a C++ compiler ala ImportC++, the issue is just that the docs haven't been updated yet to reflect it.

What do you think about this? Is this really true? Because it sure doesn't look that way to me :(

December 14, 2021

On Tuesday, 14 December 2021 at 06:21:39 UTC, Tejas wrote:

>

Hey, evilrat, I've seen people make claims that our C++ interop has reached phenomenal levels and that going any further would basically require a C++ compiler ala ImportC++, the issue is just that the docs haven't been updated yet to reflect it.

What do you think about this? Is this really true? Because it sure doesn't look that way to me :(

Unfortunately it is mostly true.
There is some missing features like above tail ref and const, there is minor mangling issues that requires pragma mangle sometimes, and other annoying little details.

Aside from that there is things that requires actual C++ compiler OR at least part of it to enable certain C++ features - like the example above with pass-by-value for classes certainly requires real C++ copy constructor, some operator overloads, use of SFINAE instead of CTFE, and I'm sure there is more of such nuances.

All this not going to happen, D spec clearly states it allows limited C++ interop by relying on linker mechanics rather than being C++ compatible language.

It is now abandoned but there was LDC fork called "Calypso", it was a mixed clang/LDC compiler that aimed to achieve seamless D/C++ interop and from what I've heard it was working just fine without all this hiccups as above.

December 15, 2021

On Tuesday, 14 December 2021 at 07:50:48 UTC, evilrat wrote:

>

There is some missing features like above tail ref and const, there is minor mangling issues that requires pragma mangle sometimes, and other annoying little details.

As far as I can tell from my limited experience, the way D approaches interacting with C++ is sound and extremely useful. Link compatibility goes a very long way.

Unfortunately it's the "annoying little details" that I immediately bumped into. Things that should be no problem at all with just link compatibility, like incorrectly mangled functions (forgetting about 'const' in return types on Windows) and this very, very annoying issue that I can't pass a class by reference but only by pointer.

I can understand that passing a class by value might be out of scope (though it would be possible, since it works for structs just as well), but with pass by reference it's really just D's boneheadedness to try to have its own way.

D is a great language, and I want to use it, but as a stand-alone language it just doesn't have the ecosystem that I need (gamedev). Using it as an integrated scripting language is for me the next best thing to benefit from it in a project that is otherwise C++. With link compatibility and auto-generating some bindings, I can get there eventually, but it's still a lot of work with many manually crafted shims. I am currently willing to invest a lot of time to try to solve this, but I won't rewrite a huge C++ code base just because D can't pass classes by reference.

Other projects will have even more constraints and even less willingness to invest time into such an undertaking and just scrap the idea early on.

C++ link compatibility was a great idea, but if D wants to expand it's user base further, in my opinion it has to polish the interop and be willing to make a few sacrifices on it's end, because that can save everyone else hundreds of hours of tedious work (and maintenance) and thus be the deciding factor for or against using D in a C++ project.

December 15, 2021

On Wednesday, 15 December 2021 at 09:36:54 UTC, Jan wrote:

>

Unfortunately it's the "annoying little details" that I immediately bumped into.

Just another example: I just learned that linking against C++ DLLs is quite limited. I ran into the issue that linking in an external variable doesn't work (even though the mangled name that D chooses is correct), because DLLs work differently than static linking does.
Someone with more in-depth knowledge told me, that Windows support in D and specifically DLL support is lacking quite a bit.

Having only link compatibility is totally fine, D currently just doesn't fulfill that promise, especially not on Windows and especially not with DLLs.

December 16, 2021
On 15/12/2021 11:54 PM, Jan wrote:
> On Wednesday, 15 December 2021 at 09:36:54 UTC, Jan wrote:
>> Unfortunately it's the "annoying little details" that I immediately bumped into.
> 
> Just another example: I just learned that linking against C++ DLLs is quite limited. I ran into the issue that linking in an external variable doesn't work (even though the mangled name that D chooses is correct), because DLLs work differently than static linking does.
> Someone with more in-depth knowledge told me, that Windows support in D and specifically DLL support is lacking quite a bit.
> 
> Having *only* link compatibility is totally fine, D currently just doesn't fulfill that promise, especially not on Windows and especially not with DLLs.

Are you sure that on the shared library side it was marked as exported?

If a symbol is not exported, there is no guarantee (nor reason to think) that it will be visible during runtime linking to said shared library/executable.

This isn't unique to D, its just how linkers work.
December 15, 2021
On Wednesday, 15 December 2021 at 11:03:27 UTC, rikki cattermole wrote:
>
> On 15/12/2021 11:54 PM, Jan wrote:
>> On Wednesday, 15 December 2021 at 09:36:54 UTC, Jan wrote:
>>> Unfortunately it's the "annoying little details" that I immediately bumped into.
>> 
>> Just another example: I just learned that linking against C++ DLLs is quite limited. I ran into the issue that linking in an external variable doesn't work (even though the mangled name that D chooses is correct), because DLLs work differently than static linking does.
>> Someone with more in-depth knowledge told me, that Windows support in D and specifically DLL support is lacking quite a bit.
>> 
>> Having *only* link compatibility is totally fine, D currently just doesn't fulfill that promise, especially not on Windows and especially not with DLLs.
>
> Are you sure that on the shared library side it was marked as exported?
>
> If a symbol is not exported, there is no guarantee (nor reason to think) that it will be visible during runtime linking to said shared library/executable.
>
> This isn't unique to D, its just how linkers work.

Yep, checked that. Even checked the .exp file and compared the mangled name there with the mangled name that D tries to use, they are the same. I know someone who worked on improving DLL support for D a while back and he said he had to fix this in D but didn't get that merged into DMD back then (since there were many other things as well) and now he doesn't have time to work on it further :(