Any insight appreciated. Details:

>dmd --version
DMD64 D Compiler v2.107.0
Copyright (C) 1999-2024 by The D Language Foundation, All Rights Reserved written by Walter Bright

References:
class ComObject
interface IUnknown

I only used class ComObject and implicitly the IUnknown interface it inherits from as a test of some COM code I was writing using comheaders.c containing the following, compiled with ImportC.

#define WINVER 0x0A00
#define _WIN32_WINNT 0x0A00
#define _WIN32_DCOM
#include <wtypes.h>
#include <oleauto.h>
#include <oaidl.h>

The file main.d compiled with it is as follows.

import std.stdio;
import comheaders;
static import com = core.sys.windows.com;

pragma(lib, "onecore"); //to fix linkage of two irrelevant symbols

void main() {
    auto COMobject = new com.ComObject();
    //auto COMobject = new ComObject();
    IUnknown* ip = cast(IUnknown*)COMobject;
    writeln(COMobject.count);
    writeln("       ip vtable: ", ip.lpVtbl);
    auto vtable = COMobject.__vptr;
    writeln("COMobject vtable: ", vtable);
    writeln("ip &AddRef: ", &ip.lpVtbl.AddRef);
    writeln("ip offset: ", cast(void*)&ip.lpVtbl.AddRef - cast(void*)ip.lpVtbl);
    auto ipaddref = cast(void*)ip.lpVtbl.AddRef;
    writeln("       ip AddRef: ", ipaddref);
    auto addref = cast(void*)(&COMobject.AddRef).funcptr;
    writeln("COMobject AddRef: ", addref);
    writeln("COMobject AddRef : ip AddRef offset: ", addref - ipaddref);
    COMobject.AddRef();
    writeln(COMobject.count);
    ip.lpVtbl.AddRef(ip);
    writeln(COMobject.count);
}

Here I make a ComObject from the statically imported core.sys.windows.com but avoid using anything else from the D windows libs. The object contains a reference count, that should be incremented with a call of AddRef. The output was as follows.

0
       ip vtable: 7FF756091A30
COMobject vtable: 7FF756091A30
ip &AddRef: 7FF756091A38
ip offset: 8
       ip AddRef: 7FF756027970
COMobject AddRef: 7FF756022EC0
COMobject AddRef : ip AddRef offset: -19120
1
1

This shows that the call of AddRef with the correct offset does nothing, and is a different function pointer to that of AddRef in the com.ComObject. So that object will apparently not work correctly with outside world code as claimed (see the above reference links). This was compiled with

dmd main.d comheaders.c vcintrinsics.lib -P/wd5105

where vcintrinsics.lib is a library I constructed to fix a problem with DMD not knowing of a series of MSVC intrinsics, i.e. to satify the linker as per here, and -P/wd5105 is to suppress a warning from MSVC when it is used by ImportC for as a C preprocessor.

I copied the source of the inconveniently named interface IUnknown in unknwn.d and of class ComObject from com.d, both in C:\D\dmd2\src\druntime\src\core\sys\windows\
into the bottom of main.d and experimented. I made a local ComObject with the commented out line above active. One change fixed the problem: using extern(C++) --- no other linking attribute worked. Here's the working code. I had to edit IUnknown which was an interface but is now a struct in comheaders.c, into IUnknown* inside QueryInterface, and E_NOINTERFACE into com.E_NOINTERFACE and define the D interface with a name different to the struct IUnknown so I made it _IUnknown_ but otherwise the source is unchanged apart from not using extern(Windows) and prefixing it all with extern(C++). Here is the rest of main.d.

import core.atomic;

extern(C++):

interface _IUnknown_ {
    HRESULT QueryInterface(IID* riid, void** pvObject);
    ULONG AddRef();
    ULONG Release();
}


class ComObject : _IUnknown_
{
    HRESULT QueryInterface(const(IID)* riid, void** ppv)
    {
        if (*riid == IID_IUnknown)
        {
            *ppv = cast(void*)cast(IUnknown*)this;
            AddRef();
            return S_OK;
        }
        else
        {   *ppv = null;
            return com.E_NOINTERFACE;
        }
    }

    ULONG AddRef()
    {
        return atomicOp!"+="(*cast(shared)&count, 1);
    }

    ULONG Release()
    {
        LONG lRef = atomicOp!"-="(*cast(shared)&count, 1);
        if (lRef == 0)
        {
            // free object

            // If we delete this object, then the postinvariant called upon
            // return from Release() will fail.
            // Just let the GC reap it.
            //delete this;

            return 0;
        }
        return cast(ULONG)lRef;
    }

    LONG count = 0;             // object reference count
}

The output is now as follows.

0
       ip vtable: 7FF76B9C0360
COMobject vtable: 7FF76B9C0360
ip &AddRef: 7FF76B9C0368
ip offset: 8
       ip AddRef: 7FF76B951580
COMobject AddRef: 7FF76B951580
COMobject AddRef : ip AddRef offset: 0
1
2

showing that finding AddRef in the ComObjects Vtable produces the same function pointer as that through the COM Interface, and both work.

My working hypothesis: both ComObject and IUnknown brought in by importing core.sys.windows.com are broken which is all of the support for COM in Phobos. Please confirm or deny.

You incorrectly declared the interface variable. Try this:

    auto COMobject = new com.ComObject();
    //auto COMobject = new ComObject();
    IUnknown ip = COMobject;
    writeln(COMobject.count);
    writeln("       ip vtable: ", ip.lpVtbl);
    auto vtable = COMobject.__vptr;
    writeln("COMobject vtable: ", vtable);
    writeln("ip &AddRef: ", &ip.lpVtbl.AddRef);
    writeln("ip offset: ", cast(void*)&ip.lpVtbl.AddRef - cast(void*)ip.lpVtbl);
    auto ipaddref = cast(void*)ip.lpVtbl.AddRef;
    writeln("       ip AddRef: ", ipaddref);
    auto addref = cast(void*)(&COMobject.AddRef).funcptr;
    writeln("COMobject AddRef: ", addref);
    writeln("COMobject AddRef : ip AddRef offset: ", addref - ipaddref);
    COMobject.AddRef();
    writeln(COMobject.count);
    ip.lpVtbl.AddRef(ip);
    writeln(COMobject.count);

On Tuesday, 19 March 2024 at 09:22:09 UTC, Kagamin wrote:

    auto COMobject = new com.ComObject();
    //auto COMobject = new ComObject();
    IUnknown ip = COMobject;

Nonsense. Of course that doesn't work! It seems you didn't read or didn't understand my post.

All you've done is written circular code that confirms compatibility with itself by remaining entirely within its own world of D only, and only using core.sys.windows.com. To make this "work" you had to import core.sys.windows.com; which I do not.

You have NOT confirmed the compatibility of core.sys.windows.com.IUnknown and core.sys.windows.com.ComObject with the outside world of actual COM.

Supposedly things defined using those can be passed through COM to outside world and will work correctly there. This is exactly what is NOT happening here.

As mentioned in my original post core.sys.windows.com is statically included so core.sys.windows.com.IUnknown is NOT IUnknown in main.d and in fact, apart from making a ComObject none of core.sys.windows.com is used at all in the original run.

IUnknown in my code is the usual COM struct defined in a Windows header and imported from comheaders.c. It's called an interface in COM terminology but it is NOT a D interface, and you can't inherit from it.

So if I just change main.d to the code you quoted above it won't compile of course.

main.d(11): Error: cannot implicitly convert expression `COMobject` of type `core.sys.windows.com.ComObject` to `IUnknown`

The outside world in C would work with an object implementing IUnknown (or another COM interface) as a pointer to a struct of type IUnknown (or similar), and main.d works with COM in exactly that way.

The documentation of the D interface IUnknown at the quoted link in my original post
says that
A COM interface is defined as one that derives from the interface core.sys.win­dows.com.IUnknown.
and
A COM interface is designed to map directly onto a Windows COM object. Any COM object can be represented by a COM interface, and any D object with a COM interface can be used by external COM clients.

This is D's internal definition, how a COM interface (outside world definition independent of programming language) is represented inside D. Conflating the inside D meanings of COM interface and IUnknown with the outside world COM definitions of those terms is the source of confusion.

In effect the documentation deliberately conflates the two by tacitly assuming that the D representation is correct so we need not make that distinction.

The D class core.sys.win­dows.com.ComObject inherits from core.sys.win­dows.com.IUnknown and so should follow those statements. My code gets one of those and behaves as an external COM client, but the result does not work.

If I fix the source of core.sys.win­dows.com.ComObject and core.sys.win­dows.com.IUnknown to have extern(C++) linkage then the self-same code works, showing that I wrote it correctly.

In order to work with COM the way that C does, when given a ComObject from the world of D, I need to treat it as a pointer to an IUnknown struct. The ComObject variable COMobject is already a reference so I need to turn it into a pointer of type IUnknown* that points to the actual ComObject that the COMobject variable refers to, all without knowing the D type of the object.

This is just simulating what would happen if such an object was passed by D though COM to e.g. an arbitrary C program. COM clients do not know the native type of the objects passed them, only the COM interface (NOT D interface) they are supposed to implement.

All the recipient knows is that the pointer is to a struct that contains a properly laid out Vtable with QueryInterface, AddRef and Release as the first three entries in that order in accordance with the standard layout of an IUnknown struct. The following code correctly achieves that outcome.

IUnknown* ip = cast(IUnknown*)COMobject

Then *ip is used as the (COM not D) interface to the externally provided COM object and it doesn't work unless the D library code is repaired.

Then it DOES work. So IUnknown and ComObject in druntime are broken. They do not lead to Vtables laid out according to the COM structs defined by Windows.

Very interesting, and thank you.

It's the cast I am using! In the notation of your code, by casting comObject directly to C_IUnknown* I am getting a different pointer than you do by first casting to drtcom.IUnknown and then casting to C_IUnknown*. There are various other subtle differences going on here too. In my code the Vtables are the same despite the cast I use. But you get your Vtable from the drtcom.IUnknown variable, not the comObject variable. The results are different. Here's a modification of your main function that shows what's going on

void main() {
    auto comObject = new drtcom.ComObject;
    auto iunknownObject = cast(drtcom.IUnknown)comObject;

    writeln("initial count ", comObject.count);

     {
         C_IUnknown* ip1 = cast(C_IUnknown*)iunknownObject;
         writeln("               ip1 vtable: ", ip1.lpVtbl);

         auto ipaddref = cast(void*)ip1.lpVtbl.AddRef;
         writeln("               ip1 AddRef: ", ipaddref);
     }

     {
         C_IUnknown* ip2 = cast(C_IUnknown*)comObject;
         writeln("               ip2 vtable: ", ip2.lpVtbl);

         auto ipaddref = cast(void*)ip2.lpVtbl.AddRef;
         writeln("               ip2 AddRef: ", ipaddref);
     }


     {
         auto vtable = iunknownObject.__vptr;
         writeln("   iunknownObject vtable: ", vtable);

         auto addref = cast(void*)(&iunknownObject.AddRef).funcptr;
         writeln("   iunknownObject AddRef: ", addref);
     }

     {
         auto vtable = comObject.__vptr;
         writeln("        comObject vtable: ", vtable);

         auto addref = cast(void*)(&comObject.AddRef).funcptr;
         writeln("        comObject AddRef: ", addref);
     }

     {
         comObject.AddRef();
         writeln("After D com object count: ", comObject.count);

         C_IUnknown* ip_old = cast(C_IUnknown*)iunknownObject;
         C_IUnknown* ip = cast(C_IUnknown*)comObject;
         writefln("ip     = %s ", ip);
         writefln("ip_old = %s", ip_old);
         ip.lpVtbl.AddRef(ip);
         writeln("After C com object count: ", comObject.count);
     }
}

The output is as follows.

initial count 0
               ip1 vtable: 7FF7EA4318A8
               ip1 AddRef: 7FF7EA3C2C90
               ip2 vtable: 7FF7EA4318E0
               ip2 AddRef: 7FF7EA3C7800
   iunknownObject vtable: 7FF7EA4318A8
   iunknownObject AddRef: 7FF7EA3C2C90
        comObject vtable: 7FF7EA4318E0
        comObject AddRef: 7FF7EA3C2D50
After D com object count: 1
ip     = 20227481000
ip_old = 20227481010
After C com object count: 1

I am confident that casting a D interface variable to a pointer works at the binary level in the expected way, especially for drtcom.IUnknown because the only reasonably simple assumption the compiler can make is that it is a valid COM object, not something that came from D.

The cast of a ComObject to a pointer produces something 16 bytes earlier than the actual start of the COM object according to the working cast, suggesting that the compiler is using some trickery in it's internal representation of COM objects made from D classes that inherit from the D interface IUnknown.

Intuitively, D may make a "normal" object with the "COM" object inside it offset by 16 bytes: two words at 64 bits. We might guess this is the two usual words in a class object, a Vtable pointer and a monitor. We might guess that a "COM" Vtable pointer follows that inside the "COM" part and it points to an offset inside the "normal" vtable so as to start with the necessary QueryInterface, AddRef, Release.

However, when casting such an object to a pointer, a pointer to the normal object is produced, NOT a pointer to the internal "COM" object. This is possible because D knows the type of the object because it is manufactured by D. This is responsible for the bad behavior shown in my initial post.

If the ComObject is created with extern(C++) we might guess that it is then behaves exactly as what it purports to be so as to be entirely compatible with C++. COM in Windows has both a C++ API and a binary compatible C API, so now everything works as expected. extern(C++) object pointers/references need to just work, so the compiler won't produce a bogus offset, whether or not the extern(C++) object is secretly embedded inside a "normal" object, which we might guess it is. There's just a different convention for pointers and references.