I'm implementing a wrapper program for wrapping generic C++ libraries - it's going very well (subclassing, virtual methods, nested classes, enums working) but I've hit an area that I don't have enough D experience to answer, and am hoping someone can point me in the right direction. My background is lots of professional Java, Python etc., but not so much writing of C++ (plenty of reading but not writing).

I need to maintain a mapping between C++ void* addresses and D wrapper Objects. The naive implementation would be

Object[void*] wrappingRegistry;

but of course that prevents the wrapped objects from being garbage collected - I need weak ref semantics.

I had a go at making it e.g.
ulong[ulong] and storing the cast(ulong) address of the D object, but it seems that I don't understand what taking the address of an obj (&obj) actually is doing - it doesn't seem to point to the memory occupied by the object but instead to the address of the variable pointing to the object?

I've had a good google around and because my understanding is poor in this area (GC innards, shared data, D pointers etc) I can't identify which of the implementations around is best for 2.064/2.065 and moving forwards. My best guess is the WeakRef found in this one:

https://github.com/phobos-x/phobosx/blob/master/source/phobosx/signal.d

by Robert in his signals2 implementation. Can anyone expain what is going on in WeakRef and InvisibleAddress?

thanks for any help you can give me.

> Object[void*] wrappingRegistry;
>
> but of course that prevents the wrapped objects from being garbage collected - I need weak ref semantics.
>
> I had a go at making it e.g.
> ulong[ulong] and storing the cast(ulong) address of the D object, but it seems that I don't understand what taking the address of an obj (&obj) actually is doing - it doesn't seem to point to the memory occupied by the object but instead to the address of the variable pointing to the object?
>

class X {};
X x;

x is an reference to an instance of X, with other words a pointer without arithmetic but with syntax sugar. &x will take the address of this pointer/reference. If you want the address of the actual instance, you can use cast(void*) for example.

On Saturday, 11 January 2014 at 20:17:14 UTC, Tobias Pankrath wrote:
>
> class X {};
> X x;
>
> x is an reference to an instance of X, with other words a pointer without arithmetic but with syntax sugar. &x will take the address of this pointer/reference. If you want the address of the actual instance, you can use cast(void*) for example.

Hi Tobias, can casting the address to void* make a difference to its value?

Here's an example of what I don't understand:

import std.stdio;
import std.string: format;

class Foo {
	int x;
}

void printAddress(Foo foo) {
	writeln("Address of parameter foo is %x".format(&foo));
	writeln("Address of parameter foo cast to void* is %x".format(cast(void*) &foo));
}

void main() {
	auto foo = new Foo();
	writeln("Address of foo is %x".format(&foo));
	writeln("Address of foo cast to void* is %x".format(cast(void*) &foo));
	printAddress(foo);
}

When run I get:

Address of foo is 7fff40ac4558
Address of foo cast to void* is 7fff40ac4558
Address of parameter foo is 7fff40ac4538
Address of parameter foo cast to void* is 7fff40ac4538

So why is the address of the parameter foo different to the address of main foo, when they both refer to the same object?

thanks

On Saturday, 11 January 2014 at 20:38:33 UTC, Abdulhaq wrote:
> On Saturday, 11 January 2014 at 20:17:14 UTC, Tobias Pankrath wrote:
>>
>> class X {};
>> X x;
>>
>> x is an reference to an instance of X, with other words a pointer without arithmetic but with syntax sugar. &x will take the address of this pointer/reference. If you want the address of the actual instance, you can use cast(void*) for example.
>
> Hi Tobias, can casting the address to void* make a difference to its value?
>

No, try this:
import std.stdio;
class X {}
void foo(X x) { writeln(cast(void*) x); }

void main() {
         X x; // null reference by default.
         writeln(cast(void*) x);
         foo(x);

         x = new X;
         writeln(cast(void*) x);
         foo(x);
}

>
> No, try this:
> import std.stdio;
> class X {}
> void foo(X x) { writeln(cast(void*) x); }
>
> void main() {
>          X x; // null reference by default.
>          writeln(cast(void*) x);
>          foo(x);
>
>          x = new X;
>          writeln(cast(void*) x);
>          foo(x);
> }

Thanks Tobias that indeed works, unfortunately storing the address as ulong in an AA does not seem to be defeating the garbage collector!

I guess I'll have to go for a WeakRef implementation, I'll try Robert's implementation in signals, but I'd really like to know how long it is likely to last as a working weak ref (in terms of the D GC changing etc.) and if I'm taking the right approach.

On Sunday, 12 January 2014 at 10:48:15 UTC, Abdulhaq wrote:
>>
>> No, try this:
>> import std.stdio;
>> class X {}
>> void foo(X x) { writeln(cast(void*) x); }
>>
>> void main() {
>>         X x; // null reference by default.
>>         writeln(cast(void*) x);
>>         foo(x);
>>
>>         x = new X;
>>         writeln(cast(void*) x);
>>         foo(x);
>> }
>
> Thanks Tobias that indeed works, unfortunately storing the address as ulong in an AA does not seem to be defeating the garbage collector!
>
> I guess I'll have to go for a WeakRef implementation, I'll try Robert's implementation in signals, but I'd really like to know how long it is likely to last as a working weak ref (in terms of the D GC changing etc.) and if I'm taking the right approach.

Sorry to reply to my own message, looking at Robert's InvisibleAddress class I think he's hiding the address from the GC by e.g. rotating the bits in the ulong. He's also made it all thread safe - I certainly don't understand the details but it's given me some ideas.

On Sunday, 12 January 2014 at 10:48:15 UTC, Abdulhaq wrote:
>>
>> No, try this:
>> import std.stdio;
>> class X {}
>> void foo(X x) { writeln(cast(void*) x); }
>>
>> void main() {
>>         X x; // null reference by default.
>>         writeln(cast(void*) x);
>>         foo(x);
>>
>>         x = new X;
>>         writeln(cast(void*) x);
>>         foo(x);
>> }
>
> Thanks Tobias that indeed works, unfortunately storing the address as ulong in an AA does not seem to be defeating the garbage collector!
>
> I guess I'll have to go for a WeakRef implementation, I'll try Robert's implementation in signals, but I'd really like to know how long it is likely to last as a working weak ref (in terms of the D GC changing etc.) and if I'm taking the right approach.

The current GC is imprecise. Everything that looks like a pointer at the bit level, is treated like one. I can't help you create weak references.

On Sunday, 12 January 2014 at 12:12:53 UTC, Tobias Pankrath wrote:
> On Sunday, 12 January 2014 at 10:48:15 UTC, Abdulhaq wrote:
>>>
>>> No, try this:
>>> import std.stdio;
>>> class X {}
>>> void foo(X x) { writeln(cast(void*) x); }
>>>
>>> void main() {
>>>        X x; // null reference by default.
>>>        writeln(cast(void*) x);
>>>        foo(x);
>>>
>>>        x = new X;
>>>        writeln(cast(void*) x);
>>>        foo(x);
>>> }
>>
>> Thanks Tobias that indeed works, unfortunately storing the address as ulong in an AA does not seem to be defeating the garbage collector!
>>
>> I guess I'll have to go for a WeakRef implementation, I'll try Robert's implementation in signals, but I'd really like to know how long it is likely to last as a working weak ref (in terms of the D GC changing etc.) and if I'm taking the right approach.
>
> The current GC is imprecise. Everything that looks like a pointer at the bit level, is treated like one. I can't help you create weak references.

No don't worry you've got me past that point of incomprehension, thanks again

Maybe this will be useful in the work:

Compile
    Windows: dmd st1.d
      Linux: dmd st1.d -L-ldl
// ---------------------------------------

// MGW 05.01.14
// Model in D a C++ object QByteArray of Qt.
//--------------------------------------------

import core.runtime;     // Load  DLL for Win
import std.stdio;        // writeln

version(linux) {
    import core.sys.posix.dlfcn;  // define dlopen() и dlsym()

    // On Linux DMD v2.063.2, these functions are not defined in core.runtime, so I had to write to.
    extern (C) void* rt_loadLibrary(const char* name) { return dlopen(name, RTLD_GLOBAL || RTLD_LAZY);  }
    void* GetProcAddress(void* hLib, string nameFun) {  return dlsym(hLib, nameFun.ptr);    }
}
version(Windows) {
	import std.c.windows.windows;  // GetProcAddress for Windows
}

// Warning!!!
// When defining constructors and member functions attribute "extern (C)" required!
alias extern (C) void function(void*, char*)       t_QByteArray_QByteArray;  t_QByteArray_QByteArray  QByteArray_QByteArray;
alias extern (C) void* function(void*, char, int)  t_QByteArray_fill;        t_QByteArray_fill        QByteArray_fill;

//T he structure of the QByteArray from the file qbytearray.h in the include directory. Because C++ inline functions missing in DLL
// there is no possibility to directly call a dozen functions.
// If you look in C++ there definition is as follows:
// inline char *QByteArray::data() { detach(); return d->data; } where d is the Data*
struct Data {
        void* rref;
        int   alloc;
        int   size;
        char* data;      // That's what we need, a pointer to an array of bytes
        char  array[1];
}

// == Experimental class DQByteArray ==
class DQByteArray {
    Data* QtObj;       // this is object: &QtObj -  size 4 byte (32 os)
    // ------------------
    // constructor D called of a constructor C++
    this(char* buf) {
        QByteArray_QByteArray(&QtObj, buf);
    }
    ~this() {
        // I can find a destructor, and here his record, but too lazy to do it ....
    }
    // As inline function is not stored in a DLL have to model it through the structure of the Data
    char* data() {
        return (*QtObj).data;
    }
    // D format: Data** == C++ format: QByteArray
    // so it became clear that such a C++object, looking at it from the D
    void* fill(char ch, int resize=-1) {
        return QByteArray_fill(&QtObj, ch, resize);
    }
}

int main(string[] args) {

// These files get QByteArray C++
version(linux)   {    auto nameQtCore = "libQtCore.so";  }
version(Windows) {    auto nameQtCore = "QtCore4.dll";   }

    auto h = Runtime.loadLibrary(nameQtCore); // Loading dll or so

    // Load function constructor QByteArray::QByteArray(char*);
    QByteArray_QByteArray = cast(t_QByteArray_QByteArray)GetProcAddress(h, "_ZN10QByteArrayC1EPKc");
    // QByteArray::fill(char*, int);
    QByteArray_fill = cast(t_QByteArray_fill)GetProcAddress(h, "_ZN10QByteArray4fillEci");
    // QByteArray::~QByteArray()

    // Create our experimental subject and consider its data
    DQByteArray ba = new DQByteArray(cast(char*)"ABC".ptr);
    printf("\n ba.data() = %s", ba.data());

    // Experience the action of the fill() and see the result
    ba.fill('Z', 5);
    printf("\n ba.data() = %s", ba.data());

    return 0;
}

On Sunday, 12 January 2014 at 16:17:23 UTC, MGW wrote:
> Maybe this will be useful in the work:
>
> Compile
>     Windows: dmd st1.d
>       Linux: dmd st1.d -L-ldl
> // ---------------------------------------
>
> // MGW 05.01.14
> // Model in D a C++ object QByteArray of Qt.
> //--------------------------------------------
>
> import core.runtime;     // Load  DLL for Win
> import std.stdio;        // writeln
>
> version(linux) {
>     import core.sys.posix.dlfcn;  // define dlopen() и dlsym()
>
>     // On Linux DMD v2.063.2, these functions are not defined in core.runtime, so I had to write to.
>     extern (C) void* rt_loadLibrary(const char* name) { return dlopen(name, RTLD_GLOBAL || RTLD_LAZY);  }
>     void* GetProcAddress(void* hLib, string nameFun) {  return dlsym(hLib, nameFun.ptr);    }
> }
> version(Windows) {
> 	import std.c.windows.windows;  // GetProcAddress for Windows
> }
>
> // Warning!!!
> // When defining constructors and member functions attribute "extern (C)" required!
> alias extern (C) void function(void*, char*)       t_QByteArray_QByteArray;  t_QByteArray_QByteArray  QByteArray_QByteArray;
> alias extern (C) void* function(void*, char, int)  t_QByteArray_fill;        t_QByteArray_fill        QByteArray_fill;
>
> //T he structure of the QByteArray from the file qbytearray.h in the include directory. Because C++ inline functions missing in DLL
> // there is no possibility to directly call a dozen functions.
> // If you look in C++ there definition is as follows:
> // inline char *QByteArray::data() { detach(); return d->data; } where d is the Data*
> struct Data {
>         void* rref;
>         int   alloc;
>         int   size;
>         char* data;      // That's what we need, a pointer to an array of bytes
>         char  array[1];
> }
>
> // == Experimental class DQByteArray ==
> class DQByteArray {
>     Data* QtObj;       // this is object: &QtObj -  size 4 byte (32 os)
>     // ------------------
>     // constructor D called of a constructor C++
>     this(char* buf) {
>         QByteArray_QByteArray(&QtObj, buf);
>     }
>     ~this() {
>         // I can find a destructor, and here his record, but too lazy to do it ....
>     }
>     // As inline function is not stored in a DLL have to model it through the structure of the Data
>     char* data() {
>         return (*QtObj).data;
>     }
>     // D format: Data** == C++ format: QByteArray
>     // so it became clear that such a C++object, looking at it from the D
>     void* fill(char ch, int resize=-1) {
>         return QByteArray_fill(&QtObj, ch, resize);
>     }
> }
>
> int main(string[] args) {
>
> // These files get QByteArray C++
> version(linux)   {    auto nameQtCore = "libQtCore.so";  }
> version(Windows) {    auto nameQtCore = "QtCore4.dll";   }
>
>     auto h = Runtime.loadLibrary(nameQtCore); // Loading dll or so
>
>     // Load function constructor QByteArray::QByteArray(char*);
>     QByteArray_QByteArray = cast(t_QByteArray_QByteArray)GetProcAddress(h, "_ZN10QByteArrayC1EPKc");
>     // QByteArray::fill(char*, int);
>     QByteArray_fill = cast(t_QByteArray_fill)GetProcAddress(h, "_ZN10QByteArray4fillEci");
>     // QByteArray::~QByteArray()
>
>     // Create our experimental subject and consider its data
>     DQByteArray ba = new DQByteArray(cast(char*)"ABC".ptr);
>     printf("\n ba.data() = %s", ba.data());
>
>     // Experience the action of the fill() and see the result
>     ba.fill('Z', 5);
>     printf("\n ba.data() = %s", ba.data());
>
>     return 0;
> }

Hi yes I think noticed in another thread that you were wrapping Qt with dynamic loading of the qt libs, interesting idea - does your code allow subclassing of the Qt classes and overriding the virtual methods? I'm taking a much more traditional approach, but there is method in my madness :-)