Thread overview | |||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
January 11, 2014 WeakRefs for a CPP->D wrapper | ||||
---|---|---|---|---|
| ||||
I'm implementing a wrapper program for wrapping generic C++ libraries - it's going very well (subclassing, virtual methods, nested classes, enums working) but I've hit an area that I don't have enough D experience to answer, and am hoping someone can point me in the right direction. My background is lots of professional Java, Python etc., but not so much writing of C++ (plenty of reading but not writing). I need to maintain a mapping between C++ void* addresses and D wrapper Objects. The naive implementation would be Object[void*] wrappingRegistry; but of course that prevents the wrapped objects from being garbage collected - I need weak ref semantics. I had a go at making it e.g. ulong[ulong] and storing the cast(ulong) address of the D object, but it seems that I don't understand what taking the address of an obj (&obj) actually is doing - it doesn't seem to point to the memory occupied by the object but instead to the address of the variable pointing to the object? I've had a good google around and because my understanding is poor in this area (GC innards, shared data, D pointers etc) I can't identify which of the implementations around is best for 2.064/2.065 and moving forwards. My best guess is the WeakRef found in this one: https://github.com/phobos-x/phobosx/blob/master/source/phobosx/signal.d by Robert in his signals2 implementation. Can anyone expain what is going on in WeakRef and InvisibleAddress? thanks for any help you can give me. |
January 11, 2014 Re: WeakRefs for a CPP->D wrapper | ||||
---|---|---|---|---|
| ||||
Posted in reply to Abdulhaq | > Object[void*] wrappingRegistry;
>
> but of course that prevents the wrapped objects from being garbage collected - I need weak ref semantics.
>
> I had a go at making it e.g.
> ulong[ulong] and storing the cast(ulong) address of the D object, but it seems that I don't understand what taking the address of an obj (&obj) actually is doing - it doesn't seem to point to the memory occupied by the object but instead to the address of the variable pointing to the object?
>
class X {};
X x;
x is an reference to an instance of X, with other words a pointer without arithmetic but with syntax sugar. &x will take the address of this pointer/reference. If you want the address of the actual instance, you can use cast(void*) for example.
|
January 11, 2014 Re: WeakRefs for a CPP->D wrapper | ||||
---|---|---|---|---|
| ||||
Posted in reply to Tobias Pankrath | On Saturday, 11 January 2014 at 20:17:14 UTC, Tobias Pankrath wrote:
>
> class X {};
> X x;
>
> x is an reference to an instance of X, with other words a pointer without arithmetic but with syntax sugar. &x will take the address of this pointer/reference. If you want the address of the actual instance, you can use cast(void*) for example.
Hi Tobias, can casting the address to void* make a difference to its value?
Here's an example of what I don't understand:
import std.stdio;
import std.string: format;
class Foo {
int x;
}
void printAddress(Foo foo) {
writeln("Address of parameter foo is %x".format(&foo));
writeln("Address of parameter foo cast to void* is %x".format(cast(void*) &foo));
}
void main() {
auto foo = new Foo();
writeln("Address of foo is %x".format(&foo));
writeln("Address of foo cast to void* is %x".format(cast(void*) &foo));
printAddress(foo);
}
When run I get:
Address of foo is 7fff40ac4558
Address of foo cast to void* is 7fff40ac4558
Address of parameter foo is 7fff40ac4538
Address of parameter foo cast to void* is 7fff40ac4538
So why is the address of the parameter foo different to the address of main foo, when they both refer to the same object?
thanks
|
January 11, 2014 Re: WeakRefs for a CPP->D wrapper | ||||
---|---|---|---|---|
| ||||
Posted in reply to Abdulhaq | On Saturday, 11 January 2014 at 20:38:33 UTC, Abdulhaq wrote:
> On Saturday, 11 January 2014 at 20:17:14 UTC, Tobias Pankrath wrote:
>>
>> class X {};
>> X x;
>>
>> x is an reference to an instance of X, with other words a pointer without arithmetic but with syntax sugar. &x will take the address of this pointer/reference. If you want the address of the actual instance, you can use cast(void*) for example.
>
> Hi Tobias, can casting the address to void* make a difference to its value?
>
No, try this:
import std.stdio;
class X {}
void foo(X x) { writeln(cast(void*) x); }
void main() {
X x; // null reference by default.
writeln(cast(void*) x);
foo(x);
x = new X;
writeln(cast(void*) x);
foo(x);
}
|
January 12, 2014 Re: WeakRefs for a CPP->D wrapper | ||||
---|---|---|---|---|
| ||||
Posted in reply to Tobias Pankrath | >
> No, try this:
> import std.stdio;
> class X {}
> void foo(X x) { writeln(cast(void*) x); }
>
> void main() {
> X x; // null reference by default.
> writeln(cast(void*) x);
> foo(x);
>
> x = new X;
> writeln(cast(void*) x);
> foo(x);
> }
Thanks Tobias that indeed works, unfortunately storing the address as ulong in an AA does not seem to be defeating the garbage collector!
I guess I'll have to go for a WeakRef implementation, I'll try Robert's implementation in signals, but I'd really like to know how long it is likely to last as a working weak ref (in terms of the D GC changing etc.) and if I'm taking the right approach.
|
January 12, 2014 Re: WeakRefs for a CPP->D wrapper | ||||
---|---|---|---|---|
| ||||
Posted in reply to Abdulhaq | On Sunday, 12 January 2014 at 10:48:15 UTC, Abdulhaq wrote:
>>
>> No, try this:
>> import std.stdio;
>> class X {}
>> void foo(X x) { writeln(cast(void*) x); }
>>
>> void main() {
>> X x; // null reference by default.
>> writeln(cast(void*) x);
>> foo(x);
>>
>> x = new X;
>> writeln(cast(void*) x);
>> foo(x);
>> }
>
> Thanks Tobias that indeed works, unfortunately storing the address as ulong in an AA does not seem to be defeating the garbage collector!
>
> I guess I'll have to go for a WeakRef implementation, I'll try Robert's implementation in signals, but I'd really like to know how long it is likely to last as a working weak ref (in terms of the D GC changing etc.) and if I'm taking the right approach.
Sorry to reply to my own message, looking at Robert's InvisibleAddress class I think he's hiding the address from the GC by e.g. rotating the bits in the ulong. He's also made it all thread safe - I certainly don't understand the details but it's given me some ideas.
|
January 12, 2014 Re: WeakRefs for a CPP->D wrapper | ||||
---|---|---|---|---|
| ||||
Posted in reply to Abdulhaq | On Sunday, 12 January 2014 at 10:48:15 UTC, Abdulhaq wrote:
>>
>> No, try this:
>> import std.stdio;
>> class X {}
>> void foo(X x) { writeln(cast(void*) x); }
>>
>> void main() {
>> X x; // null reference by default.
>> writeln(cast(void*) x);
>> foo(x);
>>
>> x = new X;
>> writeln(cast(void*) x);
>> foo(x);
>> }
>
> Thanks Tobias that indeed works, unfortunately storing the address as ulong in an AA does not seem to be defeating the garbage collector!
>
> I guess I'll have to go for a WeakRef implementation, I'll try Robert's implementation in signals, but I'd really like to know how long it is likely to last as a working weak ref (in terms of the D GC changing etc.) and if I'm taking the right approach.
The current GC is imprecise. Everything that looks like a pointer at the bit level, is treated like one. I can't help you create weak references.
|
January 12, 2014 Re: WeakRefs for a CPP->D wrapper | ||||
---|---|---|---|---|
| ||||
Posted in reply to Tobias Pankrath | On Sunday, 12 January 2014 at 12:12:53 UTC, Tobias Pankrath wrote:
> On Sunday, 12 January 2014 at 10:48:15 UTC, Abdulhaq wrote:
>>>
>>> No, try this:
>>> import std.stdio;
>>> class X {}
>>> void foo(X x) { writeln(cast(void*) x); }
>>>
>>> void main() {
>>> X x; // null reference by default.
>>> writeln(cast(void*) x);
>>> foo(x);
>>>
>>> x = new X;
>>> writeln(cast(void*) x);
>>> foo(x);
>>> }
>>
>> Thanks Tobias that indeed works, unfortunately storing the address as ulong in an AA does not seem to be defeating the garbage collector!
>>
>> I guess I'll have to go for a WeakRef implementation, I'll try Robert's implementation in signals, but I'd really like to know how long it is likely to last as a working weak ref (in terms of the D GC changing etc.) and if I'm taking the right approach.
>
> The current GC is imprecise. Everything that looks like a pointer at the bit level, is treated like one. I can't help you create weak references.
No don't worry you've got me past that point of incomprehension, thanks again
|
January 12, 2014 Re: WeakRefs for a CPP->D wrapper | ||||
---|---|---|---|---|
| ||||
Posted in reply to Abdulhaq | Maybe this will be useful in the work: Compile Windows: dmd st1.d Linux: dmd st1.d -L-ldl // --------------------------------------- // MGW 05.01.14 // Model in D a C++ object QByteArray of Qt. //-------------------------------------------- import core.runtime; // Load DLL for Win import std.stdio; // writeln version(linux) { import core.sys.posix.dlfcn; // define dlopen() и dlsym() // On Linux DMD v2.063.2, these functions are not defined in core.runtime, so I had to write to. extern (C) void* rt_loadLibrary(const char* name) { return dlopen(name, RTLD_GLOBAL || RTLD_LAZY); } void* GetProcAddress(void* hLib, string nameFun) { return dlsym(hLib, nameFun.ptr); } } version(Windows) { import std.c.windows.windows; // GetProcAddress for Windows } // Warning!!! // When defining constructors and member functions attribute "extern (C)" required! alias extern (C) void function(void*, char*) t_QByteArray_QByteArray; t_QByteArray_QByteArray QByteArray_QByteArray; alias extern (C) void* function(void*, char, int) t_QByteArray_fill; t_QByteArray_fill QByteArray_fill; //T he structure of the QByteArray from the file qbytearray.h in the include directory. Because C++ inline functions missing in DLL // there is no possibility to directly call a dozen functions. // If you look in C++ there definition is as follows: // inline char *QByteArray::data() { detach(); return d->data; } where d is the Data* struct Data { void* rref; int alloc; int size; char* data; // That's what we need, a pointer to an array of bytes char array[1]; } // == Experimental class DQByteArray == class DQByteArray { Data* QtObj; // this is object: &QtObj - size 4 byte (32 os) // ------------------ // constructor D called of a constructor C++ this(char* buf) { QByteArray_QByteArray(&QtObj, buf); } ~this() { // I can find a destructor, and here his record, but too lazy to do it .... } // As inline function is not stored in a DLL have to model it through the structure of the Data char* data() { return (*QtObj).data; } // D format: Data** == C++ format: QByteArray // so it became clear that such a C++object, looking at it from the D void* fill(char ch, int resize=-1) { return QByteArray_fill(&QtObj, ch, resize); } } int main(string[] args) { // These files get QByteArray C++ version(linux) { auto nameQtCore = "libQtCore.so"; } version(Windows) { auto nameQtCore = "QtCore4.dll"; } auto h = Runtime.loadLibrary(nameQtCore); // Loading dll or so // Load function constructor QByteArray::QByteArray(char*); QByteArray_QByteArray = cast(t_QByteArray_QByteArray)GetProcAddress(h, "_ZN10QByteArrayC1EPKc"); // QByteArray::fill(char*, int); QByteArray_fill = cast(t_QByteArray_fill)GetProcAddress(h, "_ZN10QByteArray4fillEci"); // QByteArray::~QByteArray() // Create our experimental subject and consider its data DQByteArray ba = new DQByteArray(cast(char*)"ABC".ptr); printf("\n ba.data() = %s", ba.data()); // Experience the action of the fill() and see the result ba.fill('Z', 5); printf("\n ba.data() = %s", ba.data()); return 0; } |
January 12, 2014 Re: WeakRefs for a CPP->D wrapper | ||||
---|---|---|---|---|
| ||||
Posted in reply to MGW | On Sunday, 12 January 2014 at 16:17:23 UTC, MGW wrote:
> Maybe this will be useful in the work:
>
> Compile
> Windows: dmd st1.d
> Linux: dmd st1.d -L-ldl
> // ---------------------------------------
>
> // MGW 05.01.14
> // Model in D a C++ object QByteArray of Qt.
> //--------------------------------------------
>
> import core.runtime; // Load DLL for Win
> import std.stdio; // writeln
>
> version(linux) {
> import core.sys.posix.dlfcn; // define dlopen() и dlsym()
>
> // On Linux DMD v2.063.2, these functions are not defined in core.runtime, so I had to write to.
> extern (C) void* rt_loadLibrary(const char* name) { return dlopen(name, RTLD_GLOBAL || RTLD_LAZY); }
> void* GetProcAddress(void* hLib, string nameFun) { return dlsym(hLib, nameFun.ptr); }
> }
> version(Windows) {
> import std.c.windows.windows; // GetProcAddress for Windows
> }
>
> // Warning!!!
> // When defining constructors and member functions attribute "extern (C)" required!
> alias extern (C) void function(void*, char*) t_QByteArray_QByteArray; t_QByteArray_QByteArray QByteArray_QByteArray;
> alias extern (C) void* function(void*, char, int) t_QByteArray_fill; t_QByteArray_fill QByteArray_fill;
>
> //T he structure of the QByteArray from the file qbytearray.h in the include directory. Because C++ inline functions missing in DLL
> // there is no possibility to directly call a dozen functions.
> // If you look in C++ there definition is as follows:
> // inline char *QByteArray::data() { detach(); return d->data; } where d is the Data*
> struct Data {
> void* rref;
> int alloc;
> int size;
> char* data; // That's what we need, a pointer to an array of bytes
> char array[1];
> }
>
> // == Experimental class DQByteArray ==
> class DQByteArray {
> Data* QtObj; // this is object: &QtObj - size 4 byte (32 os)
> // ------------------
> // constructor D called of a constructor C++
> this(char* buf) {
> QByteArray_QByteArray(&QtObj, buf);
> }
> ~this() {
> // I can find a destructor, and here his record, but too lazy to do it ....
> }
> // As inline function is not stored in a DLL have to model it through the structure of the Data
> char* data() {
> return (*QtObj).data;
> }
> // D format: Data** == C++ format: QByteArray
> // so it became clear that such a C++object, looking at it from the D
> void* fill(char ch, int resize=-1) {
> return QByteArray_fill(&QtObj, ch, resize);
> }
> }
>
> int main(string[] args) {
>
> // These files get QByteArray C++
> version(linux) { auto nameQtCore = "libQtCore.so"; }
> version(Windows) { auto nameQtCore = "QtCore4.dll"; }
>
> auto h = Runtime.loadLibrary(nameQtCore); // Loading dll or so
>
> // Load function constructor QByteArray::QByteArray(char*);
> QByteArray_QByteArray = cast(t_QByteArray_QByteArray)GetProcAddress(h, "_ZN10QByteArrayC1EPKc");
> // QByteArray::fill(char*, int);
> QByteArray_fill = cast(t_QByteArray_fill)GetProcAddress(h, "_ZN10QByteArray4fillEci");
> // QByteArray::~QByteArray()
>
> // Create our experimental subject and consider its data
> DQByteArray ba = new DQByteArray(cast(char*)"ABC".ptr);
> printf("\n ba.data() = %s", ba.data());
>
> // Experience the action of the fill() and see the result
> ba.fill('Z', 5);
> printf("\n ba.data() = %s", ba.data());
>
> return 0;
> }
Hi yes I think noticed in another thread that you were wrapping Qt with dynamic loading of the qt libs, interesting idea - does your code allow subclassing of the Qt classes and overriding the virtual methods? I'm taking a much more traditional approach, but there is method in my madness :-)
|
Copyright © 1999-2021 by the D Language Foundation