I've been thinking about how to generate bindings for C++ (esp. Qt!) for a little while now.  Unfortuneatly, I won't have time to implement anything until my semester is over, but with the recent talk of projects, maybe someone else will pick it up before then.

Qt bindings come up every so often; they are not going to be written manually.  The existing Qdd[1] was a port of Qt Jambi[2], which was the Java binding.  This makes some sense, since Java is also an object-oriented language that isn't C++.  Qt Jambi consists of two parts: a generator and generated code.  The generator reads the Qt headers as well as XML files that describe how to map the C++ into Java.  The generator creates both C++ and Java files that get compiled into a JNI library.  Given that D is conceptually closer to C++ than Java, I think that binding to D should be easier than Java.  Another issue here is that both Qt Jambi and QtD target Qt 4 and not version 5.

Also relevant is that the libclang bindings in Deimos[3] are 2 years and several versions old (3.1 -> 3.4 is out).  This is probably due to the lack of interest in libclang from D programmers (No surprise.  Who wants to write code in D to parse C(++)?).  It binds to libclang's C interface and may have been generated with DStep.

Here's my proposed path forward:
Write a binding generator in C++ that uses libclang to parse headers and auxillary XML / json / etc. files to guide the generation of D source. Clang provides data structures for traversing the C++ AST.  This tool could be used to bind Qt 5 and libclang so the conversion tool can eventually be migrated to D.  My hope is that using clang to avoid writing another C++ parser and D's similarity to C++ make this an easier (though still significant) undertaking than the Java bindings.

I suspect that there are several patterns in C++ and D that are implemented using slightly different constructions in each of them.  For instance, figuring out which C++ classes should map to D structs. Automatically identifying these or finding a description language for them would be a huge benefit to automation.

On the other hand, I don't yet see a way to map generic containers (such as QList<T>) into D yet.  But Qt Jambi found some way to solve this problem even when dealing with type erasure in Java.

Thoughts?

Thanks for reading my wall of text.
Paul O'Neil

[1]
QtD is not actively maintained.  It existed on
BitBucket: https://bitbucket.org/qtd/repo
before someone else forked it and moved to
GitHub: https://github.com/qtd-developers/qtd

[2]
Homepage (appears dead): http://qt-jambi.org/
Gitorious (shows recent activity): https://qt.gitorious.org/qt-jambi

[3]
Github: https://github.com/D-Programming-Deimos/libclang

[4]
Clang: http://clang.llvm.org
An example of using Clang to find CXXRecordDecl (classes, unions, structs, and the like):
http://clang.llvm.org/docs/RAVFrontendAction.html

On Friday, 14 February 2014 at 23:52:43 UTC, Paul O'Neil wrote:
> I've been thinking about how to generate bindings for C++ (esp. Qt!) for a little while now.  Unfortuneatly, I won't have time to implement anything until my semester is over, but with the recent talk of projects, maybe someone else will pick it up before then.

Why wouldn't you want to expand on DStep and add support for C++, it already uses clang?

> [1]
> QtD is not actively maintained.  It existed on
> BitBucket: https://bitbucket.org/qtd/repo
> before someone else forked it and moved to
> GitHub: https://github.com/qtd-developers/qtd

Probably worth mentioning.
There are also some recent startup efforts to generate qt bindings based on smoke qt.

https://github.com/w0rp/dqt
https://github.com/GlobecSys/smoke.d

However, its qt4.

On Sat, 2014-02-15 at 03:36 +0000, michaelc37 wrote:
> > [1]
> > QtD is not actively maintained.  It existed on
> > BitBucket: https://bitbucket.org/qtd/repo
> > before someone else forked it and moved to
> > GitHub: https://github.com/qtd-developers/qtd

There is https://bitbucket.org/michaelc37/qtd-experimental as well. This was a bit active for a while but has gone quiet recently.

> Probably worth mentioning.
> There are also some recent startup efforts to generate qt
> bindings based on smoke qt.
> 
> https://github.com/w0rp/dqt https://github.com/GlobecSys/smoke.d
> 
> However, its qt4.

If using Qt let's stick to Qt5 and not create brand new legacy APIs.

In the Pythonverse, Riverbank have some mechanism for creating their Python bindings, but I bet the tool is proprietary. PySide used some binding generation tool that appears to have to be rewritten for Qt5 so they are staying with Qt4. This has stopped the PyQt → PySide rush for licencing reasons, people are just using PyQt5 and finding ways round the licencing problems.

-- 
Russel. ============================================================================= Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder@ekiga.net 41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel@winder.org.uk London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder

On 2014-02-15 04:04, Jesse Phillips wrote:

> Why wouldn't you want to expand on DStep and add support for C++, it
> already uses clang?

Yes, it's using libclang.

-- 
/Jacob Carlborg

On 2014-02-15 00:52, Paul O'Neil wrote:

> Also relevant is that the libclang bindings in Deimos[3] are 2 years and
> several versions old (3.1 -> 3.4 is out).  This is probably due to the
> lack of interest in libclang from D programmers (No surprise.  Who wants
> to write code in D to parse C(++)?).

I originally create those bindings to be used by DStep. But never bother updating them after that. DStep contains its own bindings. I haven't updated the bindings to anything later than 3.1 since I don't use any of these features yet.

> It binds to libclang's C interface  and may have been generated with DStep.

Actually, I just did some simple search-and-replace on the original C header. libclang is the easiest binding I've created :)

> Here's my proposed path forward:
> Write a binding generator in C++ that uses libclang to parse headers and
> auxillary XML / json / etc. files to guide the generation of D source.
> Clang provides data structures for traversing the C++ AST.  This tool
> could be used to bind Qt 5 and libclang so the conversion tool can
> eventually be migrated to D.  My hope is that using clang to avoid
> writing another C++ parser and D's similarity to C++ make this an easier
> (though still significant) undertaking than the Java bindings.

Why not contribute to DStep?

> I suspect that there are several patterns in C++ and D that are
> implemented using slightly different constructions in each of them.  For
> instance, figuring out which C++ classes should map to D structs.
> Automatically identifying these or finding a description language for
> them would be a huge benefit to automation.
>
> On the other hand, I don't yet see a way to map generic containers (such
> as QList<T>) into D yet.  But Qt Jambi found some way to solve this
> problem even when dealing with type erasure in Java.

You could hard code a couple of these containers. QList<T> could be converted to T[] or a linked list. I've done that with a couple of other types: like "BOOL" is translated to "bool" and "wchar_t" is translated to "wchar".

-- 
/Jacob Carlborg

On Saturday, 15 February 2014 at 11:25:09 UTC, Jacob Carlborg wrote:
> and "wchar_t" is translated to "wchar".

That doesn't seem correct, it should be `wchar` on Windows only, and `dchar` on other systems, i.e. it should translate to core.stdc.stddef.wchar_t.

On 2014-02-15 12:54, Jakob Ovrum wrote:

> That doesn't seem correct, it should be `wchar` on Windows only, and
> `dchar` on other systems, i.e. it should translate to
> core.stdc.stddef.wchar_t.

I was mistaken. Currently DStep recognizes the "wchar_t" typedef, then it checks the size of it, translating to either "wchar" or "dchar". This is correct but will not generate cross-platform bindings. core.stdc.stddef.wchar_t seems like a better choice. Thanks.

-- 
/Jacob Carlborg

Can SWIG[1] be used to help here? It even supports D somehow. As I understand, Python uses SIP[2] to create Qt and other bindings.

[1]http://www.swig.org/
[2]http://www.riverbankcomputing.co.uk/software/sip/intro

I'm working on a C++ binding generator that is coming on well,
called Smidgen (first there was SWIG, then sip, now Smidgen...).
I have a small demo working with both VTK and the beginnings of
Qt5 GUI. I have to pick it back up (after moving house with my
family) so would like help on various things such as building it
on Windows (currently Linux 64bit only but it's only the build
system that is Linux specific, and it's cmake so should be easy
to get going on Windows).

The code is tested and just-about one-click build. Currently it's
in a subversion repository but I plan to move to github. I'd love
someone to use Smidgen to try wrapping a small C++ library as I
think it would make the job quite easy, if anyone reads this
(since I have not formally announced the thing yet) then please
let me know and I'll get the code out there).

I have targeted the sip format binding specification (as used in
sip / pyqt) so that I then get a Qt binding on the cheap (but it
will therefore be GPL only). It is going well and as I said I
have wrapped a small part of Qt GUI using it (and the PyQt sip
files) which demonstrates working:

* Using the PyQt sip files is working and I am parsing nearly all
sip tags - extending to the rest of the PyQt sip files should not
be _too_ arduous

* Subclassing a wrapped C++ class, and overriding protected and
virtual functions (e.g. paint method and mouseMoveEvent)

* Nested classes (e.g. Qt::Connection)

* Multiple packages / modules

* Enums half-done

* Sip Transfer, TransferThis and TransferBack annotations for
method arguments (which govern object ownership)

* Multiple C++ inheritance => multiple pointers (still needs a
little bit of work but generally there)

* An automated route for type conversion e.g. QString <-> D
string - needs a bit more work for e.g. QList

My TODO List
============


* non-primitive Typedefs

* Primitive types C -> D conversion
    - *.conf file
     %CToDType long = long
     %CToDType unsigned char = ubyte  (half done)

* Sip If clauses for timelines
     %Timeline {Qt_5_0_0 Qt_5_0_1 Qt_5_0_2}  (half done)


* KeepReference for arguments + tests for Transfer etc. - Easy,
in class with KeepRef
   e.g. View.setModel(model /KeepReference/) it has an extra
attribute
       -   void* setModel_SMIKeepRef
       then in setModel() {
         View_setModel_SMIX23(model.wrappedObject);
         setModel_SMIKeepRef = model;
       }

       This will make D keep a reference to the model as long as
the View instance is alive.
       Each view will have its own reference so the total can go
above 1 for a given model.

* getCastPointerForInterface can be easily improved by not
switching on a name but instead
   each class has a separate variable for each base class pointer,
that is populated in
   the constructor

   - each class has one extra pointer per interface implemented
     - override virtual void*[] getExtraPointers() {
         void*[] extraPointers = super.getExtraPointers();
         extraPointers ~= wrappedObject_Calculator;
         return extraPointers;
     }

   - in constructor this() {
         wrappedObject_Calculator =
castRectAsCalculator(wrappedObject);
     }

   - in destructor ~this() {
         deregisterWrappedObject(wrappedObject);
         deregisterWrappedObejct(wrappedObject_Calculator);
     }

* instance_wrapper et al., need to also register base class
pointers

* getWrappedObject / getClassName - how to handle this in a
x-module fashion.

* Add support for wrapping members

* Add QTest support

* int arguments that take a default enum value & enum defaults

* char** -> Use this _idea_
     this(string[] args)
     {
         //        if (m_instance != null)
         //            throw new
RuntimeException("QCoreApplication can only be initialized once");

         argc = cast(int)args.length;
         argv = toStringzArray(args);
         this(&argc, argv);

         //        m_instance.aboutToQuit.connect(m_instance,
"disposeOfMyself()");
     }

* Threading? - wrappedObjects[] should be shared as with CPP
instance tracker. Use signal.d's
   WeakRef and InvisibleAddress

* qRegisterMetaType??

* Add %UsesConverter to package.sip, so that the correct includes
are in the package and
   we don't get problems trying to include e.g. widget.h in core.so

* Converted types that are returned are not passed through
getWrapper - might cause
   duplicate D objects around same C++ object e.g. QList<QWidget>?
Is this a problem?

* In CPP wrapper, the conversion from argument can have a memory
leak (e.g. new string created
  and not deleted)  BETTER - pass new type in as a reference if
possible

* Enums don't have C++ value if it was specified - need a CPP
program that writes
   a text file e.g. Color::Red 124
                    Color::Green 234
   and then write enum wrappers which include the values using the
text lookup (at build time
   of wrapper NOT at runtime of final program)

* Operator overloading - do nicely in D

* Free function operator overloads in CPP

* Handle array arguments

* Multiple inheritance - if a method returns an instance of an
interface and it has
   been created by CPP then the *Impl is returned - it is not
currently typechecked.
   We should instead check if it's already on a registered object.

* Interfaces - allow virtual calls on the interface

* Lifetime management - VTK?

* VTK/ Qt differences - have factory methods for Method, Klass,
Package etc., and
   different wrapping flavours can have differing factories. Needs
API stability.

* Virtual functions, overloading etc. - will the right call be
made if from an inherited class I make a call
   on an inherited D class method for a virtual method - will it
call the C++ base equivalent,
   thereby not calling the correct virtual call on the derived
class? See GOTW 5,
   calling virtual funcs on a base pointer will not call the
derived class method.

* Answer to above Q - yes, so need to implement wrappers for all
inherited virtual functions
   even if not mentioned in SIP file

* Signals and slots

* Enums declared inside classes

* In CPP wrapper class remove factory function/private
constructor - it is not reqd.

* Nested classes and multiple inheritance combination probably
not working (e.g. Klass.getWrappedClassName)