On Tuesday, 18 February 2014 at 11:05:20 UTC, Moritz Maxeiner wrote:
>
> This looks very nice!
> Two questions, though:
>
> 1) Is it possible to use an already existing C API with morsel? The reason I'm asking is, because LLVM classes contain many methods where an automatic conversion is likely to cause memory leaks to appear. This is the main reason why I decided to use the C-API for my llvm-d project, instead of attempting to use automatic tools like SWIG (that and the fact that LLVM did not export C++ symbols properly for a Windows DLL, so you needed to use the C API anyway).
>

At the moment smidgen (morsel is the example C++ library I use for testing) only wraps C++ classes, on the TODO is of course to wrap C++ top level functions etc. It makes a lot of sense to add support for C functions too, but to be totally honest I have no personal need for that ATM so it would come after everything else. I would class smidgen as at beta stage at the moment in terms of the functionality it currently supports. It needs to be (ab)used by some other developers testing it out on some small APIs, and then once I feel it has settled down others could contribute code to support C APIs.

Regarding memory leaks, you're right, to wrap a C++ library there has to be manual effort on the part of the developer to tell the wrapper where object ownership is transferred etc. This cannot be automated. Smidgen is based on the sip format and it's pretty easy to annotate the API to indicate where ownership is transferred.

> 2) What is the main difference between using morsel and SWIG, other than the language used to specify the interface?

I don't know much about SWIG but the biggest difference would have to be SWIG's great maturity and extensive feature set. Smidgen has its own plusses too though:

* It's pure D and the code is tested and maintainable - i.e. if it doesn't work for someone they could probably extend/change it so that it did

* It's a very thin wrapper around the client library, so I suspect it's a bit faster at runtime than the more generic SWIG

* Did I mention it's in D :-)

I should stress that this is beta code and needs a bit of a hammering on smaller libraries before anyone spends a lot of effort with it. However, a smaller library would not take long to wrap using smidgen, so little time would be wasted if it didn't work out.

On Tuesday, 18 February 2014 at 07:49:20 UTC, Andrej Mitrovic wrote:
> On 2/18/14, Joao Matos <joao@tritao.eu> wrote:
>> Nice tools, I've also been working on a C++ binding tool myself
>> (mostly C#/.NET though).
>>
>> https://github.com/mono/CppSharp
>
> Does this use Clang's C++ API or libclang which is a C API? If the
> former, I didn't know you could call into C++ code from C#, at least
> not very easily. Was it hard to do? I guess I'm looking for any
> experiences you can share.

I actually started with the libclang C API but found out that it was too limited, so I switched to the C++ one.

Well, .NET can actually call directly into C++ via C++/CLI (it's limited to Windows though) so that's what the first parser relied on. We now have a second parser that uses pure C++ and uses CppSharp to generate C# and bootstrap itself so that it can work on all platforms.

And yes, it's been pretty hard to do, binding C++ _is_ hard, though D should have a big advantage here, it's semantics are closer to C++ than many other languages.

On Tuesday, 18 February 2014 at 11:52:24 UTC, Abdulhaq wrote:
>
> At the moment smidgen (morsel is the example C++ library I use for testing) only wraps C++ classes, on the TODO is of course to wrap C++ top level functions etc. It makes a lot of sense to add support for C functions too, but to be totally honest I have no personal need for that ATM so it would come after everything else. I would class smidgen as at beta stage at the moment in terms of the functionality it currently supports. It needs to be (ab)used by some other developers testing it out on some small APIs, and then once I feel it has settled down others could contribute code to support C APIs.
>
> Regarding memory leaks, you're right, to wrap a C++ library there has to be manual effort on the part of the developer to tell the wrapper where object ownership is transferred etc. This cannot be automated. Smidgen is based on the sip format and it's pretty easy to annotate the API to indicate where ownership is transferred.
>
>> 2) What is the main difference between using morsel and SWIG, other than the language used to specify the interface?
>
> I don't know much about SWIG but the biggest difference would have to be SWIG's great maturity and extensive feature set. Smidgen has its own plusses too though:
>
> * It's pure D and the code is tested and maintainable - i.e. if it doesn't work for someone they could probably extend/change it so that it did
>
> * It's a very thin wrapper around the client library, so I suspect it's a bit faster at runtime than the more generic SWIG
>
> * Did I mention it's in D :-)


Thanks for the comprehensive write-up, I'll have a look at
smidgen the next time I'm fiddling with llvm-d, then. It seems
promising, but I'm not yet entirely convinced that it'd fit well
with LLVM: About 70% of their C++ API not in the backend is
essentially helpers classes to deal with things either already
present or not needed in languages like D. The problem is that
the methods you actually want to access very often require
arguments with the type being one of these helper classes. And
there's no benefit of having these classes in D other than to use
instances of them as arguments to the rest of the LLVM API, since
their functionality is already present in D or not needed.
Anyway, I'll try it out when I have time and see if it beats
writing the wrapper manually.

On Saturday, 15 February 2014 at 19:57:40 UTC, Abdulhaq wrote:
> I'm working on a C++ binding generator that is coming on well,
> called Smidgen (first there was SWIG, then sip, now Smidgen...).
> I have a small demo working with both VTK and the beginnings of
> Qt5 GUI. I have to pick it back up (after moving house with my
> family) so would like help on various things such as building it
> on Windows (currently Linux 64bit only but it's only the build
> system that is Linux specific, and it's cmake so should be easy
> to get going on Windows).
>

I've tidied a few things up and the Qt5 example is on github at

https://github.com/alynch4047/sqt

It's only wrapping about 20 classes or so, from QtCore and QtWidget. I could easily include many more but it would slow down development of new binding features so I've resisted the temptation.

The main thing missing iro Qt is signals and slots - I know how I will do it, I just a need a couple of days free time.

https://github.com/w0rp/dqt

I have been working on my generator over the past few weeks, and extensively this weekend. It took many hours of work, but I just got generation of D source files from SMOKE data to work. Here is what it does and roughly how it works.

1. Earlier on, SMOKE is by a third party to generate a dynamic library. This library contains data held in a "Smoke" class defined in C++, which functions provided for creating, deleting, and getting a reference to the pointer for the class with what are *supposed* to be functions with C linkage. (I had to wrap them anyway with my own functions due to issues.)
2. Because the class members in Smoke are all C primitives, I can match the layout with a struct in D and load everything from the struct in D code. At the time of generation, I copy everything I need out of the struct into a saner representation in D, which I have called "SmokeContainer."
3. SmokeContainer is supposed to tell you everything you need to know to make generation work. So I use that data and pass it to my class which handles generation of D source files, "SmokeGenerator." It does all kinds of work to translate C++ classes from the SMOKE data to D code, with some ability for customisation like ignoring certain types or classes, replacing classes with different ones in D or just wrappers for generating them, etc.
4. The SmokeGenerator spits out the D source files and requires having a "SmokeLoader" defined in prefix D file, which loads the same SMOKE data again at runtime for finding the class method pointers needed to call C++ functions and so on.
5. You run a D program which uses the generated D source files and now you have C++ code being run from D.

While my project is actually called DQt above and the motivation for it is to get Qt working in D, I have written it all in such a way as to make the Qt specific part of the code and the general case SMOKE binding stuff loosely coupled. The only Qt specific stuff now lies in some C++ code needed for certain circumstances, (like wrapping QString) a few pre-baked D source files used as finishing touches for the generation specific to Qt, and the current test programs which generate code and run Hello World respectively. I shouldn't have too much trouble moving most of this stuff into a "dsmoke" repository instead.

If you want to try out running Hello World from the generated code yourself, and you're willing to work around my sloppy project structure, do this.

1. Build smokegen and smokeqt. I wrote two guides for Linux and Windows on this.
   1. Linux: https://github.com/w0rp/dqt/blob/master/doc/smoke_linux_build_guide.rst
   2. Windows: https://github.com/w0rp/dqt/blob/master/doc/smoke_win32_build_guide.rst
2. Use CMake to build my code. I think 'cmake . && make' will do the job on Linux, if you're in my project directory.
3. Put 'libsmoke_cwrapper' in your library path. I make it easy on myself by copying it out of the lib/ directory into src/ and by running 'export LD_LIBRARY_PATH=.' Not the best way, but it's a quick way to get it going.
4. 'rdmd -g -Llibsmoke_cwrapper.so -d -debug -w new_test.d' Will run the generator and spit some stuff into src/dqt for now.
5. 'rdmd -Llibsmoke_cwrapper.so -d smoke_test.d' will now run the Hello World program, supposing everything else went well.

I am going to continue working on this. There are some things to iron out and a lot of interesting work ahead. Hopefully I can get a somewhat usable Qt in D library out of this.

On 17 February 2014 06:05, Abdulhaq <alynch4047@gmail.com> wrote:

> On Sunday, 16 February 2014 at 02:37:50 UTC, michaelc37 wrote:
>
>>
>>> The code is tested and just-about one-click build. Currently it's in a subversion repository but I plan to move to github. I'd love someone to use Smidgen to try wrapping a small C++ library as I think it would make the job quite easy, if anyone reads this (since I have not formally announced the thing yet) then please let me know and I'll get the code out there).
>>>
>>>  yes github please.. i'd like to check this out.
>>
>
> OK I believe that smidgen is now fully hosted on github.
>
> Using smidgen as it stands I would hope that you could wrap a simple C++ library. However, no-one else has tried it yet so there are bound to be a few teething problems, please feel free to email me (anyone trying it out or just taking a look) at alynch4047 at gmail dot com if you have any questions / problems. Alternatively perhaps we could use one of the forums here??
>
> I have not yet uploaded the initial Qt5 wrapping (or VTK), I'll try to do that over the next day or two.
>

This is awesome. I have a comprehensive game engine, and I've been looking
for a solution to auto-generate my D bindings, but so far, I haven't found
anything capable of doing a proper job.
I'll definitely be checking this out. I maintain bindings by hand
currently, and I'm often fighting with keeping them in sync.

I have a couple of interesting requirements, I'll be curious to see how it works out in practise.

On Monday, 24 February 2014 at 07:40:42 UTC, Manu wrote:
> On 17 February 2014 06:05, Abdulhaq <alynch4047@gmail.com> wrote:
>
>> On Sunday, 16 February 2014 at 02:37:50 UTC, michaelc37 wrote:
>>
>>>
>>>> The code is tested and just-about one-click build. Currently it's
>>>> in a subversion repository but I plan to move to github. I'd love
>>>> someone to use Smidgen to try wrapping a small C++ library as I
>>>> think it would make the job quite easy, if anyone reads this
>>>> (since I have not formally announced the thing yet) then please
>>>> let me know and I'll get the code out there).
>>>>
>>>>  yes github please.. i'd like to check this out.
>>>
>>
>> OK I believe that smidgen is now fully hosted on github.
>>
>> Using smidgen as it stands I would hope that you could wrap a simple C++
>> library. However, no-one else has tried it yet so there are bound to be a
>> few teething problems, please feel free to email me (anyone trying it out
>> or just taking a look) at alynch4047 at gmail dot com if you have any
>> questions / problems. Alternatively perhaps we could use one of the forums
>> here??
>>
>> I have not yet uploaded the initial Qt5 wrapping (or VTK), I'll try to do
>> that over the next day or two.
>>
>
> This is awesome. I have a comprehensive game engine, and I've been looking
> for a solution to auto-generate my D bindings, but so far, I haven't found
> anything capable of doing a proper job.
> I'll definitely be checking this out. I maintain bindings by hand
> currently, and I'm often fighting with keeping them in sync.
>
> I have a couple of interesting requirements, I'll be curious to see how it
> works out in practise.

It would be great if you could give it a run, it's beta software so please email me if you run into any problems.