This will be my first posting here, but I've been obsessed with D since before there was a D2 standard. Despite that rather long period of obsession, it's only just now that I've started seriously using D as I attempt to code-up some POC projects to demonstrate that the language is mature enough for serious development, or at least the kind of development we're doing, anyway.

So, that out of the way, I suppose I should apologize in advance for what must amount to a pretty dumb question, since I truly couldn't have been the first to think of this: with regard to interfacing D with C libraries, why would we require a project like Deimos which ensures that each codebase is ported, almost entirely, by hand? Theoretically, shouldn't there be some kind of autotools-like method to establish something like, config.d.in which is then used via abstractions to properly configure the rest of the D wrapper? Obviously, D doesn't have a preprocessor, but couldn't one synthesize that with a series of judicious version() blocks? Then, whenever someone goes to build the application on their system, you'd have something to a series of autoconf-like tests to fill in the proper sections of of config.d.in and get out a config.d suitable for their system.

I'm really sorry if there's some stupidly obvious reason this couldn't work and I'm just not seeing it, or if this is some n00b question you get all the time.

On 2012-10-15 13:51, CapitalLetterC wrote:
> This will be my first posting here, but I've been obsessed with D since
> before there was a D2 standard. Despite that rather long period of
> obsession, it's only just now that I've started seriously using D as I
> attempt to code-up some POC projects to demonstrate that the language is
> mature enough for serious development, or at least the kind of
> development we're doing, anyway.
>
> So, that out of the way, I suppose I should apologize in advance for
> what must amount to a pretty dumb question, since I truly couldn't have
> been the first to think of this: with regard to interfacing D with C
> libraries, why would we require a project like Deimos which ensures that
> each codebase is ported, almost entirely, by hand? Theoretically,
> shouldn't there be some kind of autotools-like method to establish
> something like, config.d.in which is then used via abstractions to
> properly configure the rest of the D wrapper? Obviously, D doesn't have
> a preprocessor, but couldn't one synthesize that with a series of
> judicious version() blocks? Then, whenever someone goes to build the
> application on their system, you'd have something to a series of
> autoconf-like tests to fill in the proper sections of of config.d.in and
> get out a config.d suitable for their system.
>
> I'm really sorry if there's some stupidly obvious reason this couldn't
> work and I'm just not seeing it, or if this is some n00b question you
> get all the time.

The header files need to be ported so D can find the declarations of the functions and types. There are a couple of tools that can do most of the work, but not completely. Macros are specially hard to port automatically.

https://github.com/jacob-carlborg/dstep

-- 
/Jacob Carlborg

On Monday, 15 October 2012 at 12:54:51 UTC, Jacob Carlborg wrote:
> On 2012-10-15 13:51, CapitalLetterC wrote:
>> This will be my first posting here, but I've been obsessed with D since
>> before there was a D2 standard. Despite that rather long period of
>> obsession, it's only just now that I've started seriously using D as I
>> attempt to code-up some POC projects to demonstrate that the language is
>> mature enough for serious development, or at least the kind of
>> development we're doing, anyway.
>>
>> So, that out of the way, I suppose I should apologize in advance for
>> what must amount to a pretty dumb question, since I truly couldn't have
>> been the first to think of this: with regard to interfacing D with C
>> libraries, why would we require a project like Deimos which ensures that
>> each codebase is ported, almost entirely, by hand? Theoretically,
>> shouldn't there be some kind of autotools-like method to establish
>> something like, config.d.in which is then used via abstractions to
>> properly configure the rest of the D wrapper? Obviously, D doesn't have
>> a preprocessor, but couldn't one synthesize that with a series of
>> judicious version() blocks? Then, whenever someone goes to build the
>> application on their system, you'd have something to a series of
>> autoconf-like tests to fill in the proper sections of of config.d.in and
>> get out a config.d suitable for their system.
>>
>> I'm really sorry if there's some stupidly obvious reason this couldn't
>> work and I'm just not seeing it, or if this is some n00b question you
>> get all the time.
>
> The header files need to be ported so D can find the declarations of the functions and types. There are a couple of tools that can do most of the work, but not completely. Macros are specially hard to port automatically.
>
> https://github.com/jacob-carlborg/dstep

Actually, I understand perfectly well why the headers need to be ported to D code, what I was more wondering is if anybody knew of or had ever tried to use some kind of build system to automate the process to the greatest extent possible. So, for example, as a result of the configuration process certain features may be implicitly or explicitly enabled/disabled, without probing the individual user's configuration, you wouldn't know that, so how does one maximize portability, in that case? If you simply go to support the maximal set of features, you may wind up with a bunch of codepaths that go nowhere, likewise, if you choose only to support the minimal set of functionality, you may not wind up with code that's not useful to anyone, yourself included. Granted, I hadn't thought of the difficulty in porting things like macros, but I'd imagine that, in the absolute worst case, you could synthesize their behavior with templates.

But there are all sorts of issues in that same vein, like how certain structs get put together, what types are aliased with what, etc. That's why I was wondering if you could handle this analogously to how C does, with a config file that gets setup by a kind of configuration system.

On 2012-10-15 19:20, CapitalLetterC wrote:

> But there are all sorts of issues in that same vein, like how certain
> structs get put together, what types are aliased with what, etc. That's
> why I was wondering if you could handle this analogously to how C does,
> with a config file that gets setup by a kind of configuration system.

I think that would be possible but I haven't seen anyone program in D like that.

-- 
/Jacob Carlborg

On Monday, 15 October 2012 at 20:09:03 UTC, Jacob Carlborg wrote:
> I think that would be possible but I haven't seen anyone program in D like that.

One of the primary reasons I was concerned that there might be some serious flaw I'm overlooking. Oh well, that's a good enough excuse for me to begin experimentation. Thanks! :D

I don't think this is exactly what you want either, but it's certainly a lot easier than a manual process:

http://gentoo.com/di/mkconfig.html
(On Freecode) https://freecode.com/projects/mkconfig

I just now released a new version.  Freecode will be updated within 24 hours.
Check the examples/dtcltk/ directory.

There's also a simpler example within the D/ subdirectory in:
  http://gentoo.com/di/

Macros have to have their signature specified. e.g.:

    cmacro Tcl_InitMemory tcl.h void 'Tcl_Interp *'
    cmacro ScreenOfDisplay X11/Xlib.h 'Screen *' 'Display' int

 -- Brad

Manually porting of C/C++ libraries shouldn't be the way to go (a
major problem being when said library gets updated, human errors,
porting to different architectures etc).

I've had good experience using SWIG to automate the job for me,
and used it successfully to port the following libraries to D:

* ncurses
* opencv
* sfml
to name a few.

I believe this is the way forward, as opposed to maintaining
(necessary stale) ports such as Deimos.

Swig isn't perfect however, and sometimes will bail out on
constructs such as:
C++ classes with multiple inheritance, or a few obscure C or C++
constructs. Even in that case, I find it much easier to tweak the
swig interface file to get desired output rather than undertake
the huge manual conversion task. For sfml, the interface file is
just about 40 lines for example.

Improving such kind of automated tools is where we should be
concentrating efforts, not on maintaining deimos. One such
improvement I made was to allow conversion of C++ templates to D
templates (works in large majority of cases), which makes the
library much easier to use than using name mangling schemes. More
improvements would be welcome (eg support for static library
generation as opposed to dynamic load libraries), reducing the
need for tweaking swig interface files etc.

On Sunday, 21 October 2012 at 19:45:53 UTC, timotheecour wrote:
> I've had good experience using SWIG to automate the job for me,
> and used it successfully to port the following libraries to D:

Yes.  SWIG should be a lot faster than my tool.
Also I don't have any units to handle C++ code.

Completely agree. It has to be automated.

> Swig isn't perfect however, and sometimes will bail out on
> constructs such as:
> C++ classes with multiple inheritance, or a few obscure C or C++
> constructs. Even in that case, I find it much easier to tweak

If I recall, I ran into some obscure C constructs in the Tcl/Tk/X headers that I have never seen before.  I believe I had to various aliases to my "interface file" to handle those.  Also use of D reserved words had to be handled:

  substitute new new_
  substitute class class_
  substitute 'function;' 'function_;'

And some things just plain won't work in D:

  # not valid D code.
  substitute '_XPrivate .private9, .private10;' \
            '_XPrivate * private9, private10;'

On 2012-10-21 21:45, timotheecour wrote:
> Manually porting of C/C++ libraries shouldn't be the way to go (a
> major problem being when said library gets updated, human errors,
> porting to different architectures etc).

SWIG requires some kind of interface files, which I assume one must have to manually write. Then what's the difference?

> I've had good experience using SWIG to automate the job for me,
> and used it successfully to port the following libraries to D:
>
> * ncurses
> * opencv
> * sfml
> to name a few.
>
> I believe this is the way forward, as opposed to maintaining
> (necessary stale) ports such as Deimos.
>
> Swig isn't perfect however, and sometimes will bail out on
> constructs such as:
> C++ classes with multiple inheritance, or a few obscure C or C++
> constructs. Even in that case, I find it much easier to tweak the
> swig interface file to get desired output rather than undertake
> the huge manual conversion task. For sfml, the interface file is
> just about 40 lines for example.
>
> Improving such kind of automated tools is where we should be
> concentrating efforts, not on maintaining deimos. One such
> improvement I made was to allow conversion of C++ templates to D
> templates (works in large majority of cases), which makes the
> library much easier to use than using name mangling schemes. More
> improvements would be welcome (eg support for static library
> generation as opposed to dynamic load libraries), reducing the
> need for tweaking swig interface files etc.

I don't think SWIG is the right approach because:

* You have to create these interface files
* It seem to not handle all code
* No support for Objective-C

BTW, how does SWIG handle macros and documentation?

-- 
/Jacob Carlborg

> SWIG requires some kind of interface files, which I assume one must have to manually write. Then what's the difference?

These can be as small as:


sfml.i:
%{
#include <SFML/Audio.hpp>
%}
%include SFML/Config.hpp
%include SFML/Audio/Export.hpp

In practice, some additional tweaks may be needed to support certain constructs like multiple inheritance, etc, but nothing daunting (~ 50 lines of code for my sfml interface file)

> I don't think SWIG is the right approach because:
> * You have to create these interface files

=>
see above. Interface files can be very small.

> * It seem to not handle all code
I was still able to port large libraries like sfml and opencv, using tweaks to the interface files and enhancing swig to support C++template to D template conversion. There's a small number of cases where swig currently chokes, some of which could be easily fixed if there's enough demand.

> * No support for Objective-C

=>
http://www.swig.org/Doc1.1/HTML/SWIG.html#n7
One of SWIG's most recent additions is support for Objective-C parsing. This is currently an experimental feature that may be improved in future releases.

> BTW, how does SWIG handle macros and documentation?
You'd have to write a C/C++ function that does what the macro does in the interface file, which would in turn be converted to a D function.

For exporting C documentation to D, I've asked on stackoverflow: http://stackoverflow.com/questions/12413957/export-comments-in-swig-wrapper
I'm not sure it's supported ATM, but that's definitely something possible since swig has a full blown C++ parser.