On Friday, 23 January 2015 at 00:24:45 UTC, Andrei Alexandrescu wrote:
> I think it's important that we enable Calypso (https://github.com/Syniurge/Calypso) and related tooling that interfaces D with other languages, notably C++.
>
> A key topic in 2015 for D is playing well with C++. A good C++ interface will give access to a host of mature C++ libraries, starting of course with the C++ standard library. More importantly, it will provide a smooth migration path for organizations that want to do development in D whilst taking advantage of their legacy code.
>
> I'd like to open the topic of "what can we do in core D to make Calypso better".
>
> But first, I want to get better acquainted with Calypso and raise awareness of it with coworkers and the larger community. To my dismay, the github homepage provides exactly zero "look and feel" use examples beyond the mechanics of building Calypso itself.
>
> To Calypso's creator: is it possible to beef up the documentation of Calypso with a few use cases? Ideally there'd be a soup-to-nuts step by step guide of making a C++ library (either artificial or extant) interfacing with D. Also: what things on the D side would be necessary to make the interface better? Exceptions would be an obvious topic on which we have an attack already (more on it later).

Andrei thanks for finally sharing your thoughts on the matter :-)

I'll see what I can do this week-end to expand examples and write a tutorial.

As for what could be done in core D, I haven't struck anything blocking while writing Calypso. Exception catching is next after the first light of my Ogre3D demo, Clang will probably simplify handling of C++ exceptions a lot.

On 1/23/15 10:27 AM, Elie Morisse wrote:
> Exception catching is next after the first light of my Ogre3D demo,
> Clang will probably simplify handling of C++ exceptions a lot.

On that front, a coworker gave me a simple idea for integration.

1. All C++ non-polymorphic types thrown cannot be caught in D

2. We (or Calypso) define D class counterparts for polymorphic types thrown from C++, notably std::exception and subtypes. N.B. those are D classes, not D structs

3. D code can catch std::exception. The only caveat is that the caught exception cannot survive the catch statement. Given that calling into C++ is not the paramount of safety in the first place, I'd say that's a reasonable compromise.

Example:

----
import core.stdcpp.vector;
import core.stdcpp.exception;

core.stdcpp.exception g;

void fun(core.stdcpp.vector!int v)
{
    try
    {
        v.push_back(42);
    }
    catch (core.stdcpp.exception e) // fine, can catch
    {
        g = e; // ouch, will dangle
    }
}
----

Thoughts?


Andrei

On 2015-01-23 19:43, Andrei Alexandrescu wrote:

> On that front, a coworker gave me a simple idea for integration.
>
> 1. All C++ non-polymorphic types thrown cannot be caught in D
>
> 2. We (or Calypso) define D class counterparts for polymorphic types
> thrown from C++, notably std::exception and subtypes. N.B. those are D
> classes, not D structs
>
> 3. D code can catch std::exception. The only caveat is that the caught
> exception cannot survive the catch statement. Given that calling into
> C++ is not the paramount of safety in the first place, I'd say that's a
> reasonable compromise.
>
> Example:
>
> ----
> import core.stdcpp.vector;
> import core.stdcpp.exception;
>
> core.stdcpp.exception g;
>
> void fun(core.stdcpp.vector!int v)
> {
>      try
>      {
>          v.push_back(42);
>      }
>      catch (core.stdcpp.exception e) // fine, can catch
>      {
>          g = e; // ouch, will dangle
>      }
> }
> ----
>
> Thoughts?

Even with these restrictions C++ exceptions are quite complicated. I've probably said this before but in 64bit Objective-C (which uses the same exception model as C++) you can only catch C++ exceptions with the catch-all block which won't let you access the actual exception:

@try {
    v.push_back(42);
}
@catch (...) { }

Not sure if that makes it particular easier to implement.

-- 
/Jacob Carlborg

The README should be clearer now about what Calypso is supposed to do, links to the showcase example which I improved and expanded with a template partial and explicit spec example, and explains how to build it and link it to a C++ library.


(thanks to the simple partial spec example a nasty oversight about partial specializations was also fixed in the process, which may well be the #1 source of the snags I hit while importing Ogre)

On 1/24/2015 5:26 PM, Elie Morisse wrote:
> The README should be clearer now about what Calypso is supposed to do, links to
> the showcase example which I improved and expanded with a template partial and
> explicit spec example, and explains how to build it and link it to a C++ library.
>
>
> (thanks to the simple partial spec example a nasty oversight about partial
> specializations was also fixed in the process, which may well be the #1 source
> of the snags I hit while importing Ogre)

I'm sorry, but it still far from clear just what Calypso does.

Suppose I have a C++ file, test.cpp, that contains:

    int foo(unsigned *p);

How do I use Calypso to interface with that from D:

    ... what happens here ? ...
    uint x;
    foo(&x);


Next, in the README example, it says:

  $ clang++ -std=c++11 -c showcase.cpp -o showcase.cpp.o
  $ ar rcs libshowcase.a showcase.cpp.o
  $ ldc2 -cpp-args -std=c++11 -Llibshowcase.a -L-lstdc++ showcase.d

Where's Calypso in that?

On Sunday, 25 January 2015 at 10:18:34 UTC, Walter Bright wrote:
> I'm sorry, but it still far from clear just what Calypso does.
>
> Suppose I have a C++ file, test.cpp, that contains:
>
>     int foo(unsigned *p);
>
> How do I use Calypso to interface with that from D:
>
>     ... what happens here ? ...
>     uint x;
>     foo(&x);
>
>
> Next, in the README example, it says:
>
>   $ clang++ -std=c++11 -c showcase.cpp -o showcase.cpp.o
>   $ ar rcs libshowcase.a showcase.cpp.o
>   $ ldc2 -cpp-args -std=c++11 -Llibshowcase.a -L-lstdc++ showcase.d
>
> Where's Calypso in that?


Calypso is part ldc2 and, as far as I understand it, is invoked via the "-cpp-args" command line arg.

I believe the simplest example that shows the use of your function above is as follows:

-----------------------------------------
test.h
-----------------------------------------

namespace test {
  int foo(unsigned int *p);
}


-----------------------------------------
test.cpp
-----------------------------------------

#include "test.h"
int test::foo(unsigned int *p)
{
  return *p * 2;
}

-----------------------------------------
test.d
-----------------------------------------

modmap (C++) "test.h";

import (C++) test._;  //this imports global vars, funcs and typedefs
import std.stdio;

void main()
{
  uint x = 4;
  writeln("foo = ", foo(&x));
}

-----------------------------------------
BUILD
-----------------------------------------

#!/bin/bash
clang++ -std=c++11 -c test.cpp -o test.cpp.o
ar rcs libtest.a test.cpp.o
/bin/rm calypso_cache*
ldc2 -v -cpp-args -std=c++11 -Llibtest.a -L-lstdc++ test.d

-----------------------------------------
-----------------------------------------

The above was cut-n-pasted from a test dir. Tested on Linux. Produces a 'test' executable that prints 'foo = 8' when run.

I have tested more complex examples and things seem to work quite well. The one big caveat that I have run into so far with Calypso is that everything must be in a namespace...that is why test.h has the somewhat superfluous 'test' namespace. It must be there to use Calypso, as far as I can see.

This can be an issue when trying to import 'C++' libraries if they just use .h files like glorified C header files with some classes in them, or something like that, and they don't encapsulate everything in a unique namespace.

I am sure Elie will chime in, but I am definitely impressed and inspired by what he has accomplished here. Hopefully I haven't butchered the above explanation too much since I am not a Calypso expert  :)

Thanks,
Kelly

On Sunday, 25 January 2015 at 10:18:34 UTC, Walter Bright wrote:
> Next, in the README example, it says:
>
>   $ clang++ -std=c++11 -c showcase.cpp -o showcase.cpp.o
>   $ ar rcs libshowcase.a showcase.cpp.o
>   $ ldc2 -cpp-args -std=c++11 -Llibshowcase.a -L-lstdc++ showcase.d
>
> Where's Calypso in that?

On Sunday, 25 January 2015 at 11:12:50 UTC, Kelly wrote:
> Calypso is part ldc2 and, as far as I understand it, is invoked via the "-cpp-args" command line arg.

-cpp-args is only to pass arguments to Clang while generating the precompiled header.

Calypso registers itself as a "language plugin", and when parse.c encounters « import (ABC) xxx.yyy; » it asks the registered plugins if one of them handles the "ABC" language id. If that's the case it lets the plugin create the Import symbol by itself, for instance Calypso creates a cpp::Import which derives from Import and has a completely different load() method, which won't look for modules in .d files but inside the PCH generated by Clang.

Here's the LangPlugin interface:

class LangPlugin
{
public:
    // returns -1 if said lang isn't handled by this plugin, or its id number
    // to be passed to createImport otherwise
    virtual int doesHandleModmap(const utf8_t *lang) = 0;

    virtual Modmap *createModmap(int langId,
        Loc loc, Expression *arg) = 0;

    // returns -1 if said tree isn't handled by this plugin, or its id number
    // to be passed to createImport otherwise
    virtual int doesHandleImport(const utf8_t *tree) = 0;

    virtual Import *createImport(int treeId,
        Loc loc, Identifiers *packages, Identifier *id,
        Identifier *aliasId, int isstatic) = 0;

    // ===== - - - - - ===== //

    virtual Expression *getRightThis(Loc loc, Scope *sc, AggregateDeclaration *ad,
        Expression *e1, Declaration *var, int flag = 0) = 0;

    // ===== - - - - - ===== //

    virtual FuncDeclaration *buildDtor(AggregateDeclaration *ad, Scope *sc) = 0;
    virtual FuncDeclaration *buildCpCtor(StructDeclaration *sd, Scope *sc) = 0;

    // ===== - - - - - ===== //

     virtual CodeGen *codegen() = 0;
};

getRightThis, buildDtor and buildCpCtor are needed because they "override" the global functions with the same name.

About Andrei's query about a general plugin system I don't know how one could be architectured. The hooks created for Calypso are specific to C++, so the help I could get is simply to be open to those hooks (which aren't really intrusive and don't uglify the code except for one or two that could probably be done differently: https://github.com/Syniurge/Calypso/commit/debd83f49363bf6805573d141a62b25d068d8a14)

The problem with arbitrarily limiting where the hooks could occur is that it might force the plugins to copy and paste redundantly the base functions, which is bad especially for big semantic() functions.

This probably deserves more thought, maybe there's a more elegant way waiting to be found.

On Friday, 23 January 2015 at 18:43:30 UTC, Andrei Alexandrescu wrote:
> 2. We (or Calypso) define D class counterparts for polymorphic types thrown from C++, notably std::exception and subtypes. N.B. those are D classes, not D structs

Wouldn't it be easier with structs? They lack only one feature - inheritance, while classes lack many features necessary to value types: being a value type and deterministic destruction.

On 1/25/2015 7:16 AM, Elie Morisse wrote:
> On Sunday, 25 January 2015 at 10:18:34 UTC, Walter Bright wrote:
>> Next, in the README example, it says:
>>
>>   $ clang++ -std=c++11 -c showcase.cpp -o showcase.cpp.o
>>   $ ar rcs libshowcase.a showcase.cpp.o
>>   $ ldc2 -cpp-args -std=c++11 -Llibshowcase.a -L-lstdc++ showcase.d
>>
>> Where's Calypso in that?
>
> On Sunday, 25 January 2015 at 11:12:50 UTC, Kelly wrote:
>> Calypso is part ldc2 and, as far as I understand it, is invoked via the
>> "-cpp-args" command line arg.
>
> -cpp-args is only to pass arguments to Clang while generating the precompiled
> header.

You see, nobody understands where Calypso is! Is it part of clang, part of ldc, a standalone tool?

I feel like Noah:

https://www.youtube.com/watch?v=bputeFGXEjA 1:44


> Calypso registers itself as a "language plugin", and when parse.c encounters

parse.c is not part of my project. Where is parse.c?

> « import (ABC) xxx.yyy; » it asks the registered plugins if one of them handles
> the "ABC" language id. If that's the case it lets the plugin create the Import
> symbol by itself, for instance Calypso creates a cpp::Import which derives from
> Import and has a completely different load() method, which won't look for
> modules in .d files but inside the PCH generated by Clang.

A language plugin plugging into where?


> Here's the LangPlugin interface:
>
> class LangPlugin
> {
> public:
>      // returns -1 if said lang isn't handled by this plugin, or its id number
>      // to be passed to createImport otherwise
>      virtual int doesHandleModmap(const utf8_t *lang) = 0;
>
>      virtual Modmap *createModmap(int langId,
>          Loc loc, Expression *arg) = 0;
>
>      // returns -1 if said tree isn't handled by this plugin, or its id number
>      // to be passed to createImport otherwise
>      virtual int doesHandleImport(const utf8_t *tree) = 0;
>
>      virtual Import *createImport(int treeId,
>          Loc loc, Identifiers *packages, Identifier *id,
>          Identifier *aliasId, int isstatic) = 0;
>
>      // ===== - - - - - ===== //
>
>      virtual Expression *getRightThis(Loc loc, Scope *sc, AggregateDeclaration *ad,
>          Expression *e1, Declaration *var, int flag = 0) = 0;
>
>      // ===== - - - - - ===== //
>
>      virtual FuncDeclaration *buildDtor(AggregateDeclaration *ad, Scope *sc) = 0;
>      virtual FuncDeclaration *buildCpCtor(StructDeclaration *sd, Scope *sc) = 0;
>
>      // ===== - - - - - ===== //
>
>       virtual CodeGen *codegen() = 0;
> };
>
> getRightThis, buildDtor and buildCpCtor are needed because they "override" the
> global functions with the same name.
>
> About Andrei's query about a general plugin system I don't know how one could be
> architectured. The hooks created for Calypso are specific to C++, so the help I
> could get is simply to be open to those hooks (which aren't really intrusive and
> don't uglify the code except for one or two that could probably be done
> differently:
> https://github.com/Syniurge/Calypso/commit/debd83f49363bf6805573d141a62b25d068d8a14)
>
>
> The problem with arbitrarily limiting where the hooks could occur is that it
> might force the plugins to copy and paste redundantly the base functions, which
> is bad especially for big semantic() functions.
>
> This probably deserves more thought, maybe there's a more elegant way waiting to
> be found.

On Sunday, 25 January 2015 at 20:45:26 UTC, Walter Bright wrote:
>> Calypso registers itself as a "language plugin", and when parse.c encounters
>
> parse.c is not part of my project. Where is parse.c?

here? https://github.com/D-Programming-Language/dmd/blob/master/src/parse.c