ECMAScript and D (page 5)

May 01, 2002

Re: Interfacing D to Python

Posted by Patrick Down
in reply to Walter

Permalink

Patrick Down

Posted in reply to Walter

Permalink

"Walter" <walter@digitalmars.com> wrote in news:aaogn4$2pau$1 @digitaldaemon.com:
> 
> What would it take to implement in D:
> 
>     extern (Python) int some_python_function(int foo);
> 
> basically, make D really easy to interface to Python?
> 

First off you need classes too.

extern(Python) class some_python_class
{
  long some_python_method(long foo);
}

The way I usually handle this is to generate a helper
class like the following.

class some_python_class : public CPyObj
{
  public:
  some_python_class() : CPyObj() {}

  some_python_class(some_python_class& rO)
  : CPyObj(rO) {}

  int some_python_method(long p1)
  {
    CPyObj args;
    args.CreateTuple(1);
    args.SetTupleItem(1,PyInt_FromLong(p1));
    CPyObj meth = GetAttr("some_python_method");
    if(!meth.IsValid())
    {
      MethodNotFoundError("some_python_method");
      return 0;
    }
    CPyObj rtn = meth.Call(args);
    return PyInt_AsLong(rtn);
  }
};

CPyObj contains the handle to the real Python object and automatically deals with reference counting and other utility functions.

However this is all trivial compared to calling C code from
Python.  You need to build tables mapping string names to
function pointers.  You need to fill out other data structures that
define things like constructors, destructors, and where to fine
attribute and method tables.  You need code to type check and unpack
parameters from Python's variant type and call the C function with
them.  Then type need to pack up the return value into a
python object and return it.  Are you letting Python control
the lifetime of the object?  Then you need code to track the
reference count and delete it.  "Programming Python" is a good
reference it has a couple of chapters devoted to Python/C interfacing.
I will append an example to the end of this post.

In D you would need the equivalent reverse syntax.  Something this might cover it.

export(Python) int Func(int a, char[] b)
{
  return a;
}

export(Python) class CFoo
{
  export(Python) int Method(int a)
  {
    return a;
  }
}


However I will still go with my opinion that a tool
outside the compiler is a better option.  But I will say
supporting such tools with some user defined attribute
syntax like C# has might be a good thing.  In C# custom
attributes are actually classes derived from System.Attribute
that can be discovered via reflection.  But perhaps we don't
need anything as complicated.

defattr scriptable;

scriptable
int Func(int a,char[] b)
{
  return a;
}

scriptable("constructor=no")
class Foo
{
}

The D front end parser would know how to parse the attribute syntax but wouldn't do anything with the information.  A third party tool could use the D front end to create a code generator for interfacing the D module to a scripting engine, Python or anything else.

#########################

Excerpts from a Python to C example.  The example shown connects to a C++ class called CListControl derived from CBaseControl.

class CListControl
{
  public:
  virtual long AddString(const char*);

  virtual long InsertString(long,const char*);

  virtual long AddStringData(const char*,CPyObj);

  // Cut

  virtual long Get_SelectedIndex();
  virtual void Set_SelectedIndex(long);

  virtual long Get_ListSize();
  virtual void Set_ListSize(long);

  virtual long Get_Enabled();
  virtual void Set_Enabled(long);

  // Cut
};


This is rather old stuff from Python 2.0 days. 2.2 is a little bit easier to handle.  The real cpp file is over 4000 lines long so I've cut out sections.  Forgive me if it's rather ugly.  :-)

#########################

// GUIFramePy.h generated by pyintgen.py
// Wed Sep 19 17:02:33 2001

// Cut some stuff here

#include <PyObj.h>

// Cut some stuff here

struct CListControl_PyObj
{
  PyObject_HEAD
  CListControl* m_pObj;
};

// Cut some stuff here

########################

// GUIFramePy.cpp generated by pyintgen.py
// Wed Sep 19 17:02:33 2001


// Cut some stuff here

#include "listctrl.h"


#include "GUIFramePy.h"

static PyObject* pyErrorObj;

// Cut some stuff here


PyObject* CListControl_AddString(CListControl_PyObj* pObj, PyObject*
pArgs)
{
  const char* p1;
  if(!PyArg_ParseTuple(pArgs,"z" , &p1)) return 0;
  long rtn =  pObj->m_pObj->AddString(p1);
  return PyInt_FromLong(rtn);
}

PyObject* CListControl_InsertString(CListControl_PyObj* pObj, PyObject*
pArgs)
{
  long p1;
  const char* p2;
  if(!PyArg_ParseTuple(pArgs,"iz" , &p1, &p2)) return 0;
  long rtn =  pObj->m_pObj->InsertString(p1,p2);
  return PyInt_FromLong(rtn);
}

PyObject* CListControl_AddStringData(CListControl_PyObj* pObj, PyObject*
pArgs)
{
  const char* p1;
  CPyObj p2;
  if(!PyArg_ParseTuple(pArgs,"zO&" , &p1,Conv_PyObj, &p2)) return 0;
  long rtn =  pObj->m_pObj->AddStringData(p1,p2);
  return PyInt_FromLong(rtn);
}

// Cut some stuff here

PyMethodDef CListControl_methods[] = {
  { "AddString", (PyCFunction)CListControl_AddString, 1 },
  { "InsertString", (PyCFunction)CListControl_InsertString, 1 },
  { "AddStringData", (PyCFunction)CListControl_AddStringData, 1 },
  { "InsertStringData", (PyCFunction)CListControl_InsertStringData, 1 },
  { "DeleteItem", (PyCFunction)CListControl_DeleteItem, 1 },
  { "DeleteAll", (PyCFunction)CListControl_DeleteAll, 1 },
  { "SetData", (PyCFunction)CListControl_SetData, 1 },
  { "GetData", (PyCFunction)CListControl_GetData, 1 },
  { "GetText", (PyCFunction)CListControl_GetText, 1 },
  { "LoadDirectory", (PyCFunction)CListControl_LoadDirectory, 1 },
  { 0, 0, 0 }
};

PyObject* CListControl_ObjAttr(CListControl_PyObj* self, char *pAttrName)
{
  if(strcmp(pAttrName,"SelectedIndex") == 0)
    return PyInt_FromLong(self->m_pObj->Get_SelectedIndex());
  else if(strcmp(pAttrName,"ListSize") == 0)
    return PyInt_FromLong(self->m_pObj->Get_ListSize());
  else if(strcmp(pAttrName,"Enabled") == 0)
    return PyInt_FromLong(self->m_pObj->Get_Enabled());
  else if(strcmp(pAttrName,"LD_READWRITE") == 0)
    return PyInt_FromLong(DDL_READWRITE);
  else if(strcmp(pAttrName,"LD_READONLY") == 0)
    return PyInt_FromLong(DDL_READONLY);
  else if(strcmp(pAttrName,"LD_HIDDEN") == 0)
    return PyInt_FromLong(DDL_HIDDEN);
  else if(strcmp(pAttrName,"LD_SYSTEM") == 0)
    return PyInt_FromLong(DDL_SYSTEM);
  else if(strcmp(pAttrName,"LD_DIRECTORY") == 0)
    return PyInt_FromLong(DDL_DIRECTORY);
  else if(strcmp(pAttrName,"LD_ARCHIVE") == 0)
    return PyInt_FromLong(DDL_ARCHIVE);
  else if(strcmp(pAttrName,"LD_DRIVES") == 0)
    return PyInt_FromLong(DDL_DRIVES);
  else if(strcmp(pAttrName,"LD_EXCLUSIVE") == 0)
    return PyInt_FromLong(DDL_EXCLUSIVE);

  return 0;
}

PyMethodChain CListControl_chain = { CListControl_methods, &CBaseControl_chain };

PyObject* CListControl_getattr(CListControl_PyObj* self, char *pAttrName)
{
  PyObject* pyObjRtn = CListControl_ObjAttr(self,pAttrName);
  if(pyObjRtn)
    return pyObjRtn;

  // Really nasty solution to a multiple inheritance problem
  // Fix this someday
  CListControl* pTmp = self->m_pObj;
  ((CBaseControl_PyObj*)self)->m_pObj = (CBaseControl*)self->m_pObj;

  pyObjRtn = CBaseControl_ObjAttr((CBaseControl_PyObj*)self,pAttrName);

  self->m_pObj = pTmp;

  if(pyObjRtn)
    return pyObjRtn;

  return Py_FindMethodInChain(&CListControl_chain,(PyObject*)
self,pAttrName);
}


int CListControl_setattr(CListControl_PyObj* self, char *pAttrName,
PyObject* pyValue)
{
  if(strcmp(pAttrName,"SelectedIndex") == 0)
  {
    self->m_pObj->Set_SelectedIndex(static_cast<long>(PyInt_AsLong
(pyValue)));
    return 0;
  }
  else if(strcmp(pAttrName,"Enabled") == 0)
  {
    self->m_pObj->Set_Enabled(static_cast<long>(PyInt_AsLong(pyValue)));
    return 0;
  }

  return -1;
}

// Cut some stuff here




void CListControl_dealloc(CListControl_PyObj* self)
{
  MODTRACE("CListControl(%p) dereference\n",self);

  if(self->m_pObj)
    self->m_pObj->ResetPyObject();
#if 0
  delete self->m_pObj;
#endif

  PyMem_DEL(self);
}


PyTypeObject CListControl_type = {
  PyObject_HEAD_INIT(&PyType_Type)
  0,
  "CListControl",
  sizeof(CListControl_type),
  0,

  (destructor)	CListControl_dealloc,
  0,
  (getattrfunc) CListControl_getattr,
  (setattrfunc) CListControl_setattr,
  0,
  0,

  0,
  0,
  0,

  0,
  0,
  0
};

// Cut some stuff here

int Conv_CListControl(PyObject* pyObj,CListControl** pVar)
{
  if(pyObj->ob_type == &CListControl_type)
  {
    *pVar = ((CListControl_PyObj*)pyObj)->m_pObj;
    return 1;
  }
  else if(pyObj == Py_None)
  {
    *pVar = 0;
    return 1;
  }

  PyErr_SetString(pyErrorObj,"Function requires a CListControl");

  return 0;
}

// Cut some stuff here

// Module level functions
PyMethodDef AppFrame_methods[] = {
  { "RunApp", (PyCFunction)RunApp_PyFunc, 1 },
  { "ShowDialog", (PyCFunction)ShowDialog_PyFunc, 1 },
  { "FileBrowse", (PyCFunction)FileBrowse_PyFunc, 1 },
  { "ShellExec", (PyCFunction)ShellExec_PyFunc, 1 },
  {0, 0, 0}
};


void initAppFrame()
{
  PyObject* pMod;
  PyObject* pDict;

  pMod = Py_InitModule("AppFrame", AppFrame_methods);
  pDict = PyModule_GetDict(pMod);

  pyErrorObj = Py_BuildValue("s","AppFrame.error");

  PyDict_SetItemString(pDict,"error",pyErrorObj);

  if(PyErr_Occurred())
    Py_FatalError("Can't initialize module AppFrame");
  else
    InstallDataStruct();
}

On Wed, 01 May 2002 06:57:01 -0700, Russ Lewis <spamhole-2001-07-16@deming-os.org> wrote:

>Walter wrote:
[snip]
>> Anyhow, source to the D front end is now part of the distribution, and if anyone wants to morph that into an interpretted version, that would be a cool addition to D.
>
>I'm going to have to look at the code you've written for that...is it architected where somebody could remove your code and drop in an alternative version, or would it be a pretty involved port?
>
>In a perfect world, there would be a parser module that turns D code into a recursive set of structures that could then be passed to any arbitrary backend: an interpreter, a GCC frontend, a dfront translator, or even a D-- compiler.

Another speaks of the devil.  Working on it now; lexer is done, expression parsing appears done, statement parsing is halfway done, still have to do class/struct/union/function parsing, then cleaning, then testing it to hell, then documentation.  Few days perhaps for a beta?

D in, uh, restricted compilation mode, doesn't even warrant a variant from what's been discussed.  It could be done by making a kind of lint that confirms a file's validity under the stricter rules, then tells the caller whether it's all right to compile the code.

Forums