I've managed to create a DLL that crashes horribly. Hooray!

There are numerous references in a Google search to Deja Augustine's port of the Python/C API headers to D, but the site (http://www.scratch-ware.net/dfiles/) is down, and apparently has been for a while. Further searching revealed a mirror of some version of the header at http://jcc_7.tripod.com/d/pythond/ .

The header (I quickly discovered) is for version 2.3 of Python, where the current version is 2.4. It was also written for a two-year-old version of D, though no changes on this front appear to be needed. The differences in the Python API between the two versions are minimal (http://docs.python.org/whatsnew/node14.html), so I'm working on the assumption that this header will work with the 2.4 DLL with just a name change.

Anyway, I've managed to get something that compiles and even imports into a Python session without complaint, but as soon as I try to call a function in it, the interpreter locks up and crashes.

The code I have is first based on the "Simple Example" in the Python "Extending and Embedding" docs (http://docs.python.org/ext/simpleExample.html), and also takes code from the D guide to writing DLLs (http://digitalmars.com/d/dll.html).

So there are three source files involved, here: dll.d, taken directly from the D guide to DLLs; python24.d, which is basically Deja's header renamed with some minor modifications; and example.d, which reads thus:

[example.d]
import python24;
import std.c.stdio;

extern (C) PyObject *
ex_foo(PyObject *self, PyObject *args)
{
	printf("Hello, world\n");
	Py_INCREF(Py_None);
	return Py_None;
}

PyMethodDef[] example_methods = [
	{"foo", &ex_foo, 1, "foo() doc string"},
	{ null, null, 0, null }
];

export extern (C)
void
initexample()
{
	Py_InitModule("example", example_methods);
}
// EOF

I also have a python24.lib created with implib:

C:>implib /system python24.lib c:\windows\system32\python24.dll

Finally, I have an example.def:

[example.def]
LIBRARY "example.lib"
EXETYPE NT
SUBSYSTEM WINDOWS,5.0
CODE PRELOAD DISCARDABLE SHARED EXECUTE
DATA PRELOAD SINGLE WRITE

The DLL is compiled with:

C:>dmd -ofexample.dll dll.d example.d python24.d python24.lib example.def

The example.dll is generated. I then start a python session and do the following:

>>> import sys
>>> # Add the location of example.dll to the Python path
>>> sys.path.append('C:\\Projects\\pyd_test')
>>> import example
>>> example.foo
<built-in function foo>

This is good so far as it goes, but actually calling foo:

>>> example.foo()
Hello, world

And then CRASH! Windows politely informs me that python.exe has encountered a problem, and would I like to send an error report to Microsoft?

The fact that "Hello, world" is actually printed out is encouraging. However, I'm slightly stumped as to where to go next.

-Kirk McDonald

In article <e52igv$2pd$1@digitaldaemon.com>, Kirk McDonald says...
>
>There are numerous references in a Google search to Deja Augustine's
>port of the Python/C API headers to D, but the site
>(http://www.scratch-ware.net/dfiles/) is down, and apparently has been
>for a while.
>...
>The header (I quickly discovered) is for version 2.3 of Python, where
>the current version is 2.4.

Deja Augustine's version is outdated and also limited in its coverage of the Python API.  Try this: http://kinterbasdb.sourceforge.net/other/d/celerid-2006_03_19.zip

That zip includes a trivial example.  It worked fine with Python 2.4.x and the then-current version of D last time I tried it (which was March 19, 2006).

However, it doesn't work at all on non-Windows platforms because [the Digital Mars implementation of] D only supports dynamic linking on Windows.  That fact alone prevents me from using D for anything useful.

David Rushby wrote:
> In article <e52igv$2pd$1@digitaldaemon.com>, Kirk McDonald says...
> 
>>There are numerous references in a Google search to Deja Augustine's port of the Python/C API headers to D, but the site (http://www.scratch-ware.net/dfiles/) is down, and apparently has been for a while.
>>...
>>The header (I quickly discovered) is for version 2.3 of Python, where the current version is 2.4.
> 
> 
> Deja Augustine's version is outdated and also limited in its coverage of the
> Python API.  Try this:
> http://kinterbasdb.sourceforge.net/other/d/celerid-2006_03_19.zip
> 
> That zip includes a trivial example.  It worked fine with Python 2.4.x and the
> then-current version of D last time I tried it (which was March 19, 2006).
> 
> However, it doesn't work at all on non-Windows platforms because [the Digital
> Mars implementation of] D only supports dynamic linking on Windows.  That fact
> alone prevents me from using D for anything useful.


That's a really nice package. I've been spending the past day or so looking through it. ("Celerid"? Where does that name come from?) It is a pity about DMD not doing dynamic linking outside of Windows, though your header is still useful for embedding Python in a D program on a Linux system:

[test.d]
import python;

int main() {
    Py_Initialize();
    PyRun_SimpleString("print 'Hello world!'\n");
    Py_Finalize();
    return 0;
}

$ dmd test -L-lpython2.4
gcc test.o -o test -lphobos -lpthread -lm -Xlinker -lpython2.4
$ ./test
Hello, world!

Or, for that matter, on Windows.

-Kirk McDonald

In article <e55d00$nl2$1@digitaldaemon.com>, Kirk McDonald says...
> That's a really nice package.  I've been spending the past day or so looking through it. ("Celerid"? Where does that name come from?)

From "celerity", which means "swiftness of action or motion".  It's intended to refer to the fusion of the development speed of Python with the execution speed of D.

> It is a pity about DMD not doing dynamic linking outside of Windows, though your header is still useful for embedding Python in a D program on a Linux system:

Indeed, that's worth noting.  However, extending Python with a compiled language is a vastly more popular option than embedding it in the compiled language.

I can only speak for myself, but the reason I strongly prefer extending Python to embedding it is that extension is more compatible with my Lispish development style, which tends toward bottom-up design and loose coupling.  My Python programs are usually structured as loose collections of modules that can be imported and called by yet-to-be-written code, and some of these modules are also launchable as standalone programs to perform specific subtasks.

At the outset, I usually launch a given body of code both "the way it would normally be run" and via a test suite collector.  So right there are two "launch points" that I wouldn't want compiled into a single executable.  Also, I often end up importing and calling code that I wrote months earlier from a context that I couldn't have foreseen when I wrote the code.  Embedding rather than extending would complicate that approach greatly.

The performance-critical code that I would write in D instead of Python tends to appear at the lower levels of program organization.  In my programs, a given unit of such code typically concerns itself with *one thing only*, and it is then combined with other code by a high-level Python module.  The coupling between program units takes place at the Python level, where unforeseen reuse and radical reorganization are much easier.

So embedding Python in D is an interesting option, but it definitely wouldn't suit my development style.