On 21 February 2012 16:59, Michel Fortin <michel.fortin@michelf.com> wrote:
On 2012-02-21 11:03:09 +0000, Manu <turkeyman@gmail.com> said:


So I was thinking about this extern(language) thing, the obvious ones are
supported, but it would be really nice to be able to implement custom
conventions for other languages/scripting languages.

For instance, I'm thinking about Android, I have JNI binding code
everywhere, it's really ugly.
I'd love to be able to declare:
 extern(Java) int someJavaFunc(int x, float y)

And then use my function like any regular function, with the 'extern(Java)'
bit handling the JNI business behind the scenes.
I also regularly interact with javascript, lua, C#/mono, and these could
all be implemented the same way.

I'm imaging some mechanism to declare a calling convention (which would be
resolved within the extern(...) statement), and define it with a template,
something like:

callconv Java
{
 R call(T...)
 {
   // process tuple of args, make the call, return something?
 }

 R thisCall(Class, T...)
 {
   // also need a way to implementing methods... this might be enough.
 }
}

Some fancy code in there could conceivably call into any foreign language,
and this would be great!
Now when I: import java.jni;
I have the jni interface, but I also have access to extern(Java), and
that's awesome! :)

In all currently existing cases, extern(Lang) offers not only a way to declare and call foreign-language functions, but also a way to define functions to be called from the foreign language. Basically, you have two-way compatibility. How would that work with Java?

Interesting point.
Java (or any VM based language) can't hard link to the binary, so it never could try and call back in through an extern-ed function directly, it must be explicitly supplied the pointer in the first place. In that case, the API that supplies the callback to Java can produce the appropriate incoming call wrapper.

I can imagine a way to doing a hard-extern under my proposal if you were hard linking against an unsupported language language... you might need to provide a name mangler in the definition block I describe to match the foreign language, and an incoming call template would also need to be defined as a naked function with the responsibility of managing the args/stack appropriately, and calling the externed function as a regular D-call (this would probably only be some light register/stack management, I don't think it would bloat a lot). This would allow D code to also call the externed function, since the physical definition is still a D-call, you just call it directly internally.

Anyway, while I can imagine a solution, I don't think it would be needed immediately. It'll just open a further can of worms like foreign language struct layout as people try to use it in complex scenarios ;)

I don't think this reduces the usefulness of my proposal at all though, ie. being able to apply it to outgoing calls for any foreign language.

- - -

Also, I see you're thinking about methods, which means you're thinking about declaring Java classes. How is that supposed to work exactly?

Well I was thinking about that with respect to calling functions on classes that I have imported from a foreign language, not declared locally for export to a foreign language.
I clearly didn't think that through though, I'll try and clarify my thoughts to that end... I get the feeling that idea was a dead-end though.
 
I have some experience bridging Objective-C and D. I once built a complete wrapper system for Objective-C objects, each object was wrapped by a D one. It worked very well, but it generated so much bloat that it became unusable as soon as I started defining enough classes for it to be useful. See the D/Objective-C bridge: <http://michelf.com/projects/d-objc-bridge/>.

What was the primary cause of the bloat? I can't imagine my proposal causing any more bloat than the explicit jni call (or equivalent) woudl have otherwise.
 
Maybe you'll have more luck, but for me wrappers just couldn't work. So I decided making the compiler understand the Objective-C runtime and object model was a much better solution. It was fun to do and wasn't that hard actually. Here's the result: <http://michelf.com/projects/d-objc/>. Unfortunately I don't have much time to work on it anymore.

That sounds appropriate for Obj-C, but I don't think it applies to VM based languages however you slice it, since there's not actually a direct binary linkage. There's no way to know about the Java/.NET/Lua/etc object model internally, the VM owns that entirely, so you're forced to marshal through API's. That's okay though, you have to do it anyway. I'm proposing a way to unify the concept of calling functions without obscuring the actual call with junk, and still allowing the D language to annotate and optimise to its fullest extent.

More background:
<http://michelf.com/weblog/2010/dobjc-dead-end-start-anew/>


--
Michel Fortin
michel.fortin@michelf.com
http://michelf.com/