On Tuesday, 1 May 2012 at 15:56:32 UTC, Alex Rønne Petersen wrote:

> note that FFI is not anything spectacular or innovative;

Have you ever written bindings for any of these scripting languages?

On 01-05-2012 18:08, so wrote:
> On Tuesday, 1 May 2012 at 15:53:37 UTC, Alex Rønne Petersen wrote:
>> On 01-05-2012 17:43, so wrote:
>>> On Tuesday, 1 May 2012 at 15:31:05 UTC, Robert Clipsham wrote:
>>>> On 01/05/2012 16:24, so wrote:
>>>>> http://luajit.org/ext_ffi.html
>>>>> https://github.com/malkia/ufo
>>>>>
>>>>> How awesome is Mike Pall?
>>>>> I didn't dive into details of the code, but if he can do this with a
>>>>> dynamic language, why on earth D still need manual C bindings while
>>>>> having ABI compatibility? So luajit comes with a C compiler?
>>>>
>>>> Note that you can't just drop any C header file in there for that to
>>>> work (as far as I can tell), you still have to bring out individual
>>>> post-processed function declarations.
>>>
>>> https://github.com/malkia/ufo/blob/master/ffi/OpenCL.lua
>>> https://github.com/malkia/ufo/blob/master/ffi/OpenGL.lua
>>> If it can handle these 2 beasts.
>>
>> I see no preprocessor directives.
>
> They are all there as "enum".

That has nothing to do with the C preprocessor... The point here is that you can't just copy/paste a C header into LuaJIT's FFI out of the box. You have to run the preprocessor on it first, or preprocess it by hand.

>
>>>> Also, someone has written a libffi binding for D, which could probably
>>>> be adapted to work in a similar manor:
>>>>
>>>> https://github.com/lycus/libffi-d
>>>
>>> Neat. Still, having native C libraries means that you can just drop your
>>> C/C++ environment and start D. And i am sure you agree this is by far
>>> the biggest blocker for C/C++ developers.
>>
>> I'm not sure what you're trying to say here. Elaborate/rephrase?
>
> For example in my projects i implement different tasks in different
> libraries.
> All of them have C interfaces. With something like this i could just
> access these libraries as i am accessing in C/C++. Then why do i need to
> go on using C/C++? Transition would be seamless. You can say just write
> the damn bindings and be done with it. But it is neither scalable nor
> maintainable.

Let's be realistic.

In practice, you can almost never just copy/paste a C header into LuaJIT's FFI. Even a header guard (#ifndef FOO \ #define FOO \ ... \ #endif) will ruin it. The fundamental problem is the C preprocessor. The best you can do is take your C header, preprocess it with cpp, and then pass it to LuaJIT and hope nothing breaks. Usually, not even an approach like that will work because of weird compiler extensions used in headers which aren't stripped by cpp.

In an ideal world where no compiler extensions existed, this might have had a chance of working, but we all know what the situation with C and C++ is.

Yes, creating manual bindings is tedious and annoying to maintain, but it is the most foolproof approach.

-- 
- Alex

On 01-05-2012 18:22, so wrote:
> On Tuesday, 1 May 2012 at 15:56:32 UTC, Alex Rønne Petersen wrote:
>
>> note that FFI is not anything spectacular or innovative;
>
> Have you ever written bindings for any of these scripting languages?
>
>
>

No, but you're missing my point entirely. The concept of FFI is not something LuaJIT came up with, at all.

http://en.wikipedia.org/wiki/Foreign_function_interface

-- 
- Alex

On 01-05-2012 18:15, so wrote:
> On Tuesday, 1 May 2012 at 15:56:32 UTC, Alex Rønne Petersen wrote:
>
>> What are you talking about? The link you posted clearly shows that
>> LuaJIT has a C parser built in. It has everything to do with syntax
>> (note that FFI is not anything spectacular or innovative; see libffi,
>> Mono, Lisp, ...). And no, D does not "have all the structures". If it
>> did, we wouldn't be redefining them in D bindings.
>
> What am "i" talking about? How hard to understand these two things?

Very hard! You're not making it clear what it is you want!

Do you want something like LuaJIT's FFI so you can just drop a C header in and get a binding or what? You really need to make this clear. I'm not even sure what we're discussing at this point.

> ABI compatibility and "already" being able to call C natively? If you
> need any syntax you "already" have it.
>
>> What does enum have to do with the C preprocessor? Anyway, it's not
>> that simple. Any arbitrary symbol can have multiple definitions
>> depending on what path you take in the preprocessor forest.
>
> Have you ever used a C api, say OpenGL?

Yes... I wrote the libffi-d binding. You might want to have a look at your local ffi.h and ffitarget.h to see why the preprocessor is a serious problem in automated binding generation. As another example, look at gc.h from libgc.

> What are they using preprocessor for? other than enum and alias?

C libraries use the preprocessor everywhere to deal with funny differences between operating systems, libc implementations, architectures, ...

> It is that damn simple.

No, not really. See above. It may be that simple for the OpenGL headers, but frankly, OpenGL is an example of a reasonably designed library, something that can't be said for 98% of all C libraries.

> I am not talking about supporting Boost level
> preprocessor exploit. I am talking about mature "C" libraries.

"Mature" C libraries are exactly the ones that exploit the preprocessor the most to deal with the aforementioned funny differences!

Again, I direct you to ffi.h, ffitarget.h, and gc.h for "simple" examples.

-- 
- Alex

On Tuesday, 1 May 2012 at 16:31:09 UTC, Alex Rønne Petersen wrote:

> That has nothing to do with the C preprocessor... The point here is that you can't just copy/paste a C header into LuaJIT's FFI out of the box.

Well i didn't say you can. But D have great advantages over Lua to be able to do that and much more.

> Let's be realistic.
>
> In practice, you can almost never just copy/paste a C header into LuaJIT's FFI. Even a header guard (#ifndef FOO \ #define FOO \ ... \ #endif) will ruin it. The fundamental problem is the C preprocessor. The best you can do is take your C header, preprocess it with cpp, and then pass it to LuaJIT and hope nothing breaks. Usually, not even an approach like that will work because of weird compiler extensions used in headers which aren't stripped by cpp.

Again, you can't do that in lua but you should be able to do it in D. If you check some lua bindings, libraries tries to simplify the process, all the duplicate stuff they do on tons of different projects. All for nothing, and i am sure this is true for many other languages. And then you see luajit-ffi, gives both performance boost and soooo much clean code.

This discussion came up before and preprocessor was the only serious blocker. But if you support it to the level what mature C libraries use, which means simple alias and enum you have them all.

> In an ideal world where no compiler extensions existed, this might have had a chance of working, but we all know what the situation with C and C++ is.

I am not sure if C++ had quarter of its user base if it didn't work with C seamlessly.

> Yes, creating manual bindings is tedious and annoying to maintain, but it is the most foolproof approach.

In another language there may not be other exit but D, i am not sure.

On 01-05-2012 18:32, Alex Rønne Petersen wrote:
> On 01-05-2012 18:22, so wrote:
>> On Tuesday, 1 May 2012 at 15:56:32 UTC, Alex Rønne Petersen wrote:
>>
>>> note that FFI is not anything spectacular or innovative;
>>
>> Have you ever written bindings for any of these scripting languages?
>>
>>
>>
>
> No, but you're missing my point entirely. The concept of FFI is not
> something LuaJIT came up with, at all.
>
> http://en.wikipedia.org/wiki/Foreign_function_interface
>

IIRC, SDC has a feature to import C headers by using libclang, BTW.

-- 
- Alex

On 01-05-2012 18:47, so wrote:
> On Tuesday, 1 May 2012 at 16:31:09 UTC, Alex Rønne Petersen wrote:
>
>> That has nothing to do with the C preprocessor... The point here is
>> that you can't just copy/paste a C header into LuaJIT's FFI out of the
>> box.
>
> Well i didn't say you can. But D have great advantages over Lua to be
> able to do that and much more.

D still can't solve the preprocessor problem, so I don't see how it would be any easier for D than it is for LuaJIT.

>
>> Let's be realistic.
>>
>> In practice, you can almost never just copy/paste a C header into
>> LuaJIT's FFI. Even a header guard (#ifndef FOO \ #define FOO \ ... \
>> #endif) will ruin it. The fundamental problem is the C preprocessor.
>> The best you can do is take your C header, preprocess it with cpp, and
>> then pass it to LuaJIT and hope nothing breaks. Usually, not even an
>> approach like that will work because of weird compiler extensions used
>> in headers which aren't stripped by cpp.
>
> Again, you can't do that in lua but you should be able to do it in D.

No, D really can't solve the preprocessor problem. It's not as simple as you think. D does not have any more knowledge about the C compiler intended to compile the C library (or the parameters passed to it) than LuaJIT does.

> If you check some lua bindings, libraries tries to simplify the process,
> all the duplicate stuff they do on tons of different projects. All for
> nothing, and i am sure this is true for many other languages. And then
> you see luajit-ffi, gives both performance boost and soooo much clean code.
>
> This discussion came up before and preprocessor was the only serious
> blocker. But if you support it to the level what mature C libraries use,
> which means simple alias and enum you have them all.

See my reply to your other post for why this is not sufficient for the majority of C libraries.

>
>> In an ideal world where no compiler extensions existed, this might
>> have had a chance of working, but we all know what the situation with
>> C and C++ is.
>
> I am not sure if C++ had quarter of its user base if it didn't work with
> C seamlessly.

If C++ didn't have C "support", it would probably never have been adopted at all. But I never claimed otherwise.

>
>> Yes, creating manual bindings is tedious and annoying to maintain, but
>> it is the most foolproof approach.
>
> In another language there may not be other exit but D, i am not sure.

-- 
- Alex

On Tuesday, 1 May 2012 at 16:46:32 UTC, Alex Rønne Petersen wrote:
> On 01-05-2012 18:15, so wrote:
>> On Tuesday, 1 May 2012 at 15:56:32 UTC, Alex Rønne Petersen wrote:
>>
>>> What are you talking about? The link you posted clearly shows that
>>> LuaJIT has a C parser built in. It has everything to do with syntax
>>> (note that FFI is not anything spectacular or innovative; see libffi,
>>> Mono, Lisp, ...). And no, D does not "have all the structures". If it
>>> did, we wouldn't be redefining them in D bindings.
>>
>> What am "i" talking about? How hard to understand these two things?
>
> Very hard! You're not making it clear what it is you want!
>
> Do you want something like LuaJIT's FFI so you can just drop a C header in and get a binding or what? You really need to make this clear. I'm not even sure what we're discussing at this point.

Oh! Be prepared...

import capi; // and you are done.

> Yes... I wrote the libffi-d binding. You might want to have a look at your local ffi.h and ffitarget.h to see why the preprocessor is a serious problem in automated binding generation. As another example, look at gc.h from libgc.
>
> No, not really. See above. It may be that simple for the OpenGL headers, but frankly, OpenGL is an example of a reasonably designed library, something that can't be said for 98% of all C libraries.

As you said lets be realistic if we can support things like OpenGL/CL with tons of typedefs, enums, defines, function declerations (which you said seem to agree that easy). We could have 98% of mature C libraries. I don't agree with you on that 98% of mature libraries designed worse than those. Because since they target different languages they tend to have clean interfaces.

On 01-05-2012 18:56, so wrote:
> On Tuesday, 1 May 2012 at 16:46:32 UTC, Alex Rønne Petersen wrote:
>> On 01-05-2012 18:15, so wrote:
>>> On Tuesday, 1 May 2012 at 15:56:32 UTC, Alex Rønne Petersen wrote:
>>>
>>>> What are you talking about? The link you posted clearly shows that
>>>> LuaJIT has a C parser built in. It has everything to do with syntax
>>>> (note that FFI is not anything spectacular or innovative; see libffi,
>>>> Mono, Lisp, ...). And no, D does not "have all the structures". If it
>>>> did, we wouldn't be redefining them in D bindings.
>>>
>>> What am "i" talking about? How hard to understand these two things?
>>
>> Very hard! You're not making it clear what it is you want!
>>
>> Do you want something like LuaJIT's FFI so you can just drop a C
>> header in and get a binding or what? You really need to make this
>> clear. I'm not even sure what we're discussing at this point.
>
> Oh! Be prepared...
>
> import capi; // and you are done.

OK, I understand.

>
>> Yes... I wrote the libffi-d binding. You might want to have a look at
>> your local ffi.h and ffitarget.h to see why the preprocessor is a
>> serious problem in automated binding generation. As another example,
>> look at gc.h from libgc.
>>
>> No, not really. See above. It may be that simple for the OpenGL
>> headers, but frankly, OpenGL is an example of a reasonably designed
>> library, something that can't be said for 98% of all C libraries.
>
> As you said lets be realistic if we can support things like OpenGL/CL
> with tons of typedefs, enums, defines, function declerations (which you
> said seem to agree that easy). We could have 98% of mature C libraries.
> I don't agree with you on that 98% of mature libraries designed worse
> than those. Because since they target different languages they tend to
> have clean interfaces.

zlib.h, iconv.h, curses.h, arpa/inet.h, ev.h, expat.h, setjmp.h, ucontext.h, unwind.h ...

Just to name a few headers with subtle but problematic preprocessor usage.

How do you propose that we magically figure out what preprocessor definitions were given to the C compiler when the libraries were compiled? Not to mention what preprocessor definitions the compiler itself defines? And what about compiler extensions?

-- 
- Alex

On Tuesday, 1 May 2012 at 16:56:48 UTC, Alex Rønne Petersen
wrote:

>>> Yes, creating manual bindings is tedious and annoying to maintain, but
>>> it is the most foolproof approach.
>>
>> In another language there may not be other exit but D, i am not sure.

As it happens, you are safe with D even if you pick that path.
With my limited knowledge, these a few lines of code destroys
all lua binders out there. Make sure they are inlined and it
would be identical to handcrafted bindings.

WARNING: DO NOT TRY THIS WITH C++! (See the "alias F"?
ParamTuple? Yep)

import std.stdio;
import std.traits;

struct handle {
	int i=0;
	int get_arg() {
		return i++;
	}
}

alias handle* hnd;

void assign(uint N, P...)(ref P p, hnd h)
{
	static if(N)
	{
		p[N-1] = h.get_arg();
		assign!(N-1)(p, h);
	}
}

int fn(alias F)(hnd h)
{
	alias ParameterTypeTuple!(typeof(&F)) P;
	P p;

	assign!(P.length)(p, h);
	F(p);
	return 0;
}

// test >>>>
void fun0(int a)
{
	writeln("fun0 a: ", a);
}

void fun1(int a, int b)
{
	writeln("fun1 a: ", a);
	writeln("fun1 b: ", b);
}

void main()
{
// 	reg(&fn!fun0, "fun0");
// 	reg(&fn!fun1, "fun1");
	auto f0 = &fn!fun0;
	auto f1 = &fn!fun1;
	handle h;
	f0(&h);
	f1(&h);
}
// test <<<<