View mode: basic / threaded / horizontal-split · Log in · Help
May 01, 2012
Re: luajit-ffi
On Tuesday, 1 May 2012 at 15:56:32 UTC, Alex Rønne Petersen 
wrote:

> note that FFI is not anything spectacular or innovative;

Have you ever written bindings for any of these scripting 
languages?
May 01, 2012
Re: luajit-ffi
On 01-05-2012 18:08, so wrote:
> On Tuesday, 1 May 2012 at 15:53:37 UTC, Alex Rønne Petersen wrote:
>> On 01-05-2012 17:43, so wrote:
>>> On Tuesday, 1 May 2012 at 15:31:05 UTC, Robert Clipsham wrote:
>>>> On 01/05/2012 16:24, so wrote:
>>>>> http://luajit.org/ext_ffi.html
>>>>> https://github.com/malkia/ufo
>>>>>
>>>>> How awesome is Mike Pall?
>>>>> I didn't dive into details of the code, but if he can do this with a
>>>>> dynamic language, why on earth D still need manual C bindings while
>>>>> having ABI compatibility? So luajit comes with a C compiler?
>>>>
>>>> Note that you can't just drop any C header file in there for that to
>>>> work (as far as I can tell), you still have to bring out individual
>>>> post-processed function declarations.
>>>
>>> https://github.com/malkia/ufo/blob/master/ffi/OpenCL.lua
>>> https://github.com/malkia/ufo/blob/master/ffi/OpenGL.lua
>>> If it can handle these 2 beasts.
>>
>> I see no preprocessor directives.
>
> They are all there as "enum".

That has nothing to do with the C preprocessor... The point here is that 
you can't just copy/paste a C header into LuaJIT's FFI out of the box. 
You have to run the preprocessor on it first, or preprocess it by hand.

>
>>>> Also, someone has written a libffi binding for D, which could probably
>>>> be adapted to work in a similar manor:
>>>>
>>>> https://github.com/lycus/libffi-d
>>>
>>> Neat. Still, having native C libraries means that you can just drop your
>>> C/C++ environment and start D. And i am sure you agree this is by far
>>> the biggest blocker for C/C++ developers.
>>
>> I'm not sure what you're trying to say here. Elaborate/rephrase?
>
> For example in my projects i implement different tasks in different
> libraries.
> All of them have C interfaces. With something like this i could just
> access these libraries as i am accessing in C/C++. Then why do i need to
> go on using C/C++? Transition would be seamless. You can say just write
> the damn bindings and be done with it. But it is neither scalable nor
> maintainable.

Let's be realistic.

In practice, you can almost never just copy/paste a C header into 
LuaJIT's FFI. Even a header guard (#ifndef FOO \ #define FOO \ ... \ 
#endif) will ruin it. The fundamental problem is the C preprocessor. The 
best you can do is take your C header, preprocess it with cpp, and then 
pass it to LuaJIT and hope nothing breaks. Usually, not even an approach 
like that will work because of weird compiler extensions used in headers 
which aren't stripped by cpp.

In an ideal world where no compiler extensions existed, this might have 
had a chance of working, but we all know what the situation with C and 
C++ is.

Yes, creating manual bindings is tedious and annoying to maintain, but 
it is the most foolproof approach.

-- 
- Alex
May 01, 2012
Re: luajit-ffi
On 01-05-2012 18:22, so wrote:
> On Tuesday, 1 May 2012 at 15:56:32 UTC, Alex Rønne Petersen wrote:
>
>> note that FFI is not anything spectacular or innovative;
>
> Have you ever written bindings for any of these scripting languages?
>
>
>

No, but you're missing my point entirely. The concept of FFI is not 
something LuaJIT came up with, at all.

http://en.wikipedia.org/wiki/Foreign_function_interface

-- 
- Alex
May 01, 2012
Re: luajit-ffi
On 01-05-2012 18:15, so wrote:
> On Tuesday, 1 May 2012 at 15:56:32 UTC, Alex Rønne Petersen wrote:
>
>> What are you talking about? The link you posted clearly shows that
>> LuaJIT has a C parser built in. It has everything to do with syntax
>> (note that FFI is not anything spectacular or innovative; see libffi,
>> Mono, Lisp, ...). And no, D does not "have all the structures". If it
>> did, we wouldn't be redefining them in D bindings.
>
> What am "i" talking about? How hard to understand these two things?

Very hard! You're not making it clear what it is you want!

Do you want something like LuaJIT's FFI so you can just drop a C header 
in and get a binding or what? You really need to make this clear. I'm 
not even sure what we're discussing at this point.

> ABI compatibility and "already" being able to call C natively? If you
> need any syntax you "already" have it.
>
>> What does enum have to do with the C preprocessor? Anyway, it's not
>> that simple. Any arbitrary symbol can have multiple definitions
>> depending on what path you take in the preprocessor forest.
>
> Have you ever used a C api, say OpenGL?

Yes... I wrote the libffi-d binding. You might want to have a look at 
your local ffi.h and ffitarget.h to see why the preprocessor is a 
serious problem in automated binding generation. As another example, 
look at gc.h from libgc.

> What are they using preprocessor for? other than enum and alias?

C libraries use the preprocessor everywhere to deal with funny 
differences between operating systems, libc implementations, 
architectures, ...

> It is that damn simple.

No, not really. See above. It may be that simple for the OpenGL headers, 
but frankly, OpenGL is an example of a reasonably designed library, 
something that can't be said for 98% of all C libraries.

> I am not talking about supporting Boost level
> preprocessor exploit. I am talking about mature "C" libraries.

"Mature" C libraries are exactly the ones that exploit the preprocessor 
the most to deal with the aforementioned funny differences!

Again, I direct you to ffi.h, ffitarget.h, and gc.h for "simple" examples.

-- 
- Alex
May 01, 2012
Re: luajit-ffi
On Tuesday, 1 May 2012 at 16:31:09 UTC, Alex Rønne Petersen 
wrote:

> That has nothing to do with the C preprocessor... The point 
> here is that you can't just copy/paste a C header into LuaJIT's 
> FFI out of the box.

Well i didn't say you can. But D have great advantages over Lua 
to be able to do that and much more.

> Let's be realistic.
>
> In practice, you can almost never just copy/paste a C header 
> into LuaJIT's FFI. Even a header guard (#ifndef FOO \ #define 
> FOO \ ... \ #endif) will ruin it. The fundamental problem is 
> the C preprocessor. The best you can do is take your C header, 
> preprocess it with cpp, and then pass it to LuaJIT and hope 
> nothing breaks. Usually, not even an approach like that will 
> work because of weird compiler extensions used in headers which 
> aren't stripped by cpp.

Again, you can't do that in lua but you should be able to do it 
in D. If you check some lua bindings, libraries tries to simplify 
the process, all the duplicate stuff they do on tons of different 
projects. All for nothing, and i am sure this is true for many 
other languages. And then you see luajit-ffi, gives both 
performance boost and soooo much clean code.

This discussion came up before and preprocessor was the only 
serious blocker. But if you support it to the level what mature C 
libraries use, which means simple alias and enum you have them 
all.

> In an ideal world where no compiler extensions existed, this 
> might have had a chance of working, but we all know what the 
> situation with C and C++ is.

I am not sure if C++ had quarter of its user base if it didn't 
work with C seamlessly.

> Yes, creating manual bindings is tedious and annoying to 
> maintain, but it is the most foolproof approach.

In another language there may not be other exit but D, i am not 
sure.
May 01, 2012
Re: luajit-ffi
On 01-05-2012 18:32, Alex Rønne Petersen wrote:
> On 01-05-2012 18:22, so wrote:
>> On Tuesday, 1 May 2012 at 15:56:32 UTC, Alex Rønne Petersen wrote:
>>
>>> note that FFI is not anything spectacular or innovative;
>>
>> Have you ever written bindings for any of these scripting languages?
>>
>>
>>
>
> No, but you're missing my point entirely. The concept of FFI is not
> something LuaJIT came up with, at all.
>
> http://en.wikipedia.org/wiki/Foreign_function_interface
>

IIRC, SDC has a feature to import C headers by using libclang, BTW.

-- 
- Alex
May 01, 2012
Re: luajit-ffi
On 01-05-2012 18:47, so wrote:
> On Tuesday, 1 May 2012 at 16:31:09 UTC, Alex Rønne Petersen wrote:
>
>> That has nothing to do with the C preprocessor... The point here is
>> that you can't just copy/paste a C header into LuaJIT's FFI out of the
>> box.
>
> Well i didn't say you can. But D have great advantages over Lua to be
> able to do that and much more.

D still can't solve the preprocessor problem, so I don't see how it 
would be any easier for D than it is for LuaJIT.

>
>> Let's be realistic.
>>
>> In practice, you can almost never just copy/paste a C header into
>> LuaJIT's FFI. Even a header guard (#ifndef FOO \ #define FOO \ ... \
>> #endif) will ruin it. The fundamental problem is the C preprocessor.
>> The best you can do is take your C header, preprocess it with cpp, and
>> then pass it to LuaJIT and hope nothing breaks. Usually, not even an
>> approach like that will work because of weird compiler extensions used
>> in headers which aren't stripped by cpp.
>
> Again, you can't do that in lua but you should be able to do it in D.

No, D really can't solve the preprocessor problem. It's not as simple as 
you think. D does not have any more knowledge about the C compiler 
intended to compile the C library (or the parameters passed to it) than 
LuaJIT does.

> If you check some lua bindings, libraries tries to simplify the process,
> all the duplicate stuff they do on tons of different projects. All for
> nothing, and i am sure this is true for many other languages. And then
> you see luajit-ffi, gives both performance boost and soooo much clean code.
>
> This discussion came up before and preprocessor was the only serious
> blocker. But if you support it to the level what mature C libraries use,
> which means simple alias and enum you have them all.

See my reply to your other post for why this is not sufficient for the 
majority of C libraries.

>
>> In an ideal world where no compiler extensions existed, this might
>> have had a chance of working, but we all know what the situation with
>> C and C++ is.
>
> I am not sure if C++ had quarter of its user base if it didn't work with
> C seamlessly.

If C++ didn't have C "support", it would probably never have been 
adopted at all. But I never claimed otherwise.

>
>> Yes, creating manual bindings is tedious and annoying to maintain, but
>> it is the most foolproof approach.
>
> In another language there may not be other exit but D, i am not sure.

-- 
- Alex
May 01, 2012
Re: luajit-ffi
On Tuesday, 1 May 2012 at 16:46:32 UTC, Alex Rønne Petersen 
wrote:
> On 01-05-2012 18:15, so wrote:
>> On Tuesday, 1 May 2012 at 15:56:32 UTC, Alex Rønne Petersen 
>> wrote:
>>
>>> What are you talking about? The link you posted clearly shows 
>>> that
>>> LuaJIT has a C parser built in. It has everything to do with 
>>> syntax
>>> (note that FFI is not anything spectacular or innovative; see 
>>> libffi,
>>> Mono, Lisp, ...). And no, D does not "have all the 
>>> structures". If it
>>> did, we wouldn't be redefining them in D bindings.
>>
>> What am "i" talking about? How hard to understand these two 
>> things?
>
> Very hard! You're not making it clear what it is you want!
>
> Do you want something like LuaJIT's FFI so you can just drop a 
> C header in and get a binding or what? You really need to make 
> this clear. I'm not even sure what we're discussing at this 
> point.

Oh! Be prepared...

import capi; // and you are done.

> Yes... I wrote the libffi-d binding. You might want to have a 
> look at your local ffi.h and ffitarget.h to see why the 
> preprocessor is a serious problem in automated binding 
> generation. As another example, look at gc.h from libgc.
>
> No, not really. See above. It may be that simple for the OpenGL 
> headers, but frankly, OpenGL is an example of a reasonably 
> designed library, something that can't be said for 98% of all C 
> libraries.

As you said lets be realistic if we can support things like 
OpenGL/CL with tons of typedefs, enums, defines, function 
declerations (which you said seem to agree that easy). We could 
have 98% of mature C libraries. I don't agree with you on that 
98% of mature libraries designed worse than those. Because since 
they target different languages they tend to have clean 
interfaces.
May 01, 2012
Re: luajit-ffi
On 01-05-2012 18:56, so wrote:
> On Tuesday, 1 May 2012 at 16:46:32 UTC, Alex Rønne Petersen wrote:
>> On 01-05-2012 18:15, so wrote:
>>> On Tuesday, 1 May 2012 at 15:56:32 UTC, Alex Rønne Petersen wrote:
>>>
>>>> What are you talking about? The link you posted clearly shows that
>>>> LuaJIT has a C parser built in. It has everything to do with syntax
>>>> (note that FFI is not anything spectacular or innovative; see libffi,
>>>> Mono, Lisp, ...). And no, D does not "have all the structures". If it
>>>> did, we wouldn't be redefining them in D bindings.
>>>
>>> What am "i" talking about? How hard to understand these two things?
>>
>> Very hard! You're not making it clear what it is you want!
>>
>> Do you want something like LuaJIT's FFI so you can just drop a C
>> header in and get a binding or what? You really need to make this
>> clear. I'm not even sure what we're discussing at this point.
>
> Oh! Be prepared...
>
> import capi; // and you are done.

OK, I understand.

>
>> Yes... I wrote the libffi-d binding. You might want to have a look at
>> your local ffi.h and ffitarget.h to see why the preprocessor is a
>> serious problem in automated binding generation. As another example,
>> look at gc.h from libgc.
>>
>> No, not really. See above. It may be that simple for the OpenGL
>> headers, but frankly, OpenGL is an example of a reasonably designed
>> library, something that can't be said for 98% of all C libraries.
>
> As you said lets be realistic if we can support things like OpenGL/CL
> with tons of typedefs, enums, defines, function declerations (which you
> said seem to agree that easy). We could have 98% of mature C libraries.
> I don't agree with you on that 98% of mature libraries designed worse
> than those. Because since they target different languages they tend to
> have clean interfaces.

zlib.h, iconv.h, curses.h, arpa/inet.h, ev.h, expat.h, setjmp.h, 
ucontext.h, unwind.h ...

Just to name a few headers with subtle but problematic preprocessor usage.

How do you propose that we magically figure out what preprocessor 
definitions were given to the C compiler when the libraries were 
compiled? Not to mention what preprocessor definitions the compiler 
itself defines? And what about compiler extensions?

-- 
- Alex
May 01, 2012
Re: luajit-ffi
On Tuesday, 1 May 2012 at 16:56:48 UTC, Alex Rønne Petersen
wrote:

>>> Yes, creating manual bindings is tedious and annoying to 
>>> maintain, but
>>> it is the most foolproof approach.
>>
>> In another language there may not be other exit but D, i am 
>> not sure.

As it happens, you are safe with D even if you pick that path.
With my limited knowledge, these a few lines of code destroys
all lua binders out there. Make sure they are inlined and it
would be identical to handcrafted bindings.

WARNING: DO NOT TRY THIS WITH C++! (See the "alias F"?
ParamTuple? Yep)

import std.stdio;
import std.traits;

struct handle {
	int i=0;
	int get_arg() {
		return i++;
	}
}

alias handle* hnd;

void assign(uint N, P...)(ref P p, hnd h)
{
	static if(N)
	{
		p[N-1] = h.get_arg();
		assign!(N-1)(p, h);
	}
}

int fn(alias F)(hnd h)
{
	alias ParameterTypeTuple!(typeof(&F)) P;
	P p;

	assign!(P.length)(p, h);
	F(p);
	return 0;
}

// test >>>>
void fun0(int a)
{
	writeln("fun0 a: ", a);
}

void fun1(int a, int b)
{
	writeln("fun1 a: ", a);
	writeln("fun1 b: ", b);
}

void main()
{
// 	reg(&fn!fun0, "fun0");
// 	reg(&fn!fun1, "fun1");
	auto f0 = &fn!fun0;
	auto f1 = &fn!fun1;
	handle h;
	f0(&h);
	f1(&h);
}
// test <<<<
1 2 3
Top | Discussion index | About this forum | D home