Sure. You're right. Weird macros are often used for handy shortcuts
(like the aforementioned for loop header), which are not handy enough
to invent a just-as-weird way of translating them. Everything else is
either for compiler extensions (which either have built-in support in
D or are simply nit translatable) or hacks for compile-time activity
(can be replaced by more intuitive counterparts in D).
Translating _any_ C code is impossible. The C preprocessor cannot be
fully simulated in D. But translating _this_ C code is not as hard,
because it's a bad practice to abuse macros in C and no good C library
would do that.
The translator should not be a "_any_ C code translator", it should be
an "arbitrary _this_ C code translator", which makes a few assumptions
and does not guarantee 100% convertibility of the code.
The most important part of the translator should be the problem
reporter, which should point you to all the suspicious and
untranslated parts of the C code, so you can deal with them manually.

On Tue, Oct 25, 2011 at 6:36 AM, Marco Leise <Marco.Leise@gmx.de> wrote:
> Am 21.10.2011, 10:32 Uhr, schrieb Gor Gyolchanyan <gor.f.gyolchanyan@gmail.com>:
>
>> That's ALL you can do in C. fill structs and call functions
>> (fundamental type manipulation doesn't count).
>> My personal research shows the following use cases of C macros (sorted
>> by popularity in descending order):
>> 1. enum
>> 2. alias (most notably, conditionally compiled ones)
>> 3. CTFE function
>> 4. mixin template
>> 5. syntactic alias
>> 6. syntactic mixin template
>>
>> only the last 2 out of 6 cannot be translated to D.
>> An example of a syntactic alias is this very common piece of C code:
>> #ifdef __VERY_VERY_OLD_C_COMPILER__
>>    #define CONST
>> #else
>>    #define CONST const
>> #endif
>
> Maybe these cases can often be solved when, as you say, they are for
> compatibility with other compilers. The header converter would simply assume
> to be the latest and greatest of the known compilers and evaluate the code
> like this:
> - there is an #ifdef
> - if it a define from the list of "known C compilers with quirks"
>  - jump right to the else block
> The #define obviously can become more than one thing in the .d file. But
> that is determined by the instantiation site. The above CONST would probably
> be ignored when it is used in a const parameter declaration, because of D's
> transitivity and become 'immutable __gshared' when used on a global
> variable. Do you get the idea?
>

In the end, I'm sure, that manual intervention won't be necessary and
the CAPI repository will be replaced by std.c.translate or something
like that.
You'd just do this:

mixin cInclude("sqlite3.h");

and voila.

The mixin will print out all macros, which i couldn't translate and had to evaluate in-line. If some of those macros are important to the user, then translating a C header would consist of including the header and adding the missing macros.

I think this is a much better idea, then manually translating everything, because this is cheaper, covers more libraries.

On Tue, Oct 25, 2011 at 12:13 PM, Gor Gyolchanyan <gor.f.gyolchanyan@gmail.com> wrote:
> Sure. You're right. Weird macros are often used for handy shortcuts
> (like the aforementioned for loop header), which are not handy enough
> to invent a just-as-weird way of translating them. Everything else is
> either for compiler extensions (which either have built-in support in
> D or are simply nit translatable) or hacks for compile-time activity
> (can be replaced by more intuitive counterparts in D).
> Translating _any_ C code is impossible. The C preprocessor cannot be
> fully simulated in D. But translating _this_ C code is not as hard,
> because it's a bad practice to abuse macros in C and no good C library
> would do that.
> The translator should not be a "_any_ C code translator", it should be
> an "arbitrary _this_ C code translator", which makes a few assumptions
> and does not guarantee 100% convertibility of the code.
> The most important part of the translator should be the problem
> reporter, which should point you to all the suspicious and
> untranslated parts of the C code, so you can deal with them manually.
>
> On Tue, Oct 25, 2011 at 6:36 AM, Marco Leise <Marco.Leise@gmx.de> wrote:
>> Am 21.10.2011, 10:32 Uhr, schrieb Gor Gyolchanyan <gor.f.gyolchanyan@gmail.com>:
>>
>>> That's ALL you can do in C. fill structs and call functions
>>> (fundamental type manipulation doesn't count).
>>> My personal research shows the following use cases of C macros (sorted
>>> by popularity in descending order):
>>> 1. enum
>>> 2. alias (most notably, conditionally compiled ones)
>>> 3. CTFE function
>>> 4. mixin template
>>> 5. syntactic alias
>>> 6. syntactic mixin template
>>>
>>> only the last 2 out of 6 cannot be translated to D.
>>> An example of a syntactic alias is this very common piece of C code:
>>> #ifdef __VERY_VERY_OLD_C_COMPILER__
>>>    #define CONST
>>> #else
>>>    #define CONST const
>>> #endif
>>
>> Maybe these cases can often be solved when, as you say, they are for
>> compatibility with other compilers. The header converter would simply assume
>> to be the latest and greatest of the known compilers and evaluate the code
>> like this:
>> - there is an #ifdef
>> - if it a define from the list of "known C compilers with quirks"
>>  - jump right to the else block
>> The #define obviously can become more than one thing in the .d file. But
>> that is determined by the instantiation site. The above CONST would probably
>> be ignored when it is used in a const parameter declaration, because of D's
>> transitivity and become 'immutable __gshared' when used on a global
>> variable. Do you get the idea?
>>
>

The CAPI repository could be replaced by a package in Phobos with aforementioned tiny modules with a single cImport and some additions. They would be very small and conveniently available from Phobos.

On Tue, Oct 25, 2011 at 12:26 PM, Gor Gyolchanyan <gor.f.gyolchanyan@gmail.com> wrote:
> In the end, I'm sure, that manual intervention won't be necessary and
> the CAPI repository will be replaced by std.c.translate or something
> like that.
> You'd just do this:
>
> mixin cInclude("sqlite3.h");
>
> and voila.
>
> The mixin will print out all macros, which i couldn't translate and had to evaluate in-line. If some of those macros are important to the user, then translating a C header would consist of including the header and adding the missing macros.
>
> I think this is a much better idea, then manually translating everything, because this is cheaper, covers more libraries.
>
> On Tue, Oct 25, 2011 at 12:13 PM, Gor Gyolchanyan <gor.f.gyolchanyan@gmail.com> wrote:
>> Sure. You're right. Weird macros are often used for handy shortcuts
>> (like the aforementioned for loop header), which are not handy enough
>> to invent a just-as-weird way of translating them. Everything else is
>> either for compiler extensions (which either have built-in support in
>> D or are simply nit translatable) or hacks for compile-time activity
>> (can be replaced by more intuitive counterparts in D).
>> Translating _any_ C code is impossible. The C preprocessor cannot be
>> fully simulated in D. But translating _this_ C code is not as hard,
>> because it's a bad practice to abuse macros in C and no good C library
>> would do that.
>> The translator should not be a "_any_ C code translator", it should be
>> an "arbitrary _this_ C code translator", which makes a few assumptions
>> and does not guarantee 100% convertibility of the code.
>> The most important part of the translator should be the problem
>> reporter, which should point you to all the suspicious and
>> untranslated parts of the C code, so you can deal with them manually.
>>
>> On Tue, Oct 25, 2011 at 6:36 AM, Marco Leise <Marco.Leise@gmx.de> wrote:
>>> Am 21.10.2011, 10:32 Uhr, schrieb Gor Gyolchanyan <gor.f.gyolchanyan@gmail.com>:
>>>
>>>> That's ALL you can do in C. fill structs and call functions
>>>> (fundamental type manipulation doesn't count).
>>>> My personal research shows the following use cases of C macros (sorted
>>>> by popularity in descending order):
>>>> 1. enum
>>>> 2. alias (most notably, conditionally compiled ones)
>>>> 3. CTFE function
>>>> 4. mixin template
>>>> 5. syntactic alias
>>>> 6. syntactic mixin template
>>>>
>>>> only the last 2 out of 6 cannot be translated to D.
>>>> An example of a syntactic alias is this very common piece of C code:
>>>> #ifdef __VERY_VERY_OLD_C_COMPILER__
>>>>    #define CONST
>>>> #else
>>>>    #define CONST const
>>>> #endif
>>>
>>> Maybe these cases can often be solved when, as you say, they are for
>>> compatibility with other compilers. The header converter would simply assume
>>> to be the latest and greatest of the known compilers and evaluate the code
>>> like this:
>>> - there is an #ifdef
>>> - if it a define from the list of "known C compilers with quirks"
>>>  - jump right to the else block
>>> The #define obviously can become more than one thing in the .d file. But
>>> that is determined by the instantiation site. The above CONST would probably
>>> be ignored when it is used in a const parameter declaration, because of D's
>>> transitivity and become 'immutable __gshared' when used on a global
>>> variable. Do you get the idea?
>>>
>>
>

Walter Bright , dans le message (digitalmars.D:146786), a écrit :
> If you want to add a layer on top of the C API, that would be fine. std.zlib is an example of that.
> 
> But the idea of CAPI is NOT to add a layer. Not fix, extend, refactor, improve, etc.

I definitely agree with that, no attempt should be made to fix anything.

However, if you want only direct translation, the only way I see is to treat all defines as mixins. That mean all macro will become very tedious to use. Even there, there are choices to make (how do you translate multi-parameter macros ?). So I think a minimum of adaptation should be provided.

Here is an example of how this could be made:

#define square(x) ((x)*(x))

=>

// direct translation
mixin template!(string x)square
{
    enum square = '((' ~ x ~ ')*(' ~ x ~ '))';
}

// adapted version
T square(T)(T x) { return x*x; }


Which version should be added ?
Both do not do the same thing ! As you know, In the first one, if x is a
function call, it is called twice, and it the other one, it is called
only once.
So if I follow the rule: no fix, no extend, etc, I must include only the
direct translation. But the CAPI library will not be as usable as the c
one. Then I miss the purpose of CAPI: make d as usable as c when using
c libraries.

So which version should be added: I think it is both:

// direct translation
mixin template!(string x)M_square
{
    enum M_square = '((' ~ x ~ ')*(' ~ x ~ '))';
}

// adapted version
T square(T)(T x) { return mixin M_square!q{x}; }


So this is what I propose:
Direct translation have to be provided. Adapted version can be provided
alongside the direct translation of the d header. Adaptaded version must
be a direct forward call to the direct version (note here that the
adapted. Rules will have to be defined to avoid name clashes (for
example, here, I used a the direct name for the usable version, and M_
prefix for the mixin version, but we could decide other rules).


Macros are a big issue.

And I think abvious translating, such as const char* to string, or pointer-length pairs to dynamic arrays is about the same problem.


double array_sum(double* a, size_t length);

=>
double array_sum(double* a, size_t length);
// AND
double array_sum(double[] a) { return array_sum(a.ptr, a.length); }


That is very little work. The direct translation is mandatory, and the adapted translation is not. But when the translation is obvious, there is no reason for everyone to make it on it's corner. Make it in the public header and share it!

I order to remain consistent, adaptation will have to obey to very precise rules, that have to be set. No fix, no extend, no refactor, no improve, etc. Just a forward call, to have, in addition to the C API, an API that use D's power: using enums, inline functions, for defines instead of the direct mixin translation. Using D's arrays instead of C arrays, etc. could be nice too.

What translation should be provided ? What rules to translate defines ? Is translation of pointer+length pair to array worth doing ? What about stringz and strings ? Where to draw the line ?


-- 
Christophe