What is the correct way to declare and access storage managed by C?

For example, say a C module defines an array of filenames, e.g.,

char *files[] = { "one", "two", "three", 0};

The C header of course declares this as:

extern char *files[];

The way to declare it in a D module appears to be:

extern (C) {
   __gshared extern char *[] files;
}

However, trying to index into this, e.g.,

char *filename = files[1];

does not work.  In gdb if I try to examine 'filename' I see something like a reverse text string but treated as an (inaccessible) address and calling fromStringz causes a segfault.

On Wednesday, 14 March 2018 at 01:58:24 UTC, Joe wrote:
> The C header of course declares this as:
>
> extern char *files[];
>
> The way to declare it in a D module appears to be:
>
> extern (C) {
>    __gshared extern char *[] files;
> }


A D array[] should *almost never* be used in extern(C). (tbh I'm almost of the opinion that it should be an error, but since it is almost never instead of never ehhhh)


Anyway, D `array[]` is a structure consisting of a length and pointer combo.

C `array[]` is one of two things: a D `array[N]` with static size, or a D `array*`, a pointer.

In function parameters, it is... I'm pretty sure... always a pointer. So even

void foo(char* item[4]);

in C becomes

extern(C) foo(char** item);

in D.



BUT, for a global array in C, it is frequently going to be a static length array in D. So

char *files[] = { "one", "two", "three", 0};

is

extern(C) __gshared extern char*[4] files;


in D. Why? The C array variable is actually a block of memory rather than a pointer. And in D, that's represented as an array[N] with a specific size. So that's what we have here.


BTW the exact length isn't super important for an extern here. The type system would *like* to know, certainly for correct range errors, but if you declare it as the wrong length and use the .ptr, it still works like it does in C:

extern(C) __gshared extern char*[1] files; // still works

import core.stdc.stdio;

void main() {
        printf("%s\n", files.ptr[2]); // ptr bypasses the range check
}


But if you can match the length, that's ideal. BTW remember D static arrays are passed by value if assigned around; they will be copied. So use the .ptr if you want to  use the pointer of it like it is in C (notably, when passing that array to other C functions, remember, C function params are still pointers)

On Wednesday, 14 March 2018 at 02:17:57 UTC, Adam D. Ruppe wrote:
> The type system would *like* to know, certainly for correct range errors, but if you declare it as the wrong length and use the .ptr, it still works like it does in C:
>
> extern(C) __gshared extern char*[1] files; // still works
>
> import core.stdc.stdio;
>
> void main() {
>         printf("%s\n", files.ptr[2]); // ptr bypasses the range check
> }


That worked but now I have a more convoluted case: a C array of pointers to int pointers, e.g.,

int **xs[] = {x1, x2, 0};
int *x1[] = {x1a, 0};
int *x2[] = {x2a, x2b, 0};
...
int x2a[] = { 1, 3, 5, 0};

Only the first line is exposed (and without the initialization). So I tried:

extern(C) __gshared extern int**[1] xs;

The D compiler accepts that, but just about any manipulation gets screamed at, usually with Error: only one index allowed to index int. Note that I'm trying to access the ints, i.e., in C something like xs[1][0][2] to access the 5 in x2a. Do I have to mimic the intermediate C arrays?

> But if you can match the length, that's ideal.

Unfortunately, although the C array lengths are known at C compile time, they're not made available otherwise so I'm afraid the [1] trick will have to do for now.

On Sunday, 10 June 2018 at 17:59:12 UTC, Joe wrote:
> That worked but now I have a more convoluted case: a C array of pointers to int pointers, e.g.,
>
> int **xs[] = {x1, x2, 0};
> int *x1[] = {x1a, 0};
> int *x2[] = {x2a, x2b, 0};
> ...
> int x2a[] = { 1, 3, 5, 0};
>
> Only the first line is exposed (and without the initialization). So I tried:
>
> extern(C) __gshared extern int**[1] xs;
>
> The D compiler accepts that, but just about any manipulation gets screamed at, usually with Error: only one index allowed to index int. Note that I'm trying to access the ints, i.e., in C something like xs[1][0][2] to access the 5 in x2a. Do I have to mimic the intermediate C arrays?

I don't know why I didn't try this first.  It seems that the D equivalent of C's xs[1][0][2] is simply xs.ptr[[1][0][2].

On Sunday, 10 June 2018 at 17:59:12 UTC, Joe wrote:
> That worked but now I have a more convoluted case: a C array of pointers to int pointers, e.g.,
>
> int **xs[] = {x1, x2, 0};
> int *x1[] = {x1a, 0};
> int *x2[] = {x2a, x2b, 0};
> ...
> int x2a[] = { 1, 3, 5, 0};
>
> Only the first line is exposed (and without the initialization). So I tried:
>
> extern(C) __gshared extern int**[1] xs;

After a long hiatus, I'm back to working on something related to the above, but now that various other C pieces have been converted to D I'm down to converting these static arrays to D. There are two arrays that are giving me trouble. The second type is like that shown above. The first is a simpler array of pointers to int, e.g.,

int *yp = {2, 4, 0};
int *yq = {10, 12, 0};
int *ys[] = {yp, yq, 0};

In D, I first declared these as

int[] yp = [2, 4];
int[] yq = [10, 12];
__gshared int*[] ys = [ &yp, &yq ];

The compiler (ldc2) gave errors like "cannot take address of thread-local variable yp at compile time" or "static variable yp cannot be read at compile time" (when I replaced the "&yp" by "yp[0]". Eventually, I managed to get them to compile without errors by using the following:

immutable int[] yp = [2, 4];
immutable int[] yq = [10, 12];
__gshared immutable(int[])[] ys = [ yp, yq ];

I still haven't tested them (or linked them) so I don't know what other changes I'll have to make to the library (now in D) where ys is declared as:

__gshared extern int*[] ys;

Presumably changing it to

__gshared extern immutable(int[])[] ys;

should do the trick (everything is still within extern (C))? But I suspect I'll have to change several array access statements as well.

However, I've been unable to compile the other case, which now looks like this:

immutable int x1a[] = [ 9, 7, 5];
immutable(int[])[] x1 = [ x1a ];
immutable(int[])[] x2a = [ 1, 3, 5];
...
immutable(int[])[] x2 = [ x2a, x2b ];
__gshared immutable(int[])[][] xs = [ x1, x2 ];

The error is like "static variable x2a cannot be read at compile time". It seems like I would have to wrap that immutable(int[]) within another immutable, as

immutable(immutable(int[])[]) x2 ...

and extend that to xs as well.

I'll try it later but I'd like some confirmation or better yet, pointers to where this is explained in some comprehensible way, i.e., on what can you apply an address operator or array subscript and when, is the behavior differerent for immutable, const, static, thread-local and why?

On Tuesday, 18 September 2018 at 02:39:39 UTC, Joe wrote:
> On Sunday, 10 June 2018 at 17:59:12 UTC, Joe wrote:
>> That worked but now I have a more convoluted case: a C array of pointers to int pointers, e.g.,
>>
>> int **xs[] = {x1, x2, 0};
>> int *x1[] = {x1a, 0};
>> int *x2[] = {x2a, x2b, 0};
>> ...
>> int x2a[] = { 1, 3, 5, 0};
>>
>> Only the first line is exposed (and without the initialization). So I tried:
>>
>> extern(C) __gshared extern int**[1] xs;
>
> After a long hiatus, I'm back to working on something related to the above, but now that various other C pieces have been converted to D I'm down to converting these static arrays to D. There are two arrays that are giving me trouble. The second type is like that shown above. The first is a simpler array of pointers to int, e.g.,
>
> int *yp = {2, 4, 0};
> int *yq = {10, 12, 0};

This is valid C in the sense that it compiles, but I doubt it does what you think it does. This is equivalent code:

int *yp = 2;
int *yq = 10;

> int *ys[] = {yp, yq, 0};

This isn't even valid C code.

> In D, I first declared these as
>
> int[] yp = [2, 4];
> int[] yq = [10, 12];
> __gshared int*[] ys = [ &yp, &yq ];

D dynamic arrays are not equivalent to C arrays.


It's hard to see what you're trying to do with the code you posted. Have you tried instead to use a tool to translate the C headers?

On Tuesday, 18 September 2018 at 13:47:50 UTC, Atila Neves wrote:
> On Tuesday, 18 September 2018 at 02:39:39 UTC, Joe wrote:
>> The second type is like that shown above. The first is a simpler array of pointers to int, e.g.,
>>
>> int *yp = {2, 4, 0};
>> int *yq = {10, 12, 0};
>
> This is valid C in the sense that it compiles, but I doubt it does what you think it does. This is equivalent code:
>
> int *yp = 2;
> int *yq = 10;

Sorry, Atila, I got confused looking at my two cases. I should have said "an array of ints", e.g.,

int yp[] = {2, 4, 0};
int yq[] = {10, 12, 0};

>> int *ys[] = {yp, yq, 0};
>
> This isn't even valid C code.

It is, because C treats 'yp' as a pointer.

>> In D, I first declared these as
>>
>> int[] yp = [2, 4];
>> int[] yq = [10, 12];
>> __gshared int*[] ys = [ &yp, &yq ];
>
> D dynamic arrays are not equivalent to C arrays.
>
> It's hard to see what you're trying to do with the code you posted. Have you tried instead to use a tool to translate the C headers?

At this point, I've translated everything, even the code above. I had to use 'immutable(int [])' in the second and higher level arrays like 'ys' so that they could refer to 'yp' and 'yq' (without the address operators).

I still have to make several changes to the library code because these arrays were declared as dynamic arrays of pointers, of size 1, to bypass bounds checking. Now they're proper dynamic arrays, and I can use foreach on them and get other D benefits.

However, I still would like to have a deeper understanding of the "static variable yp cannot be read at compile time" error messages which went away when I declared yp immutable.

On Wednesday, 19 September 2018 at 00:46:54 UTC, Joe wrote:
> On Tuesday, 18 September 2018 at 13:47:50 UTC, Atila Neves wrote:
> Sorry, Atila, I got confused looking at my two cases. I should have said "an array of ints", e.g.,
>
> int yp[] = {2, 4, 0};
> int yq[] = {10, 12, 0};

That makes more sense.

>>> int *ys[] = {yp, yq, 0};
>>
>> This isn't even valid C code.
>
> It is, because C treats 'yp' as a pointer.

It wasn't with the definition of `yp` and `yq` you posted.

>
>>> In D, I first declared these as
>>>
>>> int[] yp = [2, 4];
>>> int[] yq = [10, 12];
>>> __gshared int*[] ys = [ &yp, &yq ];
>>
>> D dynamic arrays are not equivalent to C arrays.
>>
>> It's hard to see what you're trying to do with the code you posted. Have you tried instead to use a tool to translate the C headers?
>
> At this point, I've translated everything, even the code above. I had to use 'immutable(int [])' in the second and higher level arrays like 'ys' so that they could refer to 'yp' and 'yq' (without the address operators).

This would be the literal translation:

int[3] yp = [2, 4, 0];
int[3] yq = [10, 12, 0];
int*[3] ys;

shared static this() {
    ys = [yp.ptr, yq.ptr, null];
}


> However, I still would like to have a deeper understanding of the "static variable yp cannot be read at compile time" error messages which went away when I declared yp immutable.

I guess immutable makes everything known at compile-time? I didn't even know that was a thing. In any case, the reason why you got those error messages is because initialisation of global variables in D happens at compile-time. If you want runtime initialisation like in C++, you have to use static constructors like in my code above.