Hi there,

I've got a problem interfacing to a C library.
The following structs are used by the library's .d file that I've written.

--------------------
struct neo4j_map_entry_t
{
    neo4j_value_t key;
    neo4j_value_t value;
};

struct neo4j_value_t
{
    uint8_t _vt_off;
    uint8_t _type; /*TODO: combine with _vt_off? (both always have same value)*/
    uint16_t _pad1;
    uint32_t _pad2;
    _neo4j_value_data _vdata;
};

union _neo4j_value_data
{
    uint64_t _int;
    uintptr_t _ptr;
    double _dbl;
};
--------------------

Now I'd also like to use them in my own code. However, I fail at generating an array of map entries:

--------------------
void test()
{
    // These work
    neo4j_map_entry_t[] mapa2; // yes
    neo4j_map_entry_t entry1 = { key: neo4j_string("prop1"), value: neo4j_string("testprop1")}; // yes
    neo4j_map_entry_t entry2 = { key: neo4j_string("prop2"), value: neo4j_string("testprop2")}; // yes
    neo4j_map_entry_t* mapp; // yes

    // These don't
    neo4j_map_entry_t[2] mapa1; // no
    mapa2.length = 2; // no
    mapa2 ~= entry1; // no
    neo4j_map_entry_t[] mapa3 = [{ key: neo4j_null, value: neo4j_null}]; // no
}
--------------------

The output is:
______________________
myprogram ~master: building configuration "unittest"...
Linking...
Undefined symbols for architecture x86_64:
  "_D10neo4jTypes17neo4j_map_entry_t6__initZ", referenced from:
      _D6myprogram6myprogram5test2MFZv in myprogram.o
ld: symbol(s) not found for architecture x86_64
______________________

Why is it a linker problem? I'm not linking to the c interface but merely using D structs...

On Sunday, 27 November 2016 at 12:59:32 UTC, Timoses wrote:
> Hi there,
>
> I've got a problem interfacing to a C library.
> The following structs are used by the library's .d file that I've written.
>
> --------------------
> struct neo4j_map_entry_t
> {
>     neo4j_value_t key;
>     neo4j_value_t value;
> };
>
> struct neo4j_value_t
> {
>     uint8_t _vt_off;
>     uint8_t _type; /*TODO: combine with _vt_off? (both always have same value)*/
>     uint16_t _pad1;
>     uint32_t _pad2;
>     _neo4j_value_data _vdata;
> };
>
> union _neo4j_value_data
> {
>     uint64_t _int;
>     uintptr_t _ptr;
>     double _dbl;
> };
> --------------------
>
> Now I'd also like to use them in my own code. However, I fail at generating an array of map entries:
>
> --------------------
> void test()
> {
>     // These work
>     neo4j_map_entry_t[] mapa2; // yes
>     neo4j_map_entry_t entry1 = { key: neo4j_string("prop1"), value: neo4j_string("testprop1")}; // yes
>     neo4j_map_entry_t entry2 = { key: neo4j_string("prop2"), value: neo4j_string("testprop2")}; // yes
>     neo4j_map_entry_t* mapp; // yes
>
>     // These don't
>     neo4j_map_entry_t[2] mapa1; // no
>     mapa2.length = 2; // no
>     mapa2 ~= entry1; // no
>     neo4j_map_entry_t[] mapa3 = [{ key: neo4j_null, value: neo4j_null}]; // no
> }
> --------------------
>
> The output is:
> ______________________
> myprogram ~master: building configuration "unittest"...
> Linking...
> Undefined symbols for architecture x86_64:
>   "_D10neo4jTypes17neo4j_map_entry_t6__initZ", referenced from:
>       _D6myprogram6myprogram5test2MFZv in myprogram.o
> ld: symbol(s) not found for architecture x86_64
> ______________________
>
> Why is it a linker problem? I'm not linking to the c interface but merely using D structs...

The missing symbol is the struct initialiser for neo4j_map_entry_t. Not sure why is not being generated (it should), possibly because of the union.

That seems like a bug please report it. http://issues.dlang.org/

On Sunday, 27 November 2016 at 12:59:32 UTC, Timoses wrote:
> Why is it a linker problem? I'm not linking to the c interface but merely using D structs...

It is a linker problem because you didn't link to it... D structs have an initializer, even if they are used for interfacing with C. This is a small blob of data called something like `_D10neo4jTypes17neo4j_map_entry_t6__initZ`, or can be all zeroes, and is generated with the interface module.

So if you use the initializer from the interface module, but do not link it in, you get an undefined symbol.


Easiest fix: just link in that .d file and use the automatic features.


Why does your struct have an initializer data symbol instead of being all zeroes? I'm not entirely sure, but I think it is because the contents are structs and it didn't look inside those structs to see if they too are zeroes, it just assumed struct = fancy initialized. That's arguably an enhancement request, but I wouldn't call it a bug.

You can often bypass the initializer by using `=0` or `=void` in definitions, but not here... and that is arguably a bug, if you `=void` all fields, it should be free to zero initialize the whole thing, but it doesn't.



Why do you use the initializer? Look at each of these lines:


    neo4j_map_entry_t[] mapa2; // yes
    neo4j_map_entry_t entry1 = { key: neo4j_string("prop1"), value: neo4j_string("testprop1")}; // yes
    neo4j_map_entry_t entry2 = { key: neo4j_string("prop2"), value: neo4j_string("testprop2")}; // yes
    neo4j_map_entry_t* mapp; // yes



Those work because nothing is default initialized: pointers and arrays start as null (and thus point to no actual struct data). The middle lines have you explicitly initializing it, so again, no default needed.

    // These don't
    neo4j_map_entry_t[2] mapa1; // no

This is basically the same as `neo4j_map_entry_y a;` done twice - you create a struct, which is default initialized, which references that data.

You can bypass this with `=void` at the end of that line.

    mapa2.length = 2; // no

This also default initializes the new items you add. Just don't use that length function if you don't want to link in the other .d file.

    mapa2 ~= entry1; // no

I think the implementation does length += 1, then copy arr[$-1] = entry1, so there's a temporary default in there.

There's arguably a better implementation possible so this one *could* work. But if it doesn't now, just avoid that function (try something like std.array.uninitializedArray perhaps).

    neo4j_map_entry_t[] mapa3 = [{ key: neo4j_null, value: neo4j_null}]; // no

And I don't know with this one, since I don't know what neo4j_null is. Perhaps you just didn't do the extern symbol right, or perhaps it too is a little data blob that should be linked in.



But while this can be annoying, it isn't really a bug - it is D's auto initialize feature creating some data that you didn't link. Avoiding D-specific features and bypassing initialization with =void can skip it, but I recommend just linking in the module.

On Sunday, 27 November 2016 at 13:22:36 UTC, Nicholas Wilson wrote:
>
> The missing symbol is the struct initialiser for neo4j_map_entry_t. Not sure why is not being generated (it should), possibly because of the union.
>
> That seems like a bug please report it. http://issues.dlang.org/

Thanks for the answer, Nicholas!

I've created a small example:

source/app.d
--------------------
import mytypes;

void main()
{
    myunion[2] t;
}
--------------------
include/mytypes.d:
--------------------
module mytypes;

union myunion
{
    double b;
};
--------------------
dub.json:
--------------------
"importPaths": ["include"]
--------------------

Output:
_____________________
dmd -c -of.dub/build/application-debug-posix.osx-x86_64-dmd_2072-99D5F2E2DCAF9FF19FE5AE403C120D52/testomat.o -debug -g -w -version=Have_testomat -Iinclude source/app.d -vcolumns
Linking...
dmd -of.dub/build/application-debug-posix.osx-x86_64-dmd_2072-99D5F2E2DCAF9FF19FE5AE403C120D52/testomat .dub/build/application-debug-posix.osx-x86_64-dmd_2072-99D5F2E2DCAF9FF19FE5AE403C120D52/testomat.o -g
Undefined symbols for architecture x86_64:
  "_D7mytypes7myunion6__initZ", referenced from:
      __Dmain in testomat.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
Error: linker exited with status 1
FAIL .dub/build/application-debug-posix.osx-x86_64-dmd_2072-99D5F2E2DCAF9FF19FE5AE403C120D52/ testomat executable
dmd failed with exit code 1.
_____________________

If I change the union's variable type to "int" (or any other) it compiles just fine. So the problem seems to be the "double" value.

It also compiles just fine when I move "mytypes.d" into the source/ folder (even with "double"!!!).


Should I still report it as a bug here: https://issues.dlang.org/ ?

Or is it dub related?

On Sunday, 27 November 2016 at 14:12:31 UTC, Timoses wrote:
> If I change the union's variable type to "int" (or any other) it compiles just fine. So the problem seems to be the "double" value.

That's because int is zero initialized by default and thus doesn't need anything more than a call to zero memory function, and double isn't (it is NaN), so it gets an initializer data blob. If you make it = 0 it might work, but even then, the compiler may want to reference the initializer just because there's a non-default init value!

> Should I still report it as a bug here:

It isn't a bug, this is working as designed [1]. Making it skip the initializer with =void or =0 explicit init values is a valid enhancement request though.

1: search for "default" on this page http://dlang.org/spec/struct.html#static_struct_init

On Sunday, 27 November 2016 at 14:27:54 UTC, Adam D. Ruppe wrote:
> That's because int is zero initialized by default and thus doesn't need anything more than a call to zero memory function, and double isn't (it is NaN), so it gets an initializer data blob. If you make it = 0 it might work, but even then, the compiler may want to reference the initializer just because there's a non-default init value!


Does your answer also explain why it works when I move the mytypes.d into the source/ folder?

On Sunday, 27 November 2016 at 15:23:33 UTC, Timoses wrote:
> On Sunday, 27 November 2016 at 14:27:54 UTC, Adam D. Ruppe wrote:
>> That's because int is zero initialized by default and thus doesn't need anything more than a call to zero memory function, and double isn't (it is NaN), so it gets an initializer data blob. If you make it = 0 it might work, but even then, the compiler may want to reference the initializer just because there's a non-default init value!
>
>
> Does your answer also explain why it works when I move the mytypes.d into the source/ folder?

dub will link it in when it is in the source folder.

On Sunday, 27 November 2016 at 15:56:13 UTC, Stefan Koch wrote:
>> Does your answer also explain why it works when I move the mytypes.d into the source/ folder?
>
> dub will link it in when it is in the source folder.

exactly.

On Sunday, 27 November 2016 at 13:54:54 UTC, Adam D. Ruppe wrote:
> On Sunday, 27 November 2016 at 12:59:32 UTC, Timoses wrote:
>> [...]
>
> It is a linker problem because you didn't link to it... D structs have an initializer, even if they are used for interfacing with C. This is a small blob of data called something like `_D10neo4jTypes17neo4j_map_entry_t6__initZ`, or can be all zeroes, and is generated with the interface module.
>
> [...]

Unions are supposed to be default initialised with the first member,
a ulong and thus should be zero initialised.

> You can often bypass the initializer by using `=0` or `=void` in definitions, but not here... and that is arguably a bug, if you `=void` all fields, it should be free to zero initialize the whole thing, but it doesn't.
>
> [...]

On Sunday, 27 November 2016 at 23:25:57 UTC, Nicholas Wilson wrote:
>> `_D10neo4jTypes17neo4j_map_entry_t6__initZ`, or
> Unions are supposed to be default initialised with the first member, a ulong and thus should be zero initialised.

Right, but neo4j_map_entry_t is a struct and that's the one it is complaining about.