I'm having a lot of trouble with C++ namespaces.
The problem is, the namespace is not just attributed to the symbol in
D, but it's emulated as a named scope.

The trouble mostly appears in this situation:

file1.d
  extern(C++, NS) struct X;

file2.d
  extern(C++, NS) struct Y;

file3.d
  import file1, file2;
  X x; // nope
  Y y; // nope
  NS.X x; // NS has multiple definitions...
  NS.Y y; // NS has multiple definitions...


And various other configurations similar to this where multiple files
declare symbols in the same C++ namespace, and then something tries to
import more than one of them.
Circular imports of this sort always cause problems.

Additionally to that, I don't really want the C++ namespaces to be visible to D; they should be for mangling purposes only.

So I try code like this:
  private extern(C++, NS) struct Thing {}
  alias Thing = NS.Thing;

The idea being that NS will not be available externally, and they
should use Thing in module scope instead, but that doesnt work with
errors:
  Error: module blah class blah.NS.Thing is private

It seems aliasing a private thing into the public namespace does not make it accessible via that alias?

The only way I've managed to make any of those work is with proxy modules, which static import the C++ module, and then `alias Thing = NS.Thing` into the proxy module's scope. This is horrid, and it doubles my module count. I haven't found another pattern that's reliably workable.

Ideas?

On 11/28/2015 8:40 PM, Manu via Digitalmars-d wrote:
> I'm having a lot of trouble with C++ namespaces.
> The problem is, the namespace is not just attributed to the symbol in
> D, but it's emulated as a named scope.

It is not "emulated" in D, it is an actual named scope in D.


> The trouble mostly appears in this situation:
>
> file1.d
>    extern(C++, NS) struct X;
>
> file2.d
>    extern(C++, NS) struct Y;
>
> file3.d
>    import file1, file2;
>    X x; // nope
>    Y y; // nope
>    NS.X x; // NS has multiple definitions...
>    NS.Y y; // NS has multiple definitions...
>
>
> And various other configurations similar to this where multiple files
> declare symbols in the same C++ namespace, and then something tries to
> import more than one of them.
> Circular imports of this sort always cause problems.

D does not support C++ semantics. You cannot split namespaces into multiple files in D, nor can you add symbols to an existing namespace. For namespace NS, all the declarations in NS have to be in one file and between the { }, just like any other scope in D.


> Additionally to that, I don't really want the C++ namespaces to be
> visible to D; they should be for mangling purposes only.

We considered making them for mangling only, and rejected it as unworkable, because then two identical names in different namespaces could not be distinguished by the D symbol table system.


> So I try code like this:
>    private extern(C++, NS) struct Thing {}
>    alias Thing = NS.Thing;
>
> The idea being that NS will not be available externally, and they
> should use Thing in module scope instead, but that doesnt work with
> errors:
>    Error: module blah class blah.NS.Thing is private
>
> It seems aliasing a private thing into the public namespace does not
> make it accessible via that alias?

Aliases do not change access permissions. They are just aliases.


> The only way I've managed to make any of those work is with proxy
> modules, which static import the C++ module, and then `alias Thing =
> NS.Thing` into the proxy module's scope. This is horrid, and it
> doubles my module count. I haven't found another pattern that's
> reliably workable.
>
> Ideas?

Stop trying to bash D into behaving like C++! It'll just make for horrid code (as you discovered) and make you miserable trying. D has an interface to C++; it does not have C++ semantics.

On 29 November 2015 at 14:57, Walter Bright via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>
> D does not support C++ semantics. You cannot split namespaces into multiple files in D, nor can you add symbols to an existing namespace. For namespace NS, all the declarations in NS have to be in one file and between the { }, just like any other scope in D.

Then the feature fails. You can't put the entire STL in one file.
C++ doesn't namespace per file, it generally namespaces per project...
so this is the common case; every module define symbols in the same
C++ namespace.


>> Additionally to that, I don't really want the C++ namespaces to be visible to D; they should be for mangling purposes only.
>
>
> We considered making them for mangling only, and rejected it as unworkable, because then two identical names in different namespaces could not be distinguished by the D symbol table system.

I understand, and that's fine, but it's not particularly useful unless I can make the namespace invisible and alias the namespaced symbols into D's module (user-accessible) scope. Otherwise importing 2 modules leads to conflicting namespace and symbol resolution is all messed up.


>> So I try code like this:
>>    private extern(C++, NS) struct Thing {}
>>    alias Thing = NS.Thing;
>>
>> The idea being that NS will not be available externally, and they
>> should use Thing in module scope instead, but that doesnt work with
>> errors:
>>    Error: module blah class blah.NS.Thing is private
>>
>> It seems aliasing a private thing into the public namespace does not make it accessible via that alias?
>
>
> Aliases do not change access permissions. They are just aliases.

Maybe a special case for C++ namespaces? Or some other solution that
produces the same result.
It's a weird sort of permission this one, it's not that the symbols
are 'private'; I intend the user to use them, I just want the C++
hierarchy excluded from the symbol table and only accessibly by
controlled/explicit alias.


>> The only way I've managed to make any of those work is with proxy modules, which static import the C++ module, and then `alias Thing = NS.Thing` into the proxy module's scope. This is horrid, and it doubles my module count. I haven't found another pattern that's reliably workable.
>>
>> Ideas?
>
>
> Stop trying to bash D into behaving like C++! It'll just make for horrid code (as you discovered) and make you miserable trying. D has an interface to C++; it does not have C++ semantics.

I don't think I'm doing that at all, I'm just trying to make some C++
code linkable to D, and it doesn't work at all.
I think you misunderstood my entire post, I don't want C++ semantics
at all, I just want to mangle like C++ for linkage, but the
implementation doesn't let me do that. If anything, I'm trying to make
C++ extern's behave like D semantics, but namespaces aren't a normal
part of D, and don't really have idioms for dealing with this.

C++ tends to namespace everything with the same namespace, and that means either I implement the entire C++ library in a single module (no way), or I face this mess. The feature just doesn't work beyond a single file linkage experiment, or I completely missed how I'm supposed to use it.

Upon further wasting-my-time, I am starting to suspect the forward
referencing issue may be a bigger problem than it appears.
I have lots of aliases, and most of them work, but some of them just
don't; "Error: identifier 'Kernel' of 'ep.Kernel' is not defined" (not
a very helpful message). It is though, it's defined right next to it's
companion, which is declared at the same time, in the same place, but
I don't get an error for that one. I think the problem here is that
the companion symbol isn't involved in a cross-module reference.

I try and reduce the problem, but it always disappears. It only
appears when the number of modules and interaction between them
becomes sufficient.
I suspect that whatever it is that causes namespace forward
referencing to fail may also be leading to this problem... just a
hunch, but it's the best I've got.

On 29 November 2015 at 16:14, Manu <turkeyman@gmail.com> wrote:
> Upon further wasting-my-time, I am starting to suspect the forward
> referencing issue may be a bigger problem than it appears.
> I have lots of aliases, and most of them work, but some of them just
> don't; "Error: identifier 'Kernel' of 'ep.Kernel' is not defined" (not
> a very helpful message). It is though, it's defined right next to it's
> companion, which is declared at the same time, in the same place, but
> I don't get an error for that one. I think the problem here is that
> the companion symbol isn't involved in a cross-module reference.
>
> I try and reduce the problem, but it always disappears. It only
> appears when the number of modules and interaction between them
> becomes sufficient.
> I suspect that whatever it is that causes namespace forward
> referencing to fail may also be leading to this problem... just a
> hunch, but it's the best I've got.


Progress... now when I compile, I get a pop-up dialog box that says:
"object.Error@(0): assert(0) or HLT instruction", and then writes
"ICE: unsupported type const(char)[]" to the output.

On 29 November 2015 at 19:42, Manu <turkeyman@gmail.com> wrote:
> On 29 November 2015 at 16:14, Manu <turkeyman@gmail.com> wrote:
>> Upon further wasting-my-time, I am starting to suspect the forward
>> referencing issue may be a bigger problem than it appears.
>> I have lots of aliases, and most of them work, but some of them just
>> don't; "Error: identifier 'Kernel' of 'ep.Kernel' is not defined" (not
>> a very helpful message). It is though, it's defined right next to it's
>> companion, which is declared at the same time, in the same place, but
>> I don't get an error for that one. I think the problem here is that
>> the companion symbol isn't involved in a cross-module reference.
>>
>> I try and reduce the problem, but it always disappears. It only
>> appears when the number of modules and interaction between them
>> becomes sufficient.
>> I suspect that whatever it is that causes namespace forward
>> referencing to fail may also be leading to this problem... just a
>> hunch, but it's the best I've got.
>
>
> Progress... now when I compile, I get a pop-up dialog box that says:
> "object.Error@(0): assert(0) or HLT instruction", and then writes
> "ICE: unsupported type const(char)[]" to the output.

So, I think this compiler crash is when trying to C++ mangle types
that it doesn't know how to deal with...
interestingly, one of those types is typeof(null), which should work,
and be mangled as nullptr_t ?
Any slice type (ie, char[]) used anywhere within any scope attributed
extern(C++) seems to cause DMD to crash.

I have these extern(C++) structs and classes, and within them are a
bunch of little D inline functions and helpers to make the C++ extern
useful to D.
This is where '[]' tends to appear, and the consequent compiler
crashes. I have tried: extern (D) void helperMethod(char[] s) { ... },
but extern(D) doesn't seem to do anything here, and it still crashes.
Options?

On 29 Nov 2015 6:17 am, "Manu via Digitalmars-d" < digitalmars-d@puremagic.com> wrote:
>
> On 29 November 2015 at 14:57, Walter Bright via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
> >
> > D does not support C++ semantics. You cannot split namespaces into
multiple
> > files in D, nor can you add symbols to an existing namespace. For
namespace
> > NS, all the declarations in NS have to be in one file and between the {
},
> > just like any other scope in D.
>
> Then the feature fails. You can't put the entire STL in one file.
> C++ doesn't namespace per file, it generally namespaces per project...
> so this is the common case; every module define symbols in the same
> C++ namespace.
>
>

This (and response below about mangling) answers a recent question in a PR I made.  I had noticed the same when writing a testsuite case, only to discover you can only declare `extern(C++, std)` once.  For me, having all related code for a test close together is a nice-to-have though.

> >> Additionally to that, I don't really want the C++ namespaces to be visible to D; they should be for mangling purposes only.
> >
> >
> > We considered making them for mangling only, and rejected it as
unworkable,
> > because then two identical names in different namespaces could not be distinguished by the D symbol table system.
>
> I understand, and that's fine, but it's not particularly useful unless I can make the namespace invisible and alias the namespaced symbols into D's module (user-accessible) scope. Otherwise importing 2 modules leads to conflicting namespace and symbol resolution is all messed up.
>

Renamed imports should work here?

>
> >> So I try code like this:
> >>    private extern(C++, NS) struct Thing {}
> >>    alias Thing = NS.Thing;
> >>
> >> The idea being that NS will not be available externally, and they
> >> should use Thing in module scope instead, but that doesnt work with
> >> errors:
> >>    Error: module blah class blah.NS.Thing is private
> >>
> >> It seems aliasing a private thing into the public namespace does not make it accessible via that alias?
> >
> >
> > Aliases do not change access permissions. They are just aliases.
>
> Maybe a special case for C++ namespaces? Or some other solution that
> produces the same result.
> It's a weird sort of permission this one, it's not that the symbols
> are 'private'; I intend the user to use them, I just want the C++
> hierarchy excluded from the symbol table and only accessibly by
> controlled/explicit alias.
>

You may have already noticed, but `extern(C++, NS)` behaves more like importing a module, but rather than the module being in a separate file, it's contents are put inplace of the braces.

Like importing modules in D, the namespace name itself is not a strong encapsulation (unlike e.g D enums), so whether you use Thing or NS.Thing, both will resolve to the same symbol.  The same should also be true when importing a namespace from another module.

I think your idea with aliases was just wishful thinking, aliases themselves never worked like that.

On 29 Nov 2015 10:42 am, "Manu via Digitalmars-d" < digitalmars-d@puremagic.com> wrote:
>
> On 29 November 2015 at 16:14, Manu <turkeyman@gmail.com> wrote:
> > Upon further wasting-my-time, I am starting to suspect the forward
> > referencing issue may be a bigger problem than it appears.
> > I have lots of aliases, and most of them work, but some of them just
> > don't; "Error: identifier 'Kernel' of 'ep.Kernel' is not defined" (not
> > a very helpful message). It is though, it's defined right next to it's
> > companion, which is declared at the same time, in the same place, but
> > I don't get an error for that one. I think the problem here is that
> > the companion symbol isn't involved in a cross-module reference.
> >
> > I try and reduce the problem, but it always disappears. It only
> > appears when the number of modules and interaction between them
> > becomes sufficient.
> > I suspect that whatever it is that causes namespace forward
> > referencing to fail may also be leading to this problem... just a
> > hunch, but it's the best I've got.
>
>
> Progress... now when I compile, I get a pop-up dialog box that says:
> "object.Error@(0): assert(0) or HLT instruction", and then writes
> "ICE: unsupported type const(char)[]" to the output.

I keep telling them to fix that.  My suggestion of replacing them with meaningful errors was rejected.

On 2015-11-29 11:19, Iain Buclaw via Digitalmars-d wrote:

> I keep telling them to fix that.  My suggestion of replacing them with
> meaningful errors was rejected.

"rejected", what!! Any assertion in the compiler is a bug. Or was that not from the compiler?

-- 
/Jacob Carlborg

On 29 Nov 2015 10:57 am, "Manu via Digitalmars-d" < digitalmars-d@puremagic.com> wrote:
>
> On 29 November 2015 at 19:42, Manu <turkeyman@gmail.com> wrote:
> > On 29 November 2015 at 16:14, Manu <turkeyman@gmail.com> wrote:
> >> Upon further wasting-my-time, I am starting to suspect the forward
> >> referencing issue may be a bigger problem than it appears.
> >> I have lots of aliases, and most of them work, but some of them just
> >> don't; "Error: identifier 'Kernel' of 'ep.Kernel' is not defined" (not
> >> a very helpful message). It is though, it's defined right next to it's
> >> companion, which is declared at the same time, in the same place, but
> >> I don't get an error for that one. I think the problem here is that
> >> the companion symbol isn't involved in a cross-module reference.
> >>
> >> I try and reduce the problem, but it always disappears. It only
> >> appears when the number of modules and interaction between them
> >> becomes sufficient.
> >> I suspect that whatever it is that causes namespace forward
> >> referencing to fail may also be leading to this problem... just a
> >> hunch, but it's the best I've got.
> >
> >
> > Progress... now when I compile, I get a pop-up dialog box that says:
> > "object.Error@(0): assert(0) or HLT instruction", and then writes
> > "ICE: unsupported type const(char)[]" to the output.
>
> So, I think this compiler crash is when trying to C++ mangle types
> that it doesn't know how to deal with...
> interestingly, one of those types is typeof(null), which should work,
> and be mangled as nullptr_t ?
> Any slice type (ie, char[]) used anywhere within any scope attributed
> extern(C++) seems to cause DMD to crash.
>

Correct, you will also get the same message with shared or immutable types, and static arrays.  And I will hazard a guess at delegates and lazy parameters too.

There is no technical limitation for this, you *can* pass these data structures to C/C++ (except shared and immutable) so long as the struct layout is matching on the other side.  At the same time, it may be for the best if you pass the ptr and length explicitly.