On 7/31/2018 3:34 PM, Laeeth Isharc wrote:
> On Saturday, 28 July 2018 at 01:03:10 UTC, Walter Bright wrote:
>> On 7/27/2018 4:15 PM, Laeeth Isharc wrote:
>>> Can you think of a pragmatic solution to Atila's problem?
>>
>> One way is for the C++ => D translator to gather all the members of a namespace before trying to emit them. Since D does not impose an order on declarations (unlike C++) it is not constrained to follow the same order.
> 
> So a new post preprocessor stage that parses the produced D code and regroups to group the declarations in the same namespace together ?  Using DMD as a library or libdparse or something?

I don't know how Atila's translator works.

The htod one that I wrote would read and do the semantic processing on the entire file before walking the data structures and emitting the corresponding D code, so grouping the namespace declarations would be trivial. In fact, due to the nature of semantic processing, they would already be grouped together.

On 7/31/2018 1:43 AM, Atila Neves wrote:
> It's not the same - if I want to link to std::vector and std::string, I'd expect them to be used in D as std.vector and std.string, not std.vector and HackyDThing0.std.string.

Calling them std.string will already cause problems, because there's an std.string in Phobos.

You could call it core.stdcpp.string, analogously to core.stdc.string for string.h.

Keep in mind that with 'alias' names can behave as if they are moved to another scope.

On 7/31/2018 1:12 AM, Manu wrote:
> Given your favourite example:
> ----
> module a;
> extern(C++, ns) void foo();
> ----
> module b;
> extern(C++, ns) void foo();
> -----
> module c;
> import a, b;
> foo(); // error: ambiguous
> ns.foo(); // error, ambiguous (obviously)
> a.ns.foo(); // naturally, this works... it's the D module that
> correctly organises the symbol. 'ns' is worthless here


You can write it like this:

----
module a;
extern(C++, ns) void foo();
alias foo = ns.foo;
----
module b;
extern(C++, ns) void foo();
alias foo = ns.foo;
-----

import a,b;
a.foo();  // calls a.ns.foo()
b.foo();  // calls b.ns.foo()

> When a C++ namespace spans modules (ie, the C++ namespace encloses the
> whole library's api), translation to D looks like this:
> 
> module ns.submodule;
> extern(C++, ns) void foo();
> 
> The name is:
> ns.submodule.ns.foo(); // <- ns is already present at the head of the
> qualified name, there's no point repeating it.

So don't repeat it. There is no particular reason to use that naming convention for an ns namespace. As I wrote to Atila, using 'std' as a package name isn't going to work very well anyway because it's already used as a root package in Phobos.


> The converse case where modules are divided by their namespace (which
> naturally maps to D modules):
> 
> module cpplib.ns;
> extern(C++, ns) void foo();
> 
> Now the name is:
> cpplib.ns.ns.foo(); // <- why would anyone want that?

Why indeed. Don't do that. I don't see any reason to. We've put things like string.h in core.stdc.string, and nobody has complained they cannot just type "import string;". It works fine. Allow me to refer to:

    https://github.com/dlang/druntime/blob/master/src/core/stdcpp/exception.d

It's where std::exception goes -> core.stdcpp.exception

In it, you'll find some extern(C++,std) declarations. The file can be imported as:

    import core.stdcpp.exception;

or:

    import core.stdcpp.exception : std;

or any of a number of other ways to import it and set up aliases (if one prefers) to use as shortcuts to the name.


> But the redundancy is more than a nuisance in certain forms of scanning meta.

As I mentioned, you can use alias to refer to names in other scopes.


> And got forbid the case where C++ namespaces are embedded (namespace
> for lib name, another for the module), and we have this:
> 
> module lib_ns.module_ns;
> extrern(C++, lib_ns) extern(C++, module_ns) void foo();
> 
> The name is:
> lib_ns.module_ns.lib_ns.module_ns.foo(); // <- ....words can't even

I have no idea why it would be necessary to write such. I suspect you are unfamiliar with the uses of 'alias' to refer from one scope to names in another.


> Was that a deliberate
> strategy? I feel like that strategy's been used before.

You repeatedly impugn my motives and/or my mental stability. While that doesn't bother me, it really does not help your case. Please just stick to the technical issue. Believe it or not, I really want to help you (and Laeeth and Atila) be successful with this.

On Wed, 1 Aug 2018 at 02:10, Walter Bright via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>
> On 7/31/2018 3:34 PM, Laeeth Isharc wrote:
> > On Saturday, 28 July 2018 at 01:03:10 UTC, Walter Bright wrote:
> >> On 7/27/2018 4:15 PM, Laeeth Isharc wrote:
> >>> Can you think of a pragmatic solution to Atila's problem?
> >>
> >> One way is for the C++ => D translator to gather all the members of a namespace before trying to emit them. Since D does not impose an order on declarations (unlike C++) it is not constrained to follow the same order.
> >
> > So a new post preprocessor stage that parses the produced D code and regroups to group the declarations in the same namespace together ?  Using DMD as a library or libdparse or something?
>
> I don't know how Atila's translator works.
>
> The htod one that I wrote would read and do the semantic processing on the entire file before walking the data structures and emitting the corresponding D code, so grouping the namespace declarations would be trivial. In fact, due to the nature of semantic processing, they would already be grouped together.

None of this nuisance for Atila is necessary. You're just making
busy-work for him at someone else's expense.
Justify the design, and why it's worth the material cost to the users.

On Wednesday, August 1, 2018 4:02:39 AM MDT Walter Bright via Digitalmars-d wrote:
> On 7/31/2018 1:12 AM, Manu wrote:
> > Given your favourite example:
> > ----
> > module a;
> > extern(C++, ns) void foo();
> > ----
> > module b;
> > extern(C++, ns) void foo();
> > -----
> > module c;
> > import a, b;
> > foo(); // error: ambiguous
> > ns.foo(); // error, ambiguous (obviously)
> > a.ns.foo(); // naturally, this works... it's the D module that
> > correctly organises the symbol. 'ns' is worthless here
>
> You can write it like this:
>
> ----
> module a;
> extern(C++, ns) void foo();
> alias foo = ns.foo;
> ----
> module b;
> extern(C++, ns) void foo();
> alias foo = ns.foo;
> -----
>
> import a,b;
> a.foo();  // calls a.ns.foo()
> b.foo();  // calls b.ns.foo()

Not to say that that can't work, but I have to say that it seems pretty ugly if using extern(C++, NS) requires a bunch of aliases just to use symbols normally. If the namespace were just part of the mangling, none of those aliases would be necessary. All in all, it just seems like having the namespaces add scoping just gets in the way - especially when they would normally get the necessary scoping from being put into separate D modules.

Reading through this thread, and thinking about the issue, I don't see any advantage to the current behavior over having it just affect mangling other than the fact that that's how it currently works. All of the potential complaints that you talk about seem like they at least _should_ be a non-issue given that folks don't complain that way about D's module system for non-C++ functions - but it's certainly true that even engineers aren't always logical or reasonable.

However, if we were discussing a DIP for implementing namespaces for C++, based on my current understanding of the matter, I'd definitely argue that extern(C++) with namespaces should just affect mangling, since that approach seems much more straightforward and much more in line with how other languages are handled, particularly since the scoping aspect is already going to be handled by D's module system anyway. As it stands, it just seems like the language is forcing that all C++ functions be put inside of structs to namespace them on top of them being put into a module. And that's downright awkward, even if it can work. Certainly, it does come across like you didn't trust the D module system to do its job for some reason.

- Jonathan M Davis

On Wed, 1 Aug 2018 at 10:25, Jonathan M Davis via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>
> On Wednesday, August 1, 2018 4:02:39 AM MDT Walter Bright via Digitalmars-d wrote:
> > On 7/31/2018 1:12 AM, Manu wrote:
> > > Given your favourite example:
> > > ----
> > > module a;
> > > extern(C++, ns) void foo();
> > > ----
> > > module b;
> > > extern(C++, ns) void foo();
> > > -----
> > > module c;
> > > import a, b;
> > > foo(); // error: ambiguous
> > > ns.foo(); // error, ambiguous (obviously)
> > > a.ns.foo(); // naturally, this works... it's the D module that
> > > correctly organises the symbol. 'ns' is worthless here
> >
> > You can write it like this:
> >
> > ----
> > module a;
> > extern(C++, ns) void foo();
> > alias foo = ns.foo;
> > ----
> > module b;
> > extern(C++, ns) void foo();
> > alias foo = ns.foo;
> > -----
> >
> > import a,b;
> > a.foo();  // calls a.ns.foo()
> > b.foo();  // calls b.ns.foo()
>
> Not to say that that can't work, but I have to say that it seems pretty ugly if using extern(C++, NS) requires a bunch of aliases just to use symbols normally. If the namespace were just part of the mangling, none of those aliases would be necessary. All in all, it just seems like having the namespaces add scoping just gets in the way - especially when they would normally get the necessary scoping from being put into separate D modules.
>
> Reading through this thread, and thinking about the issue, I don't see any advantage to the current behavior over having it just affect mangling other than the fact that that's how it currently works. All of the potential complaints that you talk about seem like they at least _should_ be a non-issue given that folks don't complain that way about D's module system for non-C++ functions - but it's certainly true that even engineers aren't always logical or reasonable.
>
> However, if we were discussing a DIP for implementing namespaces for C++, based on my current understanding of the matter, I'd definitely argue that extern(C++) with namespaces should just affect mangling, since that approach seems much more straightforward and much more in line with how other languages are handled, particularly since the scoping aspect is already going to be handled by D's module system anyway. As it stands, it just seems like the language is forcing that all C++ functions be put inside of structs to namespace them on top of them being put into a module. And that's downright awkward, even if it can work. Certainly, it does come across like you didn't trust the D module system to do its job for some reason.

Thank you Jonathan.

On Wed, 1 Aug 2018 at 03:05, Walter Bright via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>
> > Was that a deliberate
> > strategy? I feel like that strategy's been used before.
>
> You repeatedly impugn my motives and/or my mental stability. While that doesn't bother me, it really does not help your case.

I'm not quite sure this is a fair connection... or certainly that's
not my intent.
But I do struggle to understand how our arguments about the
pointlessness of this complexity are so easy to dismiss, without
providing any substance at all, ever, in support of the design as is.
I can't accept workarounds involving manual intervention when the
problem shouldn't exist in the first place. This is busy-work which we
will continue to do ad-infinitum, and every time it occurs I just get
more upset and unreasonable.

> Please just stick to the technical issue.

That ball's in your court. You've never justified the design
complexity and the baggage it carries. I've prompted such
justification maybe 5 times in this thread already.
This isn't a technical debate. It never has been. If it was, we would
have arrived at "C++ namespaces are just for mangling" 6 years ago.
The evidence can't lead to any other conclusion.

> Believe it or not, I really want to help you (and Laeeth and Atila) be successful with this.

Then listen to us.
There's a blindingly obvious and ultimately simplifying solution. It
will literally resolve _every single case_ of issue or friction or
complaint i'm aware of that that has ever emerged regarding C++
namespace mangling.
It has never demonstrated to be useful, and we have never identified a
single issue that would be created in the process.
This entire class of problem will instantly disappear. And it's not
even a breaking change (ie, allow string, and then we can even solve
the additional problem where we can't refer to D reserved
names/keywords).

Unless there are actually technical reasons to wear this complexity, recommendations including "use alias everywhere", or "put everything in one file" will never satisfy.

On 8/1/2018 10:24 AM, Jonathan M Davis wrote:
> Not to say that that can't work, but I have to say that it seems pretty ugly
> if using extern(C++, NS) requires a bunch of aliases just to use symbols
> normally.

What is normal is a slippery concept, especially when one is comparing different lookup rules between languages. D modules do not a 1:1 correspondence with C++ namespaces, either (not even close).

Aliases are a normal and highly useful D tool to copy names from one scope to another.


> Certainly, it does come across like
> you didn't trust the D module system to do its job for some reason.

Reorganizing the code into modules means potentially forcing users to split code from one C++ file into multiple D files. How's that really going to work if you have a translation tool? One of the aspects of Java I didn't care for was forcing each class into its own file.

So while Manu is clearly happy with cutting up a C++ file into multiple D files, I doubt that is universal. His proposal would pretty much require that for anyone trying to work with C++ namespaces who ever has a name collision/hijack or wants to make the code robust against collision/hijacking.

An example of silent hijacking:

   extern (C++, "ab") void foo(long); // original code
   ... lots of code ...
   extern (C++, "cd") void foo(int); // added later by intern, should have been
                                     // placed in another module
   ... a thousand lines later ...
   foo(0); // OOPS! now calling cd.foo() rather than ab.foo(), D sux

You might say "nobody would ever write code like that." But that's like the C folks saying real C programmers won't write:

    int a[10];
    for (int i = 0; i <= 10; ++i)
       ...a[i]...

But we both know they do just often enough for it to be a disaster.

Now, with D:

    extern (C++, ab) void foo(long);
    foo(0);    // works!
    ---
    extern (C++, ab) void foo(long);
    extern (C++, ab) void foo(int);   // error!
    ---
    extern (C++, ab) void foo(long);
    extern (C++, cd) void foo(int);
    foo(0);    // error!

I juxtaposed the lines so it's obvious. It's not so obvious when there's a thousand lines of code between each of those lines. It's even worse when foo(long) sends a birthday card to your daughter, and foo(int) launches nuclear missiles.

Yes, this extra protection comes at a cost - you'll have to type a bit more. Just like D doesn't allow implicit declaration of variables, requires static typing, and variables are always initialized unless you explicitly use `= void`. The protection is worth it, and is part of D's philosophy.

I hope that adequately answers the "for some reason" :-)

On 7/31/2018 1:47 AM, Atila Neves wrote:
> The only good way (I don't think the mixin template and struct solutions count) to link to any of that today would be to have one enormous D file with _everything_ in it, including nested namespaces.

Why doesn't it count? The user doesn't need to write that code, the translator does. It achieves what you ask for - a declaration of foo() in the current scope, with the mangling in the C++ namespace.

All it requires from the translator is putting a boilerplate prefix and suffix onto what it already does.

I.e.,

     extern (C++, ns) int foo();

becomes:

     mixin template X() {         // boilerplate prefix

         extern (C++, ns) int foo();   // original line

     } mixin X!() x; alias foo = x.ns.foo;  // boilerplate suffix

Of course, you'd also need to write X and x as X with __LINE__ appended so unique symbols are generated. (I know you've had trouble with generating unique names in another post, but that's an independent problem we should find a way to fix. Maybe instead of __LINE__, use a checksum of the original line?)

On Wednesday, 1 August 2018 at 23:04:01 UTC, Walter Bright wrote:
> An example of silent hijacking:
>
>    extern (C++, "ab") void foo(long); // original code
>    ... lots of code ...
>    extern (C++, "cd") void foo(int); // added later by intern, should have been
>                                      // placed in another module
>    ... a thousand lines later ...
>    foo(0); // OOPS! now calling cd.foo() rather than ab.foo(), D sux
>
> You might say "nobody would ever write code like that." But that's like the C folks saying real C programmers won't write:


You can do that today, just remove the "extern(C++, ...)" part and you have the same issue. Why should C++ with namespaces be safer than just regular D ? I don't understand, if it is such a huge gigantic problem why didn't you do anything to solve this problem in regards to D then ?