On Sat, Jun 13, 2020 at 1:39 PM Manu <turkeyman@gmail.com> wrote:

> On Sat, Jun 13, 2020 at 1:30 PM Andrei Alexandrescu via Digitalmars-d < digitalmars-d@puremagic.com> wrote:
>
>> On 6/12/20 8:54 PM, Walter Bright wrote:
>> > On 6/12/2020 5:17 PM, Andrei Alexandrescu wrote:
>> >> Not sure about that part - if linkage was static by means of using the "static" keyword, multiple definitions may not be merged. (I may be wrong, please correct me.) Consider:
>> >>
>> >> static inline int fun() {
>> >>      static int x;
>> >>      return ++x;
>> >> }
>> >>
>> >> In C++, each translation unit containing a definition of fun() will have a distinct address for x. I don't see how the bodies of those functions can be merged.
>> >
>> > They are not merged in D, for the simple reason that ModuleA.fun() and
>> > ModuleB.fun() will have different (mangled) names presented to the
>> linker.
>>
>> For D the question is if they are merged if the function is defined in a .di file and imported in two other modules.
>>
>
> They are not 'merged', they just don't exist.
> The problem I've repeated many times for D is that it doesn't emit the
> function ANYWHERE, and as such, you get a "undefined symbol" error. This is
> different than C++ where you would have gotten a "multiply defined symbol"
> error, but it's exactly the same problem for the exact same reason. It just
> manifests differently because C++ has .h files which naturally duplicates
> the code into each CU and D doesn't.
>

And where I say "it's the same problem", perhaps it's better to say "it's the same issue"; about the requirement to emit inline code to each referencing CU, and what the linkonce link flags are designed to address.

On Sat, Jun 13, 2020 at 1:25 PM Andrei Alexandrescu via Digitalmars-d < digitalmars-d@puremagic.com> wrote:

> On 6/12/20 8:52 PM, Manu wrote:
> > Except in the case I described as case #3, in which it would be useful to have SOME WAY, to 'force inline' and receive an error if it failed.
>
> I recall a couple of compilers (TopSpeed, does anyone remember?) had such a feature. The warnings listing the arbitrary functions that failed whatever heuristics was utterly useless. No C++ compiler implements it today, and I don't think any should.
>

I feel like I clearly agreed here too; I gave 3 cases which are distinct
use cases. C++ implements #1 and #2, and uses the nomenclature `inline` for
those cases.
What you're talking about here is case #3, which C++ doesn't support, and I
agree C++ shouldn't try and jam this idea into `inline` because that has a
confused history, and a fairly well defined present.
That doesn't mean that it's not a useful tool though, and one that I've
wanted lots of times... but as I've made clear, I see this as a distinct
use case, and should be explicit and distinct.

This #3 use case is what `pragma(inline)` is closest to today. We almost
have the #3 thing that C++ doesn't have, and we have no way to express the
#1-2 things that C++ does have.
If it's the case that `pragma(inline)` is designed to model #3, then we're
still out in the dark with #1 and #2, which are the overwhelmingly more
common use cases.

On 6/12/2020 8:25 PM, Andrei Alexandrescu wrote:
> On 6/12/20 8:54 PM, Walter Bright wrote:
>> On 6/12/2020 5:17 PM, Andrei Alexandrescu wrote:
>>> Not sure about that part - if linkage was static by means of using the "static" keyword, multiple definitions may not be merged. (I may be wrong, please correct me.) Consider:
>>>
>>> static inline int fun() {
>>>      static int x;
>>>      return ++x;
>>> }
>>>
>>> In C++, each translation unit containing a definition of fun() will have a distinct address for x. I don't see how the bodies of those functions can be merged.
>>
>> They are not merged in D, for the simple reason that ModuleA.fun() and ModuleB.fun() will have different (mangled) names presented to the linker.
> 
> For D the question is if they are merged if the function is defined in a .di file and imported in two other modules.

If the di file is mentioned on the command line to the compiler, yes (1) instance of it appears in the executable. Otherwise, (0) instances of it appear in the executable. There are never 2 or more instances in the executable.

On Sat, Jun 13, 2020 at 1:25 PM Andrei Alexandrescu via Digitalmars-d < digitalmars-d@puremagic.com> wrote:

> On 6/12/20 8:52 PM, Manu wrote:
> > On Fri, Jun 12, 2020 at 11:25 PM Andrei Alexandrescu via Digitalmars-d <digitalmars-d@puremagic.com <mailto:digitalmars-d@puremagic.com>>
> wrote:
> >
> >     On 6/8/20 2:14 AM, Manu wrote:
> >      > In C/C++, inline says that a function will be emit to the binary
> >     only
> >      > when it is called, and the function is marked with internal
> >     linkage (it
> >      > is not visible to the linker from the symbol table)
> >
> >     By my recollection this is not the case for C++, at all.
> >
> >     * "inline" does NOT change a function's linkage in C++. You may have
> >     inline functions with internal linkage (static inline) or (default)
> >     external linkage. This is important because e.g. defining a static
> >     variable in an extern inline function will have the same address in
> all
> >     calls to the function.
> >
> >
> > It absolutely changes the linkage.
>
> No.
>

Dump a binary with and without `inline`; look at the link flags... they are different.

> I believe it uses what LLVM calls 'ChooseOne' in its code generator, I
> > don't know about 'standard' linker terminology, if such a thing exists.
>
> It does: http://eel.is/c++draft/basic.link
>
> > It's clearly in the spec too:
> > """
> >
> >  1. There may be more than one definition
> >     <
> https://en.cppreference.com/w/cpp/language/definition#One_Definition_Rule
> > of
> >     an inline function or variable in the program as long as each
> >     definition *appears in a different translation unit* and (for
> >     non-static inline functions and variables) all definitions are
> >     identical. For example, an inline function or an inline variable may
> >     be defined in a header file that is #include'd in multiple source
> files.
> >
> > """
>
> The quote does not even contain the word "linkage".
>
> I'm insisting on this because it happens so often. We need to use the terms with the same meaning, otherwise we get bogged down in silly side quests "it doesn't change linkage" - "oh but it does" and there is no progress.
>

That's because this is a C++ spec, and C++ is not a linker. The spec just
specifies the required semantics, and the compiler implements them.
The compiler happens to implement them appropriately for the linking
ecosystem (by using appropriate link flags), because that's how native code
works... you build modules and then link them with a linker which is
outside the domain of the language spec.

The whole point is that different languages can interact with each
other successfully in 'linker space'. Unless we want to have a
self-contained language island, we can't ignore that a linker will link our
code (together with other code), and the compilers output has to work
properly in that environment.
Like C++, we don't need to spec details about link flags, but the
compiler implementation still needs to implement them in those terms, cus
that's how the ecosystem has been designed to work.

I imagine any changes made here would have a similar expression in the D spec as what you read in the C++ spec... and the practical reality is the implementation will apply the appropriate link flags to the symbols.

On Sat, Jun 13, 2020 at 2:05 PM Walter Bright via Digitalmars-d < digitalmars-d@puremagic.com> wrote:

> On 6/12/2020 8:25 PM, Andrei Alexandrescu wrote:
> > On 6/12/20 8:54 PM, Walter Bright wrote:
> >> On 6/12/2020 5:17 PM, Andrei Alexandrescu wrote:
> >>> Not sure about that part - if linkage was static by means of using the "static" keyword, multiple definitions may not be merged. (I may be
> wrong,
> >>> please correct me.) Consider:
> >>>
> >>> static inline int fun() {
> >>>      static int x;
> >>>      return ++x;
> >>> }
> >>>
> >>> In C++, each translation unit containing a definition of fun() will
> have a
> >>> distinct address for x. I don't see how the bodies of those functions
> can be
> >>> merged.
> >>
> >> They are not merged in D, for the simple reason that ModuleA.fun() and
> >> ModuleB.fun() will have different (mangled) names presented to the
> linker.
> >
> > For D the question is if they are merged if the function is defined in a
> .di
> > file and imported in two other modules.
>
> If the di file is mentioned on the command line to the compiler


It's not, that's literally the point of a .di file.


> , yes (1)
> instance of it appears in the executable. Otherwise, (0) instances of it
> appear
> in the executable. There are never 2 or more instances in the executable.
>

Exactly. And this is not a useful design.

On 6/12/2020 6:16 PM, Manu wrote:
> On Sat, Jun 13, 2020 at 11:00 AM Walter Bright via Digitalmars-d <digitalmars-d@puremagic.com <mailto:digitalmars-d@puremagic.com>> wrote:
> 
>     On 6/11/2020 6:51 PM, Manu wrote:
>      > I'm fairly sure C++ uses the link flag that LLVM calls "choose one".
>      > It's in the C++ spec that all inlines collapse to the same one. We should be
>      > using the same flag.
> 
>     D totally relies on that behavior.
> 
> 
>   So emit the function to the CU where it's called and we're done here!

Next bug report: D takes too long to compile because it's recompiling code it has already compiled over and over!

Yes, this has been a bug report in the past. We fixed it.

Besides, I replied to Andrei in another message how you can resolve this - just put the 'header' file on the command line to dmd.

On 6/12/2020 9:08 PM, Manu wrote:
>     If the di file is mentioned on the command line to the compiler
> 
> It's not, that's literally the point of a .di file.

No, it isn't. A .di file is more of a convention than a feature. It's a module and does not get special treatment by the compiler.

>     , yes (1)
>     instance of it appears in the executable. Otherwise, (0) instances of it appear
>     in the executable. There are never 2 or more instances in the executable.
> 
> Exactly. And this is not a useful design.

I hate to say it, but these sorts of replies are completely useless to resolving your issues. You omitted the *why*.

Why can't you put it on the command line?

May I suggest a paradigm shift? A .d (or .di) file is not a .h file and does not work like .h files other than in a very superficial way. (In fact, .h files do not work like .h files. The C++ compiler has no idea what a .h file is. The characteristics you've imputed to it do not exist.)

It's a module. It's fundamentally different. The .h file rules you are used to do not apply.

You'd never put a .h file on the command line to a C++ compiler. Agreed. But those well-traveled notions don't apply to modules.

Am Sat, 13 Jun 2020 01:33:34 -0700 schrieb Walter Bright:

> On 6/12/2020 9:08 PM, Manu wrote:
>>     If the di file is mentioned on the command line to the compiler
>> 
>> It's not, that's literally the point of a .di file.
> 
> No, it isn't. A .di file is more of a convention than a feature. It's a module and does not get special treatment by the compiler.
> 
>>     , yes (1)
>>     instance of it appears in the executable. Otherwise, (0) instances
>>     of it appear in the executable. There are never 2 or more instances
>>     in the executable.
>> 
>> Exactly. And this is not a useful design.
> 
> I hate to say it, but these sorts of replies are completely useless to resolving your issues. You omitted the *why*.
> 
> Why can't you put it on the command line?
> 


a.di:
void foo() {}

b.d:
import a;

c.d:
import a;
void main() {foo();}


dmd -c b.d
dmd -c c.d
dmd b.o c.o => undefined reference to `_D1a3fooFZv'

dmd -c a.di b.d -ofb.o
dmd -c a.di c.d -ofc.o
dmd b.o c.o => undefined reference to `_D1a3fooFZv'

mv a.di a.d
dmd -c a.d b.d -ofb.o
dmd -c a.d c.d -ofc.o
dmd b.o c.o => multiple definition of `_D1a12__ModuleInfoZ'

dmd -c a.d b.d -ofb.o -betterC
dmd -c a.d c.d -ofc.o -betterC
dmd b.o c.o => OK

BTW: If you do dmd -c a.di you get no object file output. So .di files are treated differently


I think it's interesting that DMD seems to emit some (all?) normal functions as weak. Not sure if LDC and GDC do the same thing. Would also be interesting to see how all this interacts with staic & shared libraries, though I'm optimistic that it just works.




So basically all that's missing for Manu's inline case would be to emit pragma(inlne) functions from non-root modules. Probably a 1-line change.
-- 
Johannes

Am Fri, 12 Jun 2020 23:24:32 -0400 schrieb Andrei Alexandrescu:

> On 6/12/20 8:52 PM, Manu wrote:
>> Except in the case I described as case #3, in which it would be useful to have SOME WAY, to 'force inline' and receive an error if it failed.
> 
> I recall a couple of compilers (TopSpeed, does anyone remember?) had such a feature. The warnings listing the arbitrary functions that failed whatever heuristics was utterly useless. No C++ compiler implements it today, and I don't think any should.

Doesn't GCC's always_inline do exactly that?

https://gcc.gnu.org/onlinedocs/gcc/Common-Function-Attributes.html Generally, functions are not inlined unless optimization is specified. For functions declared inline, this attribute inlines the function independent of any restrictions that otherwise apply to inlining. Failure to inline such a function is diagnosed as an error. Note that if such a function is called indirectly the compiler may or may not inline it depending on optimization level and a failure to inline an indirect call may or may not be diagnosed.



-- 
Johannes

On Saturday, 13 June 2020 at 01:58:55 UTC, Manu wrote:
> On Sat, Jun 13, 2020 at 11:31 AM Walter Bright via Digitalmars-d < digitalmars-d@puremagic.com> wrote:
>> For example, I had to keep saying why? why? why?
>
> I've answered your "why"s so many times.
> <...>
> #2 why? - users want to add a hint to override the optimisers
> judgement

Oh cmon, Manu. This does not answer why. This is what statement that has "why?" before it.
WHAT:
Users want to add a hint to override the optimisers judgement
WHY:
Inlining has a big effect on programs performance and compiler`s heuristics based inlining algorithm doesn't always make the right decision. Manually going through the code base and marking select functions with a tag that tells inliner whether to inline that function or not can improve performance by several percent. In areas where high performance is critical and you have gain a lot by extracting even 1% of performance a tag that controls inliner is absolutely necessary. There fore D needs a way to control which function gets inlined and which is not otherwise D wont be even considered for writing code in high performance computing applications.

Now do that for other 2.
P.s. Remember (X) and (Y)?