On Monday, 8 June 2020 at 14:45:46 UTC, H. S. Teoh wrote:

> Could you just use LTO for this?  LDC's LTO, for example, lets the linker discard unreferenced symbols.

LTO is a tool that attempts to solve a problem that does not need to exist. This could be said about linkers in general.

On 6/8/20 4:42 AM, evilrat wrote:
> On Monday, 8 June 2020 at 06:14:44 UTC, Manu wrote:
>>
>> I think a first part of the conversation to understand, is that since D doesn't really have first-class `inline` (just a pragma, assumed to be low-level compiler control), I think most people bring their conceptual definition over from C/C++, and that definition is a little odd (although it is immensely useful), but it's not like what D does.
>>
>> In C/C++, inline says that a function will be emit to the binary only when
>> it is called, and the function is marked with internal linkage (it is not
>> visible to the linker from the symbol table)
>> By this definition; what inline REALLY means is that the function is not
>> placed in the binary where it is defined, it is placed in the binary where
>> it is CALLED, and each CU that calls an inline function receives their own
>> copy of the function. From here; optimisers will typically inline the call
>> if they determine it's an advantage to do so.
>>
> 
> Sorry for sticking in my nose, but in result inline in C++ simply tells the linker "this symbol could be present multiple times, just pick whichever one of them and done with it" so it won't complain about multiple symbols anymore.
> And that's why you have to slap inline to functions that have body in header in order to build.
> 
> But I'm not very certain if that's all about it, not using C++ that much since then, and never to that truly deep extent where any loose bit can destroy performance, not beyond fixing the client's code and some crappy pet projects.

This was not true the last time I found "inline abuse".

At a previous company, there was a type where a function was inline or not depending on whether you defined a macro. And when you defined that macro, the implementation was DIFFERENT (and different in a way that calling the non-inline version would be a bug).

Which meant that if the linker was free to choose whichever implementation it wanted, it would defeat the purpose of the inline mechanism (BTW, this was the first time I ever experienced an inline function, and had to research what it meant).

I think Manu's description is accurate, but I also thought it actually inlined the function call as well, even without an optimizer. Hard to tell when the end result isn't obvious.

-Steve

On Monday, 8 June 2020 at 12:35:01 UTC, Jan Hönig wrote:
> On Monday, 8 June 2020 at 10:19:16 UTC, Walter Bright wrote:
>>
>> Why does it matter where it is emitted? Why would you want multiple copies of the same function in the binary?
>
> Performance in HPC.
>
> In C++, consider an []operator. There would be a lot of function calls inside a kernel (some function with lot of loops, one billion iterations of the inner most loop easily). If then I have some kind of stencil or any array accesses, calling a function each time a top of resolving the current pointer would be very costly.

What does this have to do with whether a symbol is emitted? Emitting a symbol doesn't mean you didn't get inlining.

On 6/8/20 2:14 AM, Manu wrote:
> 3. I want to treat the function like an AST macro; I want the function inserted at the callsite, and I want to have total confidence in this mechanic. [This is about articulate mechanical control over code-gen; ie, I know necessary facts about the execution context/callstack that I expect to maintain]

This is my main use case. Not because I want AST macros (not sure I'd agree that AST macro is the right description), but because I write a lot of wrapping functions which are like "parameter modifiers". iopipe is full of them for instance.

Being able to inject a few statement before/after a call without having to dispatch to a separate function first is very useful for a wrapper. It's what the optimizer should do anyway, but making it part of the API makes a declaration that the function is a wrapper and should be treated that way.

Unfortunately, D does not implement this feature in a way that I prefer: https://issues.dlang.org/show_bug.cgi?id=15671

-Steve

On Monday, 8 June 2020 at 14:22:50 UTC, Manu wrote:
> Templates work correctly; template instances are only generated and emit to the calling CU.

Nope - they *might* be emitted into the instantiating module, but very likely aren't. Just try compiling a static lib with multiple modules instantiating equivalent templates. See https://github.com/ldc-developers/ldc/pull/3422#issuecomment-625945508 for an example.

On Monday, June 8, 2020 8:09:04 AM MDT Manu via Digitalmars-d wrote:
> On Mon, Jun 8, 2020 at 8:20 PM Walter Bright via Digitalmars-d <
> > C/C++ inline has always been a hint to the compiler, not a command.
>
> It's not a hint at all. It's a mechanical tool; it marks symbols with internal linkage, and it also doesn't emit them if it's never referenced. The compiler may not choose to ignore that behaviour, it's absolutely necessary, and very important.

It is my understanding that in C++, inline is a hint to the compiler with regards to whether a particular function call is actually inlined. There are other effects that inline has which aren't just a hint, but the behavior of actually inlining the function is a hint, not a command.

Of course, in D, it's even weirder, because it's been "give me an error if the compiler fails to inline it," and I don't know that it even functions as a hint at that point as opposed to just a way to find out that the compiler didn't do what you wanted.

- Jonathan M Davis

On Tue, Jun 9, 2020 at 12:46 AM H. S. Teoh via Digitalmars-d < digitalmars-d@puremagic.com> wrote:

> On Tue, Jun 09, 2020 at 12:09:04AM +1000, Manu via Digitalmars-d wrote: [...]
> >    I don't want a binary full of code that shouldn't be there. It's
> >    very important to be able to control what code is in your binaries.
> >    If it's not referenced, it doesn't exist.
> [...]
>
> Could you just use LTO for this?  LDC's LTO, for example, lets the linker discard unreferenced symbols.
>

I think the compiler should specify behaviour that's universally compatible, and doesn't depend on special tooling to integrate with existing workflows.

On Tue, Jun 9, 2020 at 1:00 AM Steven Schveighoffer via Digitalmars-d < digitalmars-d@puremagic.com> wrote:

> On 6/8/20 4:42 AM, evilrat wrote:
> > On Monday, 8 June 2020 at 06:14:44 UTC, Manu wrote:
> >>
> >> I think a first part of the conversation to understand, is that since D doesn't really have first-class `inline` (just a pragma, assumed to be low-level compiler control), I think most people bring their conceptual definition over from C/C++, and that definition is a little odd (although it is immensely useful), but it's not like what D does.
> >>
> >> In C/C++, inline says that a function will be emit to the binary only
> >> when
> >> it is called, and the function is marked with internal linkage (it is
> not
> >> visible to the linker from the symbol table)
> >> By this definition; what inline REALLY means is that the function is not
> >> placed in the binary where it is defined, it is placed in the binary
> >> where
> >> it is CALLED, and each CU that calls an inline function receives their
> >> own
> >> copy of the function. From here; optimisers will typically inline the
> >> call
> >> if they determine it's an advantage to do so.
> >>
> >
> > Sorry for sticking in my nose, but in result inline in C++ simply tells
> > the linker "this symbol could be present multiple times, just pick
> > whichever one of them and done with it" so it won't complain about
> > multiple symbols anymore.
> > And that's why you have to slap inline to functions that have body in
> > header in order to build.
> >
> > But I'm not very certain if that's all about it, not using C++ that much since then, and never to that truly deep extent where any loose bit can destroy performance, not beyond fixing the client's code and some crappy pet projects.
>
> This was not true the last time I found "inline abuse".
>
> At a previous company, there was a type where a function was inline or not depending on whether you defined a macro. And when you defined that macro, the implementation was DIFFERENT (and different in a way that calling the non-inline version would be a bug).
>
> Which meant that if the linker was free to choose whichever implementation it wanted, it would defeat the purpose of the inline mechanism (BTW, this was the first time I ever experienced an inline function, and had to research what it meant).
>
> I think Manu's description is accurate, but I also thought it actually inlined the function call as well, even without an optimizer. Hard to tell when the end result isn't obvious.
>

What's funny is, in most cases, whether the function is ACTUALLY inlined is
not really interesting in 2020.
What inline allows is control over the binary environment as I describe. I
read it these days as "inline to the calling CU" rather than "inline to the
calling function".

There are cases where inline is really important, and I do want an error if it fails; for instance, if you have a leaf function (does not allocate any stack memory), it's only possible to make calls from that function where the callee is inlined... and if inlining fails, your caller will lose its no-stack-frame requirement. I've had this come up numerous times, and in those cases, a really-strong-does-make-compile-error inline would be useful, but C++ doesn't have anything like that.

On Tue, Jun 9, 2020 at 9:20 AM Jonathan M Davis via Digitalmars-d < digitalmars-d@puremagic.com> wrote:

> On Monday, June 8, 2020 8:09:04 AM MDT Manu via Digitalmars-d wrote:
> > On Mon, Jun 8, 2020 at 8:20 PM Walter Bright via Digitalmars-d <
> > > C/C++ inline has always been a hint to the compiler, not a command.
> >
> > It's not a hint at all. It's a mechanical tool; it marks symbols with internal linkage, and it also doesn't emit them if it's never referenced. The compiler may not choose to ignore that behaviour, it's absolutely necessary, and very important.
>
> It is my understanding that in C++, inline is a hint to the compiler with
> regards to whether a particular function call is actually inlined. There
> are
> other effects that inline has which aren't just a hint, but the behavior of
> actually inlining the function is a hint, not a command.
>

I'm not sure it's even a hint. That behaviour is basically irrelevant. The 'other effects' you describe are what inline does in C++, and we need that too.

On Tuesday, 9 June 2020 at 00:36:18 UTC, Manu wrote:

> What's funny is, in most cases, whether the function is ACTUALLY inlined is
> not really interesting in 2020.
> What inline allows is control over the binary environment as I describe. I
> read it these days as "inline to the calling CU" rather than "inline to the
> calling function".
>
> There are cases where inline is really important, and I do want an error if it fails; for instance, if you have a leaf function (does not allocate any stack memory), it's only possible to make calls from that function where the callee is inlined... and if inlining fails, your caller will lose its no-stack-frame requirement. I've had this come up numerous times, and in those cases, a really-strong-does-make-compile-error inline would be useful, but C++ doesn't have anything like that.

Maybe it's a case where a clear disambiguation is in order? E.g. make a new

pragma(local);

...which would instruct the compiler to do what you're describing. Or, perhaps, expand the range of options for the existing pragma(inline), from the current bool to an enum of behaviors.