On 6/10/2020 3:47 PM, matheus wrote:
> Interesting: https://github.com/97-things/97-things-every-programmer-should-know/blob/master/en/thing_53/README.md 

Haha, I didn't know it was free online! Thanks for the link, and enjoy!

On 6/10/2020 1:10 PM, kinke wrote:
> TLDR: Templates, with the current emission scheme, can definitely not be used to emit a function into every referencing CU.

Yes, but in Manu's case the idea is to not need to link with a library. It's hard to see how the compiler would know that a template was instantiated in a library - meaning it'll emit an instance for at least one of the modules given on the command line.

That should resolve the particular issue Manu was having.

On Thursday, 11 June 2020 at 10:05:14 UTC, Walter Bright wrote:
> On 6/10/2020 1:10 PM, kinke wrote:
>> TLDR: Templates, with the current emission scheme, can definitely not be used to emit a function into every referencing CU.
>
> Yes, but in Manu's case the idea is to not need to link with a library. It's hard to see how the compiler would know that a template was instantiated in a library - meaning it'll emit an instance for at least one of the modules given on the command line.
>
> That should resolve the particular issue Manu was having.

The Idea is to be able to have a "header-only" library where importing is enough to use it.
Which will only (transitively) draw in  the functionality used.
While it is possible with nullary templates it is dissatisfying, dissatisfying enough to cause a busy person like Manu to write a post about it.

On 6/11/20 6:05 AM, Walter Bright wrote:
> On 6/10/2020 1:10 PM, kinke wrote:
>> TLDR: Templates, with the current emission scheme, can definitely not be used to emit a function into every referencing CU.
> 
> Yes, but in Manu's case the idea is to not need to link with a library. It's hard to see how the compiler would know that a template was instantiated in a library - meaning it'll emit an instance for at least one of the modules given on the command line.
> 
> That should resolve the particular issue Manu was having.

An example:

--- b.d:
module b;

int foo(T)()
{
   return 1;
}

enum x = foo!int();

--- a.d:
import b;

void main()
{
   auto  = foo!int();
}
---

> dmd a.d

Undefined symbols for architecture x86_64:
  "__D1b__T3fooTiZQhFNaNbNiNfZi", referenced from:
      __Dmain in a.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
Error: linker exited with status 1

comment out the enum and it works.

-Steve

On Thursday, 11 June 2020 at 10:05:14 UTC, Walter Bright wrote:
> On 6/10/2020 1:10 PM, kinke wrote:
>> TLDR: Templates, with the current emission scheme, can definitely not be used to emit a function into every referencing CU.
>
> Yes, but in Manu's case the idea is to not need to link with a library.

I know, just saying that the workaround suggested by Stefan ('just make it a dummy template') doesn't work in general - it *can* work, but only when using -allinst in combination with compiling a single object file per compiler invocation, basically excluding all static libraries compiled in one cmdline (dub default, right?).

In the linked LDC PR, I'm experimenting with a different template emission scheme, emitting each referenced template stuff into each referencing CU/object file, using linkonce_odr linkage. As a side effect, that would enable the template workaround (or open the door for some other syntactic way of emitting regular non-templated functions the same way), without -allinst or having to take care of how the compiler is invoked.

My interest in this wasn't triggered by wanting to enforce particular functions to be emitted into each referencing CU, but by
a) the current emission scheme requiring LTO in order to inline suited little templated functions when compiling static libs in one go,
b) being able to use linkonce_odr instead of weak_odr linkage for template stuff, meaning potentially less work for the linker, and potentially quite a bit less work for the optimizer. Compiling an optimized dub (a single object file) with LDC using -linkonce-templates reduces the total compile+optimize times by roughly 25%, simply because LLVM inlines most little template stuff and can then discard the unused linkonce_odr functions early in the process, without uselessly optimizing them to death (as it currently cannot discard it, as other libraries/objects might depend on some template instances as mentioned earlier).

On Monday, 8 June 2020 at 14:22:50 UTC, Manu wrote:
> On Mon, Jun 8, 2020 at 8:30 PM kinke via Digitalmars-d <
>> In its owning module, the function is emitted as a regular
>> function, as a fallback for non-inlined cases (and for when people take its address etc.). Our opinions diverge wrt. whether that's a problem - to me it's clearly no big deal, as the function is a) most likely small, and b) subject to linker stripping if unreferenced.
>
>
> a) most likely small, but still not nothing; the symbol table is public ABI
> material, and in some projects I've worked on, the symbol table is
> carefully curated.
> b) linker stripped is not reliable. We are unnecessarily inviting issues in
> some cases, and there's just no reason for that.
>
> If we're confident that link stripping is 100% reliable when a symbol is not referenced, then I have no complaint here.
>
> Can you show what case a hard-symbol in the owning CU solves? Non-inlined
> cases will still find it locally if it has internal linkage (or whatever
> that link flag is called).
> I think it's the same flag that `static` (or `inline`) in C++ specifies
> right?

Internal linkage (C(++) static), AFAIK, means that you may end up with multiple identical functions in the final linked binary. Some linkers may be able to still fold them, such as the MS linker and lld with /OPT:ICF (identical COMDAT folding).

Linkonce_odr linkage (C++ templates and `inline` functions) on the other hand means that you end up with either 0 or 1 function in the final binary, even when multiple object files define it; AFAIK, without having to rely on /OPT:REF or --gc-sections etc. At least for LLVM, linkonce_odr additionally means that the definition is discarded early if unused and might not make it to the object file at all.

Weak_odr linkage (current D templates) is similar to linkonce_odr but not 'officially' discardable (if unreferenced) for the final binary (and thus always making it to the object file). This can be overridden by /OPT:REF and --gc-sections etc., the success of which also depends on symbol visibility and/or --export-dynamic etc.

On 6/11/2020 8:16 AM, Steven Schveighoffer wrote:
> comment out the enum and it works.

Sounds good to me. Don't do that in the "header" file and it'll work.

On 6/11/2020 8:42 AM, kinke wrote:
> My interest in this wasn't triggered by wanting to enforce particular functions to be emitted into each referencing CU, but by
> a) the current emission scheme requiring LTO in order to inline suited little templated functions when compiling static libs in one go,
> b) being able to use linkonce_odr instead of weak_odr linkage for template stuff, meaning potentially less work for the linker, and potentially quite a bit less work for the optimizer. Compiling an optimized dub (a single object file) with LDC using -linkonce-templates reduces the total compile+optimize times by roughly 25%, simply because LLVM inlines most little template stuff and can then discard the unused linkonce_odr functions early in the process, without uselessly optimizing them to death (as it currently cannot discard it, as other libraries/objects might depend on some template instances as mentioned earlier).

A library consists of two parts:

1. the "header" file
2. the binary to link with, which may be absent

Take some care in what goes in 1, what goes in 2, and you'll be fine.

For example, consider core.checkedint. The functions in it are all templates so that they can be used with -betterC which doesn't link with druntime. It's a classic "header only" library. It works fine.

On Fri, Jun 12, 2020 at 3:40 AM kinke via Digitalmars-d < digitalmars-d@puremagic.com> wrote:

> On Monday, 8 June 2020 at 14:22:50 UTC, Manu wrote:
> > On Mon, Jun 8, 2020 at 8:30 PM kinke via Digitalmars-d <
> >> In its owning module, the function is emitted as a regular function, as a fallback for non-inlined cases (and for when people take its address etc.). Our opinions diverge wrt. whether that's a problem - to me it's clearly no big deal, as the function is a) most likely small, and b) subject to linker stripping if unreferenced.
> >
> >
> > a) most likely small, but still not nothing; the symbol table
> > is public ABI
> > material, and in some projects I've worked on, the symbol table
> > is
> > carefully curated.
> > b) linker stripped is not reliable. We are unnecessarily
> > inviting issues in
> > some cases, and there's just no reason for that.
> >
> > If we're confident that link stripping is 100% reliable when a symbol is not referenced, then I have no complaint here.
> >
> > Can you show what case a hard-symbol in the owning CU solves?
> > Non-inlined
> > cases will still find it locally if it has internal linkage (or
> > whatever
> > that link flag is called).
> > I think it's the same flag that `static` (or `inline`) in C++
> > specifies
> > right?
>
> Internal linkage (C(++) static), AFAIK, means that you may end up
> with multiple identical functions in the final linked binary.
> Some linkers may be able to still fold them, such as the MS
> linker and lld with /OPT:ICF (identical COMDAT folding).
>

Actually, I might be wrong about what the link flag is called.
I'm fairly sure C++ uses the link flag that LLVM calls "choose one".
It's in the C++ spec that all inlines collapse to the same one. We should
be using the same flag.

Linkonce_odr linkage (C++ templates and `inline` functions) on
> the other hand means that you end up with either 0 or 1 function in the final binary, even when multiple object files define it; AFAIK, without having to rely on /OPT:REF or --gc-sections etc. At least for LLVM, linkonce_odr additionally means that the definition is discarded early if unused and might not make it to the object file at all.
>
> Weak_odr linkage (current D templates) is similar to linkonce_odr but not 'officially' discardable (if unreferenced) for the final binary (and thus always making it to the object file). This can be overridden by /OPT:REF and --gc-sections etc., the success of which also depends on symbol visibility and/or --export-dynamic etc.
>

On Fri, Jun 12, 2020 at 8:20 AM Walter Bright via Digitalmars-d < digitalmars-d@puremagic.com> wrote:

> On 6/11/2020 8:42 AM, kinke wrote:
> > My interest in this wasn't triggered by wanting to enforce particular
> functions
> > to be emitted into each referencing CU, but by
> > a) the current emission scheme requiring LTO in order to inline suited
> little
> > templated functions when compiling static libs in one go,
> > b) being able to use linkonce_odr instead of weak_odr linkage for
> template
> > stuff, meaning potentially less work for the linker, and potentially
> quite a bit
> > less work for the optimizer. Compiling an optimized dub (a single object
> file)
> > with LDC using -linkonce-templates reduces the total compile+optimize
> times by
> > roughly 25%, simply because LLVM inlines most little template stuff and
> can then
> > discard the unused linkonce_odr functions early in the process, without uselessly optimizing them to death (as it currently cannot discard it,
> as other
> > libraries/objects might depend on some template instances as mentioned
> earlier).
>
> A library consists of two parts:
>
> 1. the "header" file
> 2. the binary to link with, which may be absent
>
> Take some care in what goes in 1, what goes in 2, and you'll be fine.
>
> For example, consider core.checkedint. The functions in it are all
> templates so
> that they can be used with -betterC which doesn't link with druntime. It's
> a
> classic "header only" library. It works fine.
>

We're not talking about templates.