If I have a bunch of templates that are only used for code generation, are they removed in the binary(since they are not needed)?

On Thursday, 1 August 2013 at 17:09:07 UTC, JS wrote:
> If I have a bunch of templates that are only used for code generation, are they removed in the binary(since they are not needed)?

templates don't exist in binaries.

On Thu, Aug 01, 2013 at 07:12:51PM +0200, John Colvin wrote:
> On Thursday, 1 August 2013 at 17:09:07 UTC, JS wrote:
> >If I have a bunch of templates that are only used for code generation, are they removed in the binary(since they are not needed)?
> 
> templates don't exist in binaries.

Templates are like cookie molds, you use them to generate lots of
(almost) identical cookies, but you never serve the mold to the
customer. ;-)


T

-- 
He who does not appreciate the beauty of language is not worthy to bemoan its flaws.

On Thursday, 1 August 2013 at 17:47:00 UTC, H. S. Teoh wrote:
> On Thu, Aug 01, 2013 at 07:12:51PM +0200, John Colvin wrote:
>> On Thursday, 1 August 2013 at 17:09:07 UTC, JS wrote:
>> >If I have a bunch of templates that are only used for code
>> >generation, are they removed in the binary(since they are not
>> >needed)?
>> 
>> templates don't exist in binaries.
>
> Templates are like cookie molds, you use them to generate lots of
> (almost) identical cookies, but you never serve the mold to the
> customer. ;-)
>
>
> T

But what about the functions that exist in them?

e.g.,

template A()
{
    void A()
    {
        B();
    }
    void B() { }
}

is everything in the template removed 100% or is there junk that the compiler doesn't remove?


Oh... and I'm not talking about theoretical... I'm talking about what dmd actually does.

On 08/01/2013 10:45 AM, H. S. Teoh wrote:
> On Thu, Aug 01, 2013 at 07:12:51PM +0200, John Colvin wrote:
>> On Thursday, 1 August 2013 at 17:09:07 UTC, JS wrote:
>>> If I have a bunch of templates that are only used for code
>>> generation, are they removed in the binary(since they are not
>>> needed)?
>>
>> templates don't exist in binaries.
>
> Templates are like cookie molds, you use them to generate lots of
> (almost) identical cookies, but you never serve the mold to the
> customer. ;-)
>
>
> T
>

I think the question is whether the instantiations of such templates are removed. If the instance is used only to initialize an enum, the function shouldn't stay in the binary.

Even if the function remains, I think the linker takes care of removing unused functions.

Ali

On Thursday, 1 August 2013 at 17:53:38 UTC, Ali Çehreli wrote:
> On 08/01/2013 10:45 AM, H. S. Teoh wrote:
>> On Thu, Aug 01, 2013 at 07:12:51PM +0200, John Colvin wrote:
>>> On Thursday, 1 August 2013 at 17:09:07 UTC, JS wrote:
>>>> If I have a bunch of templates that are only used for code
>>>> generation, are they removed in the binary(since they are not
>>>> needed)?
>>>
>>> templates don't exist in binaries.
>>
>> Templates are like cookie molds, you use them to generate lots of
>> (almost) identical cookies, but you never serve the mold to the
>> customer. ;-)
>>
>>
>> T
>>
>
> I think the question is whether the instantiations of such templates are removed. If the instance is used only to initialize an enum, the function shouldn't stay in the binary.
>
> Even if the function remains, I think the linker takes care of removing unused functions.
>
> Ali

I have a lot of code generation templates(only for code). I want to make sure they don't hang around in the binary(even strings they use) as there is no point and it is potentially dangerous(security issues).

On Thursday, 1 August 2013 at 17:57:28 UTC, JS wrote:
> On Thursday, 1 August 2013 at 17:53:38 UTC, Ali Çehreli wrote:
>> On 08/01/2013 10:45 AM, H. S. Teoh wrote:
>>> On Thu, Aug 01, 2013 at 07:12:51PM +0200, John Colvin wrote:
>>>> On Thursday, 1 August 2013 at 17:09:07 UTC, JS wrote:
>>>>> If I have a bunch of templates that are only used for code
>>>>> generation, are they removed in the binary(since they are not
>>>>> needed)?
>>>>
>>>> templates don't exist in binaries.
>>>
>>> Templates are like cookie molds, you use them to generate lots of
>>> (almost) identical cookies, but you never serve the mold to the
>>> customer. ;-)
>>>
>>>
>>> T
>>>
>>
>> I think the question is whether the instantiations of such templates are removed. If the instance is used only to initialize an enum, the function shouldn't stay in the binary.
>>
>> Even if the function remains, I think the linker takes care of removing unused functions.
>>
>> Ali
>
> I have a lot of code generation templates(only for code). I want to make sure they don't hang around in the binary(even strings they use) as there is no point and it is potentially dangerous(security issues).

The linker will get rid of anything not needed. Marking things as package (or more restrictive) will make sure they don't stay in librarys when not used.

Ultimately, if you're worried about it, get the linker to print a list of all functions, or check the assembly.

On Thu, Aug 01, 2013 at 07:52:28PM +0200, JS wrote:
> On Thursday, 1 August 2013 at 17:47:00 UTC, H. S. Teoh wrote:
> >On Thu, Aug 01, 2013 at 07:12:51PM +0200, John Colvin wrote:
> >>On Thursday, 1 August 2013 at 17:09:07 UTC, JS wrote:
> >>>If I have a bunch of templates that are only used for code generation, are they removed in the binary(since they are not needed)?
> >>
> >>templates don't exist in binaries.
> >
> >Templates are like cookie molds, you use them to generate lots of
> >(almost) identical cookies, but you never serve the mold to the
> >customer. ;-)
> >
> >
> >T
> 
> But what about the functions that exist in them?

Like I said, cookie molds. You use the mold to press cookies, but only the cookies are served, not the mold. The mold may be very complicated, containing subcookies attached to bigger cookies, but whatever is pressed (i.e., instantiated) is what's served on the dinner plate. The mold always remains in the kitchen.


> template A()
> {
>     void A()
>     {
>         B();
>     }
>     void B() { }
> }
> 
> is everything in the template removed 100% or is there junk that the compiler doesn't remove?

There is nothing to remove. If you instantiated the template, then you get a copy of everything in it. The number of copies equals the number of distinct instantiations. The template itself is just a mold, an abstract entity that doesn't exist in binary form. What it does is to serve as a mold (hence, "template") to make code. So if you use to make 10 copies of the code, that's what you'll get in your executable. If your template contains 5 functions, then each instantiation produces 5 copies of those functions. Simple as that.

Of course, not all code produces binary data -- enum and alias definitions don't produce any binary code, for example -- they're just logical entities that only exist at compile time. So if you have an enum inside a template, it will get copied however many times you instantiate the template, but none of those copies end up in the executable because they're just declarations, not actual code or data.


> Oh... and I'm not talking about theoretical... I'm talking about what dmd actually does.

See for yourself: run a disassembler on the result (preferably filtered through ddemangle so the symbols are actually readable) and see what's included and what's not. On Posix, you can use objdump or nm (if you use nm, though, keep in mind that not all symbols correspond with actual code/data, but they do take up a little space in the executable -- at least enough to store their names).


T

-- 
MACINTOSH: Most Applications Crash, If Not, The Operating System Hangs

On Thursday, 1 August 2013 at 17:53:38 UTC, Ali Çehreli wrote:
> I think the question is whether the instantiations of such templates are removed. If the instance is used only to initialize an enum, the function shouldn't stay in the binary.
>
> Even if the function remains, I think the linker takes care of removing unused functions.
>
> Ali

Unfortunately, you are wrong. No symbol optimization is done by DMD is linker can't figure this out on its own, it requires specific object file layout to do reference counting for symbols.

It is really easy to check - make template function with some anchor value inside (OXDEADBEEF) and compile it with this template used exclusively in constraints or CTFE. Then strip and objdump - you'll find anchors in resulting binary for every instantiation.

On Thursday, 1 August 2013 at 18:09:54 UTC, H. S. Teoh wrote:
> See for yourself: run a disassembler on the result (preferably filtered
> through ddemangle so the symbols are actually readable) and see what's
> included and what's not. On Posix, you can use objdump or nm (if you use
> nm, though, keep in mind that not all symbols correspond with actual
> code/data, but they do take up a little space in the executable -- at
> least enough to store their names).

Done it, see my previous comment here. I do have an idea how to fix it, but
wasn't able to get though dmd code so far :(