Hi all,

  Is it possible to strip unused code from an executable, during compilation?
  In gcc it was possible to include something like: -ffunction-sections -fdata-sections -Wl,--gc-sections
  Look at the following example:

--- lib.d ---
module lib;

import std.c.stdio;

void func1( int x ){
  printf("%d\n",x);
}

int func2( int x ){
  printf("%d\n",x);
  return x;
}


--- test.d ---
import std.c.stdio;
import lib;

int main(){
  func1(2);
}


-----------------
Compilation:

dmd test.d lib.d   - generates a 292651 byte file!!! (why so big?)

-> func2() is NOT used, but is included into the code:
nm test |grep func2
0804bb2c T _D3lib5func2FiZi

After stripping the code, I still obtain a 157444 byte file!!! (still large...)

Question: How can I remove unused code, such that my executable does not grow too much in size?
(I'm almost sure that inside the executable there are still many internal and non-used functions also)

Thanks!

Best,
Tiago Gasiba



-- 
Tiago Gasiba (M.Sc.) - http://www.gasiba.de
Everything should be made as simple as possible, but not simpler.

On Mon, 21 Nov 2005 17:26:50 +0100, Tiago Gasiba <tiago.gasiba@gmail.com> wrote:

>After stripping the code, I still obtain a 157444 byte file!!! (still large...)

I tested this once myself, with a simple "int main() return 0" and the file was excessively large. I would think unused library code could be excluded from the file. The GC isn't *that* large.

I'm an amateur though and don't know diddly compared to you or most anyone else in this NG. hopefully someone else can provide some insight.

Chris

Tiago Gasiba wrote:
> Hi all,
> 
>   Is it possible to strip unused code from an executable, during compilation?
>   In gcc it was possible to include something like: -ffunction-sections -fdata-sections -Wl,--gc-sections
>   Look at the following example:
> 
> --- lib.d ---
> module lib;
> 
> import std.c.stdio;
> 
> void func1( int x ){
>   printf("%d\n",x);
> }
> 
> int func2( int x ){
>   printf("%d\n",x);
>   return x;
> }
> 
> 
> --- test.d ---
> import std.c.stdio;
> import lib;
> 
> int main(){
>   func1(2);
> }
> 
> 
> -----------------
> Compilation:
> 
> dmd test.d lib.d   - generates a 292651 byte file!!! (why so big?)

Much of Phobos appears to be linked in whether you use it or not.  When I try this against Ares the resulting executable is 64,028 bytes.

> Question: How can I remove unused code, such that my executable does not grow too much in size?
> (I'm almost sure that inside the executable there are still many internal and non-used functions also)

Isn't there a version of strip for Win32?  I could have sworn I saw one at some point.


Sean

On Mon, 21 Nov 2005 10:03:56 -0800, Sean Kelly <sean@f4.ca> wrote:
>Isn't there a version of strip for Win32?  I could have sworn I saw one at some point.

I am not aware of strip, but does it do something similar to upx?
http://upx.sourceforge.net
I haven't tested upx on D programs yet, but I know it's very well
known for making VB apps about 15% of their original size.

Chris

Chris wrote:
> On Mon, 21 Nov 2005 10:03:56 -0800, Sean Kelly <sean@f4.ca> wrote:
> 
>>Isn't there a version of strip for Win32?  I could have sworn I saw one at some point.
> 
> 
> I am not aware of strip, but does it do something similar to upx?
> http://upx.sourceforge.net
> I haven't tested upx on D programs yet, but I know it's very well
> known for making VB apps about 15% of their original size.
> 
> Chris

UPX is an executable compressor, not a symbol stripper.  It guarantees no data loss in the compressed data.  It does this by compressing the executable code and data and redirecting the executable's main entry point to a pre-cooked decompression routine which decompresses all the data into memory and then resumes execution at the normal entry point. *phew*

James Dunne wrote:
> Chris wrote:
> 
>> On Mon, 21 Nov 2005 10:03:56 -0800, Sean Kelly <sean@f4.ca> wrote:
>>
>>> Isn't there a version of strip for Win32?  I could have sworn I saw one at some point.
>>
>>
>>
>> I am not aware of strip, but does it do something similar to upx?
>> http://upx.sourceforge.net
>> I haven't tested upx on D programs yet, but I know it's very well
>> known for making VB apps about 15% of their original size.
>>
>> Chris
> 
> 
> UPX is an executable compressor, not a symbol stripper.  It guarantees no data loss in the compressed data.  It does this by compressing the executable code and data and redirecting the executable's main entry point to a pre-cooked decompression routine which decompresses all the data into memory and then resumes execution at the normal entry point. *phew*

Besides AFAIK some versions of UPX don't support the digitalmars .exe-format. Maybe this has been fixed by now. On Linux it's possible to compact dmd-generated files quite a lot with strip & upx. OTOH upx does some implicit stripping at least when used on djgpp .exes.

No, it's not possible to do this during compilation.  It is the compiler's job to take all of the methods/functions and put them in the object file. The compiler doesn't necessarily know what symbols are going to be used and what aren't.

This is a task for the linker- one that I'm surprised it doesn't do already by default.  There's probably a reason for this, but I don't know what it is... anyone?

~John Demme

Tiago Gasiba wrote:

> Hi all,
> 
>   Is it possible to strip unused code from an executable, during
>   compilation? In gcc it was possible to include something like:
>   -ffunction-sections -fdata-sections -Wl,--gc-sections Look at the
>   following example:
> 
> --- lib.d ---
> module lib;
> 
> import std.c.stdio;
> 
> void func1( int x ){
>   printf("%d\n",x);
> }
> 
> int func2( int x ){
>   printf("%d\n",x);
>   return x;
> }
> 
> 
> --- test.d ---
> import std.c.stdio;
> import lib;
> 
> int main(){
>   func1(2);
> }
> 
> 
> -----------------
> Compilation:
> 
> dmd test.d lib.d   - generates a 292651 byte file!!! (why so big?)
> 
> -> func2() is NOT used, but is included into the code:
> nm test |grep func2
> 0804bb2c T _D3lib5func2FiZi
> 
> After stripping the code, I still obtain a 157444 byte file!!! (still
> large...)
> 
> Question: How can I remove unused code, such that my executable does not grow too much in size? (I'm almost sure that inside the executable there are still many internal and non-used functions also)
> 
> Thanks!
> 
> Best,
> Tiago Gasiba
> 
> 
> 

Perhaps when linking an object file with a static library, it might be more difficult to relink the static library and eliminate parts of it than to treat the static library as one big linking unit and link it in as a whole.

I'm no expert in linkers by any means, but I can't think of why this couldn't or shouldn't be done.

John Demme wrote:
> No, it's not possible to do this during compilation.  It is the compiler's
> job to take all of the methods/functions and put them in the object file. The compiler doesn't necessarily know what symbols are going to be used and
> what aren't.
> 
> This is a task for the linker- one that I'm surprised it doesn't do already
> by default.  There's probably a reason for this, but I don't know what it
> is... anyone?
> 
> ~John Demme
> 
> Tiago Gasiba wrote:
> 
> 
>>Hi all,
>>
>>  Is it possible to strip unused code from an executable, during
>>  compilation? In gcc it was possible to include something like:
>>  -ffunction-sections -fdata-sections -Wl,--gc-sections Look at the
>>  following example:
>>
>>--- lib.d ---
>>module lib;
>>
>>import std.c.stdio;
>>
>>void func1( int x ){
>>  printf("%d\n",x);
>>}
>>
>>int func2( int x ){
>>  printf("%d\n",x);
>>  return x;
>>}
>>
>>
>>--- test.d ---
>>import std.c.stdio;
>>import lib;
>>
>>int main(){
>>  func1(2);
>>}
>>
>>
>>-----------------
>>Compilation:
>>
>>dmd test.d lib.d   - generates a 292651 byte file!!! (why so big?)
>>
>>-> func2() is NOT used, but is included into the code:
>>nm test |grep func2
>>0804bb2c T _D3lib5func2FiZi
>>
>>After stripping the code, I still obtain a 157444 byte file!!! (still
>>large...)
>>
>>Question: How can I remove unused code, such that my executable does not
>>grow too much in size? (I'm almost sure that inside the executable there
>>are still many internal and non-used functions also)
>>
>>Thanks!
>>
>>Best,
>>Tiago Gasiba
>>
>>
>>

Jari-Matti Mäkelä schrieb:
> 
> Besides AFAIK some versions of UPX don't support the digitalmars .exe-format. Maybe this has been fixed by now. On Linux it's possible to compact dmd-generated files quite a lot with strip & upx. OTOH upx does some implicit stripping at least when used on djgpp .exes.

In Linux UPX works fine! Further, gdc also strips the code perfectly:

gdc -ffunction-sections -fdata-sections -c lib.d
gdc -ffunction-sections -fdata-sections -c test.d
gdc -ffunction-sections -fdata-sections -Wl,--gc-sections -o test test.o lib.o
ll
insgesamt 412
- -rw-------  1 gasiba users    140 2005-11-22 10:19 lib.d
- -rw-------  1 gasiba users   1400 2005-11-22 10:19 lib.o
- -rwx------  1 gasiba users 404962 2005-11-22 10:20 test
- -rw-------  1 gasiba users     62 2005-11-22 10:19 test.d
- -rw-------  1 gasiba users   1244 2005-11-22 10:19 test.o
nm test | grep func2  -> finds nothing!
strip test
ll test
- -rwx------  1 gasiba users 120092 2005-11-22 10:21 test
upx --best test
ll test
- -rwx------  1 gasiba users 53979 2005-11-22 10:21 test

Before UPX compression, the file is still very large! ca 120K !

IMHO the problem is that the linker might only be able to remove entire sections and not only pieces of code. The following was taken (long time back) from another post somewhere on the internet (don't remember from where):

<snip>
The trick is to add the -ffunction-sections option to gcc (and maybe -fdata-sections), and add the --gc-sections option to ld.

The -ffunction-sections option to gcc will tell it to put every function inits own section, for example "void foo(void)" will end up in ".text.foo".
Likewise the -fdata-sections option will tell gcc to put every data/bss variable in its own section by adding the dot-variable name as a suffix to the default section.

The --gc-sections option to ld will tell it to remove all sections that are not referenced. <snip>

Maybe DMD is not putting every piece of code in its own section and GDC is (because it uses GCC code!).
This would be an interesting feature to have (in DMD).
Although GDC does a great job with striping the code, it totally fails to compile my libraries, with internal compiler errors (due to the high usage of complex number arithmetic - where GDC is still a bit buggy).

Best,
Tiago

- --
Tiago Gasiba (M.Sc.) - http://www.gasiba.de
Everything should be made as simple as possible, but not simpler.

Phobos is huge it is clear, but there are still problems with Ares.

Under win32 the library make does not work:
- references to \bin\dmd\ etc. which is hardcoded and not the "DM standard"
- namespace conflicts (maybe these are burden in DMD deeply AFAIK)

I have tried to link the precompiled ares library (phobos.lib) to oe of my live D projects and some modules were missing. Ex. The Error class, some ToString, ToInt, etc.

I know all of these small can be patched easily but not all of us are "system programmer"/hacker.

Ares would be very nice. Some minor changes are only needed. I am looking forward to see newer versions.

Tamás Nagy

>Much of Phobos appears to be linked in whether you use it or not.  When I try this against Ares the resulting executable is 64,028 bytes.
>
>> Question: How can I remove unused code, such that my executable does not grow too much in size?
>> (I'm almost sure that inside the executable there are still many internal and non-used functions also)
>
>Isn't there a version of strip for Win32?  I could have sworn I saw one at some point.
>
>Sean