Okay this might be of topic, if so sorry for posting this.

I'm about to build a little SDL-application which should run completely
from a single binary.
So my idea was to simple append a .zip-file with all other needed
files (images etc..) to it and extract it in RAM with std.zip.

The big problem is: How does the program know where in the binary the zip-file
begins?
Naturally this information is only available *after* compilation
(its the size of the plain binary then) but how can I insert
it to the program after it has been compiled?

I could reserve the space for a size-variable and compile with the dropped-in value a second time but maybe someone has a better idea?

TIA
Henning

Henning Hasemann schrieb:
> Okay this might be of topic, if so sorry for posting this.
> 
> I'm about to build a little SDL-application which should run completely
> from a single binary.
> So my idea was to simple append a .zip-file with all other needed
> files (images etc..) to it and extract it in RAM with std.zip.
> 
> The big problem is: How does the program know where in the binary the zip-file
> begins?
> Naturally this information is only available *after* compilation
> (its the size of the plain binary then) but how can I insert
> it to the program after it has been compiled?
> 
> I could reserve the space for a size-variable and compile with the dropped-in value a second time but maybe someone has a better idea?
> 
> TIA
> Henning


There are two binary to D converter i know of:
* www.dprogramming.com
* in the tioport project, the bintod folder

So you can compile the zip into a ubyte[] and link it to your sources, then your app can simply read it.

Henning Hasemann wrote:
> I'm about to build a little SDL-application which should run completely from a single binary. So my idea was to simple append a .zip-file with all other needed files (images etc..) to it and extract it in RAM with std.zip.
> 
> The big problem is: How does the program know where in the binary the zip-file begins? Naturally this information is only available *after* compilation (its the size of the plain binary then) but how can I insert it to the program after it has been compiled?
> 
> I could reserve the space for a size-variable and compile with the dropped-in
>  value a second time but maybe someone has a better idea?
> 

http://www.digitalmars.com/d/expression.html#ImportExpression

auto z = new ZipArchive(import("foo.zip"));

(Note: This was typed before I read the post by Deewiant reminding me that the relatively new import-expression is pretty much what I'm hacking together here, and much cleaner (even though it returns char[] instead of void[]/ubyte[]/byte[]). I'll post anyway, just so I don't feel I wasted too much time :( )

Henning Hasemann wrote:
> I'm about to build a little SDL-application which should run completely
> from a single binary.
> So my idea was to simple append a .zip-file with all other needed
> files (images etc..) to it and extract it in RAM with std.zip.
> 
> The big problem is: How does the program know where in the binary the zip-file
> begins?
> Naturally this information is only available *after* compilation
> (its the size of the plain binary then) but how can I insert
> it to the program after it has been compiled?
> 
> I could reserve the space for a size-variable and compile with the dropped-in
> value a second time but maybe someone has a better idea?

A hack that should work: use an assembler to include the .zip into an object file and link it in. Also, Manually construct an array data structure pointing to it (with correct length).
Technically, this isn't what you asked for -- this won't be appending but embedding. IMHO it's a "cleaner" solution though.

Step-by-step: (Note: I didn't test this, so there may be some mistakes in these steps)

1) Copy-paste the following:
=====
section .rodata
zip_start:
    incbin "resources.zip"
zip_end:

global zip_data
zip_data:
    dd zip_end - zip_start  ; Length
    dd zip_start            ; Pointer
=====
to a file (let's say zip-resources.asm).
(For a 64-bit compiler (such as GDC for AMD64) replace both instances of 'dd' by 'dq')
For object files other than ELF (the Linux default), you may need to add an underscore prefix to zip_data (or pass "--prefix _" to nasm when assembling) if you get link errors without it.

2) Install nasm (from nasm.sourceforge.net or your Linux package manager).
(It should also work with yasm. In fact, yasm may be required to generate 64-bit object files unless you want to use the CVS version of nasm. I will be assuming nasm for the rest though, since that's what I know)

3) To assemble the file:
nasm -f elf -o zip-resources.o zip-resources.asm
3a) For windows, replace 'nasm' with 'nasmw'.
3b) For DMD/win, replace 'elf' with 'obj'. You may also want to change the extension to ".obj" (also in the next step) to better fit the naming scheme used by DMD/win itself.
3c) For GDC/win(mingw), replace 'elf' with 'coff'.
(Note: I'm not sure about 3b & 3c, but I /think/ those are the options to use)
You'll probably want to add this command to your build system (to be executed after the resources.zip is updated, obviously). I'm not sure how to do this with build/rebuild, but it's trivial with a makefile.

4) Add zip-resources.o to your linker command line as one of the object files to link.

5) In your source, add the following line:
	extern extern(C) void[] zip_data;
It's 'extern' because it's defined in another file, 'extern(C)' because I don't feel like figuring out what the name mangles to ;), and void[] because that's all std.zip requires and manipulating it manually should be discouraged. Once the new constness system is implemented & working, "final invariant" may be added as well if you want to be nice and const-correct.

6) Elsewhere, add
	auto resources = new std.zip.ZipArchive(zip_data);


Of course you're free to rename variables and assembly labels if you prefer, just be sure to change them in all places where they're mentioned.

> There are two binary to D converter i know of:
> * www.dprogramming.com
> * in the tioport project, the bintod folder
> 
> So you can compile the zip into a ubyte[] and link it to your sources, then your app can simply read it.

Thanks a lot I tried the bintod from dprogramming, works great! Exactly what I looked for!

Henning

Thank you all for your fast answers,
thanks Frits for your detailed work though it shouldnt
be necessary to do it this "hard way".

Ill also try the import() thingy from deewiant,
should save one memory and time consuming step compared to bintod.

Henning

On Sun, 20 May 2007 19:17:00 +0200
Henning Hasemann <hhasemann@web.de> wrote:

> 
> Thank you all for your fast answers,
> thanks Frits for your detailed work though it shouldnt
> be necessary to do it this "hard way".
> 
> Ill also try the import() thingy from deewiant,
> should save one memory and time consuming step compared to bintod.

whoot!
This is dramatically faster regarding compile time *and* easier
than bintod.
(Clear if you imagine that bintod converts the binary to hex and
the compiler converts it back to bin wheras import simply inserts
the binary data).

Henning

Frits van Bommel schrieb:
> (Note: This was typed before I read the post by Deewiant reminding me that the relatively new import-expression is pretty much what I'm hacking together here, and much cleaner (even though it returns char[] instead of void[]/ubyte[]/byte[]). I'll post anyway, just so I don't feel I wasted too much time :( )

Hehe, i think i can now drop my bintod in tioport :)
Thats cool.

Reply to Henning,

>> There are two binary to D converter i know of:
>> * www.dprogramming.com
>> * in the tioport project, the bintod folder
>> So you can compile the zip into a ubyte[] and link it to your
>> sources, then your app can simply read it.
>> 
> Thanks a lot I tried the bintod from dprogramming, works great!
> Exactly what I looked for!
> 
> Henning
> 

another solution would be:

auto zipstream = new MemoryStream(import("zipfile.zip));

the import("...") loads the file at compile time and stuffs it into the binary as a constant string, that with the memory stream lets you read it like a file

Reply to myself,
> 
> auto zipstream = new MemoryStream(import("zipfile.zip));
> 


note to self: read the whole thread befor posting :b