http://d.puremagic.com/issues/show_bug.cgi?id=2137

           Summary: Data not compressed on fly when adding to zip archive
           Product: D
           Version: 2.014
          Platform: PC
        OS/Version: All
            Status: NEW
          Severity: minor
          Priority: P2
         Component: Phobos
        AssignedTo: bugzilla@digitalmars.com
        ReportedBy: dsimcha@yahoo.com


The fact that, when an ArchiveMember is added via addMember() to a ZipArchive, the data in expandedData is not compressed on the fly, makes it very difficult to create large zip files without running out of memory.  I have created a simple workaround  by moving the following code from the build() method to the addMember() method:

de.crc32 = std.zlib.crc32(0, cast(void[])de.expandedData);
de.compressedData =
cast(ubyte[])std.zlib.compress(cast(void[]))de.expandedData;
de.compressedData = de.compressedData[2 .. de.compressedData.length - 4];
de.compressedSize = de.compressedData.length;
de.expandedSize = de.expandedData.length;

I also added the following to free the memory previously occupied by expandedData:

de.expandedData=null;

This is actually proving to be a significant issue for me, as I am attempting to use std.zip to archive large amounts of DNA sequence, which compresses extremely well, but won't fit in memory uncompressed.


-- 

http://d.puremagic.com/issues/show_bug.cgi?id=2137





------- Comment #1 from bugzilla@digitalmars.com  2008-08-15 01:51 -------
Could you please email me the diffs? Thanks!


-- 

http://d.puremagic.com/issues/show_bug.cgi?id=2137





------- Comment #2 from dsimcha@yahoo.com  2008-08-15 09:34 -------
Created an attachment (id=269)
 --> (http://d.puremagic.com/issues/attachment.cgi?id=269&action=view)
Diffs to make std.zip compress on fly.


-- 

http://d.puremagic.com/issues/show_bug.cgi?id=2137





------- Comment #3 from dsimcha@yahoo.com  2008-08-15 09:34 -------
Here they are.  Note that I had changed some style stuff to make the code more readable for me when I was trying to understand it, so a little of that may show up in the diffs.  Also, I hadn't bothered to include the switch statement to get store mode to work because I didn't need it, but this should be trivial to put back.


--