Thread overview
std.zlib bug
Oct 03, 2005
Artem Rebrov
Oct 03, 2005
Derek Parnell
Oct 04, 2005
Artem Rebrov
Oct 04, 2005
Artem Rebrov
Oct 04, 2005
Sean Kelly
October 03, 2005
I think Compress and UnCompress classes should be used as in following example
---------------------------------
import std.file;
import std.zlib;

void main()
{
	Compress c = new Compress;
	UnCompress u = new UnCompress;
	void[] q,w,e,wa,et; int i;

	q = read("in.dat");

	for(i=0;i+512<q.length;i+=512)
		{
		w=c.compress(q[i..i+512]);
		wa~=w; et=""; et~=w;

		w=c.flush(Z_SYNC_FLUSH);
		wa~=w; et~=w;

		e~=u.uncompress(et);
		}
	
	w=c.compress(q[i..q.length]);
	wa~=w; et=""; et~=w;
    	
	w=c.flush();
	wa~=w; et~=w;
	e~=u.uncompress(et);
	e~=u.flush();

	write("out_c.dat",wa);		
	write("out_u.dat",e);
	e = uncompress(wa);
	write("out_ux.dat",e);	

}
---------------------------------
But it does not work. Look at the .diff. With these changes it works properly. (dmd v0.133 win32)
---------------------------------
--- zlib.d	Sat Sep 24 17:25:30 2005
+++ zlib_new.d	Tue Oct  4 01:28:34 2005
@@ -287,8 +287,9 @@
 	{   delete destbuf;
 	    error(err);
 	}
-	destbuf.length = zs.total_out;
-	return destbuf;
+	//destbuf.length = zs.total_out;
+	destbuf.length = destbuf.length - zs.avail_out;
+    return destbuf;
     }

     void[] flush()
@@ -303,16 +304,22 @@
     }
     body
     {
-	void[] destbuf;
+	void[] destbuf, tmpbuf;
 	int err;

 	if (!inited)
 	    return null;

+	/*  //old code
 	destbuf = new void[zs.avail_in];
 	zs.next_out = cast(ubyte*) destbuf;
 	zs.avail_out = destbuf.length;
+	*/
+    tmpbuf = new void[zs.avail_out]; // may be  zs.avail_out+<some constnt>
+	zs.next_out = cast(ubyte*) tmpbuf;
+	zs.avail_out = tmpbuf.length;

+	/* //old code
 	err = deflate(&zs, mode);
 	if (err != Z_STREAM_END)
 	{
@@ -322,6 +329,27 @@
 	    error(err);
 	}
 	destbuf = cast(void[])((cast(ubyte *)destbuf)[0 .. zs.next_out - cast(ubyte*)destbuf]);
+	*/
+	while( (err = deflate(&zs, mode)) != Z_STREAM_END)
+		{
+		if(err == Z_OK)
+			{
+			if(zs.avail_out != 0 && mode != Z_FINISH)
+				break;
+			else if(zs.avail_out == 0)
+				{
+				destbuf ~= tmpbuf;
+				zs.next_out = cast(ubyte*) tmpbuf;
+				zs.avail_out = tmpbuf.length;
+				continue;
+				}
+			err = Z_BUF_ERROR;
+			}
+		delete destbuf;
+		error(err);
+		}
+	destbuf ~= tmpbuf[0..(tmpbuf.length-zs.avail_out)];
+	
 	if (mode == Z_FINISH)
 	{
 	    err = deflateEnd(&zs);
@@ -410,7 +438,8 @@
 	{   delete destbuf;
 	    error(err);
 	}
-	destbuf.length = zs.total_out;
+	//destbuf.length = zs.total_out;
+	destbuf.length = destbuf.length - zs.avail_out;
 	return destbuf;
     }


-- 
Using Opera's revolutionary e-mail client: http://www.opera.com/mail/
October 03, 2005
On Tue, 04 Oct 2005 02:34:51 +0400, Artem Rebrov wrote:

> I think Compress and UnCompress classes should be used ...

Why are they even classes? Are not class meant to be representation of things rather than activities? What is a Compress? What is an UnCompress? These are processes, actions, work, and not objects. The closest thing would be that 'Compress' is a Storage mechanism for compressed data and that it can do (at least) two activities: compress data into itself and decompress data that it already has.

-- 
Derek
(skype: derek.j.parnell)
Melbourne, Australia
4/10/2005 9:10:14 AM
October 04, 2005
On Tue, 04 Oct 2005 03:14:46 +0400, Derek Parnell <derek@psych.ward> wrote:

> On Tue, 04 Oct 2005 02:34:51 +0400, Artem Rebrov wrote:
>
>> I think Compress and UnCompress classes should be used ...
>
> Why are they even classes? Are not class meant to be representation of
> things rather than activities? What is a Compress? What is an UnCompress?
> These are processes, actions, work, and not objects. The closest thing
> would be that 'Compress' is a Storage mechanism for compressed data and
> that it can do (at least) two activities: compress data into itself and
> decompress data that it already has.
>

Just look at dmd\src\phobos\std\zlib.d in the DMD distribution


-- 
Using Opera's revolutionary e-mail client: http://www.opera.com/mail/
October 04, 2005
On Tue, 04 Oct 2005 22:01:36 +0400, Artem Rebrov <ar_other@mail.ru> wrote:

> On Tue, 04 Oct 2005 03:14:46 +0400, Derek Parnell <derek@psych.ward> wrote:
>
>> On Tue, 04 Oct 2005 02:34:51 +0400, Artem Rebrov wrote:
>>
>>> I think Compress and UnCompress classes should be used ...
>>
>> Why are they even classes? Are not class meant to be representation of
>> things rather than activities? What is a Compress? What is an UnCompress?
>> These are processes, actions, work, and not objects. The closest thing
>> would be that 'Compress' is a Storage mechanism for compressed data and
>> that it can do (at least) two activities: compress data into itself and
>> decompress data that it already has.
>>
>
> Just look at dmd\src\phobos\std\zlib.d in the DMD distribution
>
>
(Un)Compress classes holds state of (un)compressor.
When I can't compress whole data I split it to chunks. I transmit these
chunks to other side of the connection (for example) and uncompress
them one by one. The goal is to be able to decompress the first chunk
before I can receive the last one.



-- 
Using Opera's revolutionary e-mail client: http://www.opera.com/mail/
October 04, 2005
In article <op.sx4xxhtpncj208@comp>, Artem Rebrov says...
>
>(Un)Compress classes holds state of (un)compressor.
>When I can't compress whole data I split it to chunks. I transmit these
>chunks to other side of the connection (for example) and uncompress
>them one by one. The goal is to be able to decompress the first chunk
>before I can receive the last one.

For what it's worth, I typically implement this functionality as a stream filter in C++.  It would probably be worth doing something similar in D.


Sean