https://github.com/WalterBright/phobos/blob/std_compress/std/compress.d

I wrote this to add components to compress and expand ranges.

Highlights:

1. doesn't do any memory allocation
2. can handle arbitrarily large sets of data
3. it's lazy
4. takes an InputRange, and outputs an InputRange

Comments welcome.

On Tuesday, 4 June 2013 at 03:44:05 UTC, Walter Bright wrote:
> https://github.com/WalterBright/phobos/blob/std_compress/std/compress.d
>
> I wrote this to add components to compress and expand ranges.
>
> Highlights:
>
> 1. doesn't do any memory allocation
> 2. can handle arbitrarily large sets of data
> 3. it's lazy
> 4. takes an InputRange, and outputs an InputRange
>
> Comments welcome.

Nice! What happens if R is not a ubyte range?

On 6/3/2013 10:41 PM, Diggory wrote:
> Nice! What happens if R is not a ubyte range?

It'll work with char and ubyte, too. Anything else you'll need to cast or use an adapter.

A)
there already is std.zlib; why not have:
std.compress.zlib: public import std.zlib
std.compress.lzw: put this new module there instead of in std.compress
std.compress.image.png
std.compress.image.jpg

B)
rename:
std.compress.lzwCompress => std.compress.lzw.compress
std.compress. lzwExpand => std.compress.lzw.uncompress

which is more consistent with compress/uncompress from std.zlib

C)
maybe add a link to
https://en.wikipedia.org/wiki/Lempel%E2%80%93Ziv%E2%80%93Welch or other
source

D)
CircularBuffer belongs somewhere else; maybe std.range or std.container





On Mon, Jun 3, 2013 at 8:44 PM, Walter Bright <newshound2@digitalmars.com>wrote:

> https://github.com/**WalterBright/phobos/blob/std_** compress/std/compress.d<https://github.com/WalterBright/phobos/blob/std_compress/std/compress.d>
>
> I wrote this to add components to compress and expand ranges.
>
> Highlights:
>
> 1. doesn't do any memory allocation
> 2. can handle arbitrarily large sets of data
> 3. it's lazy
> 4. takes an InputRange, and outputs an InputRange
>
> Comments welcome.
>

On 6/3/2013 11:40 PM, Timothee Cour wrote:
> D)
> CircularBuffer belongs somewhere else; maybe std.range or std.container

I have mixed feelings about that. If you'll notice, std.compress doesn't have any imports! I wanted to make at least one module that doesn't pull in 100% of everything in Phobos (one of my pet peeves).

On Tuesday, 4 June 2013 at 08:00:03 UTC, Walter Bright wrote:
> On 6/3/2013 11:40 PM, Timothee Cour wrote:
>> D)
>> CircularBuffer belongs somewhere else; maybe std.range or std.container
>
> I have mixed feelings about that. If you'll notice, std.compress doesn't have any imports! I wanted to make at least one module that doesn't pull in 100% of everything in Phobos (one of my pet peeves).

If that is an issue, it is an issue in DMD, not in module. Modules are supposed to use each other extensively, that is the very reason to have them!

On 6/4/2013 1:00 AM, Walter Bright wrote:
> On 6/3/2013 11:40 PM, Timothee Cour wrote:
>> D)
>> CircularBuffer belongs somewhere else; maybe std.range or std.container
>
> I have mixed feelings about that. If you'll notice, std.compress doesn't have
> any imports! I wanted to make at least one module that doesn't pull in 100% of
> everything in Phobos (one of my pet peeves).

Note also I didn't document it, so it is private and can be moved.

On Tuesday, 4 June 2013 at 08:00:03 UTC, Walter Bright wrote:
> I have mixed feelings about that. If you'll notice, std.compress doesn't have any imports! I wanted to make at least one module that doesn't pull in 100% of everything in Phobos (one of my pet peeves).

I think this is a workaround, not a proper solution.

It probably means Phobos' granularity is horribly wrong.

On Tuesday, 4 June 2013 at 08:03:15 UTC, Walter Bright wrote:
> On 6/4/2013 1:00 AM, Walter Bright wrote:
>> On 6/3/2013 11:40 PM, Timothee Cour wrote:
>>> D)
>>> CircularBuffer belongs somewhere else; maybe std.range or std.container
>>
>> I have mixed feelings about that. If you'll notice, std.compress doesn't have
>> any imports! I wanted to make at least one module that doesn't pull in 100% of
>> everything in Phobos (one of my pet peeves).
>
> Note also I didn't document it, so it is private and can be moved.

Then it should be private. You should also mangle the name so that it doesn't pollute the unqualified symbol namespace (either that or fix visibility of private symbols).

On 6/4/2013 1:08 AM, Jakob Ovrum wrote:
> On Tuesday, 4 June 2013 at 08:00:03 UTC, Walter Bright wrote:
>> I have mixed feelings about that. If you'll notice, std.compress doesn't have
>> any imports! I wanted to make at least one module that doesn't pull in 100% of
>> everything in Phobos (one of my pet peeves).
>
> I think this is a workaround, not a proper solution.

Yes, it is.

> It probably means Phobos' granularity is horribly wrong.

Yup.

Phobos is hard to work on because of the complexity of everything importing and depending on everything else all in mutually referential cycles. I deliberately set out to create compress as a non-trivial module that did not do that.

I hope that splitting things up into packages will improve things.