Current std.base64 is not Boost License.

So, I wrote base64 module.

http://bitbucket.org/repeatedly/scrap/src/tip/base64.d

This module is based on RFC4648 and faster than std.base64.
In addition, User can encode / decode modified Base64 format using
Base64Impl.

What do you think?


Masahiro

On Sun, Oct 10, 2010 at 5:06 PM, Masahiro Nakagawa <repeatedly at gmail.com>wrote:

> What do you think?
>
> Masahiro
>

So the only reason to re-write it was to change the license?

Some suggestions:

(API)

1. Convert the whole thing to a range based design.
ubyte[] <-> dchar range

2. Accept ubyte[] or void[] as well as string

3. Add a constant for no padding
maybe
enum NoPadding = '\0';

I think
alias Base64!('!', '=', Base64.NoPadding);
is a lot clearer and easier to use.

4. Rename Base64Impl to Base64 and add default arguments for the standard character/padding settings.

(Style)

5. Reduce use of *ptr++ style code in favour of foreach/ranges etc.

6. Use assumeUnique instead of casting arrays to immutable.

I think the entire thing should be @safe unless REALLY needed for speed.

Hope I've been helpful.
Daniel.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.puremagic.com/pipermail/phobos/attachments/20101010/57b0a594/attachment.html>

Good work. I was discussing with Walter a while ago and we agreed that Base64 is the perfect candidate for a range interface, or even a streaming interface.

Could you adapt the interface such that Base64 accepts an input range and offers an input range? This is a sample of client code:

foreach (char[] s; base64encoder(stdin.byChunk(10000)) {
     ...
}

and

foreach (ubyte[] data; base64decoder(stdin.byLine()) {
     ...
}


Andrei

On 10/10/10 2:06 CDT, Masahiro Nakagawa wrote:
> Current std.base64 is not Boost License.
>
> So, I wrote base64 module.
>
> http://bitbucket.org/repeatedly/scrap/src/tip/base64.d
>
> This module is based on RFC4648 and faster than std.base64.
> In addition, User can encode / decode modified Base64 format using
> Base64Impl.
>
> What do you think?
>
>
> Masahiro
> _______________________________________________
> phobos mailing list
> phobos at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/phobos

On Sun, 10 Oct 2010 19:20:00 +0900, Daniel Murphy <yebblies at gmail.com> wrote:

> On Sun, Oct 10, 2010 at 5:06 PM, Masahiro Nakagawa <repeatedly at gmail.com>wrote:
>
>> What do you think?
>>
>> Masahiro
>>
>
> So the only reason to re-write it was to change the license?

License is the most important thing.

1. License issue
2. RFC version is 2045
3. Supporting format is MIME only

> Some suggestions:
>
> (API)
>
> 1. Convert the whole thing to a range based design.
> ubyte[] <-> dchar range

I read Andrei's reply, OK.

> 2. Accept ubyte[] or void[] as well as string

I will change encode and decode signatures.

     string encode(in string) -> string  encode(in void[])
     string decode(in string) -> ubyte[] decode(in string)

> 3. Add a constant for no padding
> maybe
> enum NoPadding = '\0';
>
> I think
> alias Base64!('!', '=', Base64.NoPadding);
> is a lot clearer and easier to use.

Good! I will add NoPadding.

> 4. Rename Base64Impl to Base64 and add default arguments for the standard character/padding settings.

     template Foo(char A1 = 'a')
     {
         void func() { writeln(A1); }
     }

     Foo.func()     // Compilation Error!
     Foo!().func()  // OK

Right?

> (Style)
>
> 5. Reduce use of *ptr++ style code in favour of foreach/ranges etc.
>
> 6. Use assumeUnique instead of casting arrays to immutable.
>
> I think the entire thing should be @safe unless REALLY needed for speed.

My first implementation doesn't use pointer, but such implementation is
slow :(
I think Base64 needs speed(encode and decode are often called in some
application).

assumeUnique isn't pure function...

> Hope I've been helpful.
> Daniel.

Thanks for the review :)


Masahiro

OK. I will implement the range interface. Please wait.


Masahiro

On Sun, 10 Oct 2010 22:29:49 +0900, Andrei Alexandrescu <andrei at erdani.com> wrote:

> Good work. I was discussing with Walter a while ago and we agreed that Base64 is the perfect candidate for a range interface, or even a streaming interface.
>
> Could you adapt the interface such that Base64 accepts an input range and offers an input range? This is a sample of client code:
>
> foreach (char[] s; base64encoder(stdin.byChunk(10000)) {
>      ...
> }
>
> and
>
> foreach (ubyte[] data; base64decoder(stdin.byLine()) {
>      ...
> }
>
>
> Andrei
>
> On 10/10/10 2:06 CDT, Masahiro Nakagawa wrote:
>> Current std.base64 is not Boost License.
>>
>> So, I wrote base64 module.
>>
>> http://bitbucket.org/repeatedly/scrap/src/tip/base64.d
>>
>> This module is based on RFC4648 and faster than std.base64.
>> In addition, User can encode / decode modified Base64 format using
>> Base64Impl.
>>
>> What do you think?
>>
>>
>> Masahiro
>> _______________________________________________
>> phobos mailing list
>> phobos at puremagic.com
>> http://lists.puremagic.com/mailman/listinfo/phobos
> _______________________________________________
> phobos mailing list
> phobos at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/phobos

On Sun, Oct 10, 2010 at 7:36 PM, Masahiro Nakagawa <repeatedly at gmail.com> wrote:
>
> I will change encode and decode signatures.
>
> ? ?string encode(in string) -> string ?encode(in void[])
> ? ?string decode(in string) -> ubyte[] decode(in string)
>

I think it should rather be type-consistent:

string encode(in string) -> string  encode(in ubyte[])
string decode(in string) -> ubyte[] decode(in string)

As others have said, I'd like this to use ranges instead if possible, and I'd like the option to supply a destination range as well.  The majority of work I do with this sort of thing encodes in-place into existing buffers.

On Oct 10, 2010, at 12:06 AM, Masahiro Nakagawa wrote:

> Current std.base64 is not Boost License.
> 
> So, I wrote base64 module.
> 
> http://bitbucket.org/repeatedly/scrap/src/tip/base64.d
> 
> This module is based on RFC4648 and faster than std.base64.
> In addition, User can encode / decode modified Base64 format using Base64Impl.
> 
> What do you think?
> 
> 
> Masahiro
> _______________________________________________
> phobos mailing list
> phobos at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/phobos

I agree. Last night, I thought about encode / decode with buffer. In range, each memory allocation on loop is bad...


Masahiro

On Mon, 11 Oct 2010 11:45:08 +0900, Sean Kelly <sean at invisibleduck.org> wrote:

> As others have said, I'd like this to use ranges instead if possible, and I'd like the option to supply a destination range as well.  The majority of work I do with this sort of thing encodes in-place into existing buffers.
>
> On Oct 10, 2010, at 12:06 AM, Masahiro Nakagawa wrote:
>
>> Current std.base64 is not Boost License.
>>
>> So, I wrote base64 module.
>>
>> http://bitbucket.org/repeatedly/scrap/src/tip/base64.d
>>
>> This module is based on RFC4648 and faster than std.base64.
>> In addition, User can encode / decode modified Base64 format using
>> Base64Impl.
>>
>> What do you think?
>>
>>
>> Masahiro
>> _______________________________________________
>> phobos mailing list
>> phobos at puremagic.com
>> http://lists.puremagic.com/mailman/listinfo/phobos
>
> _______________________________________________
> phobos mailing list
> phobos at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/phobos

If you define encoding and decoding on a range, there's no need to allocate for every pass through the loop. You reuse the buffer.

@Sean: I doubt you'll see any performance improvement if you encode in-place vs. in a separate buffer.


Andrei

On 10/10/10 22:25 CDT, Masahiro Nakagawa wrote:
> I agree. Last night, I thought about encode / decode with buffer. In range, each memory allocation on loop is bad...
>
>
> Masahiro
>
> On Mon, 11 Oct 2010 11:45:08 +0900, Sean Kelly <sean at invisibleduck.org> wrote:
>
>> As others have said, I'd like this to use ranges instead if possible, and I'd like the option to supply a destination range as well. The majority of work I do with this sort of thing encodes in-place into existing buffers.
>>
>> On Oct 10, 2010, at 12:06 AM, Masahiro Nakagawa wrote:
>>
>>> Current std.base64 is not Boost License.
>>>
>>> So, I wrote base64 module.
>>>
>>> http://bitbucket.org/repeatedly/scrap/src/tip/base64.d
>>>
>>> This module is based on RFC4648 and faster than std.base64.
>>> In addition, User can encode / decode modified Base64 format using
>>> Base64Impl.
>>>
>>> What do you think?
>>>
>>>
>>> Masahiro
>>> _______________________________________________
>>> phobos mailing list
>>> phobos at puremagic.com
>>> http://lists.puremagic.com/mailman/listinfo/phobos
>>
>> _______________________________________________
>> phobos mailing list
>> phobos at puremagic.com
>> http://lists.puremagic.com/mailman/listinfo/phobos
> _______________________________________________
> phobos mailing list
> phobos at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/phobos

It's more an effort to avoid any allocations at all, really, though if the algorithm preallocates a sufficiently sized buffer then that would be acceptable too.  For the actual code I write, I actually decode base64 in-place into the buffer being decoded, which would be a bit weird with ranges anyway.

On Oct 10, 2010, at 9:02 PM, Andrei Alexandrescu wrote:

> If you define encoding and decoding on a range, there's no need to allocate for every pass through the loop. You reuse the buffer.
> 
> @Sean: I doubt you'll see any performance improvement if you encode in-place vs. in a separate buffer.
> 
> 
> Andrei
> 
> On 10/10/10 22:25 CDT, Masahiro Nakagawa wrote:
>> I agree. Last night, I thought about encode / decode with buffer. In range, each memory allocation on loop is bad...
>> 
>> 
>> Masahiro
>> 
>> On Mon, 11 Oct 2010 11:45:08 +0900, Sean Kelly <sean at invisibleduck.org> wrote:
>> 
>>> As others have said, I'd like this to use ranges instead if possible, and I'd like the option to supply a destination range as well. The majority of work I do with this sort of thing encodes in-place into existing buffers.
>>> 
>>> On Oct 10, 2010, at 12:06 AM, Masahiro Nakagawa wrote:
>>> 
>>>> Current std.base64 is not Boost License.
>>>> 
>>>> So, I wrote base64 module.
>>>> 
>>>> http://bitbucket.org/repeatedly/scrap/src/tip/base64.d
>>>> 
>>>> This module is based on RFC4648 and faster than std.base64.
>>>> In addition, User can encode / decode modified Base64 format using
>>>> Base64Impl.
>>>> 
>>>> What do you think?
>>>> 
>>>> 
>>>> Masahiro
>>>> _______________________________________________
>>>> phobos mailing list
>>>> phobos at puremagic.com
>>>> http://lists.puremagic.com/mailman/listinfo/phobos
>>> 
>>> _______________________________________________
>>> phobos mailing list
>>> phobos at puremagic.com
>>> http://lists.puremagic.com/mailman/listinfo/phobos
>> _______________________________________________
>> phobos mailing list
>> phobos at puremagic.com
>> http://lists.puremagic.com/mailman/listinfo/phobos
> _______________________________________________
> phobos mailing list
> phobos at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/phobos