I have finally finished the Tiger hashing algorithm.

I have followed the format of std.md5 for the most part. The only difference being that I don't have a printDigest function (this prints the digest using printf), instead I have a digestString function which returns the digest in string form. I could add printDigest I just don't see the point.

So.. before I release it into the wild for all to see..

- Where do I put it? dsource.org? in deimos?

- What kind of license etc do I need on it? I assume I need to credit the authors of the algorithm, the reference implementation doesn't seem to have any license etc in it.

- What should the module be called, we have std.md5 so std.tiger seems right, but perhaps std.crypto.md5 and std.crypto.tiger would be better?


In writing this I have created some toHex() functions similar to the toString() ones in std.string, perhaps they can be considered for inclusion in std.string?

One thought however.. often you want to pad the result with zero's like you can with printf i.e. %08x should toHex and toString can be modified to do this? (with a default parameter) or should people have to go:

char p[] = toHex(128ul);
char r[8];

r[] = '0';
r[r.length-p.length..r.length] = p;

My digestString function does something similar, if toHex were changed to allow this then I could simply call toHex from digestString instead of having to do it myself.


One final thing, when I compile tiger with a main fn that simply prints DONE and exits, then run it, it runs, uses 1MB memory, prints DONE, then keeps running for another 90 seconds, uses another 14MB of memory, before returning to the command prompt?

What is it doing?

This only happens in a debug build.

My compile/link options are:
  d:\D\dmd\bin\dmd.exe -c "$(InputPath)" -g -gt -debug -unittest -od"$(IntDir)"
  d:\D\dmd\bin\dmd.exe -g "$(IntDir)\*.obj" -of"$(OutDir)\$(WkspName).exe"

Im using MSDEV the $() above are replaced by the file names involved.

Regan.

-- 
Using M2, Opera's revolutionary e-mail client: http://www.opera.com/m2/

In article <opr9nob2gu5a2sq9@digitalmars.com>, Regan Heath says...
>
>I have finally finished the Tiger hashing algorithm.

Excellent!


>I have followed the format of std.md5 for the most part. The only difference being that I don't have a printDigest function (this prints the digest using printf), instead I have a digestString function which returns the digest in string form. I could add printDigest I just don't see the point.

I'd suggest renaming digestString() as toString() in line with everything else,
and leave it at that. If people want to print it they can do so by hand.




>So.. before I release it into the wild for all to see..
>
>- Where do I put it? dsource.org? in deimos?

Yay! Certainly. I think you have to talk to Brad to get write access (as I don't know how to grant it), but it would definitely be welcome there. Put it like this, the future etc.crypto module will definitely incorporate Tiger (and MD5). Whether it does so by importing it from somewhere else or wrapping it (as would be the case with MD5), or by being part of Deimos, is immaterial, in a way, but from my point of view it would definitely be nicer to have everything all in the same place.

In the next few weeks (sorry, should have done it by now but I got sidetracked by Unicode) I'm going to re-document my existing stuff in Deimos using doxygen, and put together a build system, so that people can just download something that works without too much effort. (Though I confess, I haven't doxygenated it YET). It would certainly be nice, though, if everything in Deimos used doxygen-generated documentation. Of course, that is NOT a requirement for entry. You can always add that later.



>- What kind of license etc do I need on it? I assume I need to credit the authors of the algorithm, the reference implementation doesn't seem to have any license etc in it.

If you wrote it, you can put any kind of license you want on it. The only restriction as far as Deimos is concerned is that everything in Deimos must be open source, but the license, I leave up to you. BSD-style is always a good fave, but it's your choice. (Use my IPMA if you like - may only be distributed by people with as sense of humor!)

It is always courteous to credit the authors of the algorithm. In most jurisdictions in the world, this is not necessary (though I wouldn't be surprised if America were an exception), since, in most parts of the world, an ALGORITHM (as opposed to an implementation) is not something anyone can "own". However, as I say, it is polite everywhere, and politeness is always nice.




>- What should the module be called, we have std.md5 so std.tiger seems right, but perhaps std.crypto.md5 and std.crypto.tiger would be better?

Except that, in many ways, std.md5 is going to end up being the exception. That's kind of OK though, since MD5 is going to be end up as one of the weaker hashing algorithms. There's also a std.rand, but that doesn't mean that all cryptographically secure random number functions should end up in the root namespace. It's maybe good that std has ONE hashing algorithm, but it will get awfully crowded in there if they ALL get thrown in there.

Eventually, there will be a comprehensive crypto package called etc.crypto. Walter says that this might one day become std.crypto. I would be hesitant to name anything "std." directly, unless it's actually in Phobos - I think that would cause confusion. If it were to go in Deimos, it would ideally be called etc.crypto.hash.tiger (all hashing algorithms will go in etc.crypto.hash; all key generation functions in etc.crypto.kgf; all symmetric cyphers in ... - well, you get the idea).

Eventually, there will be an architecture which allows a user to mix and match their own suite of crypto functionality. That's why it's important that things like that be properly organized. Now, I realize that that architecture isn't there yet, and so asking you to put in an extra directory level may sound a bit dumb, but there is logic behind that request.

And within that architecture, there will be etc.crypto.hash.md5 - even if it's just a wrapper around std.md5 - because such consistency is part of what will make the architecture work.



>In writing this I have created some toHex() functions similar to the toString() ones in std.string, perhaps they can be considered for inclusion in std.string?

Only Walter can answer that one, but it seems to me that there is a problem with this, which everyone involved might like to seriously consider before going ahead with it. The problem is that there is a difference - an IMPORTANT difference - between an array of ubytes, and an array of chars. A hashing algorithm should always require an array of ubytes (or voids) - never chars. Converting a ubyte[] array to a hex string is easy, but converting D's notion of a string to a hex string is a much, much trickier prospect, because to convert from char[] to ubyte[], you first need to specify an encoding (UTF-8, UTF-16BE, UTF-16LE, etc...). Consequently, I think toHex() must be defined as one of:

char[] toHex(ubyte[]);
char[] toHex(void[]);

but definitely NOT:

char[] toHex(char[]); // not good

For this reason, std.string may not be the best place for it.



>One thought however.. often you want to pad the result with zero's like you can with printf i.e. %08x should toHex and toString can be modified to do this? (with a default parameter) or should people have to go:
>
>char p[] = toHex(128ul);
>char r[8];
>
>r[] = '0';
>r[r.length-p.length..r.length] = p;

For consistency with toString(), the default should be no padding. But obviously you can anything you like with non-default.



Arcane Jill

In article <caorf7$1t12$1@digitaldaemon.com>, Arcane Jill says...

I probably should have been clearer here, but this is very important, because you wouldn't BELIEVE the number of security holes that have cropped up around the world in various places because people haven't understood encodings properly.

D makes a guarantee that a char[] array WILL be in UTF-8 encoding. However, there is *NO* guarantee of any given encoding for a ubyte[] array. It MIGHT be in UTF-8, or it might be in ISO-8859-1, or some other such. Accordingly, there is *NO* way to convert from a ubyte[] to char[] without taking a grave risk.

D provides toUTF16(), toUTF32(), and so on, to convert between char[], wchar[]
and dchar[]. It can do this because it KNOWS the encoding at both ends. With a
ubyte[], you don't have that luxury.

The hash of a string will be different, depending on its encoding. If you have a ubyte[] array containing ISO-8859-1 text (say, a line read from a text file), its hash may be DIFFERENT from the hash of that same text re-encoded in UTF-8.

Accordingly, it makes absolutely no sense for there to exist a function which
can convert char[]s to hex strings. (It DOES of course make sense to convert
ubyte[]s to hex strings). At best, you could have a suite of function
toHexUTF8(), toHexISO_8859_1(), etc., but even that's ugly.

Arcane Jill

> I think you have to talk to Brad to get write access (as I don't
> know how to grant it)

Should we open the Deimos repository up for all to commit changes?  Or should we go developer by developer for a while?

BA

P.S.  Maybe this should be moved to the Deimos forum on dsource.org instead of bothering the NG here?

In article <caq39k$pd3$1@digitaldaemon.com>, Brad Anderson says...
>
> > I think you have to talk to Brad to get write access (as I don't
>> know how to grant it)
>
>Should we open the Deimos repository up for all to commit changes?  Or should we go developer by developer for a while?

Do you mean, like, so that people other than I can make changes to Int, for example? I'd rather they didn't, just yet. It's not "ready", in my mind, yet. But that will happen soon.

But I reckon that anyone who's got a package they want to put into Deimos should be able to do that. I don't have any problem with that. So I guess, yeah - people actively developing for now.


>P.S.  Maybe this should be moved to the Deimos forum on dsource.org instead of bothering the NG here?

Suits me either way. But there are definitely things that need to be discussed, and for which dsource is the best place - things like getting a build system together so we can do proper releases of Deimos, things like agreeing an overall architecture. That sort of discussion is probably better suited to there. On the other hand, talking /about/ Deimos seems fine, here.

Deimos is supposed to be a parallel to Phobos, a place for packages which the developers of those packages feel would be appropriate in Phobos (but we can't get them into Phobos because we're not Walter). Call it a testing ground for potential std-candidates. Once the things there reach a certain level of maturity, presumably Walter can make the decision to move them over (or not, his choice). And so, to that end, I don't see any reason why anyone with a candidate project shouldn't be allowed to put it there.

I guess what I'm trying to say is, getting your stuff into Phobos is hard, so getting your stuff into Deimos should be easy. If Deimos is to be a testing ground for Phobos-potentials, then it doesn't make sense to me that there should be any kind of bar to putting stuff in there. All that I would ask of developer is that they chat on the forum first, so that we can all be clear on things like the directory structure and so on, and to avoid duplication of effort.

The only other thing to say, really, is that I don't want it to be in competition with other worthwhile projects. For example, there would be no point in adding streams stuff to Deimos when Mango is already so good, and likely to become the defacto standard (which doesn't preclude Mango itself becoming part of Deimos, obviously).

That's pretty much it really. I don't want to be a gatekeeper, just a developer. And I think that the developers currently or potentially involved are sensible enough to keep hands off each other's projects voluntarily, so I guess the procedure would be:

(1) talk about your project for a bit on the Deimos forum to make sure you know where your project is going and that you're not duplicating effort;

(2) get write access (somehow - I don't know how that part works)

(3) upload your project, but voluntarily refrain from interfering with anyone
else's project until they're happy for that to happen (which must happen
eventually, of course, this being open source).

(4) get chatting on the Deimos forum about documentation/release issues/etc.

Is that okay? Does that make sense?

Arcane Jill

On Wed, 16 Jun 2004 07:05:11 +0000 (UTC), Arcane Jill <Arcane_member@pathlink.com> wrote:

>> I have followed the format of std.md5 for the most part. The only
>> difference being that I don't have a printDigest function (this prints the
>> digest using printf), instead I have a digestString function which returns
>> the digest in string form. I could add printDigest I just don't see the
>> point.
>
> I'd suggest renaming digestString() as toString() in line with everything else,
> and leave it at that. If people want to print it they can do so by hand.

I considered that. The digest is a ulong[3] so I'd be writing:

char[] toString(ulong[3] digest) {
}

which, to me, looks like a generic function for getting a string of an array of longs.. not a function for getting a string form of a digest.


I then considered using a typedef i.e.

typedef ulong[3] TigerDigest

so I could write

char[] toString(TigerDigest digest) {
}

is that a better option?


Basically I chose what I thought was the most obvious char[] digestString(ulong[3] digest).

>> So.. before I release it into the wild for all to see..
>>
>> - Where do I put it? dsource.org? in deimos?
>
> Yay! Certainly. I think you have to talk to Brad to get write access (as I don't
> know how to grant it), but it would definitely be welcome there. Put it like
> this, the future etc.crypto module will definitely incorporate Tiger (and MD5).
> Whether it does so by importing it from somewhere else or wrapping it (as would
> be the case with MD5), or by being part of Deimos, is immaterial, in a way, but
> from my point of view it would definitely be nicer to have everything all in the
> same place.

Agreed.

> In the next few weeks (sorry, should have done it by now but I got sidetracked
> by Unicode) I'm going to re-document my existing stuff in Deimos using doxygen,
> and put together a build system, so that people can just download something that
> works without too much effort. (Though I confess, I haven't doxygenated it YET).
> It would certainly be nice, though, if everything in Deimos used
> doxygen-generated documentation. Of course, that is NOT a requirement for entry.
> You can always add that later.

If everyone uses the same documentation system it will make life much easier in the future, I'm going to have to look into doxygen and add it to tiger too.

>> - What kind of license etc do I need on it? I assume I need to credit the
>> authors of the algorithm, the reference implementation doesn't seem to
>> have any license etc in it.
>
> If you wrote it, you can put any kind of license you want on it. The only
> restriction as far as Deimos is concerned is that everything in Deimos must be
> open source, but the license, I leave up to you. BSD-style is always a good
> fave, but it's your choice. (Use my IPMA if you like - may only be distributed
> by people with as sense of humor!)

I'll prob use BSD.

> It is always courteous to credit the authors of the algorithm. In most
> jurisdictions in the world, this is not necessary (though I wouldn't be
> surprised if America were an exception), since, in most parts of the world, an
> ALGORITHM (as opposed to an implementation) is not something anyone can "own".
> However, as I say, it is polite everywhere, and politeness is always nice.

That is what I figured.

>> - What should the module be called, we have std.md5 so std.tiger seems
>> right, but perhaps std.crypto.md5 and std.crypto.tiger would be better?
>
> Except that, in many ways, std.md5 is going to end up being the exception.
> That's kind of OK though, since MD5 is going to be end up as one of the weaker
> hashing algorithms. There's also a std.rand, but that doesn't mean that all
> cryptographically secure random number functions should end up in the root
> namespace. It's maybe good that std has ONE hashing algorithm, but it will get
> awfully crowded in there if they ALL get thrown in there.

I agree. Perhaps we should petition Walter to move them, now, as it's not really too late.
Or place a stub where they are and move them.
Or like you mention place a stub in the new place importing the originals.

Moving would be best (less possible confusion).

> Eventually, there will be a comprehensive crypto package called etc.crypto.
> Walter says that this might one day become std.crypto. I would be hesitant to
> name anything "std." directly, unless it's actually in Phobos - I think that
> would cause confusion. If it were to go in Deimos, it would ideally be called
> etc.crypto.hash.tiger (all hashing algorithms will go in etc.crypto.hash; all
> key generation functions in etc.crypto.kgf; all symmetric cyphers in ... - well,
> you get the idea).

Yep. Sounds good.

> Eventually, there will be an architecture which allows a user to mix and match
> their own suite of crypto functionality. That's why it's important that things
> like that be properly organized. Now, I realize that that architecture isn't
> there yet, and so asking you to put in an extra directory level may sound a bit
> dumb, but there is logic behind that request.

Nah. I could see the logic immediately.

> And within that architecture, there will be etc.crypto.hash.md5 - even if it's
> just a wrapper around std.md5 - because such consistency is part of what will
> make the architecture work.

Agreed.

>> In writing this I have created some toHex() functions similar to the
>> toString() ones in std.string, perhaps they can be considered for
>> inclusion in std.string?
>
> Only Walter can answer that one, but it seems to me that there is a problem with
> this, which everyone involved might like to seriously consider before going
> ahead with it. The problem is that there is a difference - an IMPORTANT
> difference - between an array of ubytes, and an array of chars. A hashing
> algorithm should always require an array of ubytes (or voids) - never chars.
> Converting a ubyte[] array to a hex string is easy, but converting D's notion of
> a string to a hex string is a much, much trickier prospect, because to convert
> from char[] to ubyte[], you first need to specify an encoding (UTF-8, UTF-16BE,
> UTF-16LE, etc...). Consequently, I think toHex() must be defined as one of:
>
> char[] toHex(ubyte[]);
> char[] toHex(void[]);
>
> but definitely NOT:
>
> char[] toHex(char[]); // not good
>
> For this reason, std.string may not be the best place for it.

Err.. actually my functions look like this.

char[] toHex(ubyte ub);
char[] toHex(ushort us);
char[] toHex(uint u);
char[] toHex(ulong u);

>> One thought however.. often you want to pad the result with zero's like
>> you can with printf i.e. %08x should toHex and toString can be modified to
>> do this? (with a default parameter) or should people have to go:
>>
>> char p[] = toHex(128ul);
>> char r[8];
>>
>> r[] = '0';
>> r[r.length-p.length..r.length] = p;
>
> For consistency with toString(), the default should be no padding. But obviously
> you can anything you like with non-default.

Agreed.

-- 
Using M2, Opera's revolutionary e-mail client: http://www.opera.com/m2/

On Wed, 16 Jun 2004 13:26:13 -0500, Brad Anderson <brad@dsource.dot.org> wrote:

>  > I think you have to talk to Brad to get write access (as I don't
>> know how to grant it)
>
> Should we open the Deimos repository up for all to commit changes?  Or should we go developer by developer for a while?
>
> BA

I think developer by developer is good. That way people will ask to be included, so we have some idea how many are contributing.

> P.S.  Maybe this should be moved to the Deimos forum on dsource.org instead of bothering the NG here?

If we're talking about implementation of code, here is best. If we're talking about management of deimos, there is best.

IMO.

Regan

-- 
Using M2, Opera's revolutionary e-mail client: http://www.opera.com/m2/

In article <opr9ph9y105a2sq9@digitalmars.com>, Regan Heath says...

>Err.. actually my functions look like this.
>
>char[] toHex(ubyte ub);
>char[] toHex(ushort us);
>char[] toHex(uint u);
>char[] toHex(ulong u);

Cool. Guess I misunderstood you. For some reason I thought were suggesting a signature of:

char[] toHex(char[] a)

which would be bad (although I would have no objection to anyone adding:

char[] toHex(void[] a)

). Anyway, now that I've been corrected, it seems to me that std.conv would be the right place for them - except that we can't put things into std., so I'd suggest etc.conv. (Normally, I'm reluctant to suggest putting things into the root of etc, but this seems sufficiently general to warrant an exception. You might have to allow that file to be writable by all though, in case other people come up with nice simple functions that also belong there.

Jill

hmm you don't like things in root?
I probably committed my Rational and Vec stuff in root...
I usually favor flatter hierarchies for whatever reason.
If you'd prefer something else perhaps you could do the appropriate svn moves or tell me what you'd prefer....
--Daniel
Arcane Jill wrote:
> In article <opr9ph9y105a2sq9@digitalmars.com>, Regan Heath says...
> 
> 
>>Err.. actually my functions look like this.
>>
>>char[] toHex(ubyte ub);
>>char[] toHex(ushort us);
>>char[] toHex(uint u);
>>char[] toHex(ulong u);
> 
> 
> Cool. Guess I misunderstood you. For some reason I thought were suggesting a
> signature of:
> 
> char[] toHex(char[] a)
> 
> which would be bad (although I would have no objection to anyone adding:
> 
> char[] toHex(void[] a)
> 
> ). Anyway, now that I've been corrected, it seems to me that std.conv would be
> the right place for them - except that we can't put things into std., so I'd
> suggest etc.conv. (Normally, I'm reluctant to suggest putting things into the
> root of etc, but this seems sufficiently general to warrant an exception. You
> might have to allow that file to be writable by all though, in case other people
> come up with nice simple functions that also belong there.
> 
> Jill
> 
> 

In article <cas93s$11sr$2@digitaldaemon.com>, Daniel Horn says...
>
>hmm you don't like things in root?

I'm open to persuasion, but let's move this discussion over to Deimos as it really doesn't belong here.

Jill