Hi all.  Just in case the subject didn't make it quite obvious, I have several questions :P

1) Documentation Style

I'm starting to write some code in D (see #2), and was wondering if there was an official/sanctioned documentation style to use ala javadoc/Python docstrings/etc.

2) SHA1

I noticed that the standard library has an implementation of MD5, but no SHA1.  Since MD5 was recently broken, and because I didn't know any better (:P), I decided to write an SHA1 implementation in D.  I was wondering if there was any interest in this for Phobos?

I've got a port of the RFC3174 reference code done, and I plan on refactoring it so that it's similar to the MD5 module.  I was also thinking about writing a Stream class that progressively computes the hash as you write to it.

Which was what gave me an idea for...

3) Stream filters

I don't know if there's any interest in something like this, but here goes.  We already have Streams in the standard library, so how about Stream Filters.  They would basically just be subclasses of the standard Stream class (so you can pass them to existing routines).  The difference is that they would perform some operation on data that gets read/written through them.

I had a quick Google around, and it looks like Java and PHP already have something like this, so it could be useful.  The first application that came to mind would be hashing the contents of a file as you write it out, instead of having to either use two streams, or an SHA1 context and a stream.

4) Malloc and the GC
(Sorry if this appears in another post... I doesn't seem to be posting my messages properly...)

Last question (for now), so I'll make it quick: can I allocate memory with malloc, add a root to the GC, and trust that the GC will be able to clean it up, or is mixing memory allocation systems a no-no?

Anyway, thanks for your time.  Let's hope it posts this time :P

	-- Daniel

<snip>
> 2) SHA1
>
> I noticed that the standard library has an implementation of MD5, but no SHA1.  Since MD5 was recently broken, and because I didn't know any better (:P), I decided to write an SHA1 implementation in D.  I was wondering if there was any interest in this for Phobos?
>
> I've got a port of the RFC3174 reference code done, and I plan on refactoring it so that it's similar to the MD5 module.  I was also thinking about writing a Stream class that progressively computes the hash as you write to it.
>
> Which was what gave me an idea for...
<snip>

i think there should be a part of phobos that would be like a built in
crypto module.
one way hashes SHA1, MD5, RipeMD160, Tiger, etc..
symmetric ciphers like AES and NESSIE candidates (and chosen)
and asymmetric ciphers like RSA, Rabin, DH, etc..

- Asaf.

"Daniel Keep" <daniel.keep@dummy.gmail.com> wrote in message news:cn2e73$be2$1@digitaldaemon.com...
> Hi all.  Just in case the subject didn't make it quite obvious, I have several questions :P
>
> 1) Documentation Style
>
> I'm starting to write some code in D (see #2), and was wondering if there was an official/sanctioned documentation style to use ala javadoc/Python docstrings/etc.

It ranges from javadoc to doxygen to simple // comments

[snip]

> 3) Stream filters
>
> I don't know if there's any interest in something like this, but here goes.  We already have Streams in the standard library, so how about Stream Filters.  They would basically just be subclasses of the standard Stream class (so you can pass them to existing routines).  The difference is that they would perform some operation on data that gets read/written through them.
>
> I had a quick Google around, and it looks like Java and PHP already have something like this, so it could be useful.  The first application that came to mind would be hashing the contents of a file as you write it out, instead of having to either use two streams, or an SHA1 context and a stream.

See the std.stream.BufferedStream for an example of a D filter stream. There currently isn't a class for generic filter streams but such a class would probably just declare the source stream and maybe it would also includes some logic about opening and closing the source stream. I remember people tossing around ideas for std.stream and this was one of them - though I don't remember seeing any implementations.

> 4) Malloc and the GC
> (Sorry if this appears in another post... I doesn't seem to be posting
> my messages properly...)
>
> Last question (for now), so I'll make it quick: can I allocate memory with malloc, add a root to the GC, and trust that the GC will be able to clean it up, or is mixing memory allocation systems a no-no?

The GC can't free memory it didn't allocate. The "addRoot" function just takes a pointer to a GC resource and adds it to a list that is scanned by the GC. This is useful when a pointer is passed to an "external resource" that the GC doesn't scan and so without calling addRoot the memory would be collected. The addRange function adds an entire chunk of memory to the scan list.

> Anyway, thanks for your time.  Let's hope it posts this time :P
>
> -- Daniel

Ben Hinkle wrote:
[snip] :P
> 
> The GC can't free memory it didn't allocate. The "addRoot" function just
> takes a pointer to a GC resource and adds it to a list that is scanned by
> the GC. This is useful when a pointer is passed to an "external resource"
> that the GC doesn't scan and so without calling addRoot the memory would be
> collected. The addRange function adds an entire chunk of memory to the scan
> list.
> 

Oh... sh^H^Hbother.  I was afraid of that.  I have a really fast piece of code... that spends more time zeroing out an array then it does in actual processing -_-

I would just code it in straight assembler, and then link it in, but then I'd have the same problem.  I suppose that barring some magical way to get the GC to give me uninitialized memory, I'm stuffed, eh?

It's a pity I agree with Walter's reasons for automatically initializing memory... because otherwise I could have a go at him for it :P

In article <cn2rui$v8j$1@digitaldaemon.com>, Daniel Keep says...
>
>I would just code it in straight assembler, and then link it in, but then I'd have the same problem.  I suppose that barring some magical way to get the GC to give me uninitialized memory, I'm stuffed, eh?
>
>It's a pity I agree with Walter's reasons for automatically initializing memory... because otherwise I could have a go at him for it :P

You could always create some kind of allocator class.  Use alloca, malloc, or just hold on to GC-allocated memory for reuse.


Sean

"Daniel Keep" <daniel.keep@dummy.gmail.com> wrote in message news:cn2e73$be2$1@digitaldaemon.com...
> 2) SHA1
>
> I noticed that the standard library has an implementation of MD5, but no SHA1.  Since MD5 was recently broken, and because I didn't know any better (:P), I decided to write an SHA1 implementation in D.  I was wondering if there was any interest in this for Phobos?

The reason SHA1 was not added to Phobos is the government spec for it says it would need an export license to ship it. I've been informed that this no longer applies, but if the person that informed me was mistaken, it's my head on the chopping block :-(

Export license?  Hmm...  Then I wonder how everyone else gets away with it :P

None the less, I'd love to see SHA-1, and all the other crypto stuff in Phobos, so I'll try and do some digging... see if I can't find anything.

Walter wrote:
> "Daniel Keep" <daniel.keep@dummy.gmail.com> wrote in message
> news:cn2e73$be2$1@digitaldaemon.com...
> 
>>2) SHA1
>>
>>I noticed that the standard library has an implementation of MD5, but no
>>SHA1.  Since MD5 was recently broken, and because I didn't know any
>>better (:P), I decided to write an SHA1 implementation in D.  I was
>>wondering if there was any interest in this for Phobos?
> 
> 
> The reason SHA1 was not added to Phobos is the government spec for it says
> it would need an export license to ship it. I've been informed that this no
> longer applies, but if the person that informed me was mistaken, it's my
> head on the chopping block :-(
> 
> 

Hmm... hadn't thought of that.  I suppose I could write a class that mimics regular arrays, but it seems like it wouldn't be as useful as a normal, run of the mill array (and not extendable, if I understand things correctly).

What I may end up doing is making the function an in-place operation (so the programmer has to manually duplicate arrays if need be), and make a version available for normal arrays, pointers to data, and for funky "kinda array" objects :)

Anyway, thanks for the tip.

Sean Kelly wrote:
> In article <cn2rui$v8j$1@digitaldaemon.com>, Daniel Keep says...
> 
>>I would just code it in straight assembler, and then link it in, but then I'd have the same problem.  I suppose that barring some magical way to get the GC to give me uninitialized memory, I'm stuffed, eh?
>>
>>It's a pity I agree with Walter's reasons for automatically initializing memory... because otherwise I could have a go at him for it :P
> 
> 
> You could always create some kind of allocator class.  Use alloca, malloc, or
> just hold on to GC-allocated memory for reuse.
> 
> 
> Sean
> 
> 

Daniel Keep schrieb:
> Export license?  Hmm...  Then I wonder how everyone else gets away with it :P
>
> None the less, I'd love to see SHA-1, and all the other crypto stuff in Phobos, so I'll try and do some digging... see if I can't find anything.

We could use the Debain way. NON-US developer(s) implement SHA-1 (and all the other crypto
stuff) and it's hosted on s NON-US server.

If a server is needed, I'd be willing to host it on a German server.

Thomas

I don't think that will be neccecary.  I've been digging all day, and my eyes hurt from all the monospaced documents I've been reading, but I think I've nutted out the core of the matter.

Basically, you *can* freely export cryptography software from the US, provided:

1) It's source is publicly available
2) You notify the US government that you are exporting crypto. software (I'm still not 100% sure on the specifics, but it looks as though you just have to email them w/ your URL)
3) You don't *consciously* export to any blacklisted countries (North Korea, Iran, etc.)

The Debian mailing list has been very helpful in this little quest :P

Also, I checked my Debian install, and sha1sum is included in the main package--ie: it's being exported from the US.  So we shouldn't have any problems with SHA-1 (specifically, anyway).

The following is quoted from the revised EAR section 740.13(e)(5), taken from http://lists.debian.org/debian-legal/2002/06/msg00247.html:

    (5) Notification requirement. You must provide BIS written
    notification of the Internet location (e.g., URL or Internet
    address) of the source code or a copy of the source code by the time
    of export.
    Submit the notification by email to BIS at crypt@bis.doc.gov, and
    provide a copy of the notification to the ENC Encryption Request
    Coordinator at enc@ncsc.mil.

There is also an excellent discussion on the Debian mailing list about the inclusion of crypto code into the "main" distribution (which gets hosted on US sites) here: http://lists.debian.org/debian-policy/2001/01/msg00036.html

Information on US crypto. policies can be found here: http://www.bxa.doc.gov/Encryption/Default.htm

So, I think we just need to take a careful look at what information we need to email to the addresses mentioned above, and we should be golden.

I was thinking about possibly posting to the Debian mailing list and/or the Mono mailing list (another project with lots of open source crypto code) with regards to what they had to do.  But that might seem a little rude :P

Anyway, I'll keep looking into this.  Hope this is of help.

	-- Daniel

Thomas Kuehne wrote:
> Daniel Keep schrieb:
> 
>>Export license?  Hmm...  Then I wonder how everyone else gets away with
>>it :P
>>
>>None the less, I'd love to see SHA-1, and all the other crypto stuff in
>>Phobos, so I'll try and do some digging... see if I can't find anything.
> 
> 
> We could use the Debain way. NON-US developer(s) implement SHA-1 (and all the other crypto
> stuff) and it's hosted on s NON-US server.
> 
> If a server is needed, I'd be willing to host it on a German server.
> 
> Thomas
> 
>