bcs wrote:
> On 11/20/2011 08:10 AM, Piotr Szturmaj wrote:
>> bcs wrote:
>>> On 11/04/2011 04:27 AM, Piotr Szturmaj wrote:
>>>> bcs wrote:
>>>>> Are you re-implementing the function kernels your self or are you
>>>>> using
>>>>> an existing implementation? Given what I've heard about things like
>>>>> side-channel attacks using processing times to recover keys, I'd
>>>>> rather
>>>>> not see Phobos use anything written by less than the best expert
>>>>> available.
>>>>
>>>> Until now, I implemented some hash functions. There are no branching
>>>> instructions in their transform() routines, so theoretically processing
>>>> time is independent of the function input.
>>>
>>> From my very incomplete memory I found the source I was looking for (I
>>> googled for "aes interperative dance") here
>>> http://www.moserware.com/2009/09/stick-figure-guide-to-advanced.html
>>> Look for "Foot-Shooting Prevention Agreement" in one of the images
>>> ~20-25% of the way down.
>>>
>>> tl;dr; It mentions "cache-based, timing, and other side channel
>>> attacks". Unless you can explain to me what those are, in painful
>>> detail, I don't think we should trust you to avoid them. Get a good
>>> vetted C implementation and wrap it with a nice D API and call it a day.
>  >
>> Sorry for late answer.
>
> Np, but I still have a number of concerns:
>
> What is the advantage to implementing the kernels of any of these
> functions in D? Will the code be faster? Smaller? More secure? More
> maintainable? (On the other hand, the value of doing the API code in D
> goes with no debate.)

The first advantage is that Phobos will be independent of any crypto libraries. The second one is that there will be no licensing issues. All crypto code will be under Boost license like the rest of Phobos.

> How many people in the D community have the experience and know-how to
> review the security of an implementation? If there are less than 2 or 3
> people who can do that, can we afford to include native kernels? We
> can't have just one and if we have only two and one leaves for some
> reason the code becomes un-maintainable for lack of a reviewer. *I*
> wouldn't be comfortable with less than about 4-5.

I know Regan Heath who wrote some crypto code for Tango. Also, I suspect that D _will_ gain more (crypto) contributors, especially after joining GCC.

Minimum number of contributors/reviewers requirement in open-source project is at least unfortunate in my opinion. Nevertheless, I respect your thoughts. But imagine what could happen if Walter waited for contributors instead of starting D project on his own?

Please realize that we do not implement every possible crypto algorithm at once. We need to start with something like hashing, then add encryption and other cryptographic primitives.

On Tue, 22 Nov 2011 16:16:21 -0000, Piotr Szturmaj <bncrbme@jadamspam.pl>
wrote:
> bcs wrote:
>> On 11/20/2011 08:10 AM, Piotr Szturmaj wrote:
>>> bcs wrote:
>>>> On 11/04/2011 04:27 AM, Piotr Szturmaj wrote:
>>>>> bcs wrote:
>>>>>> Are you re-implementing the function kernels your self or are you
>>>>>> using
>>>>>> an existing implementation? Given what I've heard about things like
>>>>>> side-channel attacks using processing times to recover keys, I'd
>>>>>> rather
>>>>>> not see Phobos use anything written by less than the best expert
>>>>>> available.
>>>>>
>>>>> Until now, I implemented some hash functions. There are no branching
>>>>> instructions in their transform() routines, so theoretically processing
>>>>> time is independent of the function input.
>>>>
>>>> From my very incomplete memory I found the source I was looking for (I
>>>> googled for "aes interperative dance") here
>>>> http://www.moserware.com/2009/09/stick-figure-guide-to-advanced.html
>>>> Look for "Foot-Shooting Prevention Agreement" in one of the images
>>>> ~20-25% of the way down.
>>>>
>>>> tl;dr; It mentions "cache-based, timing, and other side channel
>>>> attacks". Unless you can explain to me what those are, in painful
>>>> detail, I don't think we should trust you to avoid them. Get a good
>>>> vetted C implementation and wrap it with a nice D API and call it a day.
>>  >
>>> Sorry for late answer.
>>
>> Np, but I still have a number of concerns:
>>
>> What is the advantage to implementing the kernels of any of these
>> functions in D? Will the code be faster? Smaller? More secure? More
>> maintainable? (On the other hand, the value of doing the API code in D
>> goes with no debate.)
>
> The first advantage is that Phobos will be independent of any crypto libraries. The second one is that there will be no licensing issues. All crypto code will be under Boost license like the rest of Phobos.

Ultimately I think it comes down to the question of whether we want/expect to have native D implementations of things, or whether certain projects/libraries will always be too large to maintain in D.  The answer to that, comes down to whether we think D will gain a sufficient user base to include enough people able to produce and maintain those libraries, or .. whether D will gain sufficient importance that existing developers in those libraries produce D bindings themselves.

Under the assumption that D will gain sufficient traction it makes sense to start implementing what we can in D, now.  At the same time, in order to gain that traction D needs bindings to existing libraries.  The latter is probably a slightly higher priority at present, but it's not a reason not to develop native D implementations at the same time.

If we're lucky having initially incomplete native D implementations will actually encourage people with those skills to contribute to D, as there is a certain sort of satisfaction in doing so.

>> How many people in the D community have the experience and know-how to
>> review the security of an implementation? If there are less than 2 or 3
>> people who can do that, can we afford to include native kernels? We
>> can't have just one and if we have only two and one leaves for some
>> reason the code becomes un-maintainable for lack of a reviewer. *I*
>> wouldn't be comfortable with less than about 4-5.
>
> I know Regan Heath who wrote some crypto code for Tango. Also, I suspect that D _will_ gain more (crypto) contributors, especially after joining GCC.

I wrote those as a crypto novice and haven't had any cause to advance
since then.  There are some obvious things to watch out for, for example
the hashing routines should not make copies of the input data, or if they
do they should 'scrub' that memory clean afterwards.  But, that's the
limit of my knowledge, there are bound to be more advanced problems and
solutions that I'm simply not aware of.

Regan

-- 
Using Opera's revolutionary email client: http://www.opera.com/mail/

On 11/22/2011 08:16 AM, Piotr Szturmaj wrote:
> bcs wrote:

>> How many people in the D community have the experience and know-how to
>> review the security of an implementation? If there are less than 2 or 3
>> people who can do that, can we afford to include native kernels? We
>> can't have just one and if we have only two and one leaves for some
>> reason the code becomes un-maintainable for lack of a reviewer. *I*
>> wouldn't be comfortable with less than about 4-5.
>
> I know Regan Heath who wrote some crypto code for Tango. Also, I suspect
> that D _will_ gain more (crypto) contributors, especially after joining
> GCC.

"Wrote some crypto code" is a rather weak recommendation. Depending on how you interpret it, that would recommend *me*. A better recommendation would be "Mr Y gets paid by security company X to do crypto analysis" or "Mrs Z has published several well review papers on vulnerabilities in this kind of code".

> Minimum number of contributors/reviewers requirement in open-source
> project is at least unfortunate in my opinion. Nevertheless, I respect
> your thoughts. But imagine what could happen if Walter waited for
> contributors instead of starting D project on his own?
>
> Please realize that we do not implement every possible crypto algorithm
> at once. We need to start with something like hashing, then add
> encryption and other cryptographic primitives.

I have no problem with that comment. My concern revolves around the point that the implementation of cryptographic primitives has security implications. I'm worried that we don't have the resources to demonstrate that our implementation is at least as good as the currently available implementation. I'd rather Phobos not include a given primitive than contain one of unknown quality. What I'd like to see is that the crypto package quickly contain interfaces for all the primitives we can find pre-vetted Boost licensed implementations for. At that point I would have no issue with as methodical and meticulous effort to divest ourselves of external dependencies as we can get access to the expertises needed to vet our own implementations (to the same level as the code they are replacing).

Yes, I'm intentionally being paranoid here but this is security and paranoia is part of the job description darn-it.

On Fri, Nov 25, 2011 at 10:31 PM, bcs <bcs@example.com> wrote:

> On 11/22/2011 08:16 AM, Piotr Szturmaj wrote:
>
>> bcs wrote:
>>
>
>  How many people in the D community have the experience and know-how to
>>> review the security of an implementation? If there are less than 2 or 3 people who can do that, can we afford to include native kernels? We can't have just one and if we have only two and one leaves for some reason the code becomes un-maintainable for lack of a reviewer. *I* wouldn't be comfortable with less than about 4-5.
>>>
>>
>> I know Regan Heath who wrote some crypto code for Tango. Also, I suspect that D _will_ gain more (crypto) contributors, especially after joining GCC.
>>
>
> "Wrote some crypto code" is a rather weak recommendation. Depending on how you interpret it, that would recommend *me*. A better recommendation would be "Mr Y gets paid by security company X to do crypto analysis" or "Mrs Z has published several well review papers on vulnerabilities in this kind of code".
>
>
>  Minimum number of contributors/reviewers requirement in open-source
>> project is at least unfortunate in my opinion. Nevertheless, I respect your thoughts. But imagine what could happen if Walter waited for contributors instead of starting D project on his own?
>>
>> Please realize that we do not implement every possible crypto algorithm at once. We need to start with something like hashing, then add encryption and other cryptographic primitives.
>>
>
> I have no problem with that comment. My concern revolves around the point that the implementation of cryptographic primitives has security implications. I'm worried that we don't have the resources to demonstrate that our implementation is at least as good as the currently available implementation. I'd rather Phobos not include a given primitive than contain one of unknown quality. What I'd like to see is that the crypto package quickly contain interfaces for all the primitives we can find pre-vetted Boost licensed implementations for. At that point I would have no issue with as methodical and meticulous effort to divest ourselves of external dependencies as we can get access to the expertises needed to vet our own implementations (to the same level as the code they are replacing).
>
> Yes, I'm intentionally being paranoid here but this is security and paranoia is part of the job description darn-it.
>

How about putting a disclaimer on the module warning the code hasn't been through a rigorous security audit and point them at well established C libraries if they need that sort of assurance.

On 11/26/2011 04:19 PM, Brad Anderson wrote:
>
> How about putting a disclaimer on the module warning the code hasn't
> been through a rigorous security audit and point them at well
> established C libraries if they need that sort of assurance.

What does that gain over implementing the first itteration in terms of well established C libraries and then replacing that with native implementations as the code goes been through a rigorous security audit?

Or how about do both as API compatible implementations? That would work for people who need the proven security and people who can't afford external dependencies as well as allow them to be swapped out for each other with minimal effort once the native code is proven.

On Sun 27 Nov 2011 10:27:58 AM CST, bcs wrote:
> On 11/26/2011 04:19 PM, Brad Anderson wrote:
>>
>> How about putting a disclaimer on the module warning the code hasn't been through a rigorous security audit and point them at well established C libraries if they need that sort of assurance.
>
> What does that gain over implementing the first itteration in terms of well established C libraries and then replacing that with native implementations as the code goes been through a rigorous security audit?
>
> Or how about do both as API compatible implementations? That would work for people who need the proven security and people who can't afford external dependencies as well as allow them to be swapped out for each other with minimal effort once the native code is proven.
>

I do like this idea.
swap implementations by simply swapping import and linking?
nice.

On Sun, Nov 27, 2011 at 9:27 AM, bcs <bcs@example.com> wrote:

> On 11/26/2011 04:19 PM, Brad Anderson wrote:
>
>>
>> How about putting a disclaimer on the module warning the code hasn't been through a rigorous security audit and point them at well established C libraries if they need that sort of assurance.
>>
>
> What does that gain over implementing the first itteration in terms of well established C libraries and then replacing that with native implementations as the code goes been through a rigorous security audit?
>
> Or how about do both as API compatible implementations? That would work for people who need the proven security and people who can't afford external dependencies as well as allow them to be swapped out for each other with minimal effort once the native code is proven.
>

That's even better but isn't the issue over bundling incompatibly licensed
libraries with phobos?  Nothing is stopping someone from writing bindings
for these libraries as some random library on D Source or Github already.
 An agreed upon API would be very nice in any case.

Jude Young wrote:
> On Sun 27 Nov 2011 10:27:58 AM CST, bcs wrote:
>> On 11/26/2011 04:19 PM, Brad Anderson wrote:
>>>
>>> How about putting a disclaimer on the module warning the code hasn't
>>> been through a rigorous security audit and point them at well
>>> established C libraries if they need that sort of assurance.
>>
>> What does that gain over implementing the first itteration in terms of
>> well established C libraries and then replacing that with native
>> implementations as the code goes been through a rigorous security audit?
>>
>> Or how about do both as API compatible implementations? That would
>> work for people who need the proven security and people who can't
>> afford external dependencies as well as allow them to be swapped out
>> for each other with minimal effort once the native code is proven.
>>
>
> I do like this idea.
> swap implementations by simply swapping import and linking?
> nice.

This was my goal... to write native implementation along with OpenSSL wrapper and add 'useOpenSSL' version identifier. Would that satisfy everyone?

On 11/27/2011 12:14 PM, Brad Anderson wrote:
>
> That's even better but isn't the issue over bundling incompatibly
> licensed libraries with phobos?  Nothing is stopping someone from
> writing bindings for these libraries as some random library on D Source
> or Github already.

If we can't find something with a licence allowing bundling then we just include the D language bits (including bindings) and note that along with where to get the lib.

> An agreed upon API would be very nice in any case.

That is necessary (or at least very desirable) in any case as it would allow swapping one cipher for another just as easily as it would allow swapping one implementation for another.

On 11/27/2011 12:15 PM, Piotr Szturmaj wrote:
> Jude Young wrote:
>> On Sun 27 Nov 2011 10:27:58 AM CST, bcs wrote:
>>> On 11/26/2011 04:19 PM, Brad Anderson wrote:
>>>>
>>>> How about putting a disclaimer on the module warning the code hasn't
>>>> been through a rigorous security audit and point them at well
>>>> established C libraries if they need that sort of assurance.
>>>
>>> What does that gain over implementing the first itteration in terms of
>>> well established C libraries and then replacing that with native
>>> implementations as the code goes been through a rigorous security audit?
>>>
>>> Or how about do both as API compatible implementations? That would
>>> work for people who need the proven security and people who can't
>>> afford external dependencies as well as allow them to be swapped out
>>> for each other with minimal effort once the native code is proven.
>>>
>>
>> I do like this idea.
>> swap implementations by simply swapping import and linking?
>> nice.
>
> This was my goal... to write native implementation along with OpenSSL
> wrapper and add 'useOpenSSL' version identifier. Would that satisfy
> everyone?

Yes, though I'd prefer to see them distinct and non-mutually exclusive. For one things, someone may well consider the native implementation of one primitive vetted before they consider another to be. Both results could be had by the suitable application of aliases.