On Mon, 05 May 2014 07:39:13 +0000
Paulo Pinto via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
> Sometimes I wonder how much money have C design decisions cost the industry in terms of anti-virus, static and dynamic analyzers tools, operating systems security enforcements, security research and so on.
>
> All avoidable with bound checking by default and no implicit conversions between arrays and pointers.

Well, a number of years ago, the folks who started the codebase of the larger products at the company I work at insisted on using COM everywhere, because we _might_ have to interact with 3rd parties, and they _might_ not want to use C++. So, foolishly, they mandated that _nowhere_ in the codebase should any C++ objects be passed around except by pointer. They then had manual reference counting on top of that to deal with memory management. That decision has cost us man _years_ in time working on reference counting-related bugs. Simply using smart pointers instead would probably have saved the company millions. COM may have its place, but forcing a whole C++ codebase to function that way was just stupid, especially when pretty much none of it ever had to interact directly with 3rd party code (and even if it had, it should have been done through strictly defined wrapper libraries; it doesn't make sense that 3rd parties would hook into the middle of your codebase).

Seemingly simple decisions can have _huge_ consequences - especially when that decision affects millions of lines of code, and that's definitely the case with some of the decisions made for C. Some of them may have be unavoidable give the hardware situation and programming climate at the time that C was created, but we've been paying for them ever since. And unfortunately, the way things are going at this point, nothing will ever really overthrow C. We'll have to deal with it on some level for a long, long time to come.

- Jonathan M Davis

On Monday, 5 May 2014 at 08:04:24 UTC, Jonathan M Davis via Digitalmars-d wrote:
> On Mon, 05 May 2014 07:39:13 +0000
> Paulo Pinto via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>> Sometimes I wonder how much money have C design decisions cost
>> the industry in terms of anti-virus, static and dynamic analyzers
>> tools, operating systems security enforcements, security research
>> and so on.
>>
>> All avoidable with bound checking by default and no implicit
>> conversions between arrays and pointers.
>
> Well, a number of years ago, the folks who started the codebase of the larger
> products at the company I work at insisted on using COM everywhere, because we
> _might_ have to interact with 3rd parties, and they _might_ not want to use
> C++. So, foolishly, they mandated that _nowhere_ in the codebase should any
> C++ objects be passed around except by pointer. They then had manual reference
> counting on top of that to deal with memory management. That decision has cost
> us man _years_ in time working on reference counting-related bugs. Simply
> using smart pointers instead would probably have saved the company millions.
> COM may have its place, but forcing a whole C++ codebase to function that way
> was just stupid, especially when pretty much none of it ever had to interact
> directly with 3rd party code (and even if it had, it should have been done
> through strictly defined wrapper libraries; it doesn't make sense that 3rd
> parties would hook into the middle of your codebase).


So the decision was made and CComPtr<> and _com_ptr_t<> weren't used?!

I feel your pain. Well, now the codebase is WinRT ready. :)

>
> Seemingly simple decisions can have _huge_ consequences - especially when that
> decision affects millions of lines of code, and that's definitely the case
> with some of the decisions made for C. Some of them may have be unavoidable
> give the hardware situation and programming climate at the time that C was
> created, but we've been paying for them ever since. And unfortunately, the way
> things are going at this point, nothing will ever really overthrow C. We'll
> have to deal with it on some level for a long, long time to come.
>
> - Jonathan M Davis


I doubt those decisions really made sense, given the other system programming languages at the time. Most of them did bounds checking, had no implicit conversions and operating systems were being written with them.

Algol 60 reference compiler did not allow disabling bounds checking at all, for example.

Quote from Tony Hoare's ACM award article[1]:
"A consequence of this principle is that every occurrence of
every subscript of every subscripted variable was on every occasion
checked at run time against both the upper and the lower declared
bounds of the array. Many years later we asked our customers whether
they wished us to provide an option to switch off these checks in the
interests of efficiency on production runs. Unanimously, they urged
us not to--they already knew how frequently subscript errors occur
on production runs where failure to detect them could be disastrous.
I note with fear and horror that even in 1980 language designers and
users have not learned this lesson. In any respectable branch of
engineering, failure to observe such elementary precautions would have
long been against the law."


[1]http://www.labouseur.com/projects/codeReckon/papers/The-Emperors-Old-Clothes.pdf

On Sunday, 4 May 2014 at 21:18:24 UTC, Daniele M. wrote:
> And then comes my next question: except for that malloc-hack, would it have been possible to write it in @safe D? I guess that if not, module(s) could have been made un-@safe. Not saying that a similar separation of concerns was not possible in OpenSSL itself, but that D could have made it less development-expensive in my opinion.

TDPL SafeD visions notwithstanding, @safe is very very limiting.

I/O is forbidden so simple Hello Worlds are right out, let alone advanced socket libraries.

On Monday, 5 May 2014 at 09:32:40 UTC, JR wrote:
> On Sunday, 4 May 2014 at 21:18:24 UTC, Daniele M. wrote:
>> And then comes my next question: except for that malloc-hack, would it have been possible to write it in @safe D? I guess that if not, module(s) could have been made un-@safe. Not saying that a similar separation of concerns was not possible in OpenSSL itself, but that D could have made it less development-expensive in my opinion.
>
> TDPL SafeD visions notwithstanding, @safe is very very limiting.
>
> I/O is forbidden so simple Hello Worlds are right out, let alone advanced socket libraries.

I/O is not forbidden, it's just that writeln and friends currently can't be made safe, but that is being worked on AFAIK. While I/O usually goes through the OS, the system calls can be manually verified and made @trusted.

On Mon, 05 May 2014 10:24:27 +0000
via Digitalmars-d <digitalmars-d@puremagic.com> wrote:

> On Monday, 5 May 2014 at 09:32:40 UTC, JR wrote:
> > On Sunday, 4 May 2014 at 21:18:24 UTC, Daniele M. wrote:
> >> And then comes my next question: except for that malloc-hack, would it have been possible to write it in @safe D? I guess that if not, module(s) could have been made un-@safe. Not saying that a similar separation of concerns was not possible in OpenSSL itself, but that D could have made it less development-expensive in my opinion.
> >
> > TDPL SafeD visions notwithstanding, @safe is very very limiting.
> >
> > I/O is forbidden so simple Hello Worlds are right out, let alone advanced socket libraries.
>
> I/O is not forbidden, it's just that writeln and friends currently can't be made safe, but that is being worked on AFAIK. While I/O usually goes through the OS, the system calls can be manually verified and made @trusted.

As the underlying OS calls are all C functions, there will always be @system code involved in I/O, but in most cases, we should be able to wrap those functions in D functions which are @trusted.

Regarldess, I would think that SSL could be implemented without sockets - that is, all of its operations should be able to operate on arbitrary data regardless of whether that data is sent over a socket or not. And if that's the case, then even if the socket operations themselves had to be @system, then everything else should still be able to be @safe.

Most of the problems with @safe stem either from library functions that don't use it like they should, or because the compiler does not yet do a good enough job with attribute inference on templated functions. Both problems are being addressed, so the situation will improve over time. Regardless, there's nothing fundamentally limited about @safe except for operations which are actually unsafe with regards to memory, and any case where something isn't @safe when it's actually memory safe should be and will be fixed (as well as any situation which isn't memory safe but is considered @safe anyway - we do unfortunately still have a few of those).

- Jonathan M Davis

On 2014-05-04 4:34 AM, Daniele M. wrote:
> I have read this excellent article by David A. Wheeler:
>
> http://www.dwheeler.com/essays/heartbleed.html
>
> And since D language was not there, I mentioned it to him as a possible
> good candidate due to its static typing and related features.
>
> However, now I am asking the community here: would a D implementation
> (with GC disabled) of OpenSSL have been free from Heartbleed-type
> vulnerabilities? Specifically
> http://cwe.mitre.org/data/definitions/126.html and
> http://cwe.mitre.org/data/definitions/20.html as David mentions.
>
> I find this perspective very interesting, please advise :)

I'm currently working on a TLS library using only D. I've shared the ASN.1 parser here: https://github.com/globecsys/asn1.d

The ASN.1 format allows me to compile the data structures to D from the tls.asn1 in the repo I linked to. It uses the equivalent of D template structures extensively with what's called an Information Object Class.

Obviously, when it's done I need a DER serializer/deserializer which I intend on editing MsgPackD, and then I can do a handshake (read a ASN.1 certificate) and encrypt/decrypt AES/RSA using the certificate information and this cryptography library: https://github.com/apartridge/crypto .

I've never expected any help so I'm not sure what the licensing will be. I'm currently working on the generation step for the ASN.1 to D compiler, it's very fun to make a compiler in D.

On 5/5/14, 2:32 AM, JR wrote:
> On Sunday, 4 May 2014 at 21:18:24 UTC, Daniele M. wrote:
>> And then comes my next question: except for that malloc-hack, would it
>> have been possible to write it in @safe D? I guess that if not,
>> module(s) could have been made un-@safe. Not saying that a similar
>> separation of concerns was not possible in OpenSSL itself, but that D
>> could have made it less development-expensive in my opinion.
>
> TDPL SafeD visions notwithstanding, @safe is very very limiting.
>
> I/O is forbidden so simple Hello Worlds are right out, let alone
> advanced socket libraries.

Sounds like a library bug. Has it been submitted? -- Andrei

05-May-2014 18:59, Etienne пишет:
> On 2014-05-04 4:34 AM, Daniele M. wrote:
>> I have read this excellent article by David A. Wheeler:
>>
>> http://www.dwheeler.com/essays/heartbleed.html
>>
>> And since D language was not there, I mentioned it to him as a possible
>> good candidate due to its static typing and related features.
>>
>> However, now I am asking the community here: would a D implementation
>> (with GC disabled) of OpenSSL have been free from Heartbleed-type
>> vulnerabilities? Specifically
>> http://cwe.mitre.org/data/definitions/126.html and
>> http://cwe.mitre.org/data/definitions/20.html as David mentions.
>>
>> I find this perspective very interesting, please advise :)
>
> I'm currently working on a TLS library using only D. I've shared the
> ASN.1 parser here: https://github.com/globecsys/asn1.d

Cool, keep us posted.

>
> The ASN.1 format allows me to compile the data structures to D from the
> tls.asn1 in the repo I linked to. It uses the equivalent of D template
> structures extensively with what's called an Information Object Class.
>
> Obviously, when it's done I need a DER serializer/deserializer which I
> intend on editing MsgPackD, and then I can do a handshake (read a ASN.1
> certificate) and encrypt/decrypt AES/RSA using the certificate
> information and this cryptography library:
> https://github.com/apartridge/crypto .
>
> I've never expected any help so I'm not sure what the licensing will be.
> I'm currently working on the generation step for the ASN.1 to D
> compiler, it's very fun to make a compiler in D.

Aye, D seems to be a nice choice for writing compilers.

-- 
Dmitry Olshansky

On Monday, 5 May 2014 at 14:59:13 UTC, Etienne wrote:
> On 2014-05-04 4:34 AM, Daniele M. wrote:
>> I have read this excellent article by David A. Wheeler:
>>
>> http://www.dwheeler.com/essays/heartbleed.html
>>
>> And since D language was not there, I mentioned it to him as a possible
>> good candidate due to its static typing and related features.
>>
>> However, now I am asking the community here: would a D implementation
>> (with GC disabled) of OpenSSL have been free from Heartbleed-type
>> vulnerabilities? Specifically
>> http://cwe.mitre.org/data/definitions/126.html and
>> http://cwe.mitre.org/data/definitions/20.html as David mentions.
>>
>> I find this perspective very interesting, please advise :)
>
> I'm currently working on a TLS library using only D. I've shared the ASN.1 parser here: https://github.com/globecsys/asn1.d
>
> The ASN.1 format allows me to compile the data structures to D from the tls.asn1 in the repo I linked to. It uses the equivalent of D template structures extensively with what's called an Information Object Class.
>
> Obviously, when it's done I need a DER serializer/deserializer which I intend on editing MsgPackD, and then I can do a handshake (read a ASN.1 certificate) and encrypt/decrypt AES/RSA using the certificate information and this cryptography library: https://github.com/apartridge/crypto .
>
> I've never expected any help so I'm not sure what the licensing will be. I'm currently working on the generation step for the ASN.1 to D compiler, it's very fun to make a compiler in D.

This is a quite radical approach, I am very interested to see its development! Have you thought about creating an SSL/TLS implementations tester instead? With the compiled information I see this goal quite well in range.

On Monday, 5 May 2014 at 10:41:41 UTC, Jonathan M Davis via Digitalmars-d wrote:
> On Mon, 05 May 2014 10:24:27 +0000
> via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>
>> On Monday, 5 May 2014 at 09:32:40 UTC, JR wrote:
>> > On Sunday, 4 May 2014 at 21:18:24 UTC, Daniele M. wrote:
>> >> And then comes my next question: except for that malloc-hack,
>> >> would it have been possible to write it in @safe D? I guess
>> >> that if not, module(s) could have been made un-@safe. Not
>> >> saying that a similar separation of concerns was not possible
>> >> in OpenSSL itself, but that D could have made it less
>> >> development-expensive in my opinion.
>> >
>> > TDPL SafeD visions notwithstanding, @safe is very very limiting.
>> >
>> > I/O is forbidden so simple Hello Worlds are right out, let
>> > alone advanced socket libraries.
>>
>> I/O is not forbidden, it's just that writeln and friends
>> currently can't be made safe, but that is being worked on AFAIK.
>> While I/O usually goes through the OS, the system calls can be
>> manually verified and made @trusted.
>
> As the underlying OS calls are all C functions, there will always be @system
> code involved in I/O, but in most cases, we should be able to wrap those
> functions in D functions which are @trusted.
>
> Regarldess, I would think that SSL could be implemented without sockets - that
> is, all of its operations should be able to operate on arbitrary data
> regardless of whether that data is sent over a socket or not. And if that's
> the case, then even if the socket operations themselves had to be @system,
> then everything else should still be able to be @safe.
>
> Most of the problems with @safe stem either from library functions that don't
> use it like they should, or because the compiler does not yet do a good enough
> job with attribute inference on templated functions. Both problems are being
> addressed, so the situation will improve over time. Regardless, there's
> nothing fundamentally limited about @safe except for operations which are
> actually unsafe with regards to memory, and any case where something isn't
> @safe when it's actually memory safe should be and will be fixed (as well as
> any situation which isn't memory safe but is considered @safe anyway - we do
> unfortunately still have a few of those).
>
> - Jonathan M Davis

You nailed it. If we wanted to translate the theoretical exercise into something real, it would be nice to have an implementation of PolarSSL that works on ring buffers only, then leave network layer integration to clients. Much cleaner separation of concerns.