View mode: basic / threaded / horizontal-split · Log in · Help
December 24, 2011
Carmack about static analysis
A new blog post by the very good John Carmack, I like how well readable this post is:
http://altdevblogaday.com/2011/12/24/static-code-analysis/

If you don't know who he is:
http://en.wikipedia.org/wiki/John_Carmack


Beside the main point of the article that is to use more static analysis tools, here are some comments:


>Anything that isn't crystal clear to a static analysis tool probably isn't clear to your fellow programmers, either.<

It seems that elsewhere Carmack has said something related:
>if the compiler can't figure out that it's safe, then it's probably hard for a human to figure that out too, and this likely to get it wrong.<

But human minds and the logic of lint programs are two cognitive (or pre-cognitive) systems that work in very different ways. Often what's obvious and easy for a mammalian brain can't be seen by a lint tool and what a lint tool finds easy to spot is hard to find for a human brain. So they are both needed. The difference in the way computers and people "think" is one of reasons of the usefulness of computers for our society.

Here there are some bad examples about /Analyze:
http://randomascii.wordpress.com/2011/09/13/analyze-for-visual-studio-the-ugly-part-5/
In the same blog you see many other similar cases.


>NULL pointers are the biggest problem in C/C++, at least in our code.  The dual use of a single value as both a flag and an address causes an incredible number of fatal issues.<

A big problem not solved in D.


>Printf format string errors were the second biggest issue in our codebase, heightened by the fact that passing an idStr instead of idStr::c_str() almost always results in a crash, but annotating all our variadic functions with /analyze annotations so they are properly type checked kills this problem dead.<

A problem that people didn't want to reduce in D.


>A lot of the serious reported errors are due to modifications of code long after it was written.  An incredibly common error pattern is to have some perfectly good code that checks for NULL before doing an operation, but a later code modification changes it so that the pointer is used again without checking.  Examined in isolation, this is a comment on code path complexity, but when you look back at the history, it is clear that it was more a failure to communicate preconditions clearly to the programmer modifying the code.<

D has a space reserved for preconditions, so I think this is a smaller problem in D compared to C++.


>There was a paper recently that noted that all of the various code quality metrics correlated at least as strongly with code size as error rate, making code size alone give essentially the same error predicting ability.  Shrink your important code.<

In theory functional-style is a good to shrink the code, but in D functional-style code is a jungle of (({}){()}) so it's hard to write and hard to read.

Bye,
bearophile
December 24, 2011
Re: Carmack about static analysis
On Saturday, 24 December 2011 at 12:42:41 UTC, bearophile wrote:
>>Printf format string errors were the second biggest issue in
> A problem that people didn't want to reduce in D.

printf and writef are entirely different beasts.
December 24, 2011
Re: Carmack about static analysis
On Sat, 24 Dec 2011 23:42:41 +1100, bearophile <bearophileHUGS@lycos.com>  
wrote:

> A new blog post by the very good John Carmack, I like how well readable  
> this post is:
> http://altdevblogaday.com/2011/12/24/static-code-analysis/
>
> ... human minds and the logic of lint programs are two cognitive (or  
> pre-cognitive) systems that work in very different ways. Often what's  
> obvious and easy for a mammalian brain can't be seen by a lint tool and  
> what a lint tool finds easy to spot is hard to find for a human brain.  
> So they are both needed. The difference in the way computers and people  
> "think" is one of reasons of the usefulness of computers for our society.


But isn't lint written by humans? I'm pretty certain 'lint' is not a  
cognitive entity, but just a program written by people, who as we all  
agree, can and do make mistakes. I'm under the impression that a //lint//  
program is an attempt to emulate a very pedantic (if not anal-retentive)  
person who in real life would have no real friends, due to their  
obsessive–compulsive habit for extreme nit-picking.

So in saying that, I would have to side with John Carmack in so far as  
thinking that if a near-perfect lint program cannot understand some source  
code, then a regular coder would also have difficulty in groking it.

In short, it does no harm to make one's code as readable (such that your  
intention is exceedingly obvious) as possible.

> Here there are some bad examples about /Analyze:
> http://randomascii.wordpress.com/2011/09/13/analyze-for-visual-studio-the-ugly-part-5/
> In the same blog you see many other similar cases.


The article you reference is primarily saying that this specific lint-like  
functionality contains bugs. It is not an argument to convince us to  
abandon lint functionality.


> In theory functional-style is a good to shrink the code, but in D  
> functional-style code is a jungle of (({}){()}) so it's hard to write  
> and hard to read.

I once, long ago, suggested to Walter/Andrei that D is approaching the  
ASCII equivalent of APL. The actual text of D source code can be a  
hindrance to reading the code, let alone understanding what has been  
written. I know it is hyperbole, but some D source modules are close to  
write-only code, meaning that it takes specialist/expert D knowledge to  
understand some source code. I'm not so certain that this is a desirable  
feature of any programming language.

-- 
Derek Parnell
Melbourne, Australia
December 24, 2011
Re: Carmack about static analysis
On 12/24/2011 06:42 AM, bearophile wrote:
> A new blog post by the very good John Carmack, I like how well
> readable this post is:
> http://altdevblogaday.com/2011/12/24/static-code-analysis/

Nice! One nice snippet that I can't quote strongly enough:

"Automation is necessary.  It is common to take a sort of smug
satisfaction in reports of colossal failures of automatic systems, but
for every failure of automation, the failures of humans are legion."

This is a very important tidbit. We've improved participation of
automation to D's development process a fair amount, but we still have a
long time to go.

"Anything that isn’t crystal clear to a static analysis tool probably 
isn’t clear to your fellow programmers, either."

This I disagree with; I think it comes with the assumption that static 
analysis tools Carmack tried don't even try to approach certain 
problems. There are plenty of things that are relatively simple but 
extremely difficult to automatically prove correct.


Andrei
December 24, 2011
Re: Carmack about static analysis
On 12/24/2011 09:11 AM, Derek wrote:
> On Sat, 24 Dec 2011 23:42:41 +1100, bearophile
> <bearophileHUGS@lycos.com> wrote:
>> In theory functional-style is a good to shrink the code, but in D
>> functional-style code is a jungle of (({}){()}) so it's hard to write
>> and hard to read.

As I mentioned once, the proposition of writing functional code in D is 
quite a bit different from other languages. In languages dedicated to 
pure or almost pure functional semantics, there may literally be no 
other way to achieve anything except in functional style; if there is, 
the style is almost invariably stilted.

In D, functional style code must compete with a well honed imperative 
paradigm. And as unpleasant as that may be, sometimes code that uses 
mutation may be "better" than functional code by some metrics.

> I once, long ago, suggested to Walter/Andrei that D is approaching the
> ASCII equivalent of APL. The actual text of D source code can be a
> hindrance to reading the code, let alone understanding what has been
> written. I know it is hyperbole, but some D source modules are close to
> write-only code, meaning that it takes specialist/expert D knowledge to
> understand some source code. I'm not so certain that this is a desirable
> feature of any programming language.

We recently discussed this a bit in connection with the standard 
library. It definitely could be written in a manner that is easier to 
understand for the more casual reader, but that would mean reducing the 
capabilities (e.g. no reduce with multiple parameters) and making it slower.

That being said, I personally find contemporary application D code very 
readable. It's a pleasure to get to it after using C++ at work.

Anyhow, is there anything you have in mind that we have the chance of 
improving at this point?


Thanks,

Andrei
December 24, 2011
Re: Carmack about static analysis
On Saturday, 24 December 2011 at 15:33:04 UTC, Andrei 
Alexandrescu wrote:
> Anyhow, is there anything you have in mind that we have the 
> chance of improving at this point?
>
>
> Thanks,
>
> Andrei

For one, we should follow up on:

foo!(a => a * 2)(bar);

vs.

foo!((a) { return a * 2; })(bar);
December 24, 2011
Re: Carmack about static analysis
On Sun, 25 Dec 2011 02:33:01 +1100, Andrei Alexandrescu  
<SeeWebsiteForEmail@erdani.org> wrote:


> We recently discussed this a bit in connection with the standard  
> library. It definitely could be written in a manner that is easier to  
> understand for the more casual reader, but that would mean reducing the  
> capabilities (e.g. no reduce with multiple parameters) and making it  
> slower.

It's the syntax and not the semantics that troubles me. I'm sure that some  
syntax changes could be possible to make readability improvements with no  
loss of capabilities.

> ... I personally find contemporary application D code very readable.  
> It's a pleasure to get to it after using C++ at work.


I'm sure you are totally correct; I'm not really a C++ coder. And I'm sure  
you also process the specialist/expert level of D knowledge to make  
reading contemporary D code a non-issue. But when compared to spoken  
language text, D code can appear quite obtuse to average coders. And I  
believe this is main do to the very high use of non-alphabetic symbols and  
a level of overloading of both punctuation characters and reserved words.


> Anyhow, is there anything you have in mind that we have the chance of  
> improving at this point?

I have nothing concrete right now, and I suspect nothing that a C/C++/D  
aficionado would appreciate, so I'll remain silent about this and try  
harder to appreciate D syntax. There is no point in starting  
bike-shed/flame-wars about syntax issues. I understand Walter's viewpoints  
on the desire for C compatibility and on the use of reserved words.

-- 
Derek Parnell
Melbourne, Australia
December 24, 2011
Re: Carmack about static analysis
On 2011-12-24 13:42, bearophile wrote:
>> There was a paper recently that noted that all of the various code quality metrics correlated at least as strongly with code size as error rate, making code size alone give essentially the same error predicting ability.  Shrink your important code.<
>
> In theory functional-style is a good to shrink the code, but in D functional-style code is a jungle of (({}){()}) so it's hard to write and hard to read.

There have been suggestion for a new lambda syntax, among them the one 
used by C# and Scala:

foo(a => a * 2);

-- 
/Jacob Carlborg
December 24, 2011
Re: Carmack about static analysis
On 12/24/2011 6:35 AM, Adam D. Ruppe wrote:
> On Saturday, 24 December 2011 at 12:42:41 UTC, bearophile wrote:
>>> Printf format string errors were the second biggest issue in
>> A problem that people didn't want to reduce in D.
>
> printf and writef are entirely different beasts.

D's charter is not to fix bad usage of C functions like printf and memcpy. Use 
the corresponding D features like writef and array copy, and you won't need 
static analysis on them.
December 24, 2011
Re: Carmack about static analysis
On 12/24/2011 11:30 AM, Walter Bright wrote:
> On 12/24/2011 6:35 AM, Adam D. Ruppe wrote:
>> On Saturday, 24 December 2011 at 12:42:41 UTC, bearophile wrote:
>>>> Printf format string errors were the second biggest issue in
>>> A problem that people didn't want to reduce in D.
>>
>> printf and writef are entirely different beasts.
>
> D's charter is not to fix bad usage of C functions like printf and
> memcpy. Use the corresponding D features like writef and array copy, and
> you won't need static analysis on them.

You still might if you want to make sure they work without running them.

Andrei
« First   ‹ Prev
1 2 3 4 5
Top | Discussion index | About this forum | D home