Hello Walter,

> Consider the Bible. It's long and complicated, and by careful
> examination of it you can find a verse here and there to justify *any*
> behavior.
> 

This is true of any long document if you are willing to string together enough quotes and ignore enough of it. I'd bet you could get the Declaration of Intendance out of Playboy if you wanted to.

> D is complicated, and is founded on principles that are not orthogonal
> - they are often at odds with each other. Any attempt to take one
> particular aspect of D's behavior and use it as a rule to impose
> elsewhere is surely doomed to conflict with some other rule.
> 
> The only reasonable way forward is to evaluate each idea not only in
> terms of all of D's principles, but also on its own merits, and throw
> in one's best judgment.
> 
> Nearly a decade with D has now shown that some ideas and choices were
> dead wrong, but others were more right than I even dreamed <g>.

You sort of tangentially hit what I think is the key to avoiding creating meaning out of whole cloth; take the document as a whole, look for themes and trends (any document worth reading will have them), use them to answer your questions.

BCS wrote:
> For open source and libs, yes. For proprietary code bases, I'd say it's about a wash. Having it in another file could make the language/code base easier to read and also allow a much more powerful rules language (because it doesn't have to fit in the host language). And because only you will be maintaining the code, needing another tool (that you already have) and another build step isn't much of an issue.


Of course you can make third party tools work, but I think there is a lot of merit and psychological advantage to building it into the base language. For example, unit testing and document creation are usually 3rd party tools, but with D they are built in, and I think them being built in has been a huge success.

Hi,

Walter Bright wrote:
> 
> There's a lot of hoopla about these static checkers, but I'm not impressed by them based on what I can find out about them. What do you know about what these checkers do that is not on this list? Any other kinds of checking that would be great to implement?

The development edition of VS2008 also has a static code analyzer. There
is a list of issued warnings for C/C++ with examples at:

http://msdn.microsoft.com/en-us/library/a5b9aa09.aspx

There are a lot more messages generated for managed code:

http://msdn.microsoft.com/en-us/library/ee1hzekz.aspx

The analyzer does inter-procedural-analysis, so it (sometimes) knows, what range of values function arguments can have.

I've run the analysis on the dmd source code, but deactivated some
warnings regarding memory leaks and variable hiding to not get flooded
with messages. After that, code analysis added about 150 warnings to the
1000 already issued by the compiler (mostly "signed/unsigned mismatch"
and "unreferenced local variable").

The code analyzer warnings often deal with
- "Dereferencing NULL pointer" (I've added __assume statements to
assert, so the compiler does not warn null pointers, if there is an
"assert(pointer)" somewhere before the indirection). Most of these
warnings are not so easy to verify, but some are probably correct.

- "Using uninitialized memory": mostly after switch-statements without
variable initialization in default.

- "Invalid data: accessing 'variable', the readable size is 'X' bytes,
but 'Y' bytes might be read": as far as I could see, these are often
false positives, because the analyzer could not deal with variable sized
arrays at the end of structs (you can add annotation to help the
analyzer, though, but I have not tried). Even with constant size arrays, some of the warnings are plain wrong.

- stdlib/windows-api calls (e.g. printf arguments, ignored return codes)

I guess, that you'll have to annotate the code a lot if you want to have the compiler always do the analyzing and get reasonable results. I've also seen C# code with a lot of attributes just to suppress warnings from the analyzer.

Rainer

Walter Bright wrote:
> Nick Sabalausky wrote:
>> "Walter Bright" <newshound1@digitalmars.com> wrote in message
>>> 2. possible dereference of NULL pointers (some reaching definitions
>>> of a pointer are NULL)
>>> 2. Optimizer collects the info, but ignores this, because people are
>>> annoyed by false positives.
>>>
>>
>> If you mean something like this:
>>
>> Foo f;
>> if(cond)
>>     f = new Foo();
>> f.bar();
>>
>> Then I *want* the compiler to tell me. C# does this and I've never been annoyed by it, in fact I've always appreciated it. I'm not aware of any other C# user that has a problem with that either. If that's not what you mean though, then could you elaborate?
> 
> The problem crops up when there are two connected variables:
> 
>   void foo(bool flag)
>   {
>     char* p = null;
>     if (flag)
>     p = "hello";
>     ...
>     if (flag)
>     bar(*p);
>   }
> 
> The code is logically correct, there is no null pointer dereference possible. However, the data flow analysis will see the *p and see two reaching definitions for p: null and "hello", even though only one actually reaches.
> 
> Hence the false positive. To eliminate the false error report, the user would have to insert a redundant null check.
> 
> Does this happen in practice? Yes.

	I don't know about Coverity/clang, but I have used Klocwork a
couple of times and it will work properly in the example you have
given. I.e it sees that both conditions are linked and that you
don't use p if you haven't initialized it.

	Of course, if the condition is more complex or if the condition
might be modified by another thread (even if you know it isn't),
there comes a point at which it will give a warning.

		Jerome
-- 
mailto:jeberger@free.fr
http://jeberger.free.fr
Jabber: jeberger@jabber.fr

Rainer Schuetze wrote:
> I've run the analysis on the dmd source code, but deactivated some
> warnings regarding memory leaks and variable hiding to not get flooded
> with messages. After that, code analysis added about 150 warnings to the
> 1000 already issued by the compiler (mostly "signed/unsigned mismatch"
> and "unreferenced local variable").

Could you turn it all on, run it through the dmd source code, and send me the results? I'd like to see if it detected any real bugs.

BCS wrote:
> Hello Walter,
> 
>> Consider the Bible. It's long and complicated, and by careful
>> examination of it you can find a verse here and there to justify *any*
>> behavior.
>>
> 
> This is true of any long document if you are willing to string together enough quotes and ignore enough of it. I'd bet you could get the Declaration of Intendance out of Playboy if you wanted to.

What is this Declaration of Intendance you speak of?

As for ignoring context...well, I'm willing to discuss this, but not here.

Nick Sabalausky wrote:
> static void Main(string[] args)
> {
>     Foo f;
>     if(args.Count() > 2) { f = new Foo(); }
> 
>     if(args.Count() > 2)
>     {
>        f.bar(); // ERROR: Use of unassgned local variable 'f'
>     }
> 
>     Foo f2;
>     createFoo(ref f2); // ERROR: Use of unassgned local variable 'f2'
>     f2.bar();
> }
> 
> static void createFoo(ref Foo f)
> {
>     f = new Foo();
> }
> ---------------------------------------------
> 
> The first one is rather strange coding though and makes it easy to hide errors anyway. And the second one's a tad odd too, plus I don't see any harm in solving that with "Foo f2=null": it would at least be a hell of a lot better than the compiler doing that very same "=null" automatically. I know Walter doesn't agree, but I'd much rather have a few slightly inconvinient false positives (or would it really be a false negative?) than even a mere possibility for a hidden error.

The second one is an error; createFoo might use its argument before assigning. You should have marked its argument as out instead, which would not yield an error.

In point of fact, that's a common pattern in C#. Dictionaries define a method bool TryGetValue(key, out value):

DateTime date;
if (dict.TryGetValue("key", out date))
{
    // use date
}

On Sat, Oct 3, 2009 at 4:00 AM, Nick Sabalausky <a@a.a> wrote:
> "Christopher Wright" <dhasenan@gmail.com> wrote in message news:ha5mtp$f3c$2@digitalmars.com...
>> BCS wrote:
>>>
>>> This is true of any long document if you are willing to string together enough quotes and ignore enough of it. I'd bet you could get the Declaration of Intendance out of Playboy if you wanted to.
>>
>> What is this Declaration of Intendance you speak of?
>>
>
> That's the one they meant to write.

Nicely played.

Hello Christopher,

> BCS wrote:
> 
>> Hello Walter,
>> 
>>> Consider the Bible. It's long and complicated, and by careful
>>> examination of it you can find a verse here and there to justify
>>> *any* behavior.
>>> 
>> This is true of any long document if you are willing to string
>> together enough quotes and ignore enough of it. I'd bet you could get
>> the Declaration of Intendance out of Playboy if you wanted to.
>> 
> What is this Declaration of Intendance you speak of?

OK so I use spell check as a crutch (and I was trying to pay attention to a lecture when I typed that).

> As for ignoring context...well, I'm willing to discuss this, but not
> here.

Note: I'm not claiming that some document is saying anything, but, that with enough cut and paste, any long enough document can be made to say anything you want and that sometimes this requiters that you ignore context. The implication is that a selected set of excerpts from some document can't be taken as proof that the document says or doesn't say anything.

There is a simple and elegant solution to the problem of false positives in static analysis.  Simple provide a way for the programmer to say, "I know this looks dangerous, but I know what I'm doing, so trust me on this."  This could be done by adding some new syntax, either a new keyword or a reused old keyword, or by changing the semantics of void initialization.

For example, this would be an error:
  T t;
  if (i < 10) {
    t = new T;
  }
  if (i < 10) {
    t.f();
  }

This would be legal:
  T t = unchecked;
  if (i < 10) {
    t = new T;
  }
  if (i < 10) {
    t.f();
  }

But this would be an error:
  T t = unchecked;
  t.f();

As would this:
  T t = unchecked;
  f(t);

'unchecked' has the following properties:
  - It is an error to use the value of a variable that is initialized as
'unchecked'.
  - 'unchecked' tells the static analysis that the programmer knows what
he's doing.  The static analysis should therefore be liberal rather than
conservative.
  - At runtime, 'unchecked' is equivalent to 'null' for non-nullable
reference types.  This increases the chance that the error will be
detected at runtime.

On the other hand, variables without any explicit initializer at all
will have the following properties:
  - It is an error to use the value of the variable.
  - The static analysis is conservative by default.  If it thinks the
variable could be used uninitialized, an error is reported.
  - Since the static analysis guarantees that the variable will not be
used uninitialized, there is no need to initialize the variable at runtime.


-- 
Rainer Deyke - rainerd@eldwood.com