On 5/26/2022 7:35 AM, deadalnix wrote:
> So I upgraded to 2.100
> 
> It is now reporting DIP100 deprecation by default.
> 
> Fantastic. Every single one of them is a false positive so far. I now face the situation where I will have deprecation warning forever, or add attribute soup to the program.
> 
> This is the opposite of progress. Stop it.
> 
> I'd be happy to add attributes for something that could actually track ownership/lifetime. DIP1000 is not that. Adding attributes for that is not worth it.

Many others report the same experience.

Not so long ago, I was asked to try out one of the popular C++ code static analyzers. Since at the time the D backend was still written in C++, I thought "great! let's see if it finds any bugs in the backend!".

It found zero bugs and 1000 false positives.

Does that make it a useless tool? Not at all. The thing is, the backend is 35 year old code. All the bugs had already been squeezed out of it! The false positives were things like printf formats (like printing a pointer as a %x instead of %p), pointer aliasing issues (modern code should use unions instead of casts), irrelevant portability issues, etc.

The same issue came up when const was added to D. It didn't fix existing, working, debugged code, either.

It's come up as well when C and C++ tightened their language specs.

The additional semantic checking is for:

1. new code - so you don't have to debug it, the compiler tells you right off the bat

2. self-documentation - so you know what a function plans to do with a pointer passed to it

3. when a pointer is marked `scope`, you don't have to double check it. The compiler checks it for you.

After all, you pointed out where D allows implicit conversion of a mutable delegate to an immutable one. The fix is:

    https://github.com/dlang/dmd/pull/14164

but it breaks existing, working, debugged code. What do you suggest we do about that? Remove immutable annotations? Ignore immutable annotations? Or fix user code to be const-correct? (I made it opt-in by putting it behind the dip1000 switch. It will eventually become the default.)

As for lifetimes, yes, I know that dip1000 only addresses one level of lifetimes. The fix for all levels is the @live stuff, which has already been merged. But we've decided on fixing all the dip1000 issues before putting a major effort into @live. The @live implementation depends on dip1000, and you'll be pleased to note that it only is turned on a function-by-function bases iff that function is marked @live. It's completely opt-in.

BTW, if you use templates, the compiler will infer the correct dip1000 attributes for you.

BTW, dip1000 is a complete solution for pointers to stack based objects. Just not for malloc'd objects.

On Thursday, 26 May 2022 at 18:00:00 UTC, Walter Bright wrote:
> Many others report the same experience.
>
> Not so long ago, I was asked to try out one of the popular C++ code static analyzers. Since at the time the D backend was still written in C++, I thought "great! let's see if it finds any bugs in the backend!".
>
> It found zero bugs and 1000 false positives.
>
> Does that make it a useless tool? Not at all. The thing is, the backend is 35 year old code. All the bugs had already been squeezed out of it! The false positives were things like printf formats (like printing a pointer as a %x instead of %p), pointer aliasing issues (modern code should use unions instead of casts), irrelevant portability issues, etc.
>
> The same issue came up when const was added to D. It didn't fix existing, working, debugged code, either.
>
> It's come up as well when C and C++ tightened their language specs.
>
> The additional semantic checking is for:
>
> 1. new code - so you don't have to debug it, the compiler tells you right off the bat
>
> 2. self-documentation - so you know what a function plans to do with a pointer passed to it
>
> 3. when a pointer is marked `scope`, you don't have to double check it. The compiler checks it for you.
>
> After all, you pointed out where D allows implicit conversion of a mutable delegate to an immutable one. The fix is:
>
>     https://github.com/dlang/dmd/pull/14164
>
> but it breaks existing, working, debugged code. What do you suggest we do about that? Remove immutable annotations? Ignore immutable annotations? Or fix user code to be const-correct? (I made it opt-in by putting it behind the dip1000 switch. It will eventually become the default.)
>
> As for lifetimes, yes, I know that dip1000 only addresses one level of lifetimes. The fix for all levels is the @live stuff, which has already been merged. But we've decided on fixing all the dip1000 issues before putting a major effort into @live. The @live implementation depends on dip1000, and you'll be pleased to note that it only is turned on a function-by-function bases iff that function is marked @live. It's completely opt-in.
>
> BTW, if you use templates, the compiler will infer the correct dip1000 attributes for you.

You wrote something similar to this a while back, and while there is some merit to it, I think there are a few wrong turn in there that make the argument bogus.

To begin with, I don't expect the same level of analysis from a static analyzer than from the compiler. I can ignore the static analyzer if it is wrong, I cannot ignore the compiler, after all, I need it to compile my code. False positive are therefore much more acceptable from the tool than the compiler.

Second, I expect the constraint checked by the compiler to provide me with useful invariant I can rely upon. For instance, if some data is immutable, and that it is really an invariant in the program, then I can know this data can be shared safely as nobody else is going to modify it.  The existence of the invariant limits my options on one axis - I cannot mutate this data - while opening my option in another axis - i can share this data safely without synchronization.

If immutable instead meant immutable in most places, you you can mutate it with this weird construct, then it is effectively useless as a language construct, because it restrict my expressiveness on one axis without granting me greater expressiveness on another.

Breaking the invariant must be breaking the type system, which is only possible in @system code and comes with all the appropriate warnings.

The problem with DIP1000, is that it doesn't provide me with invariant I can rely upon, because it is unable to track more than one level indirection. It can only detect some violation of the invariant, but not all (in fact, probably not the majority). As a result, it doesn't allow me to build upon it to make something greater.

It belongs in a static analysis tool.

On 5/26/2022 3:54 PM, deadalnix wrote:
> To begin with, I don't expect the same level of analysis from a static analyzer than from the compiler. I can ignore the static analyzer if it is wrong, I cannot ignore the compiler, after all, I need it to compile my code. False positive are therefore much more acceptable from the tool than the compiler.

The static analyzer should be built in to the language when looking for bugs, not stylistic issues.


> Second, I expect the constraint checked by the compiler to provide me with useful invariant I can rely upon. For instance, if some data is immutable, and that it is really an invariant in the program, then I can know this data can be shared safely as nobody else is going to modify it.  The existence of the invariant limits my options on one axis - I cannot mutate this data - while opening my option in another axis - i can share this data safely without synchronization.
> 
> If immutable instead meant immutable in most places, you you can mutate it with this weird construct, then it is effectively useless as a language construct, because it restrict my expressiveness on one axis without granting me greater expressiveness on another.

Immutable means immutable in D, all the way. I've fended off many attempts to turn it into "logical const" and add "mutable" overrides. Most of the complaints about immutable and const in D is they are relentless and brutal, not that they don't work.

Yes, we find holes in it now and then, and we plug them.


> Breaking the invariant must be breaking the type system, which is only possible in @system code and comes with all the appropriate warnings.
> 
> The problem with DIP1000, is that it doesn't provide me with invariant I can rely upon,

It does for stack based allocation.


> because it is unable to track more than one level indirection. It can only detect some violation of the invariant, but not all (in fact, probably not the majority).

It's designed to track all attempts to escape pointers to the stack, and I mean all as in 100%. It will not allow building multilevel data structures on the stack. Yes, some implementation bugs have appeared and a lot has been fixed (with Dennis' help). Some mistakes in the design have been discovered and adjustments made.

So far (fingers crossed) no *fundamental* problems with it have been found.


> As a result, it doesn't allow me to build upon it to make something greater.

@live builds upon it and relies on it. I agree @live has yet to prove itself, but focus is on ImportC and dip1000 for the moment.

On Thursday, 26 May 2022 at 23:25:06 UTC, Walter Bright wrote:
> Immutable means immutable in D, all the way. I've fended off many attempts to turn it into "logical const" and add "mutable" overrides. Most of the complaints about immutable and const in D is they are relentless and brutal, not that they don't work.
>
> Yes, we find holes in it now and then, and we plug them.
>

Yes, this is an example of something that is done right and provided value as a result. DIP1000 and @live just aren't.