On 26/08/2024 5:55 AM, Bruce Carneal wrote:
> The lesson I take from the DIP 1000 history is that we need something that is simpler to explain, something that is much easier to use correctly, something that models the problem more clearly.

I have another observation that I believe to be the final nail in DIP1000's current scope of design.

What people refer to it as, is DIP1000. Its DIP number.

This includes both Walter and Dennis.

It is not called scoped pointers or escape analysis.

The human mind is highly associative, the fact that it is not connecting the dots on this is a major concern that it is not in fact solving a problem that people have, let alone model in their mind.

This should of happened in the last 10 years, yet it hasn't.

On Sunday, August 25, 2024 11:55:04 AM MDT Bruce Carneal via Digitalmars-d wrote:
> The lesson I take from the DIP 1000 history is that we need something that is simpler to explain, something that is much easier to use correctly, something that models the problem more clearly.
>
> Bug reporting/fixing is great, but sometimes the bug pattern indicates a rethink is in order.  I believe this is one of those times.

I am strongly of the opinion that DIP 1000 is far more complex than it's worth and that we'd be better off just dropping it. If you're primarily using the GC, then DIP 1000 doesn't even help you, because it's focused on stuff like trying to make taking the address of a local variable @safe instead of just leaving that up to the programmer like we always have. D can already be memory-safe quite easily so long as you minimize stuff like taking the address of local variables, and we have @trusted to deal with those issues. The GC allows us to treat the vast majority of stuff as @safe, and @trusted lets us deal with the rest.

The main problem with the current situation IMHO is that slicing static arrays is not treated the same as taking the address of a local variable even though it's exactly the same thing except that the length of the array is passed along with the address. If we treat that as @system (and preferably also remove the implicit slicing of static arrays), then the main @safety hole that we currently have that I'm aware of is plugged. And if we have additional @safety holes, we can address each of those individually by treating stuff that's @system as @system instead of complicating the language further to try to treat more as @safe without needing @trusted.

The problem of course is then the folks who want to be able to do stuff like take the address of a local variable and have the compiler be smart enough to determine that that's @safe - particularly since if they're doing it correctly, it will be. However, I _really_ don't think that the complexity that comes with DIP 1000 is worth fixing that problem. Unless a D program is actively trying to not use the GC, only a small portion of the program is going to be doing stuff like taking the address of a local, and normally, taking the address of a local is restricted enough with what it's doing (since it pretty much has to be if you don't want the address to escape) that it shouldn't be hard to manually verify that no escaping is taking place. And if it is hard to verify manually, the odds are that you're going to have to cast away scope to shut up the compiler with DIP 1000 anyway, because the compiler won't have been able to verify it either.

If someone is able to come up with an alternate solution which is much simpler, then great! It would be awesome to be able to treat more stuff as @safe without needing @trusted. But as Walter has pointed out, a lot of the complexity that comes with DIP 1000 is because D is a complex language, and I'm not convinced that it's even possible to come up with a simple solution to this problem unless we do something like get rid of separate compilation so that the compiler can examine all of the code and track the lifetimes of everything (which we're obviously not going to do, since that has a very large cost in other areas like compilation times and integrating with C/++ code).

D is already able to make a lot of code memory-safe thanks to the fact that it has a GC, and I think that we should stop digging this rabbit hole that's trying to make malloc and taking the address of locals memory-safe without requiring @trusted. It's just not worth it.

Personally, every time that I've even tried to use DIP 1000, I've given up, because it's just too much of a pain. And honestly, I suspect that if DIP 1000 ever gets made the default, I will either just be casting away scope everywhere that the compiler infers it and slap @trusted on that code to shut the compiler up, or I'll just give up on making that code @safe. I'm perfectly fine with manually verifying the rare case where I need to take the address of a local variable or slice a static array, and I do _not_ want to deal with figuring out where and how I need to slap scope everywhere to make the compiler happy - especially when it's then going to start complaining about stuff that worked perfectly fine and was quite memory safe prior to scope getting involved.

- Jonathan M Davis

On 8/25/2024 6:28 PM, Jonathan M Davis wrote:
> I'm
> perfectly fine with manually verifying the rare case where I need to take
> the address of a local variable or slice a static array, and I do _not_ want
> to deal with figuring out where and how I need to slap scope everywhere to
> make the compiler happy - especially when it's then going to start
> complaining about stuff that worked perfectly fine and was quite memory safe
> prior to scope getting involved.

If you never take the address of a local, or a ref to a local, dip1000 is not going to complain about your code!

For example:
```
struct S { @safe ref int bar() { } }

@safe
int* foo(int i)
{
    S s;
    s.bar();
    return null;
}
```

compiles without error with -dip1000.

The following does error:

```
@safe int* foo(int i)
{
    return bar(&i);
}

@trusted
int* bar(int* p) { return p; }
```

```
reference to local variable `i` assigned to non-scope parameter `p` calling  `bar`
```

Perhaps that error check on a trusted function call should be suppressed.

On Monday, August 26, 2024 12:20:18 AM MDT Walter Bright via Digitalmars-d wrote:
> On 8/25/2024 6:28 PM, Jonathan M Davis wrote:
> > I'm
> > perfectly fine with manually verifying the rare case where I need to take
> > the address of a local variable or slice a static array, and I do _not_
> > want to deal with figuring out where and how I need to slap scope
> > everywhere to make the compiler happy - especially when it's then going
> > to start complaining about stuff that worked perfectly fine and was quite
> > memory safe prior to scope getting involved.
>
> If you never take the address of a local, or a ref to a local, dip1000 is not going to complain about your code!

Sure, but in the cases where I do take the address of a local, I don't want to deal with the compiler inferring anything as scope. Then I'd have to start worrying about whether the functions that I pass the pointer to are scope, and if I mark up any code as scope to make the compiler happy, that quickly becomes viral, whereas right now, I can just take the address of a local and happily pass it around without any complaints. I just have to mark the function as @trusted, and if the code is complex enough that it's not easy to see whether a pointer is escaping, then I'm just not going to do something like take the address of a local.

Where it comes up more frequently for me though is static arrays. There are plenty of cases where it makes sense to create a static array to avoid allocations and slice it to actually operate on the data when there is no need to return any such slice, so it's easy to avoid any escaping. But using DIP 1000 means that scope has to be used all over the place. Part of the reason that I gave up on doing anything with DIP 1000 and dxml is because some of the tests use static arrays and slice them, and if DIP 1000 is enabled, then scope is needed all over the place. It's not helping. It just makes it so that I have to mark up a bunch of code with scope to make the compiler happy.

I don't want to have to mark up code with scope just to handle the case where someone might decide to do something like pass in the slice of a static array with code where it's obvious whether anything is escaping. It quickly becomes viral, and I'm not at all convinced that it's actually making things safer given that if the code is complex enough that you need the compiler to tell you that stuff isn't escaping, then the compiler is probably going to fail at figuring it out somewhere in the process anyway, forcing you to cast away scope. At that point, it's better to just not do something like take the address of a local or slice a static array.

And while to an extent, you can just avoid using scope, because DIP 1000 infers scope in various places, you sometimes end up with it whether you like it or not. So, if DIP 1000 ever becomes the default, it's going to affect folks who do not want the help.

I fully expect that if DIP 1000 becomes the default, I will either be casting away scope whenever it gets inferred, or if that's enough of a pain, with my own code, I'll just give up on @safe entirely and mark everything as @system. I very much like the idea of @safe catching cases where I accidentally do something @system, but I don't want to have to mark up my code all over the place just so that the compiler can make a bit more code @safe without @trusted. The ROI isn't even vaguely there IMHO - especially with a language that already makes it possible to make the vast majority of code @safe thanks to the GC. Right now, the issue of taking the address of locals is quite isolated, because it really only affects code around the place where you take the address (at the cost of some basic thinking to make sure that you're not doing something stupid with the pointer), whereas with DIP 1000, we're going to have cases where scope gets used all over the place just in case the pointer or array that's passed in has the address of a local variable. Instead of it being just the caller's problem, it becomes a problem for every function that's being called with that pointer or slice.

And the situation is going to be particularly bad for library writers, since then even if you don't want to bother with scope, it's going to cause problems for users of your library who do want to use scope if you don't go to the extra effort of using it anywhere and everywhere that might need it for the calling code to be able to pass in something that's scope. And the fact that scope gets inferred is likely to cause problems in templated code in particular, because then anyone who uses scope in their code and passes it to your library that doesn't use scope will get it inferred in some places (and potentially in a _lot_ of places if member functions are marked with scope), causing compilation errors. It seems to me that scope risks becoming an utter nightmare with ranges in particular if anyone decides that they want to mark any functions on their ranges with scope given how common it is for range-based code to wrap ranges with other ranges using templated code.

For my own libraries, I'll probably just say tough luck and say that folks will have to cast away scope or simply not use it if they want to use my libraries, whereas if I have to deal with it for Phobos, that will significantly reduce my interest in doing work for Phobos that I don't feel that I really need to do, because it's just going to be too much of a pain to deal with.

>From where I sit, DIP 1000 is an extremely viral feature that provides
minimal benefit. If anything, I'd like to see fewer attributes in D, not more, and while scope does technically already exist, its use is quite limited without DIP 1000, whereas with DIP 1000, there's a real risk that it's going to have to be used all over the place - particularly in library code.

- Jonathan M Davis