Yes, a bad allocator is still a bad allocator. There is nothing we can do to guard against that. Only something like address sanitizer could prevent bad things from happening.

Unfortunately there is also nothing stopping the implementation in phobos or libc from doing the same thing either. Its not really worth considering at this level. Either by mistake or on purpose a memory allocator can corrupt memory without the D compiler being able to discover it, @safe has nothing to do with it.

On Monday, 23 January 2023 at 08:49:50 UTC, Richard (Rikki) Andrew Cattermole wrote:
> Basically right now we're missing the lifetime checks surrounding borrowed & function parameter. Everything else is do-able right now, even if it isn't as cheap as it could be (like RC eliding).

Have you seen the borrow method [1] used by SafeRefCounted? It is already possible, in the current D language, to prevent a container or smart pointer from leaking references. The syntax is awkward, because you have to use a callback, but it can be done.

Lifetime issues are not the blocker here. The blocker is being able to give deallocate/free a safe interface, so that it can be used safely in a generic or polymorphic context, where the specific implementation is not known in advance.

[1]: https://dlang.org/library/std/typecons/borrow.html

On Monday, 23 January 2023 at 16:39:07 UTC, Richard (Rikki) Andrew Cattermole wrote:
> Yes, a bad allocator is still a bad allocator. There is nothing we can do to guard against that. Only something like address sanitizer could prevent bad things from happening.
>
> Unfortunately there is also nothing stopping the implementation in phobos or libc from doing the same thing either. Its not really worth considering at this level. Either by mistake or on purpose a memory allocator can corrupt memory without the D compiler being able to discover it, @safe has nothing to do with it.

Please read the original thread linked in Atila's first post. It is not very long, and I responded to these exact objections in that thread already.

On 24/01/2023 5:39 AM, Paul Backus wrote:
> On Monday, 23 January 2023 at 08:49:50 UTC, Richard (Rikki) Andrew Cattermole wrote:
>> Basically right now we're missing the lifetime checks surrounding borrowed & function parameter. Everything else is do-able right now, even if it isn't as cheap as it could be (like RC eliding).
> 
> Have you seen the borrow method [1] used by SafeRefCounted? It is already possible, in the current D language, to prevent a container or smart pointer from leaking references. The syntax is awkward, because you have to use a callback, but it can be done.

Yes I'm aware of this method. I've talked about it with Robert Schadek during last DConf Online who argued that this is the only way forward. This is not the way forward as far as I'm concerned. It is a major step backwards in usability as it is not how people work with arrays or data types in general.

> Lifetime issues are not the blocker here. The blocker is being able to give deallocate/free a safe interface, so that it can be used safely in a generic or polymorphic context, where the specific implementation is not known in advance.

I can't agree with that. We need to move people away from calling into allocators directly! They are an advanced concept, that is easy to get wrong especially when creating them. Which is what composing them like std.experimental.allocators does.

Ultimately, I think its ok for allocators to not be @safe, they are simply too easy to get wrong without any way to prevent this at the compiler level. You use them when you want to do something more advanced without any hand holding. Use data structures like a vector type to make it @safe instead, which is why I think lifetime is the only thing blocking it atm.

On 24/01/2023 5:41 AM, Paul Backus wrote:
> Please read the original thread linked in Atila's first post. It is not very long, and I responded to these exact objections in that thread already.

Yeah I read it at the time, but did so again at your request.

On Monday, 23 January 2023 at 16:59:16 UTC, Richard (Rikki) Andrew Cattermole wrote:
> On 24/01/2023 5:39 AM, Paul Backus wrote:
>> Have you seen the borrow method [1] used by SafeRefCounted? It is already possible, in the current D language, to prevent a container or smart pointer from leaking references. The syntax is awkward, because you have to use a callback, but it can be done.
>
> Yes I'm aware of this method. I've talked about it with Robert Schadek during last DConf Online who argued that this is the only way forward. This is not the way forward as far as I'm concerned. It is a major step backwards in usability as it is not how people work with arrays or data types in general.

I agree that it's bad UX, but what's the alternative? Implement a borrow checker in D? It'll take 5-10 years and won't even work properly when it's done. At that point, you may as well just tell people to switch to Rust.

>> Lifetime issues are not the blocker here. The blocker is being able to give deallocate/free a safe interface, so that it can be used safely in a generic or polymorphic context, where the specific implementation is not known in advance.
>
> I can't agree with that. We need to move people away from calling into allocators directly! They are an advanced concept, that is easy to get wrong especially when creating them. Which is what composing them like std.experimental.allocators does.
>
> Ultimately, I think its ok for allocators to not be @safe, they are simply too easy to get wrong without any way to prevent this at the compiler level. You use them when you want to do something more advanced without any hand holding. Use data structures like a vector type to make it @safe instead, which is why I think lifetime is the only thing blocking it atm.

I am not advocating for typical D users to call into allocators directly. The people who benefit from allocators having a @safe interface are authors of generic container and smart pointer libraries.

It is currently *impossible* to write a @safe vector type that accepts a user-supplied allocator. You can only do it if you hard-code a dependency on a specific allocator (or a specific predefined set of allocators) whose behavior you know in advance.

If your argument is that we should do exactly that, and simply give up on supporting user-supplied allocators entirely, then I can accept that as a reasonable position. Certainly it would be the quickest and easiest way to un-block progress on containers, and we can always revisit the issue in the future if necessary.

On 24/01/2023 6:32 AM, Paul Backus wrote:
> On Monday, 23 January 2023 at 16:59:16 UTC, Richard (Rikki) Andrew Cattermole wrote:
>> On 24/01/2023 5:39 AM, Paul Backus wrote:
>>> Have you seen the borrow method [1] used by SafeRefCounted? It is already possible, in the current D language, to prevent a container or smart pointer from leaking references. The syntax is awkward, because you have to use a callback, but it can be done.
>>
>> Yes I'm aware of this method. I've talked about it with Robert Schadek during last DConf Online who argued that this is the only way forward. This is not the way forward as far as I'm concerned. It is a major step backwards in usability as it is not how people work with arrays or data types in general.
> 
> I agree that it's bad UX, but what's the alternative? Implement a borrow checker in D? It'll take 5-10 years and won't even work properly when it's done. At that point, you may as well just tell people to switch to Rust.

A lot of it is already done with DIP1000.

When you call a function and pass a borrowed value in, that part is complete.

What we are missing is the initial borrow action and guaranteeing it being tied to another object in terms of life.

This only really needs to cover within a function body, so the DFA here should actually be really easy.

Perhaps something like this (note ref not actually required for pointer types):

```d
struct Thing(T) {
	ref T get() scope { ... }
}

{
	Thing thing;
	scope ref got = thing.get; // owner = thing
	func(got); // ok parameter is scope

	thing = Thing.init;// Error: thing must out live got variable, but thing is being assigned to.
	return got; // Error: scope variable thing must out live got variable, but got is being returned.
}

void func(scope ref T value) {}
```

For the rest, I'm glad that we are converging on a possible position :)