On 10/23/2022 4:55 PM, Timon Gehr wrote:
> Missed a 0 there.

We forgive you :-) since missing a 0 only matters on one's paycheck.

On 10/23/2022 2:27 PM, Per Nordlöw wrote:
> Would be interesting to hear Walter's opinion on built-in sum-types and obstacles in implementing them in dmd.

I haven't thought much about them yet.

On 24.10.22 02:29, Walter Bright wrote:
> On 10/23/2022 4:54 PM, Timon Gehr wrote:
>> As soon as people try to use it in production they will run into some issues with expressiveness. For example, DIP1000 does not even support storing objects with different lifetimes in different fields of the same struct. (Real case that came up on Discord today.)
> 
> That's right. DIP1000 does not track lifetimes at all.

It kind of does a bit with return annotations:

```d
int* foo(return scope int* x)@safe{
    auto y=x;
    int t;
    auto z=&t;
    return y; // ok, but can't return z
}
```

> It only does scoped lifetimes.

My use cases with storing references in structs and region allocators can be dealt with with scoped lifetimes only, but they still do not work with DIP1000.

In any case, I'd also consider that lifetime tracking, it's just not very precise. The main problem though is lack of modularity. In the example above, `y` and `z` are different, and the compiler understands that they are different within that one function, but there is no way to preserve that difference when passing both of them though a function further down the stack at the same time.

> Tracking requires data flow analysis, which is in the purview of @live.
> ...

I guess my issue here is that @live, while it may use some methods that may be helpful in another context, does not enable any of the use cases I described either. It does not make @safe any safer and it also does not make DIP1000 any more modular.

>> The underlying issue is that DIP1000 lacks a modular way to track different lifetimes. E.g., DIP100 does not allow having an array allocated on a region allocator, containing references pointing to objects allocated on a distinct region allocator with correct lifetime tracking. As soon as you store things in the array and want to get them back out, if it works at all, their lifetimes will be limited by the lifetime of the region allocator that backs the array storage, even if their own allocator actually is longer-lived.
> 
> That's right.
> 
> 
>> DIP1000 is very keen to conflate different lifetimes and to cut the longer one off based on the shorter one.
> 
> Yup.
> 
> DIP1000 does not handle what you describe, and isn't designed to. However, it is still very useful for the vast bulk of cases.

I agree that it is quite useful for some cases. It just does not seem to offer all that much yet in terms of actually enabling safe manual memory management for the entire application.

> It's also carefully set up to be conservative, i.e. it will always err on the side of safety.
> 
> To loosen up the safety is the job of @trusted containers, similar to Rust's unsafe code blocks, because Rust's expressiveness is limited, too.

Yes, it is also limited (additional safety guarantees always come with some limitations), but it can do everything I brought up above. Furthermore, the unsafe blocks can usually be hidden away in libraries with expressive interfaces that are able to distinguish distinct lifetimes pretty well.

On 10/23/2022 6:11 PM, Steven Schveighoffer wrote:
> This seems like a goofy workaround, still makes me feel like the rejection in @trusted code is a bug.

Yes, it looks like a bug. Please file it and tag it with the 'safe' keyword.

On 10/23/2022 6:50 PM, Timon Gehr wrote:
> It kind of does a bit with return annotations:
> 
> ```d
> int* foo(return scope int* x)@safe{
>      auto y=x;
>      int t;
>      auto z=&t;
>      return y; // ok, but can't return z
> }
> ```

It isn't actually tracking lifetimes, it just copies the attributes from x to y. z gets the scope attribute because it is initialized with the address of a stack variable.


> I guess my issue here is that @live, while it may use some methods that may be helpful in another context, does not enable any of the use cases I described either. It does not make @safe any safer and it also does not make DIP1000 any more modular.

@live's purpose is to prevent:

1. use after free
2. multiple free's
3. no free

To go further than that will require the user to construct a container that encapsulates whatever it does, and is likely going to include some @trusted functionality. Which is more or less what Rust does.


> I agree that it is quite useful for some cases. It just does not seem to offer all that much yet in terms of actually enabling safe manual memory management for the entire application.

Without the 1,2,3 above, you're right that DIP1000 is not a complete solution. But it is a necessary precondition for doing 1,2,3.

On Sunday, 23 October 2022 at 23:02:41 UTC, Timon Gehr wrote:
> On 10/23/22 09:08, Imperatorn wrote:
>> On Sunday, 23 October 2022 at 02:42:53 UTC, Mike Parker wrote:
>>> On Sunday, 23 October 2022 at 01:02:16 UTC, Paul Backus wrote:
>>>
>>>> My prediction: if Walter gets bored of ImportC and decides to make pattern matching his next project, we'll get it. If he doesn't, we won't.
>>>>
>>>
>>> Walter's primary focus right now is shoring up DIP 1000.
>> 
>> That's good to know. Would be great news if Dip1000 would be finalized ☀️
>
> I don't expect it to be "finalized" very soon. It lacks expressiveness, which will be improved by small, incremental changes. Which also means it won't be stable all that soon.

But, he's working on it ;)

"hope is the last thing tha..."

On 10/24/22 06:26, Walter Bright wrote:
> On 10/23/2022 6:50 PM, Timon Gehr wrote:
>> It kind of does a bit with return annotations:
>>
>> ```d
>> int* foo(return scope int* x)@safe{
>>      auto y=x;
>>      int t;
>>      auto z=&t;
>>      return y; // ok, but can't return z
>> }
>> ```
> 
> It isn't actually tracking lifetimes, it just copies the attributes from x to y. z gets the scope attribute because it is initialized with the address of a stack variable.
> ...

Well, yes, this is essentially how to track lifetimes.

Right now, there are two kinds of lifetimes (attached to declaration using annotations). Some lifetimes outlive the current function and others are strictly lexical.

To get more expressiveness, you could:

- have more than one lifetime that exceeds the current function's lifetime. I.e., multiple distinct return annotations, ideally an arbitrary number, by allowing the return annotation to be parameterized by a lifetime variable.

- have such parameterized scope return annotations on fields of aggregates.

None of this requires a fundamentally different approach than what DIP1000 does. What's missing is:

- a way to parameterize functions and aggregates that is completely erased at runtime (generally useful, not only for lifetimes; e.g., this is how to fix `inout`.)
- return storage class parameterized by a lifetime variable

By default, people write their code in the current DIP1000 subset, but library authors get more expressiveness to build containers that actually work well with DIP1000.

> 
>> I guess my issue here is that @live, while it may use some methods that may be helpful in another context, does not enable any of the use cases I described either. It does not make @safe any safer and it also does not make DIP1000 any more modular.
> 
> @live's purpose is to prevent:
> 
> 1. use after free
> 2. multiple free's
> 3. no free
> ...

Yes, as some sort of linter within @system/@trusted code. DIP1000 is for @safe code. (I don't really see why @system and @trusted code should have access to more static analysis muscle that @safe code, but it's not even strictly needed in order to make DIP1000 vastly more expressive.)

I think there is currently more demand for a type system that does not get in your way in @safe code than for a linter that helps you get malloc/free in @system/@trusted code right.

> To go further than that will require the user to construct a container that encapsulates whatever it does, and is likely going to include some @trusted functionality. Which is more or less what Rust does.
> ...

Yes, but the magic is in expressive interfaces that actually allow the implementation to be @trusted because the type checker can correctly restrict @safe code in a way that allows the @trusted code to make assumptions about the behavior of said @safe code.

Note that @live loses all guarantees at interface boundaries, but those are what actually matters for modular safety in the style of Rust. DIP1000 does not lose guarantees at interface boundaries, which means this is the more promising approach that people will want to extend.

> 
>> I agree that it is quite useful for some cases. It just does not seem to offer all that much yet in terms of actually enabling safe manual memory management for the entire application.
> 
> Without the 1,2,3 above, you're right that DIP1000 is not a complete solution. But it is a necessary precondition for doing 1,2,3.
> 

DIP1000+@live is also not a "complete solution". One (less important) reason is because @live does not address 1,2,3 in @safe code. Another (more important) reason is that DIP1000 is not really expressive enough to create some common containers encapsulating @trusted functionality. (E.g., region allocators.)

I agree with the abstract motivation for DIP1000, it's just that I predict that the actual implementation will in practice be too restrictive to achieve the stated goals.