On Thursday, 19 September 2019 at 10:17:43 UTC, Walter Bright wrote:
> A bit of friendly advice - berating me is not a great way to sell me on your ideas. My natural reaction to that is to look hard for ways to reject it. If your idea really is better, then you lose, I lose, and the D community loses.

A bit of friendly advice: if you don't want people to berate you, don't spend months ignoring things they're trying to communicate to you, especially when they are putting significantly more visible effort analyzing your points than you put into writing them.

Seriously, I don't care. I've spent months trying to communicate the flaws of your proposal to you. I did it with as much politeness and respect as someone can have while criticizing someone else's plans. I wrote examples, counter-examples, tried to understand your general goal, tried to formulate my points as clearly as I could, wrote a beginning of a counter-proposal (which I talked to you about, this isn't anything new), etc.

I'm pretty sure I have put more effort into communicating with you than any contributor on this forum for the last two years. You've ignored or dismissed everything I said.

So, I don't want to be hostile, I don't want to be rude, and I try to say this with stoicism.

But I maintain what I said earlier: you make it really, *really* difficult and painful to communicate with you. From what I've seen and hear from D contributors, I'm not isolated in that sentiment.

On Thursday, 19 September 2019 at 09:42:06 UTC, Walter Bright wrote:
> D's transitive const has also proved difficult to retrofit into existing code (like the DMD source code).

> That's right, it won't be. Just like if you call an @trusted function from an @safe one, it won't be safe, either (at least the compiler can't check it for you).

I think those two comparisons are interesting, because @trusted and const have different semantics, as far incrementally fitting them goes.

const (and @pure) starts from the "leaves", the small functions that don't call anything else. As your refactoring progresses, more and more functions become pure, closer to the root, until ideally everything but your main() is pure.

@trusted, on the other hand, is something you're trying to get rid of. You really, really don't want a codebase where half your code is marked as @trusted, because that means half your code needs to be audited to find any potential memory vulnerability.

The way you've described @live, it would be something between the two. Like, it's an attribute you add to functions that you incrementally convert to the new paradigm; but it's also something you really don't want half your code to have, because then you need to manually audit every non-@live code interacting with @live code to make sure memory corruption doesn't happen.

Not sure if I'm explaining this clearly.

> > https://gist.github.com/PoignardAzur/9896ddb17b9f6d6f3d0fa5e6fe1a7088
>
> I haven't studied that. Is it a 100% solution? I know there's one being worked on for C++, but it's an 80-90% solution.

It's 100% memory-safe, transitive, mechanically checkable, malicious-non-@trusted-code proof.

I've been trying to nail down the semantics, and I think[1] it requires lifetime annotations (I need to take the time to write a demonstration why at some point), but that's nothing that wouldn't apply to the @live proposal as well.

[1] https://forum.dlang.org/thread/qssaruktegnbtsdjeyri@forum.dlang.org

On Thursday, 19 September 2019 at 09:42:06 UTC, Walter Bright wrote:

> > [1]
> https://gist.github.com/PoignardAzur/9896ddb17b9f6d6f3d0fa5e6fe1a7088
>
> I haven't studied that. Is it a 100% solution? I know there's one being worked on for C++, but it's an 80-90% solution.
>
> D needs to have a 100%, mechanically checkable, solution.
>
> We have 100% for transitive const and function purity. It's hard to get code to pass, but worth it when one succeeds. The Ownership/Borrowing solution is 100% which is what is appealing about it.

There are two 100% solutions that I am aware of.  One is Rust's borrow checker, which I'm assuming is already familar to all, and the other is Pony's reference capabilities [1].  As the ownership and borrowing vision for D is being polished, I think it would be worthwhile to become familiar with Pony's model even if only to provide contrast and solidify existing ideas.

Graydon Hoare [2] and Niko Matsakis [3], both rock stars in the Rust community, provided some interesting commentary on Pony's model.

[1] - https://tutorial.ponylang.io/reference-capabilities/reference-capabilities.html
[2] - https://www.reddit.com/r/rust/comments/34rszb/pony_type_and_memory_safe_language/cqxkmm8/
[3] - https://www.reddit.com/r/rust/comments/34rszb/pony_type_and_memory_safe_language/cqxvxky/

I'm really excited about where you're going with all of this, but this DIP does seem like an odd thing to do in isolation from a complete ownership/borrowing model.  It still seems to me that a complete ownership/borrowing/reference-capabilities solution, in the general case, would already handle the use cases addressed by this DIP.

Mike

On Thursday, 19 September 2019 at 02:54:57 UTC, Walter Bright wrote:
> On 9/18/2019 5:15 AM, Olivier FAURE wrote:
>> Any proposal which doesn't have data flow analysis won't provide actual memory safety.
>
> We plug the holes one by one.

Lol, this sounds like a sinking ship. I like your reference.

On 9/19/2019 6:26 AM, Mike Franklin wrote:
> On Thursday, 19 September 2019 at 09:42:06 UTC, Walter Bright wrote:
> 
>> > [1]
>> https://gist.github.com/PoignardAzur/9896ddb17b9f6d6f3d0fa5e6fe1a7088
>>
>> I haven't studied that. Is it a 100% solution? I know there's one being worked on for C++, but it's an 80-90% solution.
>>
>> D needs to have a 100%, mechanically checkable, solution.
>>
>> We have 100% for transitive const and function purity. It's hard to get code to pass, but worth it when one succeeds. The Ownership/Borrowing solution is 100% which is what is appealing about it.
> 
> There are two 100% solutions that I am aware of.  One is Rust's borrow checker, which I'm assuming is already familar to all, and the other is Pony's reference capabilities [1].  As the ownership and borrowing vision for D is being polished, I think it would be worthwhile to become familiar with Pony's model even if only to provide contrast and solidify existing ideas.
> 
> Graydon Hoare [2] and Niko Matsakis [3], both rock stars in the Rust community, provided some interesting commentary on Pony's model.
> 
> [1] - https://tutorial.ponylang.io/reference-capabilities/reference-capabilities.html
> [2] - https://www.reddit.com/r/rust/comments/34rszb/pony_type_and_memory_safe_language/cqxkmm8/ 
> 
> [3] - https://www.reddit.com/r/rust/comments/34rszb/pony_type_and_memory_safe_language/cqxvxky/

Thanks for the references.


> I'm really excited about where you're going with all of this, but this DIP does seem like an odd thing to do in isolation from a complete ownership/borrowing model.  It still seems to me that a complete ownership/borrowing/reference-capabilities solution, in the general case, would already handle the use cases addressed by this DIP.

An OB system would build on and extend this DIP, not replace it. It's the next step in:

1. DIP25 built on and extended the existing checks for returning the address of a local.

2. DIP1000 built on and extended DIP15 by adding checking for pointers, not just references.

3. DIP1021 built on and extended DIP1000 by adding the notion of OB for function parameters. Look at the implementation of DIP1021 - it is most definitely leveraging the code for 1 and 2.

A full OB system will leverage the code added for 3, too.

DIP1021 also has the advantage of trying out OB on a limited scale, and gently introducing D users to the concepts behind it.

Code modified to work with DIP25, DIP1000, and DIP1021 will not need a do-over for OB, it'll just be further down the same road. As you're well aware, upgrading Phobos to DIP1000 was a bit of a struggle, but I hope you'll agree it was well worth it. I suspect that going from pre-DIP25 to OB in one step would be too wrenching of a change.

On 9/19/2019 5:43 AM, Olivier FAURE wrote:
> On Thursday, 19 September 2019 at 09:42:06 UTC, Walter Bright wrote:
>> D's transitive const has also proved difficult to retrofit into existing code (like the DMD source code).
> 
>> That's right, it won't be. Just like if you call an @trusted function from an @safe one, it won't be safe, either (at least the compiler can't check it for you).
> 
> I think those two comparisons are interesting, because @trusted and const have different semantics, as far incrementally fitting them goes.
> 
> const (and @pure) starts from the "leaves", the small functions that don't call anything else. As your refactoring progresses, more and more functions become pure, closer to the root, until ideally everything but your main() is pure.
> 
> @trusted, on the other hand, is something you're trying to get rid of. You really, really don't want a codebase where half your code is marked as @trusted, because that means half your code needs to be audited to find any potential memory vulnerability.

I've been looking into making the DMD source code @safe. Because of all the interconnected dependencies, it's looking like the only way is to make everything @trusted, and then piece by piece make it @safe.

Because DMD does lazy semantic analysis (for very good reasons), going const has been a frustrating experience and is unlikely to ever happen.


> The way you've described @live, it would be something between the two. Like, it's an attribute you add to functions that you incrementally convert to the new paradigm; but it's also something you really don't want half your code to have, because then you need to manually audit every non-@live code interacting with @live code to make sure memory corruption doesn't happen.

Yes, but code must have a transition path. DMD would still be in C++ if I couldn't translate it function by function, keeping the entire codebase working and passing the test suite at each step. I did a number of ugly things to make it happen - but now that it's in D I gradually go back and fix the compromises.

It's like how the first intercontinental railroad was built. It was all quick-and-dirty, slap dash, unsafe, and ramshackle. It was also built entirely with muscle. But once it was done, and trains could be run on it, suddenly it was incredibly useful, money was coming in, etc., and piece by piece the crappy construction was fixed. It could not have been built otherwise.


> Not sure if I'm explaining this clearly.
> 
>> > https://gist.github.com/PoignardAzur/9896ddb17b9f6d6f3d0fa5e6fe1a7088
>>
>> I haven't studied that. Is it a 100% solution? I know there's one being worked on for C++, but it's an 80-90% solution.
> 
> It's 100% memory-safe, transitive, mechanically checkable, malicious-non-@trusted-code proof.
> 
> I've been trying to nail down the semantics, and I think[1] it requires lifetime annotations (I need to take the time to write a demonstration why at some point), but that's nothing that wouldn't apply to the @live proposal as well.
> 
> [1] https://forum.dlang.org/thread/qssaruktegnbtsdjeyri@forum.dlang.org

Is there a language that uses this, or is it entirely new?

On Thursday, 19 September 2019 at 20:34:47 UTC, Walter Bright wrote:
> [snip]
> Is there a language that uses this, or is it entirely new?

I understood it as being a mixture of pony’s reference capabilities and Rust’s borrow checker, but I could be wrong.

I don’t think the addition of DIP 1021 precludes the future addition of unique the way I read it. The addition of unique might need to take DIP 1021 into account.

On 19.09.19 11:09, Olivier FAURE wrote:
> 
> No, Rust isn't the only possible model for memory safety. Reference-counting isn't the only possible memory-safe data structure. The "foobar(rcObject, &rcObject.data)" problem that has been floated around for a few years isn't the only challenge to implementing a safe GC-less model.
> 
> The reason I'm strongly opposed to this DIP, is that it should be based on a thorough analysis of possible memory safety models, advantages of different models over each other, and what model would be most suitable for D and why.
> 
> Instead, it acts like the analysis was already done and everybody agreed the Rust model was the best and all that's left is implementation details.

I'd be happy and excited if that was the extent of the problems! @live is not the Rust model! Rust does not have a language built-in notion of unique pointer, and it is not necessary (nor sufficient) to have it if your borrowing works. Mixing up GC pointers and unique pointers is a terrible idea. Why is @live trying to _remove_ GC from D?

On 19.09.19 12:17, Walter Bright wrote:
> A bit of friendly advice - berating me is not a great way to sell me on your ideas. My natural reaction to that is to look hard for ways to reject it. If your idea really is better, then you lose, I lose, and the D community loses.
> 
> A much better method is to start by:
> 
> 1. explaining that your method meets the requirement of 100% mechanically checkable memory safety, that it's not just a collection of "best practices".
> ...

Your own answer to this was "we plug the holes one by one". Do you see the double standard? Anyway, I have actually done this but you just ignored the explanation and went back to claiming that my suggestions do not meet this requirement. I simply can't spend tens of hours in debates every week anymore.

> 2. pointing to an existing implementation in another language that proves point (1) would be very helpful.
> ...

You are pointing to Rust, but you are not doing what Rust does. And anyway, Rust does not support GC. Do you really want to remove GC from D?

> ...
> 
> P.S. I've encouraged people for the last 10 years to submit proposals on how to make D memory safe. Nothing happened but a few hand-wavy suggestions. I gave up waiting, now I'm moving forward one way or another. If that's what it takes to get people motivated to come up with better proposals, that's good!

The issue is that you make it seem like you are unable to recognize valid criticism, let alone better (i.e., non-terrible) proposals. I don't really have time to create a polished DIP and I'm not even sure you would acknowledge it as such. All I can say is that I'll severely regret my investment in D if @live moves forward. This is not at all where I expected the language to go. Where is the pragmatism?

On 20.09.19 03:17, Timon Gehr wrote:
> 
> 
> P.S. I've encouraged people for the last 10 years to submit proposals on how to make D memory safe.

BTW: The problem is not that D is not memory safe. The "problem" is that D relies on GC to be memory safe. There may be some @trusted code that uses manual memory management and violates @safe D memory safety, but that is not the fault of the language specification. To solve the problem you need to give more tools to this currently broken @trusted code so it can restrict @safe code suitably in it's usage patterns, not destroy GC and all the actually @safe code that has been written so far.