July 18, 2019
On Wednesday, 17 July 2019 at 23:26:20 UTC, Walter Bright wrote:
> On 7/17/2019 11:59 AM, Olivier FAURE wrote:
>> The current example isn't good enough. It's not @safe (it calls free() directly), it's not RAII (the structure doesn't manage its own memory),
>
> That's not the point. A container with an @safe interface can do this, for example, simply by appending to the container contents the pointer to the old contents can be invalidated. The point of the example is to illustrate the problem clearly, not obfuscate it with the complexities of a container object.

You are not limited to one example. Showing an implementation using a common RAII pattern to illustrate why it isn't sufficient and why it requires a new feature to replace/complement it. RAII is part of the reason why you should never be calling free directly to begin with. And the borrowing feature from rust very closely resembles std::unique_ptr from C++.

> More generally, any time there are two mutable pointers to the same memory object, one can invalidate the other. This is the principle pointed out in the "Prior Work" section. The DIP is not intended to be a tutorial on this, which is why there's a link to more explanation. It has nothing to do with RAII, and thinking about it in terms of RAII will miss the generality of the concept.

Showing an example that doesn't use malloc/free or something similar to it, would be more beneficial then. As malloc/free generally is replaced by RAII and isn't/shouldn't be used directly anyways.

How does the feature differentiate between GC and non-gc allocated pointers? If I label one function @live won't this limit all pointer operations, even though it is only really relevant to non-gc pointers? This seems like two different features clashing against one another, both trying to do the same thing but interfering with one another.


July 18, 2019
On 7/18/2019 4:31 AM, Olivier FAURE wrote:
> Rust as it exists today is a vast language designed almost from the ground up to accommodate the borrowing model. It relies on several features that do not (and may never) exist in D, so saying "we're going to use a memory model like the one Rust has" is extremely vague,

The only one mutable pointer XOR one or more const pointers is not a difficult concept. A sufficient description of it is included in the DIP, and a link provided for those who want to learn more.

Also, the DIP does not say "we're going to use a memory model like the one Rust has."


> and doesn't explain how various corner cases will be handled.

Please enumerate them.

Keep in mind that the argument tracking code is already in the compiler, it was needed for DIP25 and DIP1000. This is pointed out in the DIP.


> This is especially true for this DIP, which explicitly doesn't include function-wide data flow analysis like Rust does.

Right, it's restricted to function calls.

July 18, 2019
On 7/18/2019 4:54 PM, Exil wrote:
> You are not limited to one example. Showing an implementation using a common RAII pattern to illustrate why it isn't sufficient and why it requires a new feature to replace/complement it.

One example is all that's necessary. Adding a bunch of irrelevant code to it adds nothing.

What part of the proposal are you not understanding?

(If one doesn't understand that simple example, a far more complex one won't help.)


>> More generally, any time there are two mutable pointers to the same memory object, one can invalidate the other. This is the principle pointed out in the "Prior Work" section. The DIP is not intended to be a tutorial on this, which is why there's a link to more explanation. It has nothing to do with RAII, and thinking about it in terms of RAII will miss the generality of the concept.
> 
> Showing an example that doesn't use malloc/free or something similar to it, would be more beneficial then.

Any container that does a realloc() when something is added to it will trigger the problem.


> As malloc/free generally is replaced by RAII and isn't/shouldn't be used directly anyways.

It's irrelevant if it is used directly or indirectly, as the result is the same. Since RAII is irrelevant to the issue, adding an RAII object would only serve to confuse the reader into thinking it is about RAII.

In fact, the DIP is not necessary if people strictly adhere to RAII.


> How does the feature differentiate between GC and non-gc allocated pointers?

It doesn't.


> If I label one function @live won't this limit all pointer operations, even though it is only really relevant to non-gc pointers?

@live is a separate proposal.
July 19, 2019
On Friday, 19 July 2019 at 01:32:34 UTC, Walter Bright wrote:
>
> One example is all that's necessary. Adding a bunch of irrelevant code to it adds nothing.
>
> What part of the proposal are you not understanding?
>

For me it's unclear if the examples as given would become compile time errors because it states "The checks would only be enforced for @safe code." and the examples are not @safe.

Would the examples give errors in @system code or are they just there to illustrate buggy code and the idea being these could be made @safe with future work? (or both?)

Also, does it also check simple non-heap references, for example would this become compile time error?

---
@safe void f(scope ref int a, scope ref int b)
{
     a=1;
     assert(b==1);
}
@safe void main()
{
    int i = 0;
    f(i, i);
    assert(i==1);
}
---

Horrible code, but it compiles and runs with -dip1000 -dip25 currently. https://run.dlang.io/is/XdQBzJ

If it's going to be an error, a simple example like this in the Breaking Changes and Deprecations section could help clear up whats changing from a practical sense.
July 19, 2019
On Friday, 19 July 2019 at 02:59:59 UTC, David Bennett wrote:
>
> Also, does it also check simple non-heap references, for example would this become compile time error?
>
> ---
> @safe void f(scope ref int a, scope ref int b)
> {
>      a=1;
>      assert(b==1);
> }
> @safe void main()
> {
>     int i = 0;
>     f(i, i);
>     assert(i==1);
> }
> ---
>

Actually re-reading the DIP again it's says "multiple mutable references to the same object" not "multiple references to the same mutable object" so I'm guessing my example above would still compile with -dip1021 ?

If that's so I'm guessing this would be the simplest thing that would stop compiling?

---
@safe void f(scope ref int* a, scope ref int* b)
{
    *a=1;
    a = null;
    assert(*b==1);

}
@safe void main()
{
    int i = 0;
    scope int* a = &i;
    scope int* b = &i;
    f(a, b);
    assert(i==1);
    assert(a is null);
    assert(*b==1);
}
---


July 19, 2019
On 7/18/2019 6:32 PM, Walter Bright wrote:
> On 7/18/2019 4:54 PM, Exil wrote:
>> You are not limited to one example. Showing an implementation using a common RAII pattern to illustrate why it isn't sufficient and why it requires a new feature to replace/complement it.
> 
> One example is all that's necessary. Adding a bunch of irrelevant code to it adds nothing.

But what the heck. Open the file:

https://github.com/dlang/dmd/blob/master/src/dmd/root/outbuffer.d

Look at line 408, peekSlice(). It returns a slice of data[].

Now look at line 132, writeByte(). It calls reserve(1). Line 50 reallocates data[], invalidating the pointer returned by peekSlice().

Write a function that passes both the OutBuffer reference, and the result of peekSlice(). In that function, call writeByte(). Boom.

The example in the DIP is the same thing, minus the 470 lines of other stuff.
July 19, 2019
On 7/18/2019 7:59 PM, David Bennett wrote:
> For me it's unclear if the examples as given would become compile time errors because it states "The checks would only be enforced for @safe code." and the examples are not @safe.

Any container is going to wrap a call to free() in @trusted.


> Also, does it also check simple non-heap references, for example would this become compile time error?
> 
> ---
> @safe void f(scope ref int a, scope ref int b)
> {
>       a=1;
>       assert(b==1);
> }
> @safe void main()
> {
>      int i = 0;
>      f(i, i);
>      assert(i==1);
> }
> ---

Yes, because you are passing two mutable references to the same memory.


> If it's going to be an error, a simple example like this in the Breaking Changes and Deprecations section could help clear up whats changing from a practical sense.

Since the DIP disallows passing two mutable pointers to the same memory object, any code that does will break.
July 19, 2019
On 7/18/2019 8:31 PM, David Bennett wrote:
> If that's so I'm guessing this would be the simplest thing that would stop compiling?
> 
> ---
> @safe void f(scope ref int* a, scope ref int* b)
> {
>      *a=1;
>      a = null;
>      assert(*b==1);
> 
> }
> @safe void main()
> {
>      int i = 0;
>      scope int* a = &i;
>      scope int* b = &i;
>      f(a, b);
>      assert(i==1);
>      assert(a is null);
>      assert(*b==1);
> }
> ---

That will still compile, because as the DIP says, it only looks at the function call, it is not doing data flow analysis on preceding or succeeding statements.

July 19, 2019
Consider this contrived example:

someLibrary.d
-------------
module someLibrary;

import core.stdc.stdlib;

int a;
int b;

ref int foo() @safe
{
    if ((rand() % 2) == 0)
    	return a;
    else
    	return b;
}

ref int bar() @safe
{
    if ((rand() % 4) == 0)
    	return a;
    else
    	return b;
}

main.d
------
import someLibrary;

void doSomething(scope ref int a, scope ref int b) @safe
{
    // whatever...
}

void main() @safe
{
    doSomething(foo(), bar());
}

How can the compiler statically determine whether `foo` and `bar` will return a reference to the same data?

It seems the language or type system must provide this guarantee, which would require a full-fledged ownership/borrowing system to accurately enforce what this DIP is proposing.

July 19, 2019
On Friday, 19 July 2019 at 07:59:14 UTC, Walter Bright wrote:
> https://github.com/dlang/dmd/blob/master/src/dmd/root/outbuffer.d
>
> Look at line 408, peekSlice(). It returns a slice of data[].
>
> Now look at line 132, writeByte(). It calls reserve(1). Line 50 reallocates data[], invalidating the pointer returned by peekSlice().
>
> Write a function that passes both the OutBuffer reference, and the result of peekSlice(). In that function, call writeByte(). Boom.

Alright, I'm going to cut to what I think is the heart of the problem.

    void someFunction()
    {
        auto buffer = OutBuffer(someData);
        auto slice = buffer.peekSlice();

        buffer.writeByte();
        slice[0] = 42;          // Boom
    }

As you're aware, nothing about your DIP stops the above case from happening. So I don't understand what the point of this DIP even is, given that for every memory leak that you your DIP prevents, I could give you another way to express it so that your DIP lets it pass.

It doesn't close a loophole in @safe, because the core problem is that writeByte() isn't actually @safe; presumably, your function that takes the OutBuffer reference and the result of peekSlice() wouldn't be @safe either, so it wouldn't even be stopped by your DIP.

> The example in the DIP is the same thing, minus the 470 lines of other stuff.

Nobody is asking you to write a 470-lines example. People are asking you to write at least one example that would behave differently with your DIP than it behaves now.

Like, you insist on that, like the fact that we're asking for examples means we don't understand your DIP and thus we're not fit to judge it, or like we're being unreasonable and we're only asking because we're being capricious or something.

Speaking personally, I'm not asking for examples because I don't understand your proposal. The reason I'm asking for examples is that examples are a good medium to support communication, and to suss out the deeper reasons people might disagree about an idea.

I'm pretty sure I do understand your proposal, and I think it's fundamentally flawed. I think you're trying to gradually implement the Rust memory model, and you think that non-aliased function arguments are a good first step, that will help determine how a full implementation should be rolled out. I think that approach is unworkable, because the Rust memory model is kind of an all-or-nothing feature, and gradually implementing it like you intend isn't really feasible; I think DIP 1021, if it passes, will barely be noticed by anyone, except as an annoyance, and won't bring any useful feedback for designing a more coherent memory model. Because, again, DIP 1021 doesn't actually bring any additional memory safety, because "use after free" in real code almost never occurs in a way that could be stopped by DIP 1021.