Hi there,

I have been through the discussions on the forum regarding scope, ownership, GC and so on.  After a serious thought about this, I'd like to discuss about a possible solution for this matter. My concern is to know if my solution could work and to discuss the possible impact on the current D language.


After watching the Dconf 2018 session about "Safe Memory Management", I told myself that, if D can guarantee an exception in the case we delete an already deleted object, then it's a major step forward. So let's assume that.

Now, let's distinguish two kind of references, one being "owner" of the object it targets, and other that are smooth references to the same object.

We could say that a given object will always belong to one and only one owner. This owner will be in charge of deleting the object.

There will be 2 cases. If the owner is a local variable, as soon as the variable is out of scope, the object is deleted. If the owner is an attribute of a extra object, the deletion happens after the extra object is deleted.

The syntax could look like it (don't pay much attention to the key word I use, it's only to illustrate).


void main()
{
   auto r := new Rectangle(5,6); // r owns the object.
   auto r4 := new Rectangle(8,7); // r4 owns the object.
   auto f = new Foo();

   // now r2 owns the object. Deletion still happens when r2 is out of scope.
   auto r2 := r;

   auto r3 = r2; // Smooth reference

   // fonction1 only borrow the object, so it will not delete it.
   fct1( r2 );

   auto rr := f.fct2( in r2 );

   // We can STILL use r3 at our own risk (we don't know the r2's destiny).
   // Considering the design and UT will prevent bugs.

   auto w = r3.getWidth(); // An exception is raised if r3 is not valid.

   f.fct3( in r2 ); // Compilation error, we cannot give what we no longer own.

   f.fct3( in rr ); // Ok.

   // implicit code :
   // delete r4;
   // delete f , then f deletes rr by owership.
}

class Foo
{
   Rectangle my_r;

   out Rectangle fct2( in Rectangle r )
   {
       return out r;
   }

   fct3( in Rectangle r )
   {
      this.my_r := r; // Foo is the owner of r now.
   }

}

This way, all the libraries would explicitly tell where the memory is managed. All classes could keep their attribute objects under management if they are designed to.

I cannot see a case where we could miss a memory deallocation because the compiler will add the delete for us at the expected spot.

Of course, we can easily run into invalid smooth references (kind of NPE). But if D provides safe memory management, well, we can handle it.

Tell me guys what you think :)

The message above is repost of :

https://forum.dlang.org/post/pfjotkcazuiuhlvzijih@forum.dlang.org

So I can reply to Chris M. here.

------------------------------------------------------------------

Yes, Chris, I got inspired by Rust :)  But Rust goes too far and it lowers the productivity. Nevertheless, it demonstrates that we can make the compiler do a great deal of things regarding MM.
I have also got inspired by how we could make "a better C" and the main issue in C is that it's easy to get lost in MM responsibility.

Regarding D, the problem is that if you use the GC, you are in competition with Go. So it would be vital for D to get rid of the GC (or at least, make it optional, even with Phobos).

I will go deeper and look at dip25 and dip1000.

On Wednesday, 11 July 2018 at 22:59:50 UTC, xray wrote:
> The message above is repost of :
>
> https://forum.dlang.org/post/pfjotkcazuiuhlvzijih@forum.dlang.org
>
> So I can reply to Chris M. here.
>
> ------------------------------------------------------------------
>
> Yes, Chris, I got inspired by Rust :)  But Rust goes too far and it lowers the productivity. Nevertheless, it demonstrates that we can make the compiler do a great deal of things regarding MM.
> I have also got inspired by how we could make "a better C" and the main issue in C is that it's easy to get lost in MM responsibility.
>
> Regarding D, the problem is that if you use the GC, you are in competition with Go. So it would be vital for D to get rid of the GC (or at least, make it optional, even with Phobos).
>
> I will go deeper and look at dip25 and dip1000.

I feel the following should be disallowed, since we've moved some checking to runtime. Ideally this system would all happen at compile-time.

auto r3 = r2; // Smooth reference
auto w = r3.getWidth(); // An exception is raised if r3 is not valid.

On Thursday, 12 July 2018 at 14:13:25 UTC, Chris M. wrote:
> On Wednesday, 11 July 2018 at 22:59:50 UTC, xray wrote:
>> The message above is repost of :
>>
>> https://forum.dlang.org/post/pfjotkcazuiuhlvzijih@forum.dlang.org
>>
>> So I can reply to Chris M. here.
>>
>> ------------------------------------------------------------------
>>
>> Yes, Chris, I got inspired by Rust :)  But Rust goes too far and it lowers the productivity. Nevertheless, it demonstrates that we can make the compiler do a great deal of things regarding MM.
>> I have also got inspired by how we could make "a better C" and the main issue in C is that it's easy to get lost in MM responsibility.
>>
>> Regarding D, the problem is that if you use the GC, you are in competition with Go. So it would be vital for D to get rid of the GC (or at least, make it optional, even with Phobos).
>>
>> I will go deeper and look at dip25 and dip1000.
>
> I feel the following should be disallowed, since we've moved some checking to runtime. Ideally this system would all happen at compile-time.
>
> auto r3 = r2; // Smooth reference
> auto w = r3.getWidth(); // An exception is raised if r3 is not valid.

If we disallow the use of smooth references, we fall into the same paradigm as Rust. Then it's going to be hard to implement data structures with many references to the same object and the language becomes less flexible.

But yes, I assume we are in Safe Memory Management so that we can do :

if ( isValidRef(r3) ) {
   auto w = r3.getWidth();
}

Also, I have started to look at the dip-1000. At first glance, "scope" is an approach that makes sense but it does not seem to fit with the "ownership" concept that I suggest, ...unless someone has a brilliant idea to reconcile all those concepts.

On Wednesday, 11 July 2018 at 22:46:45 UTC, xray wrote:
> [ .. ]
>
> After watching the Dconf 2018 session about "Safe Memory Management", I told myself that, if D can guarantee an exception in the case we delete an already deleted object, then it's a major step forward. So let's assume that.

This can be done with a library solution.

> Now, let's distinguish two kind of references, one being "owner" of the object it targets, and other that are smooth references to the same object.

> We could say that a given object will always belong to one and only one owner. This owner will be in charge of deleting the object.

So something like this?

https://github.com/atilaneves/automem/blob/master/source/automem/unique.d

> void main()
> {
> [...]
> }

The only thing I got from this are that "smooth references" are like Rust's borrows. Which just gave me the idea to add this member function to `Unique`:

scope ref T borrow();

I have to think about @safety guarantees but it should be ok with DIP 1000.

Atila

On Thursday, 12 July 2018 at 21:31:04 UTC, xray wrote:
> On Thursday, 12 July 2018 at 14:13:25 UTC, Chris M. wrote:
>> On Wednesday, 11 July 2018 at 22:59:50 UTC, xray wrote:
>>> [...]
>>
>> I feel the following should be disallowed, since we've moved some checking to runtime. Ideally this system would all happen at compile-time.
>>
>> auto r3 = r2; // Smooth reference
>> auto w = r3.getWidth(); // An exception is raised if r3 is not valid.
>
> If we disallow the use of smooth references, we fall into the same paradigm as Rust. Then it's going to be hard to implement data structures with many references to the same object and the language becomes less flexible.
>
> But yes, I assume we are in Safe Memory Management so that we can do :
>
> if ( isValidRef(r3) ) {
>    auto w = r3.getWidth();
> }
>
> Also, I have started to look at the dip-1000. At first glance, "scope" is an approach that makes sense but it does not seem to fit with the "ownership" concept that I suggest, ...unless someone has a brilliant idea to reconcile all those concepts.

I was concerned there may have been a conflict between your idea and DIP1000, but now that I look closer that may not be true. Maybe it could even help with smooth references?

auto r1 := new Ref();
scope r2 = r1; // we know r2 will not outlive r1

On Friday, 13 July 2018 at 12:43:20 UTC, Atila Neves wrote:
>
> The only thing I got from this are that "smooth references" are like Rust's borrows. Which just gave me the idea to add this member function to `Unique`:
>
> scope ref T borrow();
>
> I have to think about @safety guarantees but it should be ok with DIP 1000.
>
> Atila

Sounds interesting.

I imagine you could specialize this depending on mutability. Rust allows only one mutable borrow, but eliminated immutable borrows, but you can't mix them. You could also place some restrictions, like dis-allow borrows, only allow immutable borrows, etc.

On Friday, 13 July 2018 at 14:47:59 UTC, jmh530 wrote:
>
> Sounds interesting.
>
> I imagine you could specialize this depending on mutability. Rust allows only one mutable borrow, but eliminated

I swear I must have dyslexia or something. Eliminated should be unlimited.

On Friday, 13 July 2018 at 14:47:59 UTC, jmh530 wrote:
> On Friday, 13 July 2018 at 12:43:20 UTC, Atila Neves wrote:
>>
>> The only thing I got from this are that "smooth references" are like Rust's borrows. Which just gave me the idea to add this member function to `Unique`:
>>
>> scope ref T borrow();
>>
>> I have to think about @safety guarantees but it should be ok with DIP 1000.
>>
>> Atila
>
> Sounds interesting.
>
> I imagine you could specialize this depending on mutability. Rust allows only one mutable borrow, but eliminated immutable borrows, but you can't mix them. You could also place some restrictions, like dis-allow borrows, only allow immutable borrows, etc.

Rust can do that because it enforces it at compile-time. A D solution wouldn't be able to do anything more with immutable borrows.

On Friday, 13 July 2018 at 17:12:26 UTC, Atila Neves wrote:
>
> Rust can do that because it enforces it at compile-time. A D solution wouldn't be able to do anything more with immutable borrows.

Hmm, thinking on this a little more...it does seem difficult...but I don't think the problem is with immutable borrows. I think the issue is with the exclusivity of Rust's borrowing.

D's immutable is transitive so if you're using immutable at some point, then no one else can modify it anyway. So you should only be able to immutably borrow something that's immutable anyway.

Rust, by contrast, allows immutable borrows of mutable data. In some sense what Rust does corresponds more to const (or maybe head const), but it's more than just const. A Rust immutable borrow of mutable data prevents the mutable data from being modified during the borrow. The Rust example below involves an immutable borrow of mutable data, but it fails to compile because you modify x while it is borrowed. If you put y in a separate scope, then it compiles because x is no longer being borrowed after y exits the scope.

fn main() {
    let mut x = 5;
    let y = & x;
    x += 1;
}

This exclusivity also affects mutable borrows as well. Below does not compile because y controls x. Rust's mutable borrows are also exclusive. Only one is allowed at a time. So that same trickiness is applied here.

fn main() {
    let mut x = 5;
    let y = &mut x;
    x += 1;
}

The only thing that made sense to me about implementing this at compile-time was a template parameter that could disable things like opAssign.
https://run.dlang.io/is/FvJvFv
But being able to change the template parameter is tricky. You can cast it at run-time, but other that that it's beyond me.

On a separate note, I didn't have any success working with automem and const/immutable types.
https://run.dlang.io/is/el4h3e