Walter posted an example implementation of a reference counted array [1], that utilizes the features introduced in DIP25 [2]. Then, in the threads about reference counted objects, several people posted examples [3, 4] that broke the suggested optimization of eliding `opAddRef()`/`opRelease()` calls in certain situations.

A weakness of the same kind affects DIP25, too. The core of the problem is borrowing (ref return as in DIP25), combined with manual (albeit hidden) memory management. An example to illustrate:

    struct T {
        void doSomething();
    }
    struct S {
        RCArray!T array;
    }
    void main() {
        auto s = S(RCArray!T([T()])); // s.array's refcount is now 1
        foo(s, s.array[0]);           // pass by ref
    }
    void foo(ref S s, ref T T) {
        s.array = RCArray!T([]);      // drop the old s.array
        t.doSomething();              // oops, t is gone
    }

Any suggestions how to deal with this? As far as I can see, there are the following options:

1) Make borrowing (return ref) @system. This would defeat the purpose of DIP25.

2) Disallow (by convention) borrowing for refcounted objects. Again, this would make DIP25 pointless, and strictly speaking, anything that relies on convention cannot be @safe. And there's no guarantee that it doesn't affect other things besides RC.

3) Introduce a full linear type system. A _very_ large and invasive change, and probably cumbersome to work with.

4) Live with it. Accept that it's not possible to get the last bit of @safe-ty without extreme and unjustifiable costs. Make it @safe nevertheless, and formulate usage guides that people are expected to follow.

5) Make `RCArray` a special type whose purpose is known to the compiler, and implement complicated checks to verify @safe-ty. Again, kind of defeats the purpose, and adds complexity to the language and implementation.

6) Restrict borrowing to situations where it's @safe. Or better, allow it everywhere, but make it @system where necessary. I think problems can only happen at function boundaries (what happens inside a function can be checked statically), but I'd have to think about it more.

7) Anything else? Are there some small details (in whatever part of the language) that can be adjusted to get us additional guarantees?

Option 6) currently appears the most promising to me.

Comments?

[1] http://forum.dlang.org/thread/mcg8qq$1mbr$1@digitalmars.com
[2] http://wiki.dlang.org/DIP25
[4] http://forum.dlang.org/post/ilfwmeobprkcorpiqoui@forum.dlang.org
[5] http://forum.dlang.org/post/mcqk4s$6qb$1@digitalmars.com

On 3/1/15 7:44 AM, "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm@gmx.net>" wrote:
> Walter posted an example implementation of a reference counted array
> [1], that utilizes the features introduced in DIP25 [2]. Then, in the
> threads about reference counted objects, several people posted examples
> [3, 4] that broke the suggested optimization of eliding
> `opAddRef()`/`opRelease()` calls in certain situations.
>
> A weakness of the same kind affects DIP25, too. The core of the problem
> is borrowing (ref return as in DIP25), combined with manual (albeit
> hidden) memory management. An example to illustrate:
>
>      struct T {
>          void doSomething();
>      }
>      struct S {
>          RCArray!T array;
>      }
>      void main() {
>          auto s = S(RCArray!T([T()])); // s.array's refcount is now 1
>          foo(s, s.array[0]);           // pass by ref
>      }
>      void foo(ref S s, ref T T) {
>          s.array = RCArray!T([]);      // drop the old s.array
>          t.doSomething();              // oops, t is gone
>      }

Thanks for pointing this out - it's a real problem. -- Andrei

On Sunday, 1 March 2015 at 15:44:49 UTC, Marc Schütz wrote:
> Walter posted an example implementation of a reference counted array [1], that utilizes the features introduced in DIP25 [2]. Then, in the threads about reference counted objects, several people posted examples [3, 4] that broke the suggested optimization of eliding `opAddRef()`/`opRelease()` calls in certain situations.
>
> A weakness of the same kind affects DIP25, too. The core of the problem is borrowing (ref return as in DIP25), combined with manual (albeit hidden) memory management. An example to illustrate:
>
>     struct T {
>         void doSomething();
>     }
>     struct S {
>         RCArray!T array;
>     }
>     void main() {
>         auto s = S(RCArray!T([T()])); // s.array's refcount is now 1
>         foo(s, s.array[0]);           // pass by ref
>     }
>     void foo(ref S s, ref T T) {
>         s.array = RCArray!T([]);      // drop the old s.array
>         t.doSomething();              // oops, t is gone
>     }
>
> Any suggestions how to deal with this? As far as I can see, there are the following options:

See:
http://forum.dlang.org/post/bghjqvvrdcfqmoiyyuqz@forum.dlang.org
...and:
http://forum.dlang.org/post/cviwlkugnothraubcfgy@forum.dlang.org

On Sunday, 1 March 2015 at 16:54:15 UTC, Zach the Mystic wrote:
> On Sunday, 1 March 2015 at 15:44:49 UTC, Marc Schütz wrote:
>> Walter posted an example implementation of a reference counted array [1], that utilizes the features introduced in DIP25 [2]. Then, in the threads about reference counted objects, several people posted examples [3, 4] that broke the suggested optimization of eliding `opAddRef()`/`opRelease()` calls in certain situations.
>>
>> A weakness of the same kind affects DIP25, too. The core of the problem is borrowing (ref return as in DIP25), combined with manual (albeit hidden) memory management. An example to illustrate:
>>
>>    struct T {
>>        void doSomething();
>>    }
>>    struct S {
>>        RCArray!T array;
>>    }
>>    void main() {
>>        auto s = S(RCArray!T([T()])); // s.array's refcount is now 1
>>        foo(s, s.array[0]);           // pass by ref
>>    }
>>    void foo(ref S s, ref T T) {
>>        s.array = RCArray!T([]);      // drop the old s.array
>>        t.doSomething();              // oops, t is gone
>>    }
>>
>> Any suggestions how to deal with this? As far as I can see, there are the following options:
>
> See:
> http://forum.dlang.org/post/bghjqvvrdcfqmoiyyuqz@forum.dlang.org
> ...and:
> http://forum.dlang.org/post/cviwlkugnothraubcfgy@forum.dlang.org

That's not applicable here though. The compiler doesn't know we're doing reference counting, so it cannot insert AddRef/Release calls.

On Sunday, 1 March 2015 at 17:48:15 UTC, Marc Schütz wrote:
> That's not applicable here though. The compiler doesn't know we're doing reference counting, so it cannot insert AddRef/Release calls.

Why can't structs also implement opAddRef/Release? Why is it restricted to classes?

On Sunday, 1 March 2015 at 17:48:15 UTC, Marc Schütz wrote:
>> See:
>> http://forum.dlang.org/post/bghjqvvrdcfqmoiyyuqz@forum.dlang.org
>> ...and:
>> http://forum.dlang.org/post/cviwlkugnothraubcfgy@forum.dlang.org
>
> That's not applicable here though. The compiler doesn't know we're doing reference counting, so it cannot insert AddRef/Release calls.

Maybe the compiler can detect a split-pass and insert the appropriate calls to the correlate opAdd/opRelease functions for structs according to the same rules as for classes. Isn't split-passing a rare business in general?

On 3/1/2015 7:44 AM, "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm@gmx.net>" wrote:
> A weakness of the same kind affects DIP25, too. The core of the problem is
> borrowing (ref return as in DIP25), combined with manual (albeit hidden) memory
> management. An example to illustrate:
>
>      struct T {
>          void doSomething();
>      }
>      struct S {
>          RCArray!T array;
>      }
>      void main() {
>          auto s = S(RCArray!T([T()])); // s.array's refcount is now 1
>          foo(s, s.array[0]);           // pass by ref
>      }
>      void foo(ref S s, ref T T) {
>          s.array = RCArray!T([]);      // drop the old s.array
>          t.doSomething();              // oops, t is gone
>      }

The trouble seems to happen when there are two references to the same object passed to a function. I.e. there can be only one "borrowed" ref at a time.

I'm thinking this could be statically disallowed in @safe code.

On 2015-03-01 19:21:57 +0000, Walter Bright said:

> The trouble seems to happen when there are two references to the same object passed to a function. I.e. there can be only one "borrowed" ref at a time.
> 
> I'm thinking this could be statically disallowed in @safe code.

That's actually not enough. You'll have to block access to global variables too:

	S s;

	void main() {
		s.array = RCArray!T([T()]);   // s.array's refcount is now 1
		foo(s.array[0]);           // pass by ref
	}
	void foo(ref T t) {
		s.array = RCArray!T([]);      // drop the old s.array
		t.doSomething();              // oops, t is gone
	}


-- 
Michel Fortin
michel.fortin@michelf.com
http://michelf.com/

On Sunday, 1 March 2015 at 20:51:35 UTC, Michel Fortin wrote:
> On 2015-03-01 19:21:57 +0000, Walter Bright said:
>
>> The trouble seems to happen when there are two references to the same object passed to a function. I.e. there can be only one "borrowed" ref at a time.
>> 
>> I'm thinking this could be statically disallowed in @safe code.
>
> That's actually not enough. You'll have to block access to global variables too:
>
> 	S s;
>
> 	void main() {
> 		s.array = RCArray!T([T()]);   // s.array's refcount is now 1
> 		foo(s.array[0]);           // pass by ref
> 	}
> 	void foo(ref T t) {
> 		s.array = RCArray!T([]);      // drop the old s.array
> 		t.doSomething();              // oops, t is gone
> 	}

Globals to impures, that is.

On 3/1/2015 12:51 PM, Michel Fortin wrote:
> That's actually not enough. You'll have to block access to global variables too:

Hmm. That's not so easy to solve.