On 3 March 2015 at 06:37, Walter Bright via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
> On 3/1/2015 12:51 PM, Michel Fortin wrote:
>>
>> That's actually not enough. You'll have to block access to global variables too:
>>
>>      S s;
>>
>>      void main() {
>>          s.array = RCArray!T([T()]);   // s.array's refcount is now 1
>>          foo(s.array[0]);           // pass by ref
>>      }
>>      void foo(ref T t) {
>>          s.array = RCArray!T([]);      // drop the old s.array
>>          t.doSomething();              // oops, t is gone
>>      }
>
>
> Thinking about it, there are many other ways this can happen. At the moment, I'm stuck thinking of a solution other than requiring foo() to be pure. Anyone have ideas?

My immediate impression on this problem:

s.array[0] is being passed to foo from main. s does not belong to main
(is global), and main does not hold have a reference to s.array.
Shouldn't main just need to inc/dec array around the call to foo when
passing un-owned references down the call tree.
It seems to me that there always needs to be a reference _somewhere_
on the stack for anything being passed down the call tree (unless the
function is pure). Seems simplest to capture a stack ref at the top
level, then as it's received as arguments to each callee, it's
effectively owned by those functions and they don't need to worry
anymore.

So, passing global x to some function; inc/dec x around the function call that it's passed to...? Then the stack has its own reference, and the global reference can go away safely.

On Tuesday, 3 March 2015 at 08:04:25 UTC, Manu wrote:
> So, passing global x to some function; inc/dec x around the function
> call that it's passed to...? Then the stack has its own reference, and
> the global reference can go away safely.

Yes, the sane thing to do is to improve the general type system and general optimizer. The special casing D is going for will lead to no good. New non-trivial special case features just punches more holes in the type system. Which is the opposite of what is needed to bring D to a stable state.

By trivial I mean syntax sugar, which always is ok, but a type system should be general with no special casing in it.

What you need to do is to a way to implement smart pointers as non-ref-capable types, and the ability to do moves to transfer ownership up the call stack, and good general opimizations in the backends for it. AFAIK the current type system is too weak to enforce it. The type system also lacks head-const-ref, which often is needed for safe "manual optimization"?

The special casing effort is largely wasted because one cannot have efficient ARC without whole program/module optimization anyway. Swift ARC does better when optimizing larger units. With whole program optimization, stronger typing and smart inlining the RC performance issues can be reduced more efficiently.

It would be better to focus on ways to tighten the type system and how to utilize stronger typing for optimization of larger units (whole module/program). Special casing in the language makes optimization algorithms harder to write. Long term evil.

On 3/2/2015 9:58 PM, weaselcat wrote:
> Borrowing 'a' from a struct would make the parent struct immutable during the
> borrow scope of 'a', I believe.

Right, now consider that struct is a leaf in a complex graph of data structures.

On Tuesday, 3 March 2015 at 05:12:15 UTC, Walter Bright wrote:
> On 3/2/2015 6:04 PM, weaselcat wrote:
>> On Tuesday, 3 March 2015 at 01:56:09 UTC, Walter Bright wrote:
>>> On 3/2/2015 4:40 PM, deadalnix wrote:
>>>> After moving resources, the previous owner can no longer be used.
>>>
>>> How does that work with the example presented by Marc?
>>
>> He couldn't pass s and a member of s because s is borrowed as mutable.
>> He would have to pass both as immutable.
>
> A pointer to s could be obtained otherwise and passed.

No. Rust compiler forbids you from obtaining the pointer
otherwise. Namely, it is a compile time error to make an alias of
a mutably borrowed pointer.

On Monday, 2 March 2015 at 22:42:44 UTC, weaselcat wrote:
> I don't think you were advocating for this but,
> +1 for a borrow system similar to Rust.

We're working on it ;-)

On Monday, 2 March 2015 at 22:58:19 UTC, Walter Bright wrote:
> On 3/2/2015 1:09 PM, "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm@gmx.net>" wrote:
>> "I have discovered a marvellous solution, but this post is too short to describe
>> it."
>
> Fortunately, Fermat (er, Andrei) was able to pass along his dazz idea to me this afternoon, before he expired to something he had to attend to. We were both in the dumps about this problem last night, but I think he solved it.
>
> His insight was that the deletion of the payload occurred before the end of the lifetime of the RC object, and that this was the source of the problem. If the deletion of the payload occurs during the destructor call, rather than the postblit, then although the ref count of the payload goes to zero, it doesn't actually get deleted.
>
> I.e. the postblit manipulates the ref count, but does NOT do payload deletions. The destructor checks the ref count, if it is zero, THEN it does the payload deletion.
>
> Pretty dazz idea, dontcha think? And DIP25 still stands unscathed :-)
>
> Unless, of course, we missed something obvious.

Clever :-) But there are two things:

The object is still accessible after its refcount went to zero, and can therefore potentially be resurrected. Probably not a problem, but needs to be taken into account, in particular in with respect to the freelist. That's tricky, because an object can be released and resurrected several times, and care must be taken that it will not end up in the freelist and get destroyed multiple times. And that hasn't even touched on thread-safety yet.

The bigger problem is that it's relying on a convention. The RC wrapper needs to be constructed in a particular way that's easy to get wrong and that the compiler has no way to check for us.

On Tuesday, 3 March 2015 at 09:05:46 UTC, Walter Bright wrote:
> On 3/2/2015 9:58 PM, weaselcat wrote:
>> Borrowing 'a' from a struct would make the parent struct immutable during the
>> borrow scope of 'a', I believe.
>
> Right, now consider that struct is a leaf in a complex graph of data structures.

Then you still cannot have more than one mutable reference to the entire graph. Because that is impractical, Rust uses unsafe (i.e. @trusted in D speak) accessors that "cast away" the ownership, but do so in a way that doesn't violate the guarantees.

For example, the type system doesn't allow you to get mutable references to the left and right children of a binary tree node. But there can be an accessor method that internally does some unsafe magic to return a tuple with mutable references to them, annotated with the information that they are mutably borrowed from the node. Both child refs are mutable, and the parent node is inaccessible as long as they exist.

On 3/3/15 5:05 AM, "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm@gmx.net>" wrote:
> The object is still accessible after its refcount went to zero, and can
> therefore potentially be resurrected. Probably not a problem, but needs
> to be taken into account, in particular in with respect to the freelist.
> That's tricky, because an object can be released and resurrected several
> times, and care must be taken that it will not end up in the freelist
> and get destroyed multiple times. And that hasn't even touched on
> thread-safety yet.

Could you please give an example?

> The bigger problem is that it's relying on a convention. The RC wrapper
> needs to be constructed in a particular way that's easy to get wrong and
> that the compiler has no way to check for us.

Agreed.


Andrei

On 3/3/15 5:45 AM, "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm@gmx.net>" wrote:
> On Tuesday, 3 March 2015 at 09:05:46 UTC, Walter Bright wrote:
>> On 3/2/2015 9:58 PM, weaselcat wrote:
>>> Borrowing 'a' from a struct would make the parent struct immutable
>>> during the
>>> borrow scope of 'a', I believe.
>>
>> Right, now consider that struct is a leaf in a complex graph of data
>> structures.
>
> Then you still cannot have more than one mutable reference to the entire
> graph. Because that is impractical, Rust uses unsafe (i.e. @trusted in D
> speak) accessors that "cast away" the ownership, but do so in a way that
> doesn't violate the guarantees.
>
> For example, the type system doesn't allow you to get mutable references
> to the left and right children of a binary tree node. But there can be
> an accessor method that internally does some unsafe magic to return a
> tuple with mutable references to them, annotated with the information
> that they are mutably borrowed from the node. Both child refs are
> mutable, and the parent node is inaccessible as long as they exist.

Well... the bigger problem is that it's relying on a convention. The accessor method needs to be constructed in a particular way that's easy to get wrong and that the compiler has no way to check for us.

:o)


Andrei

On Tuesday, 3 March 2015 at 08:04:25 UTC, Manu wrote:
> My immediate impression on this problem:
>
> s.array[0] is being passed to foo from main. s does not belong to main
> (is global), and main does not hold have a reference to s.array.
> Shouldn't main just need to inc/dec array around the call to foo when
> passing un-owned references down the call tree.
> It seems to me that there always needs to be a reference _somewhere_
> on the stack for anything being passed down the call tree (unless the
> function is pure). Seems simplest to capture a stack ref at the top
> level, then as it's received as arguments to each callee, it's
> effectively owned by those functions and they don't need to worry
> anymore.
>
> So, passing global x to some function; inc/dec x around the function
> call that it's passed to...? Then the stack has its own reference, and
> the global reference can go away safely.

This is my position too.

There is another problem being discussed now, however, having to do with references to non-rc'd subcomponents of an Rc'd type.