On Saturday, 27 September 2014 at 09:38:35 UTC, Dmitry Olshansky wrote:
> As usual, structs are value types, so this feature can be mis-used, no two thoughts abouts it. It may need a bit of improvement in user-friendliness, compiler may help there by auto-detecting common misuse.
>
> Theoretically class-es would be better choice, except that question of allocation pops up immediately, then consider for instance COM objects.
>
> The good thing w.r.t. to memory about structs - they are themselves already allocated "somewhere", and it's only ref-counted payload that is allocated and destroyed in a user-defined way.
>
> And now for the killer reasons to go for struct is the following:
>
> Compiler _already_ does all of life-time management and had numerous bug fixes to make sure it does the right thing. In contrast there is nothing for classes that tracks their lifetimes to call proper hooks.

This cannot be stressed enough.

>
> Let's REUSE that mechanism we have with structs and go as lightly as possible on  untested LOCs budget.
>
> Full outline, of generic to the max, dirt-cheap implementation with a bit of lowering:
>
> ARC or anything close to it, is implemented as follows:
> 1. Any struct that have @ARC attached, must have the following methods:
> 	void opInc();
> 	bool opDec(); // true - time to destroy
> It also MUST NOT have postblit, and MUST have destructor.
>
> 2. Compiler takes user-defined destructor and creates proper destructor, as equivalent of this:
> 	if(opDec()){
> 		user__defined_dtor;
> 	}
> 3. postblit is defined as opInc().
>
> 4. any ctor has opInc() appended to its body.
>
> Everything else is taken care of by the very nature of the structs.

AFAICS we don't gain anything from this, because it just automates certain things that can already be done manually in a suitably implemented wrapper struct. I don't think automation is necessary here, because realistically, how many RC wrappers will there be? Ideally just one, in Phobos.

> Now this is enough to make ref-counted stuff a bit simpler to write but not much beyond. So here the next "consequences" that we can then implement:
>
> 4. Compiler is expected to assume anywhere in fully inlined code, that opInc()/opDec() pairs are no-op. It should do so even in debug mode (though there is less opportunity to do so without inlining). Consider it an NRVO of the new age, required optimization.
>
> 5. If we extend opInc/opDec to take an argument, the compiler may go further and batch up multiple opInc-s and opDec-s, as long as it's safe to do so (e.g. there could be exceptions thrown!):
>
> Consider:
>
> auto a = File("some-file.txt");
> //pass to some structs for future use
> B b = B(a);
> C c = C(a);
> a = File("other file");
>
> May be (this is overly simplified!):
>
> File a = void, b = void, c = void;
> a = File.user_ctor("some-file.txt")'
> a.opInc(2);
> b = B(a);
> c = C(a);
> a = File.user_ctor("other file");
> a.opInc();

I believe we can achieve the same efficiency without ARC with the help of borrowing and multiple alias this. Consider the cases where inc/dec can be elided:

   RC!int a;
   // ...
   foo(a);
   // ...
   bar(a);
   // ...

Under the assumption that foo() and bar() don't want to keep a copy of their arguments, this is a classical use case for borrowing. No inc/dec is necessary, and none will happen, if RC!int has an alias-this-ed method returning a scoped reference to its payload.

On the other hand, foo() and bar() could want to make copies of the refcounted variable. In this case, we still wouldn't need an inc/dec, but we need a way to express that. The solution is another alias-this-ed method that returns a (scoped) BorrowedRC!int, which does not inc/dec on construction/destruction, but does so on copying. (It's probably possible to reuse RC!int for this, a separate type is likely not necessary.)

The other opportunity is on moving:

    void foo() {
        RC!int a;
        // ....
        bar(a);    // last statement in foo()
    }

Here, clearly `a` isn't used after the tail call. Instead of copy & destroy, the compiler can resort to a move (bare bitcopy). In contrast to C++, this is allowed in D.

This covers most opportunities for elision of the ref counting. It only leaves a few corner cases (e.g. `a` no longer used after non-tail calls, accumulated inc/dec as in your example). I don't think these are worth complicating the compiler with ARC.

On Saturday, 27 September 2014 at 10:23:20 UTC, Marc Schütz wrote:
> On the other hand, foo() and bar() could want to make copies of the refcounted variable. In this case, we still wouldn't need an inc/dec, but we need a way to express that. The solution is another alias-this-ed method that returns a (scoped) BorrowedRC!int, which does not inc/dec on construction/destruction, but does so on copying. (It's probably possible to reuse RC!int for this, a separate type is likely not necessary.)

Yepp, it's possible, it turned out to work quite naturally:
http://wiki.dlang.org/User:Schuetzm/scope#Reference_counting

27-Sep-2014 14:23, "Marc Schütz" <schuetzm@gmx.net>" пишет:
> On Saturday, 27 September 2014 at 09:38:35 UTC, Dmitry Olshansky wrote:

>> The good thing w.r.t. to memory about structs - they are themselves
>> already allocated "somewhere", and it's only ref-counted payload that
>> is allocated and destroyed in a user-defined way.
>>
>> And now for the killer reasons to go for struct is the following:
>>
>> Compiler _already_ does all of life-time management and had numerous
>> bug fixes to make sure it does the right thing. In contrast there is
>> nothing for classes that tracks their lifetimes to call proper hooks.
>
> This cannot be stressed enough.
>
>>
>> Let's REUSE that mechanism we have with structs and go as lightly as
>> possible on  untested LOCs budget.
>>
>> Full outline, of generic to the max, dirt-cheap implementation with a
>> bit of lowering:
>>
>> ARC or anything close to it, is implemented as follows:
>> 1. Any struct that have @ARC attached, must have the following methods:
>>     void opInc();
>>     bool opDec(); // true - time to destroy
>> It also MUST NOT have postblit, and MUST have destructor.
>>
>> 2. Compiler takes user-defined destructor and creates proper
>> destructor, as equivalent of this:
>>     if(opDec()){
>>         user__defined_dtor;
>>     }
>> 3. postblit is defined as opInc().
>>
>> 4. any ctor has opInc() appended to its body.
>>
>> Everything else is taken care of by the very nature of the structs.
>
> AFAICS we don't gain anything from this, because it just automates
> certain things that can already be done manually in a suitably
> implemented wrapper struct. I don't think automation is necessary here,
> because realistically, how many RC wrappers will there be? Ideally just
> one, in Phobos.

You must be missing something big. Ref-counting ain't singular thing, it's a strategy with a multitude of implementations, see my other post.


>> Now this is enough to make ref-counted stuff a bit simpler to write
>> but not much beyond. So here the next "consequences" that we can then
>> implement:
>>
>> 4. Compiler is expected to assume anywhere in fully inlined code, that
>> opInc()/opDec() pairs are no-op. It should do so even in debug mode
>> (though there is less opportunity to do so without inlining). Consider
>> it an NRVO of the new age, required optimization.
>>
>> 5. If we extend opInc/opDec to take an argument, the compiler may go
>> further and batch up multiple opInc-s and opDec-s, as long as it's
>> safe to do so (e.g. there could be exceptions thrown!):
>>
>> Consider:
>>
>> auto a = File("some-file.txt");
>> //pass to some structs for future use
>> B b = B(a);
>> C c = C(a);
>> a = File("other file");
>>
>> May be (this is overly simplified!):
>>
>> File a = void, b = void, c = void;
>> a = File.user_ctor("some-file.txt")'
>> a.opInc(2);
>> b = B(a);
>> c = C(a);
>> a = File.user_ctor("other file");
>> a.opInc();
>
> I believe we can achieve the same efficiency without ARC with the help
> of borrowing and multiple alias this.

Problem is - there is no borrowing yet in the compiler, or maybe you mean something more simple.

> Consider the cases where inc/dec
> can be elided:
>
>     RC!int a;
>     // ...
>     foo(a);
>     // ...
>     bar(a);
>     // ...
>
> Under the assumption that foo() and bar() don't want to keep a copy of
> their arguments, this is a classical use case for borrowing. No inc/dec
> is necessary, and none will happen, if RC!int has an alias-this-ed
> method returning a scoped reference to its payload.

Interesting. However scope must work first, also passing an RC!int by borrowing is this:

void func(scope(A) a)

or what? how does it transform scope? (Sorry I haven't followed your proposals for scope)


> On the other hand, foo() and bar() could want to make copies of the
> refcounted variable. In this case, we still wouldn't need an inc/dec,
> but we need a way to express that. The solution is another alias-this-ed
> method that returns a (scoped) BorrowedRC!int, which does not inc/dec on
> construction/destruction, but does so on copying. (It's probably
> possible to reuse RC!int for this, a separate type is likely not
> necessary.)

Who would make sure original RC still exists?

> The other opportunity is on moving:
>
>      void foo() {
>          RC!int a;
>          // ....
>          bar(a);    // last statement in foo()
>      }

We should already have it with structs by their nature.

>
> Here, clearly `a` isn't used after the tail call. Instead of copy &
> destroy, the compiler can resort to a move (bare bitcopy). In contrast
> to C++, this is allowed in D.
>
> This covers most opportunities for elision of the ref counting. It only
> leaves a few corner cases (e.g. `a` no longer used after non-tail calls,
> accumulated inc/dec as in your example). I don't think these are worth
> complicating the compiler with ARC.

I don't mind having working scope and borrowing but my proposal doesn't require them.

-- 
Dmitry Olshansky

On 9/27/14, 1:11 AM, Foo wrote:
>> Consider:
>>
>> struct MyRefCounted
>>     void opInc();
>>     void opDec();
>>     int x;
>> }
>>
>> MyRefCounted a;
>> a.x = 42;
>> MyRefCounted b = a;
>> b.x = 43;
>>
>> What is a.x after this?
>>
>>
>> Andrei
>
> a.x == 42
> a.ref_count == 1 (1 for init, +1 for copy, -1 for destruction)
> b.x == 43
> b.ref_count == 1 (only init)

So then when does the counter get ever incremented? -- Andrei

On 9/27/14, 2:38 AM, Dmitry Olshansky wrote:
>
> Okay it serves no good for me to make these tiny comments while on the go.
>
> As usual, structs are value types, so this feature can be mis-used, no
> two thoughts abouts it. It may need a bit of improvement in
> user-friendliness, compiler may help there by auto-detecting common misuse.

I still don't understand what "this feature" is after reading your long post twice.

So structs are still value types and you replace postblit/destroy with calls to opInc/opDec? That's it? How does this enable anything more interesting than ctors/dtors?


Andrei

On 9/27/14, 2:43 AM, Dmitry Olshansky wrote:
> Refcounting is process of add(x), and sub(x), and calling destructor
> should the subtract call report zero. Everything else is in the hands of
> the creator.

I literally have no idea what you are discussing here. -- Andrei

27-Sep-2014 23:14, Andrei Alexandrescu пишет:
> On 9/27/14, 2:38 AM, Dmitry Olshansky wrote:
>>
>> Okay it serves no good for me to make these tiny comments while on the
>> go.
>>
>> As usual, structs are value types, so this feature can be mis-used, no
>> two thoughts abouts it. It may need a bit of improvement in
>> user-friendliness, compiler may help there by auto-detecting common
>> misuse.
>
> I still don't understand what "this feature" is after reading your long
> post twice.
>
> So structs are still value types and you replace postblit/destroy with
> calls to opInc/opDec? That's it? How does this enable anything more
> interesting than ctors/dtors?
>

Compiler is aware that opInc and opDec are indeed ref-countinng ops, meaning that opInc + opDec = no op. I claim that this is enough to get "ARC" going.


-- 
Dmitry Olshansky

27-Sep-2014 23:15, Andrei Alexandrescu пишет:
> On 9/27/14, 2:43 AM, Dmitry Olshansky wrote:
>> Refcounting is process of add(x), and sub(x), and calling destructor
>> should the subtract call report zero. Everything else is in the hands of
>> the creator.
>
> I literally have no idea what you are discussing here. -- Andrei

That proposed scheme is completely abstract as to what exactly adding X to a ref-count does. Or what exactly subtracting X from ref-count does.
The only invariant is that there is something that will tell us after yet another decrement that ref-count is zero and we should free resources.

As I said elsewhere - automagically embedding ref-count is too limiting, there is whole lot of ways to implement count including intrusive and non-intrusive.


-- 
Dmitry Olshansky

On 9/27/14, 12:50 PM, Dmitry Olshansky wrote:
> 27-Sep-2014 23:14, Andrei Alexandrescu пишет:
>> On 9/27/14, 2:38 AM, Dmitry Olshansky wrote:
>>>
>>> Okay it serves no good for me to make these tiny comments while on the
>>> go.
>>>
>>> As usual, structs are value types, so this feature can be mis-used, no
>>> two thoughts abouts it. It may need a bit of improvement in
>>> user-friendliness, compiler may help there by auto-detecting common
>>> misuse.
>>
>> I still don't understand what "this feature" is after reading your long
>> post twice.
>>
>> So structs are still value types and you replace postblit/destroy with
>> calls to opInc/opDec? That's it? How does this enable anything more
>> interesting than ctors/dtors?
>>
>
> Compiler is aware that opInc and opDec are indeed ref-countinng ops,
> meaning that opInc + opDec = no op. I claim that this is enough to get
> "ARC" going.

You give marginal details but still don't describe the thing. When are they called and what do they have that ctors/dtors/postblit don't?

FWIW the language always "understands" when to elide postblit/dtor calls.


Andrei

On 9/27/14, 12:53 PM, Dmitry Olshansky wrote:
> 27-Sep-2014 23:15, Andrei Alexandrescu пишет:
>> On 9/27/14, 2:43 AM, Dmitry Olshansky wrote:
>>> Refcounting is process of add(x), and sub(x), and calling destructor
>>> should the subtract call report zero. Everything else is in the hands of
>>> the creator.
>>
>> I literally have no idea what you are discussing here. -- Andrei
>
> That proposed scheme is completely abstract as to what exactly adding X
> to a ref-count does.

What is "that proposed scheme?" I feel like I'm watching a movie starting half time.

> Or what exactly subtracting X from ref-count does.
> The only invariant is that there is something that will tell us after
> yet another decrement that ref-count is zero and we should free resources.
>
> As I said elsewhere - automagically embedding ref-count is too limiting,
> there is whole lot of ways to implement count including intrusive and
> non-intrusive.

Sure, agreed. But what's the deal here?


Andrei