On 2/22/15 8:50 PM, deadalnix wrote:
> On Sunday, 22 February 2015 at 20:48:58 UTC, Andrei Alexandrescu wrote:
>>> What ??? That mean writing all library code twice, for client that want
>>> GC and for these who don't.
>>
>> I'm not 100% convinced but it seems to me RC vs. GC is a class design
>> time decision.
>>
>
> The right strategy for memory management depend on the usage you'll do
> of an object, not of the object itself.

I used to think the same. But then I considered typechecking things like:

class Widget
{
    private char name[1024];
    char[] getName() { return name[]; }
    ...
}

Such code is safe if Widget is a GC class but not if it uses reference counting.


Andrei

On 2/23/15 6:56 AM, "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm@gmx.net>" wrote:
> On Sunday, 22 February 2015 at 17:01:45 UTC, Andrei Alexandrescu wrote:
>> On 2/22/15 6:49 AM, "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm@gmx.net>"
>> wrote:
>>>
>>> No. There's also returning the reference from a member function, storing
>>> it in a passed-in reference (pointer, ref, out or slice), and passing it
>>> to other functions that in turn leak the reference, as well as throwing
>>> it. And leaking closures containing the reference.
>>>
>>> That's all that I can think of now...
>>
>> Consider
>>
>> class C { ... client code ... }
>> alias T = RefCounted!C;
>> ... more client code ...
>>
>> For reference counting to work transparently, access to the symbol "C"
>> must be restricted. RefCounted obviously needs access to it. Client
>> code should never have access to it, even in the definition of C.
>>
>> That means:
>>
>> 1. client code must not be able to declare variables of type C or
>> issue calls like "new C" etc.
>
> No, this would require the class to be specialized for refcounting. But
> the memory management method needs to be the client code's decision; it
> can decide to manage some instance by refcounting, some by Unique!C, and
> leave others to the GC. The class implementer shouldn't need to care
> about all that.

That's not feasible. Code that assumes the class object will live forever can simply do things that are not allowed to code that must assume the object will go away at some determined point. Consider this which I just posted:

class Widget
{
    private char name[1024];
    char[] getName() { return name[]; }
    ...
}

The typechecker must WHILE COMPILING WIDGET, NOT ITS CLIENT know whether getName() is okay or not.

>> 2. The type of "this" in methods of C must be RefCounted!C, not C.
>>
>> 3. Conversions of C to bases of C and interfaces must be forbidden;
>> only their RefCounted!Base versions must be allowed.
>
> These two points have undesirable consequences: All consumers such
> objects need to be aware of the exact type, which includes the
> management strategy (RC, Unique, GC). But this is a violation of the
> principle of separation of concerns: a consumer shouldn't need to have
> information about the management strategy, it should work equally with
> `RefCounted!C`, `Unique!C` and bare (GC) `C`, as long as it doesn't take
> ownership of the resource.

Well I don't know of another way.

>> 4. Returning references to direct members of C must be restricted the
>> same way they are for structs (see http://wiki.dlang.org/DIP25). A GC
>> class object does not have that restriction.
>
> This is only a partial solution that doesn't work efficiently with
> anything other then value members. Slices, pointers and classes require
> introducing an additional, useless indirection, and that's not the only
> problem with it.

So what do you propose instead? I am well aware of the disadvantages of a solution that _works_. Do you have something better?

>> I think reference counting is an important component of a complete
>> solution to resource management. D should implement world-class
>> reference counting for safe code.
>>
>> For 1-4 above, although I am a staunch supporter of library-exclusive
>> abstractions, I have reached the conclusion there is no way to
>> implement RC in safe code for D classes without changes to the
>> language. The more we realize that as a community the quicker we can
>> move to effect it.
>
> I'm not sure this is what you're implying, but do you want to restrict
> it to classes only? Why not structs and slices, too?

Structs and slices can be implemented @safe-ly after the advent of DIP25. It's classes that are the troublemaker.

> Of course I agree that a language change is necessary, but I'm convinced
> what you suggest above is not the right direction at all. (And I see
> that deadalnix has already replied and basically said the same thing.)

That does awfully little to help. I'm telling things as they are. It's difficult to disagree with e.g. "a refcounted class cannot be implicitly convertible to Object and stay refcounted".

> In general, this and related proposals tend to limit themselves on
> memory management (as witnessed by the importance that `ref` and `@safe`
> play in them). This is too narrow IMO. A well thought-out solution can
> be equally applicable to the broader field of resource management.

Looking forward to your insights.


Andrei

On Monday, 23 February 2015 at 18:16:38 UTC, Andrei Alexandrescu wrote:
> The typechecker must WHILE COMPILING WIDGET, NOT ITS CLIENT know whether getName() is okay or not.

Aye, I haven't been following this thread closely, but I thought of this case a while ago myself and it actually led me to the belief that the this pointer needs to be scope unless the class itself is designed solely for GC use; escaping anything through it must not be allowed for guaranteed memory safety if it is manually freed in any form, whether refcounting, RAII, or free.

If this is scope, then the usage site has freedom of deallocation method. If not, it must be known at design time of the class itself.

On 2/23/15 10:27 AM, Adam D. Ruppe wrote:
> On Monday, 23 February 2015 at 18:16:38 UTC, Andrei Alexandrescu wrote:
>> The typechecker must WHILE COMPILING WIDGET, NOT ITS CLIENT know
>> whether getName() is okay or not.
>
> Aye, I haven't been following this thread closely, but I thought of this
> case a while ago myself and it actually led me to the belief that the
> this pointer needs to be scope unless the class itself is designed
> solely for GC use; escaping anything through it must not be allowed for
> guaranteed memory safety if it is manually freed in any form, whether
> refcounting, RAII, or free.
>
> If this is scope, then the usage site has freedom of deallocation
> method. If not, it must be known at design time of the class itself.

That's exactly right. There should be a DConf talk about that, apparently it's a widely misunderstood topic. -- Andrei

Just brainstorming:

1. Move RC!T into the runtime because it can get special compiler attention.
2. Duplicate the const/mutable/immutable machinery for Scope/RC/GC classes ( / types).
3. RC!T is not struct RC { size_t cntr; T payload } but a T*. The allocator makes sure that the reference count is at T*-8 (or at T*+RCOffset!T or whatever).
4. RC!T and T (the GC managed type) implicitly convert to Scope!T. Scope!T makes sure that no reference escapes in an unsafe way (using the foundations in dip25).

Because of 3. we can have polymorphic functions that either return a GC allocated or an RC allocated class. It's type depends of the type of this, which leads to the equivalent of inout, let's call it @rcgc.

On Monday, 23 February 2015 at 15:35:52 UTC, Ola Fosheim Grøstad wrote:
> If you do excessive refcounting (ARC) and care about performance you actually need to let the implementor of C decide where the RC_counter is embedded...

C providing facility to be refcountable do not equate with C's user want refcounting. It means that if C's user want it to be refcounted, it is gonna be faster.

On Monday, 23 February 2015 at 17:48:27 UTC, Andrei Alexandrescu wrote:
> I used to think the same. But then I considered typechecking things like:
>
> class Widget
> {
>     private char name[1024];
>     char[] getName() { return name[]; }
>     ...
> }
>
> Such code is safe if Widget is a GC class but not if it uses reference counting.
>

This is not a valid argument against my point, simply expressing difficulties of implementation and lack of imagination of what this implementation can be.

I'm not sure how to convey the idea anymore. That is an ownership issue and solutions exist for it. Several of them have been proposed already.

It feels like we are gonna add a bazillion of stupid hacks to work around not introducing ownership into D. The return ref thing is one of them. What you are suggesting for refcounting is another.

On Monday, 23 February 2015 at 18:16:38 UTC, Andrei Alexandrescu wrote:
> That's not feasible. Code that assumes the class object will live forever can simply do things that are not allowed to code that must assume the object will go away at some determined point. Consider this which I just posted:
>
> class Widget
> {
>     private char name[1024];
>     char[] getName() { return name[]; }
>     ...
> }
>
> The typechecker must WHILE COMPILING WIDGET, NOT ITS CLIENT know whether getName() is okay or not.
>

Let's use that exemple.

> Well I don't know of another way.
>

I'm not sure if this is dishonest, but proposal has been made in this newgroup, by various person,s including myself.

Let's get around the major concept.

name is owned by the widget instance (let's call it w). That means its lifetime is the same as w's lifetime.

w's owner will provide its lifetime. If w's owner is the GC, then its lifetime is infinite. if w's owner is some refounting system, its lifetime is defined by when the refcounting system will destroy it.

getName here assumes the lifetime of the object is infinite. That's ok. It means you won't be able to call it when it is owned by a RC system. If you want to be able to do so, you'll ned have a way to specify the lifetime of the returned slice. For instance :

class Widget {
    private char name[1024];
    scope(char[]) getName() scope { return name[]; }
    ...
}

In this case, you can call the method when w is owned by a RC system, as it is now explicit to the caller that the lifetime of what is returned and the compiler can ensure the slice do not exceed its lifetime (ie w's lifetime).

This require to be able to express lifetime and preferably ownership as well. So far, what I feel from you and Walter is the will to not go into that direction and instead created a myriad of hacks for various special cases. I do think this is wrong headed and generally not the way forward. Languages should provide the most generic and powerful tool so that interesting schemes can be implemented on top of it.

On 2/23/15 5:23 PM, deadalnix wrote:
> So far, what I feel from you and Walter is the will to not go into that
> direction and instead created a myriad of hacks for various special
> cases. I do think this is wrong headed and generally not the way
> forward. Languages should provide the most generic and powerful tool so
> that interesting schemes can be implemented on top of it.

This is a free world. Walter and I are working on a DIP. Please work on yours. I can't promise we'll choose to your liking, but this is the one way you can make your point heard and understood. What doesn't work is trash talk.

I guarantee I recognize brilliance when I see it. So if you have a brilliant idea, it won't be missed. Have at it. One thing I cannot do is choose a solution that you prefer over one I prefer - this does remain a subjective topic. I can't help it. But please don't consider me an idiot because I don't like what you propose.


Thanks,

Andrei

On 2/23/15 5:43 PM, Andrei Alexandrescu wrote:
> But please don't consider me an idiot because I don't like what you
> propose.

Hmmm... actually I take that back :o). You are totally free to consider me such (or at least of intellect inferior to yours etc). What I mean is please don't consider me somehow socially or morally obligated to choose your solution just because you like it so much more than mine.

Andrei