What if I want to use an existing (GC) class as ref counted, something
like core.thread.Thread? Tacking @rc on it would break some existing
code, but not doing it requires the GC.
Same goes for specifying an allocator to use with a ref counted class.
It seems like a much better idea to decide that when using the class as
opposed to when implementing the class.

On 11/02/2015 06:28 PM, Martin Nowak wrote:
> What if I want to use an existing (GC) class as ref counted, something
> like core.thread.Thread?

I suspect converting a GC class to an @rc class would require at least changes wrt escaping addresses of members.

> Tacking @rc on it would break some existing
> code, but not doing it requires the GC.

Probably that's the general case. But if a class has no public data members and member functions that don't leak member addresses, it stands to reason conversion could break no code.

> Same goes for specifying an allocator to use with a ref counted class.

I don't understand this part, could you please detail.

> It seems like a much better idea to decide that when using the class as
> opposed to when implementing the class.

Such an approach would need to first define which subset of classes work with both GC and RC. Do we agree on that? For example, classes that escape addresses of data members work safely only with GC. Same goes about classes that escape "this".

For my money, I have long looked for a way to implement reference counting outside of the class definition. I had my mind modeled after the C++ approach, and it does seem like a very sensible idea. Lately, however, I got disabused about the notion because I figured the set of manipulations allowed to GC classes largely surpasses the set of manipulations allowed to RC classes.

I am curious how you think you can define these two subsets.


Andrei

_______________________________________________
Dlang-study mailing list
Dlang-study@puremagic.com
http://lists.puremagic.com/cgi-bin/mailman/listinfo/dlang-study

On Tuesday, 3 November 2015 at 00:36:35 UTC, Andrei Alexandrescu wrote:
> On 11/02/2015 06:28 PM, Martin Nowak wrote:
>> What if I want to use an existing (GC) class as ref counted, something
>> like core.thread.Thread?
>
> I suspect converting a GC class to an @rc class would require at least changes wrt escaping addresses of members.
>
>> Tacking @rc on it would break some existing
>> code, but not doing it requires the GC.
>
> Probably that's the general case. But if a class has no public data members and member functions that don't leak member addresses, it stands to reason conversion could break no code.
>
>> Same goes for specifying an allocator to use with a ref counted class.
>
> I don't understand this part, could you please detail.
>
>> It seems like a much better idea to decide that when using the class as
>> opposed to when implementing the class.
>
> Such an approach would need to first define which subset of classes work with both GC and RC. Do we agree on that? For example, classes that escape addresses of data members work safely only with GC. Same goes about classes that escape "this".
>
> For my money, I have long looked for a way to implement reference counting outside of the class definition. I had my mind modeled after the C++ approach, and it does seem like a very sensible idea. Lately, however, I got disabused about the notion because I figured the set of manipulations allowed to GC classes largely surpasses the set of manipulations allowed to RC classes.
>
> I am curious how you think you can define these two subsets.
>
>
> Andrei

Why not just do something like a C++ shared_ptr with compiler support?

If @rc was treated like a storage class, it could mean "this type is a compiler assisted shared_ptr". Any D type could then be @rc, including those that needed to escape an @rc aggregate. With compiler support, use of something like 'make_shared' would be implicit, avoiding the potential double allocation of a C++ shared_ptr. Also, taking the address or getting a ref of an aggregate member could implicitly yield an @rc of that type.

Example:

@rc class Foo
{
  int x;

  // error: invalid conversion from @rc(int) to int*
  // int* getNum() { return &x; }

  // ok
  @rc(int) getNum() { return &x; }
}

In the above example, taking the address of Foo::x would yield an @rc(int) which aliased the ref count and data pointers from the @rc(Foo), just like a C++ shared_ptr aliasing ctor, but hidden from the programmer.

   Bit
_______________________________________________
Dlang-study mailing list
Dlang-study@puremagic.com
http://lists.puremagic.com/cgi-bin/mailman/listinfo/dlang-study

On Monday, November 02, 2015 20:43:50 Jakob Bornecrantz wrote:
> On Saturday, 31 October 2015 at 10:19:12 UTC, Jonathan M Davis wrote:
> > On Saturday, October 31, 2015 03:10:41 Jakob Bornecrantz wrote:
> >> Have you done any investigation how this effects exception
> >> handling and
> >> dynamic upcasts?
[snip]
> > The same goes with casting. Having a root object is irrelevant to casting. As long as the class reference that you're casting refers to an object whose type is either the target type or a type derived from the target type, then the cast will succeed. And if it doesn't, the result will be null. I don't see how the existence of a root object would affect that.
> >
> > Remember that C++ doesn't have a root object, and exceptions and casting work just fine there.
>
> Correct, but vtable will effect it. Casts on classes in D are dynamic by default unless they are downcasts.
>
> In C++ dynamic_cast wont work on classes that doesn't
> have a vtables, see example http://vp.dav1d.de/5ZOm?cpp I had
> to check for myself since it was a long time I programmed
> in C++ thanks to D. :)

Since we only have one cast operator in D, I'd fully expect it to be able to sort this out. If an @rc class has no vtable, then there's no need to be concerned with dynamic casts, and if it does, then casting to other classes in its hierarchy should work as it does now - it's just that Object and any class hierarchy built on it wouldn't be involved.

> I think just slapping @rc on Throwable will probably break code that is out there in the wild.

At minimum, @rc would have to be inherited by Throwable's descendants, or it would definitely break code. Whether the difference in semantics would break code or not, I don't know. I don't _think_ so, but I could be missing something. Regardless, we really should find a way a to improve exceptions so that they don't have to be GC-allocated (whether that's via ref-counting or some other mechanism) so that exceptions can reasonably be used in @nogc code, and given the fact that 99.999999% of the time, exceptions are allocated, thrown, caught (maybe rethrown and recaught a few times), and then no longer referenced anywhere in the code (rather than kept around somewhere after the last catch block is exited), you'd think that it would make sense for us to have a mechanism for the exception to be automatically destroyed and its memory freed after that last catch block rather than leaving it as garbage for the GC to clean up who-knows-when. And as long as that mechanism doesn't destroy the exception if references to it exist after the last catch block is exited, then I wouldn't think that doing something like making Throwable @rc would break code.

So, it seems to me that we should at least explore the possibility of making Throwable and its descendants @rc and see if that will solve those problems for us. If it doesn't, then we need to find a different solution for that - or maybe it's an indicator that @rc as currently described isn't quite where we should be going.

- Jonathan M Davis

_______________________________________________
Dlang-study mailing list
Dlang-study@puremagic.com
http://lists.puremagic.com/cgi-bin/mailman/listinfo/dlang-study

On Thursday, 5 November 2015 at 05:35:48 UTC, Jonathan M Davis wrote:
> So, it seems to me that we should at least explore the possibility of making Throwable and its descendants @rc and see if that will solve those problems for us. If it doesn't, then we need to find a different solution for that - or maybe it's an indicator that @rc as currently described isn't quite where we should be going.

Just Throwable being @rc isn't enough. Exceptions contain additional data, at the least a message, which is often dynamically generated. Releasing the exception object itself would clean up only part of the garbage.
_______________________________________________
Dlang-study mailing list
Dlang-study@puremagic.com
http://lists.puremagic.com/cgi-bin/mailman/listinfo/dlang-study

On Thu, Nov 05, 2015 at 11:19:11AM +0000, Marc Schütz wrote:
> Just Throwable being @rc isn't enough. Exceptions contain additional data, at the least a message, which is often dynamically generated.

That's a bad pattern we should discourage though. Generating a string for an exception makes for less useful exceptions (extracting data from them is harder) and is just generally wasteful because you can avoid it.

Instead, we should be making exception subclasses that hold the data you'd use to generate that string. Those would be user-defined and managed however the data themselves would be... so the Exception destructor is responsible for destroying them if structs or other RC items, or leave them behind for the GC, or whatever.

The string member IMO ought to be outright deprecated and replaced with a generator function, but that's not likely to happen... but still if we need it, I'd say leave it as garbage collected and tell the user it just isn't the best way to go.

_______________________________________________
Dlang-study mailing list
Dlang-study@puremagic.com
http://lists.puremagic.com/cgi-bin/mailman/listinfo/dlang-study

> 
> 5 нояб. 2015 г., в 16:12, destructionator@gmail.com написал(а):
> 
>> On Thu, Nov 05, 2015 at 11:19:11AM +0000, Marc Schütz wrote:
>> Just Throwable being @rc isn't enough. Exceptions contain additional data,
>> at the least a message, which is often dynamically


> Instead, we should be making exception subclasses that hold the data you'd use to generate that string. Those would be user-defined and managed however the data themselves would be... so the Exception destructor is responsible for destroying them if structs or other RC items, or leave them behind for the GC, or whatever.

FYI https://github.com/D-Programming-Language/druntime/pull/1411

I believe it's one breaking change that just must happen.

> 
> _______________________________________________
> Dlang-study mailing list
> Dlang-study@puremagic.com
> http://lists.puremagic.com/cgi-bin/mailman/listinfo/dlang-study

_______________________________________________
Dlang-study mailing list
Dlang-study@puremagic.com
http://lists.puremagic.com/cgi-bin/mailman/listinfo/dlang-study

On Thursday, 5 November 2015 at 13:12:20 UTC, destructionator wrote:
> On Thu, Nov 05, 2015 at 11:19:11AM +0000, Marc Schütz wrote:
>> Just Throwable being @rc isn't enough. Exceptions contain additional data, at the least a message, which is often dynamically generated.
>
> That's a bad pattern we should discourage though. Generating a string for an exception makes for less useful exceptions (extracting data from them is harder) and is just generally wasteful because you can avoid it.

The message was just the most glaring example, but it applies to other members as well.

>
> Instead, we should be making exception subclasses that hold the data you'd use to generate that string. Those would be user-defined and managed however the data themselves would be... so the Exception destructor is responsible for destroying them if structs or other RC items, or leave them behind for the GC, or whatever.

This is not a nice solution for Phobos exceptions, though. It requires transferring ownership of data to the exception. For the message member, this means that you have to call .idup if you want to pass a string literal or a string that you actually want to be garbage collected.

How about this, though?

struct Droppable(T) {
    T payload;
    void delegate(const(T)* object) dropper;
    alias payload this;
    @disabled this(this);
    ~this() {
        if(dropper) dropper(payload);
    }
}

class Exception {
    Droppable!string msg;
    // ...
}

I think this is a very useful pattern. We can then add helpers that convert GC, RC, Unique or otherwise managed objects that create Droppable wrappers from those, setting the dropper to a function that decrements the refcount, directly releases the memory, or is a no-op. And because the dropper is a delegate, it can even be a member of an allocator.
_______________________________________________
Dlang-study mailing list
Dlang-study@puremagic.com
http://lists.puremagic.com/cgi-bin/mailman/listinfo/dlang-study

Le 5 nov. 2015 à 9:55, Marc Schütz <schuetzm@gmx.net> a écrit :
> How about this, though?
> 
> struct Droppable(T) {
>    T payload;
>    void delegate(const(T)* object) dropper;
>    alias payload this;
>    @disabled this(this);
>    ~this() {
>        if(dropper) dropper(payload);
>    }
> }
> 
> class Exception {
>    Droppable!string msg;
>    // ...
> }
> 
> I think this is a very useful pattern. We can then add helpers that convert GC, RC, Unique or otherwise managed objects that create Droppable wrappers from those, setting the dropper to a function that decrements the refcount, directly releases the memory, or is a no-op. And because the dropper is a delegate, it can even be a member of an allocator.

But how do you prevent the payload from being leaked in a way that it outlives the droppable? For instance, someone might want to log the exception message and put it into a queue for a separate logging thread to process. You're giving raw access to the string, so it's very easy mess things up if you are not careful.

-- 
Michel Fortin
michel.fortin@michelf.ca
https://michelf.ca


_______________________________________________
Dlang-study mailing list
Dlang-study@puremagic.com
http://lists.puremagic.com/cgi-bin/mailman/listinfo/dlang-study

On Friday, 6 November 2015 at 02:44:39 UTC, Michel Fortin wrote:
> But how do you prevent the payload from being leaked in a way that it outlives the droppable? For instance, someone might want to log the exception message and put it into a queue for a separate logging thread to process. You're giving raw access to the string, so it's very easy mess things up if you are not careful.

Well, that requires borrowing/sealed references:

struct Droppable(T) {
    private T payload;
    void delegate(ref T object) dropper;
    ref T borrow() return { return payload; }
    alias borrow this;
    @disabled this(this);
    ~this() {
        if(dropper) dropper(payload);
    }
}

But that's a separate topic. My point was that this is a mechanism to abstract away from specific ownership schemes without making the ownership part of the type (i.e. `Exception` can be used as-is no matter whether the message if garbage collected or refcounted or static).
_______________________________________________
Dlang-study mailing list
Dlang-study@puremagic.com
http://lists.puremagic.com/cgi-bin/mailman/listinfo/dlang-study