On Mon, Jan 07, 2019 at 08:31:06PM +0100, Jacob Carlborg via Digitalmars-d wrote:
> On 2019-01-06 04:42, Walter Bright wrote:
[...]
> > Andrei, Razvan, and I have decided that the most pragmatic way forward to support reference counting is to support copy constructors ala C++.  C++ is clearly successful with this approach, and Andrei/Razvan concluded that D's postblit just doesn't cut it. Razvan has a proposal for copy constructors, and an implementation PR that has fallen a bit behind his latest proposal.
> 
> Razvan has confirmed that copy constructors are only for structs [1]. The ARC (opAddRef/opRelease) proposal was for classes as well. So copy constructors are not a substitute as defined now.
[...]

I think the idea is probably to use struct wrappers to implement RC, i.e., you'd have a struct wrapping a class reference of the object it's managing.  As long as the struct can have semantics compatible with RC, it can be used to implement RC. At least in theory.

My suspicion, though, is that at some point further language support is going to be needed, because of interaction with D's const system. The RC count needs to be mutable, yet the managed object must be allowed to be const or immutable in a way that doesn't require onerous, incomplete workarounds (like using a Const template instead of just typing `const RC!MyClass` -- which won't work if you have the RC'd object inside another aggregate that also needs const/immutable qualifiers).

Requiring that RC objects can only be mutable is going to lead to more dissatisfaction about RC in D.


T

-- 
The easy way is the wrong way, and the hard way is the stupid way. Pick one.

On 06.01.19 01:51, Nicholas Wilson wrote:
> On Saturday, 5 January 2019 at 22:42:11 UTC, Meta wrote:
>> On Saturday, 5 January 2019 at 22:05:19 UTC, Manu wrote:
>>> Is progress possible, or is the hard reality that the language is just designed such to be resistant to a quality GC, while the ecosystem sadly tends to rely on it?
>>>
>>> Where's the ARC stuff? What happened to opAddRef/opRelease?
>>
>> As per Andrei's talk at the last Dconf, ref counting requires __mutable to play nicely with const and immutable.
> 
> I'd rather have opHeadMutable than __mutable, does the same thing but doesn't subvert the type system

I'd like opHeadMutable too, but it certainly doesn't do the same thing.

On Mon, Jan 07, 2019 at 02:33:56PM -0500, Steven Schveighoffer via Digitalmars-d wrote:
> On 1/7/19 1:59 PM, Neia Neutuladh wrote:
[...]
> > Like I said before, casting to shared would have to invoke a runtime function. That's a runtime change and compiler change, but I don't think the spec says anything about whether any cast might invoke a runtime function.
> > 
> 
> This is doable. Casting to/from shared should be a rare thing, and probably fine to hook.
> 
> However, the problem is more (for me) that the memory would have to be copied, as the memory is currently in one pool and has to be moved to another pool.
[...]

I think what Neia has in mind, which he mentioned in another post, is to have the druntime function simply pin the object, whichever pool it belongs to.  The idea being to tell the respective thread-local GC "this object might be referenced by another thread, do not collect or move".


T

-- 
"The number you have dialed is imaginary. Please rotate your phone 90 degrees and try again."

On 1/7/19 2:52 PM, H. S. Teoh wrote:
> On Mon, Jan 07, 2019 at 02:33:56PM -0500, Steven Schveighoffer via Digitalmars-d wrote:
>> On 1/7/19 1:59 PM, Neia Neutuladh wrote:
> [...]
>>> Like I said before, casting to shared would have to invoke a runtime
>>> function. That's a runtime change and compiler change, but I don't
>>> think the spec says anything about whether any cast might invoke a
>>> runtime function.
>>>
>>
>> This is doable. Casting to/from shared should be a rare thing, and
>> probably fine to hook.
>>
>> However, the problem is more (for me) that the memory would have to be
>> copied, as the memory is currently in one pool and has to be moved to
>> another pool.
> [...]
> 
> I think what Neia has in mind, which he mentioned in another post, is to
> have the druntime function simply pin the object, whichever pool it
> belongs to.  The idea being to tell the respective thread-local GC "this
> object might be referenced by another thread, do not collect or move".

Imagine I have a shared piece of data, then I cast it to thread local. Now, it is able to point at thread local data, but it lives in the shared heap. Pinned or not, this means a local collection must scan the shared heap to see if it can collect any thread-local data. This defeats the purpose of having a thread-local heap in the first place (which you should be able to scan without having to stop-the-world).

-Steve

On Mon, 07 Jan 2019 11:52:18 -0800, H. S. Teoh wrote:
> I think what Neia has in mind, which he mentioned in another post, is to have the druntime function simply pin the object, whichever pool it belongs to.  The idea being to tell the respective thread-local GC "this object might be referenced by another thread, do not collect or move".

She, not he.

Thinking about this some more, this strategy isn't as cheap or easy as I was hoping. Let's say you're using fearless:

auto a = gcExclusive!Node;
spawn({
  {
    auto tmp = a.lock;
    tmp.parent = new Object;
  }
  GC.threadLocal.collect();
  GC.threadLocal.minimize();
  {
    auto tmp = a.lock;
    writeln(tmp.parent.toString);
  }
});

Where do we insert the runtime call to mark a.parent as shared? Fearless
would have to:
1. Acquire the mutex
2. Tell the current thread's GC to add the locked object as a root
3. Run your code
4. Tell the thread's GC to pin everything it can find from `a` (by casting
back to shared)
5. Release the mutex

This effectively incurs a mark (but not sweep) cycle every time you cast to shared. If it were just pinning one object, that would be a lot easier to sell.

On Mon, Jan 07, 2019 at 03:28:26PM -0500, Steven Schveighoffer via Digitalmars-d wrote:
> On 1/7/19 2:52 PM, H. S. Teoh wrote:
[...]
> > I think what Neia has in mind, which he mentioned in another post, is to have the druntime function simply pin the object, whichever pool it belongs to.  The idea being to tell the respective thread-local GC "this object might be referenced by another thread, do not collect or move".
> 
> Imagine I have a shared piece of data, then I cast it to thread local. Now, it is able to point at thread local data, but it lives in the shared heap.  Pinned or not, this means a local collection must scan the shared heap to see if it can collect any thread-local data. This defeats the purpose of having a thread-local heap in the first place (which you should be able to scan without having to stop-the-world).
[...]

Hmph, good point.  That sux. :-(  I suppose that's where copying/moving the object comes in -- migrate it to a different pool somehow so that we can avoid stop-the-world.


T

-- 
They pretend to pay us, and we pretend to work. -- Russian saying

On Monday, January 7, 2019 1:48:04 PM MST Neia Neutuladh via Digitalmars-d wrote:
> On Mon, 07 Jan 2019 11:52:18 -0800, H. S. Teoh wrote:
> > I think what Neia has in mind, which he mentioned in another post, is to have the druntime function simply pin the object, whichever pool it belongs to.  The idea being to tell the respective thread-local GC "this object might be referenced by another thread, do not collect or move".
>
> She, not he.
>
> Thinking about this some more, this strategy isn't as cheap or easy as I was hoping. Let's say you're using fearless:
>
> auto a = gcExclusive!Node;
> spawn({
>   {
>     auto tmp = a.lock;
>     tmp.parent = new Object;
>   }
>   GC.threadLocal.collect();
>   GC.threadLocal.minimize();
>   {
>     auto tmp = a.lock;
>     writeln(tmp.parent.toString);
>   }
> });
>
> Where do we insert the runtime call to mark a.parent as shared? Fearless
> would have to:
> 1. Acquire the mutex
> 2. Tell the current thread's GC to add the locked object as a root
> 3. Run your code
> 4. Tell the thread's GC to pin everything it can find from `a` (by casting
> back to shared)
> 5. Release the mutex
>
> This effectively incurs a mark (but not sweep) cycle every time you cast to shared. If it were just pinning one object, that would be a lot easier to sell.

Casting between shared and thread-local doesn't have to be free, but it cannot be expensive, because unless you're using atomics, the way that it's designed pretty much requires that you cast away shared (after locking a mutex to protect the object) in order to operate on it. So, if casting away shared is expensive, then code using shared gets screwed.

Also, there isn't anything stopping a programmer from assigning a thread-local reference to something inside an object that has had shared cast away, and there likely isn't any reason for the programmer to cast anything to shared in that scenario. They'd just get rid of the thread-local reference to the shared object before releasing the mutex, meaning that the runtime couldn't even see that what had been assigned to one of its internals had gone from being treated as thread-local to shared. Having objects go from shared to thread-local in such code could be possible as well if a reference is taken from the object which has had shared cast away and then stored as thread-local. In fact that sort of scenario is pretty much exactly what would happen with something like a consumer-producer queue. Programmers dealing with shared need to be aware of such things and deal with them appropriately in order to avoid having problems (which is part of why casting away shared has to be @trusted), but it's perfectly legal so long as no shared object is ever treated as thread-local when it can actually be operated on by multiple threads, and no thread-local objects end up on multiple threads at the same time. Given that sort of situation, I don't see how we can have the GC accurately track whether objects are thread-local or shared. Casting is just too blunt an instrument and allows too much.

- Jonathan M Davis

On Mon, 07 Jan 2019 13:06:22 -0800, H. S. Teoh wrote:
> Hmph, good point.  That sux. :-(  I suppose that's where copying/moving the object comes in -- migrate it to a different pool somehow so that we can avoid stop-the-world.

class Node
{
  enum Type { leaf, intermediate }
  Type type;
  union { Node child; ulong data; }
}
auto node = new Node;
node.type = Node.Type.leaf;
node.data = cast(ulong)cast(void*)node;

How do you copy this?

Pinning avoids those problems, but it still involves scanning memory as if doing a collection.

On Monday, 7 January 2019 at 18:18:05 UTC, H. S. Teoh wrote:
> Yep. GC phobia is a common malady among C/C++ folks (I used to be one of them).

Guilty as charge here!

What I like most about GC is the efficiency! With scanning and a global owner, you don't need to keep ownership information anywhere. Which leads to less copying (std::string leads malloc for every string copy, GC avoids that), slices being two machine words instead of 3, etc.

On Mon, Jan 07, 2019 at 07:13:59PM +0000, Russel Winder via Digitalmars-d wrote: [...]
> Whilst GCs can, indeed do, have problems, some people are too quick to blame the GC when actually the problems are elsewhere in most of the situations.
[...]

Yep, that's typical GC phobia, fueled mostly by prejudice, fear, and ignorance, in the absence of hard evidence.  The GC is a convenient whipping boy to blame for whatever malady you're suffering from when you're not sure what the actual cause is, and a convenient excuse not to use something you don't want to use for reasons you're too embarrassed to say out loud.


T

-- 
MSDOS = MicroSoft's Denial Of Service