D classes are distinct from structs because they are intended to be bound to a reference (pointer) and not addressed as a value (inline/copying).

But for efficiency reasons, scoped classes can stack-allocate, but my understanding is that the compiler can still allocate it on the GC heap? So it is still a reference type, I guess?

But how about "emplace", does "emplace" imply the the compiler cannot put it on the GC heap? Yes, I am aware that this is library construct, but still relevant.

The reason I am asking this is because implementing reference counting in a clean fashion requires classes to be proper reference types, so that means disabling scoped classes and emplaced classes, and only allow it where a reference/pointer to a class never exceed a reference count of 1.

The core of my question is this: is it at all reasonable for "user code" to assume that a class is not allocated on the heap? Is it reasonable to write code that assumes that it is allocated "inline" as with emplace? Because if it is, then then the advantages of having classes as reference types and making them different from structs are lost.

On Thursday, 24 June 2021 at 06:50:44 UTC, Ola Fosheim Grøstad wrote:

Yes, class refs are always pointers. scope classes are deprecated (I don't think I've ever seen one); with scope c = new Object, you can have the compiler allocate a class instance on the stack for you, but c is still a ref. emplace doesn't allocate, you have to pass the memory explicitly. A class instance can also live in the static data segment (static immutable myStaticObject = new Object;); extern(C++) class instances can also live on the C++ heap/stack etc. etc.

On Thursday, 24 June 2021 at 07:28:56 UTC, kinke wrote:

I used it for my database thing where it is supposed to be destroyed reliably but also uses runtime polymorphism.

I now suggest people just stick scope(exit) .destroy(obj); any time you use it.

On Thursday, 24 June 2021 at 07:28:56 UTC, kinke wrote:

The dmd frontend uses them all the time to avoid allocation overhead for Visitors.

On Thursday, 24 June 2021 at 12:31:08 UTC, Stefan Koch wrote:

I was talking about not having seen a scope class C { ... }, not the scope storage class as in the example.

On Thursday, 24 June 2021 at 07:28:56 UTC, kinke wrote:

But the user code cannot depend on it being stack allocated? So I could replace the Object reference with a reference counting pointer and put the counter at a negative offset?

This is more of a problem. I was thinking about arrays that provide an emplace method, then one could replace emplace with heap allocation. I guess it isn't really possible to make emplace with custom memory work gracefully with reference counting with ref count at negative offset.

But it isn't required to? It certainly wouldn't work with reference counting if it is stored in read only memory...

Yes, that cannot be avoided.

On Friday, 25 June 2021 at 06:09:17 UTC, Ola Fosheim Grøstad wrote:

Well AFAIK it's mandated by the language, so an RC scheme replacing such allocations by heap ones seems like a definite step backwards - it's a useful pattern, and as Stefan pointed out, definitely in use. You could still stack-allocate but accommodate for the counter prefix in the compiler.

It's certainly possible as it's a library thing; some existing code may assume the returned reference to point to the beginning of the passed memory though (where there'd be your counter). What you'd definitely need to adjust is __traits(classInstanceSize), accomodating for the extra counter prefix.
There's very likely existing code out there which doesn't use druntime's emplace[Initializer], but does it manually.

Not required to AFAIK, but if it's not statically allocated, you'd need to allocate it at runtime via some module or CRT ctor. It's probably easier to have the compiler put it into static but writable memory, so that you can mess with the counter.

All in all, I think a more interesting/feasible approach would be abusing the monitor field of extern(D) classes for the reference counter. It's the 2nd field (of pointer size) of each class instance, directly after the vptr (pointer to vtable). I think all monitor access goes through a druntime call, so you could hook into there, disallowing any regular monitor access, and put this (AFAIK, seldomly used) monitor field to some good use.

Wrt. manual non-heap allocations (stack/data segment/emplace etc.), you could e.g. reserve the most significant bit of the counter to denote such instances and prevent them from being free'd (and possibly finalization/destruction too; this would need some more thought I suppose).

On Friday, 25 June 2021 at 07:01:31 UTC, kinke wrote:

Yes, but then I need to mark it as non-freeable.

Yes, if people don't make assumptions about where the class ends and overwrites some other object, but I suspect pointer arithmetics isn't all that common for classes in D code.

I guess, but the compiler could have a release note warning against this.

Another reason to add the ability to mark it as non-freeable.

Yes, if you don't want to support weak pointers. I think you need two counters if you want to enable the usage of weak pointers.

One reason to put it at a negative offset is that it makes it possible to make it fully compatible with shared_ptr. And then you can also have additional fields such as a weak counter or a deallocation function pointer.

I don't think maintaining the D ABI is important, so one could add additional fields to the class. Maintaining core language semantics shouldn't require ABI support I think.

On Friday, 25 June 2021 at 07:17:20 UTC, kinke wrote:

Destruction is a bit tricky. If people rely on the destructor to run when the function returns then that cannot be moved to a reference counter. For instance if they have implemented some kind of locking mechanism or transaction mechanism with classes…

The most tricky one is emplace though as you have no way of releasing the memory without an extra function pointer.

Regarding using high bits in the counter; What you would want is to have a cheap increment/decrement and instead take the hit when the object is released. So you might actually instead want to keep track of the allcation-status in the lower 3 bits and instead do ±8, but I am not sure how that affects different CPUs. The basic idea, would be to make it so you don't trigger destruction on 0, but when the result is/becomes negative.