On Friday, 25 June 2021 at 07:17:20 UTC, kinke wrote:

You cannot use the most significant bit as it will not work with some 32-bit systems. Linux with a 3G kernel position for example. Better to use the least significant bit as all allocated memory is guaranteed to be aligned. Regardless this requires compiler support for masking off this bit.

Now where going into halfway fat pointer support. Then we can just use fat pointers instead and have full freedom.

On Friday, 25 June 2021 at 17:37:13 UTC, IGotD- wrote:

One thing I have found out over the years that if you want to have full versatility, have a pointer to your free function in your fat pointer. By having this you have generic method to free your object when it goes out of scope. You have the ability to use custom allocators and even change allocators on the fly. If you for some reason don't want to free your object automatically, just put zero in that field for example.

On Friday, 25 June 2021 at 17:37:13 UTC, IGotD- wrote:

Hm. Not sure if I follow, I think we are talking about stuffing bits into the counter and not the address?

But fat pointers are 16 bytes, so quite expensive.

On Friday, 25 June 2021 at 20:22:24 UTC, Ola Fosheim Grøstad wrote:

Then I misunderstood. If it's a counter it should be fine.

Yes, that's a tradeoff but one I'm willing to take. I'm thinking even bigger managed pointers of perhaps 32 bytes which has more metadata like the allocated size. Managed languages in general have fat pointers which we see everywhere and it is not a big deal.

If you are littering pointers you perhaps should refactor your code, use an array if loads of objects of the same type. Another thing which I'm not that satisfied with D is that there is no built in method of expanding member classes into the host class like C++ which creates pointer littering and memory fragmentation.

On Friday, 25 June 2021 at 17:05:41 UTC, Ola Fosheim Grøstad wrote:

I cannot imagine how weak pointers would work without an ugly extra indirection layer. If we're on the same page, we're talking about embedding the reference counter directly in the class instance, and the class ref still pointing directly to the instance.

Weak pointers aren't in the language, so I don't see why they would matter here. I thought you were after replacing GC-allocated class instances by a simple RC scheme.

In modern C++ code I've been looking at so far, shared_ptr was used very rarely (and unique_ptr everywhere). AFAIK, the main reason being poor performance due to the extra indirection of shared_ptr. So replacing every D class ref by a shared_ptr-analogon for interop reasons would seem very backwards to me.

On Friday, 25 June 2021 at 23:55:40 UTC, kinke wrote:

So, my understanding is that C++ make_shared may allocate the reference chunk and the object in the same memory area, so that the reference count/weak counter is at a negative offset. That way you can free up the object while retaining the counter. (At some fragmentation cost, if the counter isn't freed.) This may give some cache advantages over separate allocation.

Maybe @weak could be an annotation, but I haven't given this much thought. If we think about it; You don't have to pass around weak pointers, so there is no reason for parameters to be marked weak? Only pointer-fields in structs or classes? In function bodies you would typically not want to use a weak pointer as you want to extend the lifetime of the object until the function returns.

Also you could mark a class as non-weak, to save the weak counter.

One goal could be to make a class compatible with C++ or Swift on request. Both support weak pointers. You could have multiple ref-count layout schemes as long as they all are on negative offsets. Just don't mix class hierarchies. So you could mix a D class hierarchy, C++ class-hiearchy and Swift class-hierarchy in the same codebase?

In C++ ownership should in general be kept local and one should use borrowing pointers (raw pointers) for actual computations, so only use owned pointers for transfer of ownership. But that puts more of a burden on the developer.

On Saturday, 26 June 2021 at 07:00:37 UTC, Ola Fosheim Grøstad wrote:

I'm pretty sure I haven't seen a weak pointer in C++ yet. I don't look at C++ much anymore, but I suspect they are even a lot rarer than shared_ptr. Wrt. mixed class hierarchies, being able to inherit from and instantiate C++ classes in D is of some priority and mostly works today. (Let's not get into discussing multiple inheritance here, it's hardly a show-stopper for most use cases.)

Is Swift a thing outside the Apple universe (I admittedly despise ;))? It's surely much better than their Objective-C crap, but still...

So for rare use cases like shared_ptr/weak pointer interop, a library solution (just like they are in C++) is IMO enough.

On Saturday, 26 June 2021 at 07:39:44 UTC, kinke wrote:

Weak pointers are usually not needed, but sometimes you do need them and then not having them becomes a big weakness. Usually ones strives to have ownership defined in a way that makes it possible to dismantle a graph in a structured way from a single thread, and then weak pointers are not needed at all.

Is it possible to inherit from a C++ class and get a D subclass, and is it possible to inherit from a D class and get a C++ class?

The Apple universe is pretty big, we can dislike that, but that does not change the market…

So the goal would be to put the ref-count on a negative offset so that the layout can match up with C++ or Swift. I don't think that implies any big disadvantages, but I could be wrong. The only way to find out is to hash out the alternatives.

But the best solution is to get to a place where you can hand D-objects to other languages with ease without doing a runtime conversion from one layout to another.

On Saturday, 26 June 2021 at 13:49:25 UTC, Ola Fosheim Grøstad wrote:

Sure thing, with extern(C++) class of course.

With C++, you can today, an extern(C++) class C is equivalent to and mangled as C++ C*. You can't pass it directly to some unique_ptr<C> or shared_ptr<T> of course; an according D wrapper reflecting the C++ implementation (library-dependent) would be needed anyway for correct mangling. It'd be implemented as a templated D struct, similar to how std::vector works today for the MSVC runtime: https://github.com/dlang/druntime/blob/master/src/core/stdcpp/vector.d

On Saturday, 26 June 2021 at 20:03:01 UTC, kinke wrote:

Oh, I've just seen that unique_ptr is already implemented in https://github.com/dlang/druntime/blob/master/src/core/stdcpp/memory.d, and according to the tests, working for the MSVC++ runtime, see https://github.com/dlang/druntime/pull/2723/files.