On Saturday, 13 February 2016 at 22:16:02 UTC, rsw0x wrote:
> On Saturday, 13 February 2016 at 21:10:50 UTC, Andrei Alexandrescu wrote:
>> On 02/13/2016 01:50 PM, Mathias Lang via Digitalmars-d wrote:
>>> [...]
>>
>> There's no need. I'll do the implementation with the prefix, and if you do it with a global hashtable within the same or better speed, my hat is off to you.
>
> I believe he was referring to BiBoP.


er, nevermind, sorry for the noise. I shouldn't read threads from the end to the start.

On Saturday, 13 February 2016 at 21:12:10 UTC, Andrei Alexandrescu wrote:
> On 02/13/2016 03:07 PM, Iakh wrote:
>> So you can use metadata only with global allocators,
>> until you don't need to save ref to the allocator.
>
> Well you can use other allocators if you save them so you have them available for deallocation. -- Andrei

Yeap. You did catch me. And if you don't have "@intransitive const" or
C++'s mutable you can't save ref to the allocator within object:

struct RCExternal
{
    //alias allocator = theGlobalAllocator;
    auto allocator = &theGlobalAllocator; // assign mutable to immutable

    private void[] data;

    ~this () { allocator.decRef(data.ptr); }
}

On Saturday, 13 February 2016 at 22:01:45 UTC, deadalnix wrote:
> On Saturday, 13 February 2016 at 21:10:50 UTC, Andrei Alexandrescu wrote:
>> There's no need. I'll do the implementation with the prefix, and if you do it with a global hashtable within the same or better speed, my hat is off to you.
>>
>
> That is false dichotomy. What about storing the metadata at an address that is computable from from the object's address, while not being contiguous with the object allocated ? Is substracting a constant really the only option here ? (hint, it is not)

1) False-sharing overhead:
If you look at the case where the allocated data is non-shared, you'll get better cache locality, instead of sharing overhead on the cache coherency system.

2) Unfriendly allocation size:
In general you're correct, but in the end, it depends on the type of data. If you consider the case of shared_ptr - refcount as metadata + ptr as the actual allocation - you get a nice allocation size of 2*size_t.sizeof.
As I see it, AffixAllocator is not a silver bullet, but in some very specific cases it can actaully be a very good fit. You just have to carefully consider this case-by-case. E.g. for shared types with irregular size we can just switch to a different allocator - that's the whole point of the composable allocators.

3) Increased probability/danger of buffer overflow/underflow:
First, I would say that things like slice bounds checks make D a safer language for the average user than C/C++, which should make this a bit less of a problem.
Second, I actually think that even if this suddenly leads to a lot of problems for the end users, this would bring more pressure for adding more Ada/Rust-like static analysis for D - which is a Good Thing.

On 02/13/2016 04:34 AM, Andrei Alexandrescu wrote:
> On 02/12/2016 08:42 PM, Dicebot wrote:
>> So you
>> envision this kind of metadata support to be only available via specific
>> type of allocator, AffixAllocator, and not being supported by most of
>> them?
> 
> That is correct. You want metadata, you put an AffixAllocator together. -- Andrei

Aha, I think this kills most of my here-and-now concerns if it is only planned as opt-in allocator extension. Better to do benchmarking of various approaches to layout when there is rcstring prototype in that case.

Thanks!

On 2/13/16 5:01 PM, deadalnix wrote:
> What about storing the metadata at an address that is computable from
> from the object's address, while not being contiguous with the object
> allocated ? Is substracting a constant really the only option here ?
> (hint, it is not)

I'd say, you have at your disposal a flexible allocator framework so a simple way to go about it would be to implement new allocators that take various options about storing metadata. -- Andrei

On Sunday, 14 February 2016 at 01:27:40 UTC, Andrei Alexandrescu wrote:
> On 2/13/16 5:01 PM, deadalnix wrote:
>> What about storing the metadata at an address that is computable from
>> from the object's address, while not being contiguous with the object
>> allocated ? Is substracting a constant really the only option here ?
>> (hint, it is not)
>
> I'd say, you have at your disposal a flexible allocator framework so a simple way to go about it would be to implement new allocators that take various options about storing metadata. -- Andrei

I have nothing against this as some sort of optin allocation scheme. But my understanding was that it is desirable to wire RC into this.

One thing that I really miss from most of the discussions about RC is support for weak references, which IMO should definitely be supported in the standard RC implementation. Event if they are only needed rarely, there are some things that simply do not work without them. And I don't think they can be implemented efficiently on top of ordinary RC.

For them to work natively, the lifetime of the allocated memory block and that of the reference count must be separate. So they could either simply be a separate allocation, or live in a special reference count store, similar to what Objective-C does.

On 02/14/2016 03:08 AM, Sönke Ludwig wrote:
> For them to work natively, the lifetime of the allocated memory block
> and that of the reference count must be separate.

Not necessarily. C++ makes this work for make_shared by keeping the memory allocated around (but not the object) until the last weak ref goes away.

We can do the same, but we also have a better alternative. Most of our allocators support shrink-in-place. For now I haven't exposed it as a primitive but that's short work. When the object goes away we can shrink memory in place to only the length of the metadata.

Overall: I also think weak refs should be supported, but I know they can be tacked on later.


Andrei

On Sunday, 14 February 2016 at 11:14:59 UTC, Andrei Alexandrescu wrote:
> We can do the same, but we also have a better alternative. Most of our allocators support shrink-in-place. For now I haven't exposed it as a primitive but that's short work. When the object goes away we can shrink memory in place to only the length of the metadata.

This will lead to massive memory fragmentation. When you allocate a lot of objects they tend to be of the same size...

On 02/12/2016 02:12 PM, Andrei Alexandrescu wrote:
> https://github.com/D-Programming-Language/phobos/pull/3991

Finally I got around to update this. Please find holes in my logic. Updated documentation at:

http://dtest.thecybershadow.net/artifact/website-6a8825cc07dee2db751fa4b61044073b228c20c7-b911819df047024a6d304d01957307a3/web/phobos-prerelease/std_experimental_allocator_building_blocks_affix_allocator.html

Thanks! -- Andrei