On Sunday, 9 April 2017 at 10:22:49 UTC, Atila Neves wrote:
> I did not. Thanks for telling me!
>
> The way I wrote it RefCounted!(shared T) works - RefCounted doesn't have to be shared itself, but I guess it could be.

I think the other design is slightly more correct, having a single thread own a shared value => RefCounted!(shared T), having multiple threads own a value (which is transitively shared) => shared(RefCounted!T).

The latter is also neede for `static shared RC!T rc;`.

On Sunday, 9 April 2017 at 08:56:52 UTC, Atila Neves wrote:
> http://code.dlang.org/packages/automem

You might find my own containers interesting, especially

https://github.com/nordlow/phobos-next/blob/master/src/array_ex.d

Supports all the different ways I could think an array needs to work:
https://github.com/nordlow/phobos-next/blob/master/src/array_ex.d#L64
https://github.com/nordlow/phobos-next/blob/master/src/array_ex.d#L90
https://github.com/nordlow/phobos-next/blob/master/src/array_ex.d#L1677

Does not (yet) support custom allocators, though. But that can be added, if requested.

Are now in extensive use in my (non-public) applications.

On Sunday, 9 April 2017 at 13:59:14 UTC, Andrei Alexandrescu wrote:
>> . The allocator has to be specified as part of the type: this means the user can choose how to store it in the smart pointer, which for singletons (e.g. Mallocator) or stateless allocators means they can take up zero space. If a singleton (or the default theAllocator), the allocator doesn't need to be passed in to the constructor, otherwise it does. Specifying, e.g. Mallocator also means the relevant code can be marked @nogc.
>
> After extensively studying how C++ allocator framework works, I got to the notion that making the allocator part of the type is an antipattern.

Just repeating an older argument of mine which didn't make it during the std.allocator discussions (http://forum.dlang.org/post/ubithltzbtdypaegnhvi@forum.dlang.org).

Yes, we clearly want type erasure for the Allocator, because having the Allocator as part of the type tends to push Allocator choices/forwarding everywhere into the APIs. It also prevents conversion/copying values with different allocators.
The obvious solution is to use an Allocator interface and/or a custom deleter function (deleter was needed for attribute correct destruction with polymorphic RC!Klass, see [¹]).

Now as Chandler Carruth mentions,

https://www.youtube.com/watch?v=fHNmRkzxHWs&t=3950
https://www.youtube.com/watch?v=fHNmRkzxHWs&t=4037

, an interface with dynamic dispatch would prevent optimizing away redundant allocations.
Allocations are complex enough that I'm not too worried about the virtual call overhead itself.

I think we might be able to solve this problem in D by making IAllocator.allocate pure, which tells the compiler that this function returns a fresh piece of memory without any side-effect, i.e. enough information to optimize away allocations.

Pure might be too restrictive for some allocators, but maybe this can be solved with a little type system hack (or at worse a compiler exemption).

[¹]: https://github.com/MartinNowak/phobos/commit/8cf0ec29ad65ac2a13bd6917b4ff3da0fdea5ab0#diff-4e008aedb3026d4a84f58323e53bf017R4896

On Tuesday, 11 April 2017 at 09:53:46 UTC, Martin Nowak wrote:
> I think we might be able to solve this problem in D by making IAllocator.allocate pure, which tells the compiler that this function returns a fresh piece of memory without any side-effect, i.e. enough information to optimize away allocations.
>
> Pure might be too restrictive for some allocators, but maybe this can be solved with a little type system hack (or at worse a compiler exemption).

In LDC we have an attribute for that `allocSize` (https://github.com/ldc-developers/druntime/blob/ldc/src/ldc/attributes.d#L16)

perhaps this attribute should be used across compilers and be in druntime?

On Tuesday, 11 April 2017 at 08:09:15 UTC, Martin Nowak wrote:
> On Sunday, 9 April 2017 at 10:22:49 UTC, Atila Neves wrote:
>> I did not. Thanks for telling me!
>>
>> The way I wrote it RefCounted!(shared T) works - RefCounted doesn't have to be shared itself, but I guess it could be.
>
> I think the other design is slightly more correct, having a single thread own a shared value => RefCounted!(shared T), having multiple threads own a value (which is transitively shared) => shared(RefCounted!T).
>
> The latter is also neede for `static shared RC!T rc;`.

Unfortunately I later remembered that because it has a destructor it can't be shared and not shared with the sample implementation. And I'd really like to avoid having to have two different names like Rust does with Rc and Arc.

Atila

On Tuesday, 11 April 2017 at 10:24:08 UTC, Nicholas Wilson wrote:
> In LDC we have an attribute for that `allocSize` (https://github.com/ldc-developers/druntime/blob/ldc/src/ldc/attributes.d#L16)
>
> perhaps this attribute should be used across compilers and be in druntime?

Nice, if pure required strong purity, it would be quite a huge hack, so a specific attribute seems friendlier.

On Monday, 10 April 2017 at 08:31:28 UTC, Atila Neves wrote:
>> ```d
>> import std.experimental.allocator.mallocator;
>> UniqueArray!(int, Mallocator) a;
>> a ~= [0,1];
>> ```

So the difference between std.container.Array and UniqueArray is that the latter supports allocators?

On Tuesday, 11 April 2017 at 22:32:51 UTC, Martin Nowak wrote:
> On Monday, 10 April 2017 at 08:31:28 UTC, Atila Neves wrote:
>>> ```d
>>> import std.experimental.allocator.mallocator;
>>> UniqueArray!(int, Mallocator) a;
>>> a ~= [0,1];
>>> ```
>
> So the difference between std.container.Array and UniqueArray is that the latter supports allocators?

That's the general idea, but I confess I didn't even look at Array.

Atila

On Monday, 10 April 2017 at 08:11:37 UTC, Atila Neves wrote:
> On Sunday, 9 April 2017 at 13:59:14 UTC, Andrei Alexandrescu wrote:

>> Great. Can RefCounted itself be shared? I learned this is important for composition, i.e. you want to make a RefCounted a field in another object that is itself shared, immutable etc.
>
> Since it has a destructor, no:
>
> http://forum.dlang.org/post/sqazguejrcdtjimtjxtz@forum.dlang.org
>
> The only way to do that would be to split it into two. Which I guess I could with a template mixin implementing the guts.

Syntax is not the core of the issue, it's not about just marking a destructor as shared. Making RefCounted itself shared would require implementing some form of synchronization of all the 'dereference' operations, including assignments. I.e. if we have some shared(RefCounted!T) ptr, what should happen when two threads simultaneously attempt to do ptr = shared(RefCounted!T)(someNewValue) ? Should a library implementation even consider this? Or should such synchronization be left to client's care? It seems like in this regard shared(RefCounted!T) would be no different from shared(T*), which brings me to the next point.

On Andrei's comment regarding composition: if we're thinking about the design of shared data, IMHO, copying and shared are (or at least, should be) mutually exclusive. The only exception being references (pointers), and even for those care should be taken (which is one of the reasons of even having something like a RefCounted). If you can make a copy, there is no reason to share, or, conversely, if you intend to share, why would you copy?
From that follows that a shared object should not ever have a RefCounted field. A Unique field, perhaps, but not a RefCounted. RefCounted's purpose is to be copied, not shared between threads. Unless I'm missing something.

On Wednesday, 12 April 2017 at 13:32:36 UTC, Stanislav Blinov wrote:
> Syntax is not the core of the issue, it's not about just marking a destructor as shared. Making RefCounted itself shared would require implementing some form of synchronization of all the 'dereference' operations, including assignments. I.e. if we have some shared(RefCounted!T) ptr, what should happen when two threads simultaneously attempt to do ptr = shared(RefCounted!T)(someNewValue) ? Should a library implementation even consider this? Or should such synchronization be left to client's care? It seems like in this regard shared(RefCounted!T) would be no different from shared(T*), which brings me to the next point.

If we can control memory layout, we can do what shared_ptr does and couple the reference counter with the object, then we can have just one pointer:

struct RefCounted(T)
{
  struct Wrapper
  {
    int count;
    T payload;
  }
  Wrapper* payload;
}