On Saturday, 13 February 2016 at 01:42:08 UTC, Dicebot wrote:
> On 02/13/2016 03:35 AM, Andrei Alexandrescu wrote:
>> Folks who defined refcounted types use the special AffixAllocator internally in an encapsulated manner. They define it, they use it - outside intervention is not possible. Those who use their own allocation calls and then use assumeUnique cannot use it on the new types. Or maybe I'm not understanding something.
>
> Ah probably this is the important point I have totally missed. So you envision this kind of metadata support to be only available via specific type of allocator, AffixAllocator, and not being supported by most of them?

That's the most important part of the current design of std.allocator, IMO:
1) allocators are made to be composabled - they only perform suballocations.
2) allocators are first class value types - this makes them usable in pure code. Also only `shared` Allocators objects can be used by more than one thread.

On Saturday, 13 February 2016 at 00:30:58 UTC, Andrei Alexandrescu wrote:
> On 02/12/2016 06:52 PM, deadalnix wrote:
>> [...]
>
> I think we're good there. -- Andrei

Is there somewhere where I / others can see an explanation of how "we're good"? Those sound like genuine problems.

On 2/13/16 7:40 AM, John Colvin wrote:
> On Saturday, 13 February 2016 at 00:30:58 UTC, Andrei Alexandrescu wrote:
>> On 02/12/2016 06:52 PM, deadalnix wrote:
>>> [...]
>>
>> I think we're good there. -- Andrei
>
> Is there somewhere where I / others can see an explanation of how "we're
> good"? Those sound like genuine problems.

Those are just a soup of generic considerations that could be brought about any code. -- andrei

On 2/12/16 10:10 PM, tsbockman wrote:
> Cool. Then this technique could also be used to address Timon Gehr's
> concerns about violating the type system, if necessary.

Regarding that. Not sure what he meant, but it dawned on me we've never mentioned in the language spec that using synchronization unnecessarily for non-shared data and also using no synchronization (in the same thread) is fine. -- Andrei

On 13.02.2016 03:35, Andrei Alexandrescu wrote:
> On 02/12/2016 09:21 PM, Timon Gehr wrote:
>> Const could also mean mutable. This can hence reference the same data as
>> both shared and unshared, which violates the type system.
>
> If const comes from mutable, then shared is superfluous leading to extra
> synchronization. That's suboptimal, but how does it violate the type
> system? -- Andrei

One point of 'shared' is that data which is _not_ shared cannot be mutated by some other thread while you are manipulating it.

'shared' data can be accessed by multiple threads simultaneously.
Unshared data can be accessed by only one thread at a time.
'shared' data can be passed freely among threads.

If you alias the same data as both shared and unshared, the shared version can be sent to another thread which will then modify it, violating the guarantees on the unshared reference.

On 02/13/2016 11:55 AM, Timon Gehr wrote:
> If you alias the same data as both shared and unshared, the shared
> version can be sent to another thread which will then modify it,
> violating the guarantees on the unshared reference.

Oh, I understand. So you're thinking of someone taking the address of allocator.prefix(block) and sharing it across threads? -- Andrei

On 13.02.2016 18:42, Andrei Alexandrescu wrote:
> On 02/13/2016 11:55 AM, Timon Gehr wrote:
>> If you alias the same data as both shared and unshared, the shared
>> version can be sent to another thread which will then modify it,
>> violating the guarantees on the unshared reference.
>
> Oh, I understand. So you're thinking of someone taking the address of
> allocator.prefix(block) and sharing it across threads? -- Andrei

Not necessarily. shared is transitive and prefix/suffix are arbitrary types which might contain mutable indirections.

I don't really see why this part of the allocator interface should even be typed.

On Saturday, 13 February 2016 at 18:16:43 UTC, Timon Gehr wrote:
> On 13.02.2016 18:42, Andrei Alexandrescu wrote:
>> On 02/13/2016 11:55 AM, Timon Gehr wrote:
>>> If you alias the same data as both shared and unshared, the shared
>>> version can be sent to another thread which will then modify it,
>>> violating the guarantees on the unshared reference.
>>
>> Oh, I understand. So you're thinking of someone taking the address of
>> allocator.prefix(block) and sharing it across threads? -- Andrei
>
> Not necessarily. shared is transitive and prefix/suffix are arbitrary types which might contain mutable indirections.
>
> I don't really see why this part of the allocator interface should even be typed.

I think it would be a lot simpler if we can consider allocations from unshared allocator objects to be unshareable. This rules out GCAllocator and Mallocator, but thread-local allocators can offer better performance anyway.

On Friday, 12 February 2016 at 23:29:58 UTC, Timon Gehr wrote:
> The first thing that comes to mind is that accessing a global associative array is not 'pure'.

Not necessarily. Purity depends on the allocator and how you access it (via parameter or through a static/shared instance).
I have this code in my project: (note **pure** unittest)

/// UniqueRef is encapsulates T* or T (if (is(T == A[], A))
/// It is designed to ensure unique ownership and to
/// provide transparent access to its payload
struct UniqueRef(T, alias Allocator = Mallocator)
   if (isScalarValueType!T || isDynamicArray!T)
{ /* ... */ }

///
pure nothrow @safe
// not @nogc because the alias template parameter cause GC context allocation :(
// Maybe make the Allocator a ctor parameter of UniqueRef?
unittest
{
   // Test forwarding of operators for arrays of user defined types

   auto allocator = MyStackAllocator!1024();

   assert (allocator.usedBytes == 0);
   assert (allocator.allocationCount == 0);

   {
      auto arr = UniqueRef!(Vec3[], allocator)(4);
      assert (arr.length == 4);
      assert (arr.ptr !is null);

      assert (arr.ptr == allocator.bytes.ptr);
      assert (allocator.usedBytes == 4 * Vec3.sizeof);
      assert (allocator.allocationCount == 1);

      arr[] = Vec3(3, 2, 1);
      Vec3[4] expected1 = [
         Vec3(3, 2, 1), Vec3(3, 2, 1),
         Vec3(3, 2, 1), Vec3(3, 2, 1)
      ];
      assert (arr[] == expected1);

      arr[] += Vec3(1, 1, 1);
      Vec3[4] expected2 = [
         Vec3(4, 3, 2), Vec3(4, 3, 2),
         Vec3(4, 3, 2), Vec3(4, 3, 2)
      ];
      assert (arr[] == expected2);
   }

   assert (allocator.usedBytes == 0);
   assert (allocator.allocationCount == 0);
}

I think I can accomodate a SharedRef that uses the proposed affix allocator fairly easily in my current design. And it would still be conditionally pure (depending on the payload). I just haven't found the need to share ownership in my project yet.

2016-02-12 20:12 GMT+01:00 Andrei Alexandrescu via Digitalmars-d < digitalmars-d@puremagic.com>:

> https://github.com/D-Programming-Language/phobos/pull/3991
>

The only difference between an approach based on an associative array and
> AffixAllocator is that the latter is faster (the association is fixed by
> layout).
>
>
Could you point to your benchmark / paper about this ?

The general idea sounds good, however I share deadalnix's concern over storing the metadata next to the user's data. I also fail to see how those are a "soup of generic considerations", given it only apply when using an affix.