When std.experimental.allocator was created, I was under the impression it was meant to be used to make containers. But since allocate() returns void[], casts are required before using that memory, which is unsafe.

Is my only option to wrap the casts in an @trusted helper function? or am I missing something?

 Thanks

On Saturday, 21 January 2017 at 02:47:49 UTC, bitwise wrote:
> [...]

I can see that 'TypedAllocator' exists, but with a different interface than the rest of the allocators. Why wouldn't all allocators just have a common interface? Why wouldn't Mallocator take a template parameter allowing usage with 'ubyte' or 'void' for uninitialized memory via 'allocate' or 'allocateUninitialized' functions?

For my own purposes, I would need my containers to have a default allocator(like Mallocator), and support an aligned allocator(doesn't seem to be present in std.experimental).

On Saturday, 21 January 2017 at 02:47:49 UTC, bitwise wrote:
> When std.experimental.allocator was created, I was under the impression it was meant to be used to make containers. But since allocate() returns void[], casts are required before using that memory, which is unsafe.
>
> Is my only option to wrap the casts in an @trusted helper function? or am I missing something?
>
>  Thanks

System functions that do memory allocation, give you a memory block as a void*. So a cast is anyway required to cast to the required data type.

Allocators don't cast from void* because you would need to implement the same logic in each allocator. Therefore there are functions like make/makeArray/dispose that take an allocator, cast and initialize data.

As for safity part. Allocators can't provide the same safity as the garbage collector anyway, because if you use manual memory management you get all the problems like you can pass to deallocate() a wrong pointer or to free a memory block twice. But with the right abstractions and enough testing, you can reduce such problems and make the code more trusted.

Alligned allocator isn't present in the std.experimental.allocator because IAllocator has a method alignedAllocate() (or similar) - so any allocator can be designed to provide aligned allocations.

On Saturday, 21 January 2017 at 05:06:07 UTC, Eugene Wissner wrote:
> Alligned allocator isn't present in the std.experimental.allocator because IAllocator has a method alignedAllocate()

I hadn't noticed this, but I immediately see two problems:

1) there is no alignedDeallocate() which is needed because aligned allocators usually prepend metadata containing a pointer to the actual start of the memory to be passed to free(). So deallocate() can't know if the memory came from allocate() or alignedAllocate(), and hence does not now how to properly free the memory.

2) The whole point of using an allocator is that is allocates memory in a different way, but provides a standard interface. It shouldn't be up to a container to know whether to call allocate() or alignedAllocate(). A container should simply call allocate() of whatever allocator it was given.

On Sat, 21 Jan 2017 05:52:49 +0000, bitwise wrote:

> On Saturday, 21 January 2017 at 05:06:07 UTC, Eugene Wissner wrote:
>> Alligned allocator isn't present in the std.experimental.allocator because IAllocator has a method alignedAllocate()
> 
> I hadn't noticed this, but I immediately see two problems:
> 
> 1) there is no alignedDeallocate() which is needed because aligned
> allocators usually prepend metadata containing a pointer to the actual
> start of the memory to be passed to free(). So deallocate() can't know
> if the memory came from allocate() or alignedAllocate(), and hence does
> not now how to properly free the memory.

Check the code. CAllocatorImpl's implementation of alignedAllocate returns null if the implementation doesn't support alignedAllocate. (CAllocatorImpl is the provided implementation of IAllocator.)

> 2) The whole point of using an allocator is that is allocates memory in
> a different way, but provides a standard interface. It shouldn't be up
> to a container to know whether to call allocate()
> or alignedAllocate(). A container should simply call allocate() of
> whatever allocator it was given.

If you need aligned memory, you need to use alignedAllocate. You could unconditionally wrap the given allocator using BitmappedBlock, but this adds unnecessary overhead if the user passed in an allocator that supports aligned allocation.

On Saturday, 21 January 2017 at 05:52:49 UTC, bitwise wrote:
> I hadn't noticed this, but I immediately see two problems:
>
> 1) there is no alignedDeallocate() which is needed because aligned allocators usually prepend metadata containing a pointer to the actual start of the memory to be passed to free(). So deallocate() can't know if the memory came from allocate() or alignedAllocate(), and hence does not now how to properly free the memory.
>
> 2) The whole point of using an allocator is that is allocates memory in a different way, but provides a standard interface. It shouldn't be up to a container to know whether to call allocate() or alignedAllocate(). A container should simply call allocate() of whatever allocator it was given.

yes, agreed. alignedAllocate() seems then to make the interface complexer without reason.

On Saturday, 21 January 2017 at 02:47:49 UTC, bitwise wrote:
> When std.experimental.allocator was created, I was under the impression it was meant to be used to make containers. But since allocate() returns void[], casts are required before using that memory, which is unsafe.
>
> Is my only option to wrap the casts in an @trusted helper function? or am I missing something?
>
>  Thanks

There's no need for casts. You can just use the high-level wrappers:
http://dlang.org/phobos-prerelease/std_experimental_allocator#.make

http://dlang.org/phobos-prerelease/std_experimental_allocator#.makeArray

http://dlang.org/phobos-prerelease/std_experimental_allocator#.dispose

http://dlang.org/phobos-prerelease/std_experimental_allocator#.makeMultidimensionalArray

http://dlang.org/phobos-prerelease/std_experimental_allocator#.disposeMultidimensionalArray

On Saturday, 21 January 2017 at 09:56:29 UTC, ZombineDev wrote:
> On Saturday, 21 January 2017 at 02:47:49 UTC, bitwise wrote:
>> When std.experimental.allocator was created, I was under the impression it was meant to be used to make containers. But since allocate() returns void[], casts are required before using that memory, which is unsafe.
>>
>> Is my only option to wrap the casts in an @trusted helper function? or am I missing something?
>>
>>  Thanks
>
> There's no need for casts. You can just use the high-level wrappers:
> http://dlang.org/phobos-prerelease/std_experimental_allocator#.make
>
> http://dlang.org/phobos-prerelease/std_experimental_allocator#.makeArray
>
> http://dlang.org/phobos-prerelease/std_experimental_allocator#.dispose
>
> http://dlang.org/phobos-prerelease/std_experimental_allocator#.makeMultidimensionalArray
>
> http://dlang.org/phobos-prerelease/std_experimental_allocator#.disposeMultidimensionalArray

Though, since you are implementing a container that manually manages memory, you won't be able to get away with no @trusted parts in your code. http://dlang.org/phobos-prerelease/std_experimental_allocator#.expandArray can help, but you'll still need to call dispose/deallocate somewhere yourself and you're the only that knows when is safe to do this.
If you look at how Rust's standard library is implemented, you'll see the same pattern. The users have a safe API, but underneath you'll that the library uses many 'unsafe' blocks where the compiler doesn't see the full picture which only the author of the code knows.

On Sat, 21 Jan 2017 04:01:06 +0000, bitwise wrote:

> On Saturday, 21 January 2017 at 02:47:49 UTC, bitwise wrote:
>> [...]
> 
> I can see that 'TypedAllocator' exists, but with a different interface than the rest of the allocators. Why wouldn't all allocators just have a common interface?

TypedAllocator combines several allocators, using a series of Policies to determine which one to use for a given type.

The interface for allocators is mostly given by IAllocator and mostly implemented with UFCS functions defined in std.experimental.allocator.package. The one aspect that isn't given with IAllocator (because OOP doesn't support it well) is that not all allocators have to support aligned allocations.

TypedAllocator overrides some of those default implementations in order to use its Policies. It doesn't expose the untyped variants so that you can't get around its Policies.

Unfortunately, there's no way to require that a TypedAllocator return an aligned allocation aside from using an aligned allocator for the relevant policy.

On 01/21/2017 01:06 AM, Eugene Wissner wrote:
> On Saturday, 21 January 2017 at 05:52:49 UTC, bitwise wrote:
>> I hadn't noticed this, but I immediately see two problems:
>>
>> 1) there is no alignedDeallocate() which is needed because aligned
>> allocators usually prepend metadata containing a pointer to the actual
>> start of the memory to be passed to free(). So deallocate() can't know
>> if the memory came from allocate() or alignedAllocate(), and hence
>> does not now how to properly free the memory.
>>
>> 2) The whole point of using an allocator is that is allocates memory
>> in a different way, but provides a standard interface. It shouldn't be
>> up to a container to know whether to call allocate() or
>> alignedAllocate(). A container should simply call allocate() of
>> whatever allocator it was given.
>
> yes, agreed. alignedAllocate() seems then to make the interface
> complexer without reason.

alignedAllocate provides access to OS/clib-provided primitives for aligned allocation. Those don't require a special deallocation function, see e.g. http://en.cppreference.com/w/c/memory/aligned_alloc. -- Andrei