Hi all,


Eduard, Alexandru Jercaianu and I are working on improving allocators' design and implementation. This entails a few breaking changes.

In order to make matters easier for code using allocators, Sebastian Wilzbach created a dub package freezing the existing API: http://code.dlang.org/packages/stdx-allocator. Please use that until we work the kinks out of allocators - great things are coming! - and after that feel free to upgrade code to use the new and improved allocators.

One rather disruptive piece we're working on is converting IAllocator from an interface to a reference counted structs (that in turn uses an interface internally). This is so there is deterministic control of how many users a given allocator has, so it can be destroyed deterministically.

Another possible work item was raised by https://github.com/dlang/phobos/pull/5879. Currently, allocators traffic in void[]. When I first designed allocators, I considered using ubyte[] instead. Using void[] is somewhat closer to the intent of allocators - that memory is meant to be used for storing anything. However, using void[] makes it difficult to express the fact that allocate() is a safe function for virtual all allocators. So we may switch to ubyte[] to express that at the lowest level we're trafficking chunks of octets. This is not a big design change, but it's bound to break code.


Thanks,

Andrei

On Thursday, 30 November 2017 at 19:01:02 UTC, Andrei Alexandrescu wrote:
> Hi all,
>
>
> Eduard, Alexandru Jercaianu and I are working on improving allocators' design and implementation. This entails a few breaking changes.
>
> [...]

Sounds good!

Please consider -betterC on your refactoring. Would be awesome to have a core allocators package working under betterC flag, meaning no D runtime deps.

On Thursday, 30 November 2017 at 19:01:02 UTC, Andrei Alexandrescu wrote:
> Hi all,
>
>
> Eduard, Alexandru Jercaianu and I are working on improving allocators' design and implementation. This entails a few breaking changes.
>
> In order to make matters easier for code using allocators, Sebastian Wilzbach created a dub package freezing the existing API: http://code.dlang.org/packages/stdx-allocator. Please use that until we work the kinks out of allocators - great things are coming! - and after that feel free to upgrade code to use the new and improved allocators.

I just pushed a couple of fixes [1] for older Phobos versions and stdx-allocator now works down until 2.072.2.
If someone needs an older Phobos version to work with stdx-allocator, please let me know.
Also switching to stdx-allocator is rather easy, e.g.:

```
sed -i "s/std[.]experimental/stdx/g" **/*.d
```

[1] https://github.com/wilzbach/stdx-allocator/commit/d06e4f2bae2eee5d380d145221ecb9cab04c90d7

On Thursday, 30 November 2017 at 19:01:02 UTC, Andrei Alexandrescu wrote:
> Currently, allocators traffic in void[]. When I first designed allocators, I considered using ubyte[] instead.

I experimented with using byte[] for opaque buffers, because byte is signed, one can't parse byte[] content in a meaningful way, and should cast to ubyte[] first, so byte[] is practically opaque (if you're careful).

> However, using void[] makes it difficult to express the fact that allocate() is a safe function for virtual all allocators.

Safe code should only use typed allocation (make) and never see underlying implementation.

On 2017-11-30 20:01, Andrei Alexandrescu wrote:
> Hi all,
> 
> 
> Eduard, Alexandru Jercaianu and I are working on improving allocators' design and implementation. This entails a few breaking changes.

It would be nice if the API and the GCAllocator were CTFE-able. This would allow functions that take allocators as parameters to be CTFE-able if you pass in the GCAllocator which doesn't work today.

-- 
/Jacob Carlborg

On Thursday, 30 November 2017 at 19:01:02 UTC, Andrei Alexandrescu wrote:
> Another possible work item was raised by https://github.com/dlang/phobos/pull/5879. Currently, allocators traffic in void[]. When I first designed allocators, I considered using ubyte[] instead. Using void[] is somewhat closer to the intent of allocators - that memory is meant to be used for storing anything. However, using void[] makes it difficult to express the fact that allocate() is a safe function for virtual all allocators. So we may switch to ubyte[] to express that at the lowest level we're trafficking chunks of octets. This is not a big design change, but it's bound to break code.

That argument should also apply to e.g. std.file.read.

On Friday, 1 December 2017 at 02:30:29 UTC, Seb wrote:
> On Thursday, 30 November 2017 at 19:01:02 UTC, Andrei Alexandrescu wrote:
>> Hi all,
>>
>>
>> Eduard, Alexandru Jercaianu and I are working on improving allocators' design and implementation. This entails a few breaking changes.
>>
>> In order to make matters easier for code using allocators, Sebastian Wilzbach created a dub package freezing the existing API: http://code.dlang.org/packages/stdx-allocator. Please use that until we work the kinks out of allocators - great things are coming! - and after that feel free to upgrade code to use the new and improved allocators.
>
> I just pushed a couple of fixes [1] for older Phobos versions and stdx-allocator now works down until 2.072.2.
> If someone needs an older Phobos version to work with stdx-allocator, please let me know.
> Also switching to stdx-allocator is rather easy, e.g.:
>
> ```
> sed -i "s/std[.]experimental/stdx/g" **/*.d
> ```
>
> [1] https://github.com/wilzbach/stdx-allocator/commit/d06e4f2bae2eee5d380d145221ecb9cab04c90d7


Just a friendly reminder to people that experimental isn't expected to be stable.
2.079 will come with this change, switch to stdx-allocator package if you prefer stability. stdx-allocator provides the current code and works down until 2.072.

https://dlang.org/changelog/pending.html#std-experimental-allocator-rciallocator
https://code.dlang.org/packages/stdx-allocator

Also note that only vibe.d ~>= 0.8.3-alpha.1 and vibe-core ~>= 1.4.0-alpha.1 use this package.
(though while most projects have been upgraded manually, I hope that we can get a new release of Vibe.d out there soon to avoid any issues - see [1]).

[1] https://github.com/vibe-d/vibe.d/issues/2058

On Thursday, 30 November 2017 at 19:01:02 UTC, Andrei Alexandrescu wrote:
> So we may switch to ubyte[]

Hooray!

On Thursday, 30 November 2017 at 19:01:02 UTC, Andrei Alexandrescu wrote:
> Hi all,
>
>
> Eduard, Alexandru Jercaianu and I are working on improving allocators' design and implementation. This entails a few breaking changes.

Awesome! If you have any architecture/API drafts they would interesting to review for BetterC.

Best regards,
Ilya

On Friday, 9 February 2018 at 14:24:45 UTC, 9il wrote:
> On Thursday, 30 November 2017 at 19:01:02 UTC, Andrei Alexandrescu wrote:
>> Hi all,
>>
>>
>> Eduard, Alexandru Jercaianu and I are working on improving allocators' design and implementation. This entails a few breaking changes.
>
> Awesome! If you have any architecture/API drafts they would interesting to review for BetterC.
>
> Best regards,
> Ilya

Here's what I know has happened so far:

1) RCAllocator


https://dlang.org/changelog/pending.html#std-experimental-allocator-rciallocator

2) AscendingPageAllocator

https://dlang.org/phobos-prerelease/std_experimental_allocator_building_blocks_ascending_page_allocator.html

(there's most likely more WIP)