Thread overview
August 13

I feel like I can't possibly be the first to ask, but I couldn't find any prior discussion of this:
When is std.experimental.allocator going to be moved out of experimental? Is there any roadmap for it? Is it just in limbo?

August 13

On Sunday, 13 August 2023 at 11:44:50 UTC, IchorDev wrote:

>

I feel like I can't possibly be the first to ask, but I couldn't find any prior discussion of this:
When is std.experimental.allocator going to be moved out of experimental? Is there any roadmap for it? Is it just in limbo?

We can and should do better than std.experimental.allocator

It's unreadable and too complex, and it uses classes/interface therefore not good as a base

I also think this is not a job for phobos, i'd love to see the allocator stuff put in ``core.memory`

We only need 3 function pointers (allocate, free, resize), perhaps a struct that hold them

core.memory only need 3 type of allocator: GCAllocator PageAllocator ArenaAllocator

In fact that's what i do, my core API is as simple as:

struct VTable
{
    alloc_d alloc;
    resize_d resize;
    free_d free;
}

private alias alloc_d = ubyte[] function(const void* self, size_t len, ubyte ptr_align);
private alias resize_d = bool function(const void* self, ubyte[] buf, ubyte buf_align, size_t new_len);
private alias free_d = void function(const void* self, ubyte[] buf, ubyte buf_align);

I pass around a struct that has this vtable

        heap = c_allocator;
        arena = ArenaAllocator.create(heap);

        servers.create(heap);
        entities.create(heap);

        // use arena for temp stuff
        // then clean it all in on go
        (..)
        arena.dispose();

No need to use classes or interface, a struct is all needed, effective, efficient and just few lines of code

Zig was eye opener to me, it should be as simple as possible, there is no reason to be complex or to use advanced features, the price to pay to use allocators should be 0, therefore it should work with -betterC

I am super interested in helping get stuff done, let's work together, ping me on the IRC server

More eye opening stuff:

https://www.gingerbill.org/series/memory-allocation-strategies/

August 14
Mine (-betterC) https://github.com/Project-Sidero/basic_memory/tree/main/source/sidero/base/allocators

Similar scope to one in Phobos.

On that note I'm still waiting a year+ for Atila to get back to me about talking about where to go for std.experimental.allocators.
August 13

On Sunday, 13 August 2023 at 15:25:16 UTC, Richard (Rikki) Andrew Cattermole wrote:

>

Mine (-betterC) https://github.com/Project-Sidero/basic_memory/tree/main/source/sidero/base/allocators

Similar scope to one in Phobos.

On that note I'm still waiting a year+ for Atila to get back to me about talking about where to go for std.experimental.allocators.

It's almost good, unfortunately it uses RAII.. wich is a red flag for me

The premise of an Allocator API is to give control to the user, this is lost

Also, i'd personally remove this import: import std.typecons : Ternary;, no need to tap into std just for this struct

https://github.com/dlang/phobos/blob/v2.105.0/std/typecons.d#L10472

Mine does't use RAII, and i personally think this is what a good core API should be

August 14
Yeah you're right Ternary should probably be replaced, although amazingly it has never caused problems so far.

But I cannot agree about RAII. Its a valid tool for managing lifetimes of memory allocators. Memory allocators must be able to store per instance state and that state must not accidentally die on you. If it does that is a great way to give the user a very bad day.

We are certainly aiming for different things, for instance I don't trust my (future) users when it comes to memory safety. So they get very well hand held in all aspects. Including locking of RCAllocator internally and using RC there.

But internally to API's if you need to optimize you'd use the composable allocator structs directly, rather than the expensive (in comparison) RCAllocator.

August 13
On Sunday, 13 August 2023 at 16:00:51 UTC, Richard (Rikki) Andrew Cattermole wrote:
> Yeah you're right Ternary should probably be replaced, although amazingly it has never caused problems so far.
>
> But I cannot agree about RAII. Its a valid tool for managing lifetimes of memory allocators. Memory allocators must be able to store per instance state and that state must not accidentally die on you. If it does that is a great way to give the user a very bad day.
>
> We are certainly aiming for different things, for instance I don't trust my (future) users when it comes to memory safety. So they get very well hand held in all aspects. Including locking of RCAllocator internally and using RC there.
>
> But internally to API's if you need to optimize you'd use the composable allocator structs directly, rather than the expensive (in comparison) RCAllocator.

Core API should subscribe to the premise: give memory allocation control (and therefore dealocation) back to the user

I'm not against you using RAII, however, i'm against making RAII a requirement for everyone who want to use the API

That's why i'm suggesting moving the API to ``core.memory``, you can have RAII based allocators in phobos ``std.memory``, it's more for general use

But ``core`` means it's the foundation, helps port to esoteric platforms easier, or usecases where RAII might not be the best choice / or not possible at all

Also if you want people to use D for games, you want an allocator API that doesn't use RAII, same for exceptions btw
August 14
On 14/08/2023 4:10 AM, ryuukk_ wrote:
> Also if you want people to use D for games, you want an allocator API that doesn't use RAII, same for exceptions btw

After thinking about it a bit, this would suggest to me that you are trying to solve a problem that I would outright recommend against using an abstraction to solve.

Memory allocators that only deal with fixed sized memory blocks, that are specific to a thread are going to have the best performance. If this is the case you don't need an abstraction at all.

A rough pseudo code where no RCAllocator would be used:

```d
module particles;

private __gshared FreeList!(AllocatorList!(Region!(SystemMapper, __traits(classInstanceSize, Particle)))) allocator;

final class Particle {
	int lifeLeft;

	void free() {
		allocator.dispose(this);
	}

	static Particle create() {
		return allocator.make!Particle();
	}
}
```

Of course you'd only call free/create as part of a particle manager, but this should still demonstrate that an abstraction isn't required if you understand your data and memory lifetimes well enough to see any performance improvement by using a memory allocator library.
August 13

On Sunday, 13 August 2023 at 11:44:50 UTC, IchorDev wrote:

>

I feel like I can't possibly be the first to ask, but I couldn't find any prior discussion of this:
When is std.experimental.allocator going to be moved out of experimental? Is there any roadmap for it? Is it just in limbo?

The current design of std.experimental.allocator is incompatible with @safe. Most likely, it will eventually be replaced with a new "allocators v2" package, and moved to undead.

August 14

On Sunday, 13 August 2023 at 16:00:51 UTC, Richard (Rikki) Andrew Cattermole wrote:

>

Yeah you're right Ternary should probably be replaced, although amazingly it has never caused problems so far.

But I cannot agree about RAII. Its a valid tool for managing lifetimes of memory allocators. Memory allocators must be able to store per instance state and that state must not accidentally die on you. If it does that is a great way to give the user a very bad day.

We are certainly aiming for different things, for instance I don't trust my (future) users when it comes to memory safety. So they get very well hand held in all aspects. Including locking of RCAllocator internally and using RC there.

But internally to API's if you need to optimize you'd use the composable allocator structs directly, rather than the expensive (in comparison) RCAllocator.

I have complicated feelings about all of this.
The current implementation is a bit confusing, but I think that's due to somewhat poor documentation that's overenthusiastic to explain the complicated inner-workings, which would only matter to you if you want to build your own allocator. For an average user, the API could be explained as "malloc + free, except they're type-safe and take a custom allocator as an argument":

auto x = make!Type(someAllocator);
dispose(someAllocator, x);

I agree with ryuukk_ that it should really be a core submodule, and obviously work with BetterC (but maybe still using exception throwing if we're being a bit forwards-looking).
std.experimental.allocator.building_blocks—and some other modules like it—have a lot of good code that I think shouldn't just be thrown out: https://dlang.org/library/std/experimental/allocator/building_blocks.html
I agree that the "interfaces" of the allocators should be re-done without class or interface. Some of the interface functions are explicitly optional, which would work better with a struct and function pointers. For the non-optional stuff, that's an annoyance...

August 14
On Sunday, 13 August 2023 at 16:10:32 UTC, ryuukk_ wrote:
>
> Core API should subscribe to the premise: give memory allocation control (and therefore dealocation) back to the user
>

I'm not sure about why RAII is an issue, but I fully agree with your stance about a simpler allocator, and one we can actually build upon.

Great many C libraries work with just 3 preprocessor macros for malloc, free, realloc, something like that. A standard struct with 3 pointers in druntime would be great!

We just can't build upon std.experimental.allocator if it doesn't work in betterC and WebASM.

And it should be clear by now that D programmers want dependencies with fewer files, fewer concepts, fewer new names, (and if I may less templatitis) else they won't build upon it.