Jump to page: 1 2
Thread overview
opNew operator overload
Apr 05
Dennis
Apr 05
Dennis
Apr 05
Dennis
Apr 05
bauss
Apr 05
Dennis
Apr 05
Dennis
Apr 05
Mike Shah
Apr 06
Johan
Apr 07
Mike Shah
Apr 05
monkyyy
April 05

Abstract

When a struct Allocator or class Allocator defines T opNew(T, A...)(A a) => ..., then allocator.new Item("x") will be rewritten to allocator.opNew!Item("x").

Rationale

Some D programs use a different allocator than the built-in GC allocator you get when using the new operator. To make allocation type safe, allocator libraries often offer a templated function that creates an instance of an arbitrary type using that allocator. However, these template functions have to awkwardly use a synonym for new, since it's a reserved keyword. A common choice is 'make' or 'alloc' (short for 'allocate').

Some real examples:

Since D programmers are familiar with new T() syntax, enabling it for custom allocators makes code both more pleasing to read and easier to refactor when switching from using the GC to a custom allocator. It also makes it easier to switch from one custom allocator to another when there's a unified syntax, since you don't need to learn and apply new function signatures every time.

Prior work

  • D runtime hooks have been translated to templates. new X() is already being rewritten to instances of _d_newitemT, _d_newarrayT, or _d_newclassT, so overrding the implicitly imported object.d module lets you override the allocator with a custom one. This is hacky though, and only provides the means to switch a single global allocator since there's still no way to pass an allocator parameter.
  • class allocators have been deprecated because "Classes should not be responsible for their own (de)allocation strategy". This proposal is different because it doesn't let the class decide which allocator to use, it requires explicit mentioning of the allocator object on each allocation with new.

Description

The rewrite works just like existing operator overloading.
opNew has to be defined inside an aggregate.
The typeof(new T()) will be passed as the first template parameter, while the arguments passed to new are passed as run-time arguments.

struct GcAllocator
{
    // Simple wrapper, can be swapped out later
    T opNew(T, A...)(A args) => new T(args);
}

void main()
{
    GcAllocator allocator;
    allocator.new int(3);       // a.opNew!(int*)(3)
    allocator.new int[3];       // a.opNew!(int[])(3)
    allocator.new int[](3);     // a.opNew!(int[])(3)
    allocator.new int[3][4];    // a.opNew!(int[][])(3, 4)
    allocator.new Struct;       // a.opNew!(Struct*)()
    allocator.new Struct(args); // a.opNew!(Struct*)(args)
    allocator.new Class(args);  // a.opNew!Class(args)
}

Explicit instantiation of Nested class

The .new syntax is already used for explicit instantiation of a nested class. However, I consider this an obscure legacy feature. For backwards compatibility, this will have precedence over the operator overload.
The only limitation this brings is that allocators may not contain a nested class, which shouldn't be a problem in practice.

Zero argument variant

One could imagine a zero-argument variant like this:

struct Unique(T)
{
    T t;
    Unique opNew() => new Unique(T.init);
}

void main()
{
    auto a = Unique!int.new;
}

However, there might be an expectation that it's the same as:

auto a = new Unique!int;

So I'm not sure whether it's a good idea. It could be added in the future.

With statements

The with statement does not allow you to implicitly call opNew, as that could lead to surprising results and breakage.

with (a)
    auto p = new int; // don't try a.opNew!int

destroy()

Since the GC takes care of freeing memory, there is usually no need to call object.destroy() on an object created with new. When the custom allocator does region-based memory management, there's no need to free individual objects either. When the allocator does require manual freeing of objects, one could implement destroy as a member function of the allocator.

struct Allocator
{
    T opNew(T, A...)(A args) => new T(args);
    void destroy(T)(T o) {}
}

void main()
{
    Allocator allocator;
    auto o = allocator.new Object;
    allocator.destroy(o);
}

Since destroy is a library function in object.d, which can already be shadowed by a member function, there's no need for any special consideration here.

April 06
Adapting this to have a right version might be quite a good idea too:

```d
struct Foo {
	static Bar opNewRight(Allocator, Args)(ref Allocator, Args) {
		
	}
}

struct Bar {
	Foo* _;
}
```

I'd suggest making this be more preferred than ``opNew``. So that a type can control and see its memory allocator (for i.e. storage and then deallocation).
April 05

On Saturday, 5 April 2025 at 18:23:34 UTC, Richard (Rikki) Andrew Cattermole wrote:

>

Adapting this to have a right version might be quite a good idea too

I don't see what problem that's trying to solve.

April 06
On 06/04/2025 6:29 AM, Dennis wrote:
> On Saturday, 5 April 2025 at 18:23:34 UTC, Richard (Rikki) Andrew Cattermole wrote:
>> Adapting this to have a right version might be quite a good idea too
> 
> I don't see what problem that's trying to solve.

If you have a reference counted type you may want to control the memory allocator used (i.e. data structure).

It has to be able to deallocate and that means it needs the allocator stored in its state.

April 05

On Saturday, 5 April 2025 at 18:31:17 UTC, Richard (Rikki) Andrew Cattermole wrote:

>

It has to be able to deallocate and that means it needs the allocator stored in its state.

If you need to store an allocator as a field, then that allocator must be passed as a regular constructor parameter. No need to be clever.

April 05

On Saturday, 5 April 2025 at 13:49:57 UTC, Dennis wrote:

>

Abstract

When a struct Allocator or class Allocator defines T opNew(T, A...)(A a) => ..., then allocator.new Item("x") will be rewritten to allocator.opNew!Item("x").

Rationale

[...]

Since D programmers are familiar with new T() syntax, enabling it for custom allocators makes code both more pleasing to read and easier to refactor when switching from using the GC to a custom allocator. It also makes it easier to switch from one custom allocator to another when there's a unified syntax, since you don't need to learn and apply new function signatures every time.

  • D programmers are familiar with using new to allocate with the GC. It's not obvious that giving a new, different meaning to the same synatx will make code easier to read and understand.

  • Some D programmers are also familiar with the existing obj.new T syntax for instantiating nested classes. In general, differentiating between this existing syntax and your proposed syntax will require non-local information (i.e., looking up the type of obj).

  • It's not obvious to me that refactoring from new T(args) to alloc.new T(args) is any easier than refactoring from new T(args) to alloc.make!T(args).

  • It's also not obvious to me that refactoring from alloc1.new T to alloc2.new T is any easier than refactoring from alloc1.make!T to alloc2.make!T.

Overall, I'm not convinced that this proposal offers any compelling benefits compared to the current make!T approach. The fact that it overlaps with an existing syntax (nested class new) is also a strike against it.

April 05

On Saturday, 5 April 2025 at 18:41:41 UTC, Dennis wrote:

>

On Saturday, 5 April 2025 at 18:31:17 UTC, Richard (Rikki) Andrew Cattermole wrote:

>

It has to be able to deallocate and that means it needs the allocator stored in its state.

If you need to store an allocator as a field, then that allocator must be passed as a regular constructor parameter. No need to be clever.

I agree with this.

April 05

On Saturday, 5 April 2025 at 18:43:57 UTC, Paul Backus wrote:

>
  • D programmers are familiar with using new to allocate with the GC. It's not obvious that giving a new, different meaning to the same synatx will make code easier to read and understand.

Well you're obviously not supposed to make it do anything different than allocate just like new. I don't see how this is anything different from overloading ~= for a custom array type. As long as it means appending, there should be no surprises.

>
  • Some D programmers are also familiar with the existing [obj.new T syntax for instantiating nested classes.][1]

Yeah I wish that feature didn't exist.

>
  • It's not obvious to me that refactoring from new T(args) to alloc.new T(args) is any easier than refactoring from new T(args) to alloc.make!T(args).

  • It's also not obvious to me that refactoring from alloc1.new T to alloc2.new T is any easier than refactoring from alloc1.make!T to alloc2.make!T.

It's about being standard/uniform and recognizable. Not everyone uses make. Having to parenthesize array types like make!(int[]) is annoying, so you also have makeArray!int, others use alloc, or alloc.array!int etc. When scanning D code, I can easily locate creation of objects with new syntax. Calls to make!T or constructNew!T don't stand out from other functions like read!T or parse!T.

In Java you have ArrayList where you have to use list.get(i) and list.size() instead of list[i] and list.length, which is perfectly functional and it's not a complex refactor, but when reading code it's harder to recognize where array operations happen.

In D, I love how you can use operator overloading to create library arrays that are almost drop-in replacements for language arrays. It helps when incrementally refactoring GC-based array code to use custom allocators for performance / portability (to WebAssembly in my case).

I thought it would be nice to expand this library array type love to the new operator when allocating fixed size arrays, for example auto pixels = new ubyte[4 * width * height];. That being said, I'm not 100% sold on the idea myself, I can see downsides as well. As an alternative, I guess we could officially declare make!T to be the library equivalent of new and hope it catches on.

April 05

On Saturday, 5 April 2025 at 13:49:57 UTC, Dennis wrote:

>

The with statement does not allow you to implicitly call opNew, as that could lead to surprising results and breakage.

Without with(allocator): your not actually making the syntactical leap that allows code to be part of someones else's grand plan of memory management. Making the entire enterprise worthless.

I expect that to be the outcome so therefore all libs of allocators are worthless, but in theory you should be aiming for the prize of replacing the gc with someones grand plan, id argue via compiler flag, but if your not even handling the simpler case....

April 05

On Saturday, 5 April 2025 at 19:30:49 UTC, Dennis wrote:

>

It's about being standard/uniform and recognizable. Not everyone uses make.

Not everyone will use opNew either. If you are capable of achieving uniformity with opNew, you are also capable of achieving uniformity without it.

>

Having to parenthesize array types like make!(int[]) is annoying, so you also have makeArray!int, others use alloc, or alloc.array!int etc.

I don't think those are any uglier than new int[](n), and they're certainly less confusing than new int[n].

In any case, "writing parentheses is annoying sometimes" is kind of a weak justification for introducing new syntax.

>

When scanning D code, I can easily locate creation of objects with new syntax. Calls to make!T or constructNew!T don't stand out from other functions like read!T or parse!T.

I cannot think of a single situation where I've been reading D code and felt the need to "easily locate creation of objects."

I have been in situations where I wanted to know where a specific object was being created, and grepped for new T to find it--but I could have grepped for make!T just as easily.

>

In D, I love how you can use operator overloading to create library arrays that are almost drop-in replacements for language arrays. It helps when incrementally refactoring GC-based array code to use custom allocators for performance / portability (to WebAssembly in my case).

I thought it would be nice to expand this library array type love to the new operator when allocating fixed size arrays, for example auto pixels = new ubyte[4 * width * height];.

So, let me get this straight: you're proposing that myAllocator.new ubyte[n] would (potentially) return some custom library type, instead of a ubyte[]?

That's...well, there are a lot of words I could use to describe it, but let's just say that "uniform" and "consistent" would not be among them.

« First   ‹ Prev
1 2