I wanted to make a 2D array like structure and support D slice like operations,
but I had surprisingly bad experience.

I quickly copy pasted the example from the docs: https://dlang.org/spec/operatoroverloading.html#array-ops

It's something like this:
struct Array2D(E)
{
    E[] impl;
    int stride;
    int width, height;

    this(int width, int height, E[] initialData = [])
    ref E opIndex(int i, int j)
    Array2D opIndex(int[2] r1, int[2] r2)
    auto opIndex(int[2] r1, int j)
    auto opIndex(int i, int[2] r2)
    int[2] opSlice(size_t dim)(int start, int end)
    @property int opDollar(size_t dim : 0)()
    @property int opDollar(size_t dim : 1)()
}

So basic indexing works fine:
Array2D!int foo(4, 4);
foo[0, 1] = foo[2, 3];

But array copy and setting/clearing doesn't:
int[] bar = [ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15 ];
foo[] = bar[];

And I get this very cryptic message:
(6): Error: template `example.Array2D!int.Array2D.opSlice` cannot deduce function from argument types `!()()`, candidates are:
(51):        `example.Array2D!int.Array2D.opSlice(ulong dim)(int start, int end) if (dim >= 0 && (dim < 2))`

1. WTF `!()()` and I haven't even called anything with opSlice i.e. `a .. b`?

Anyway, it doesn't overload [] with opIndex(), so fine, I add that.
T[] opIndex() { return impl; }

Now I get:
foo[] = bar[]; // or foo[] = bar;
Error: `foo[]` is not an lvalue and cannot be modified

Array copying docs say:
When the slice operator appears as the left-hand side of an assignment expression, it means that the contents of the array are the target of the assignment rather than a reference to the array. Array copying happens when the left-hand side is a slice, and the right-hand side is an array of or pointer to the same type.

2.WTF I do have slice operator left of assignment.
So I guess [] is just wonky named getter (and not an operator) for a slice object and that receives the = so it's trying to overwrite/set the slice object itself.

Next I added a ref to the E[] opIndex():
ref E[] opIndex() { return impl; }

Now foo[] = bar[] works as expected, but then I tried
foo[] = 0;
and that fails:
Error: cannot implicitly convert expression `0` of type `int` to `int[]`

3. WTF. Didn't I just get reference directly to the slice and array copy works, why doesn't array setting?

The ugly foo[][] = 0 does work, but it's so ugly/confusing that I'd rather just use a normal function.

So I added:
ref E[] opIndexAssign(E value) { impl[] = value; return impl; }

And now foo[] = 0; works, but foo[0, 1] = foo[2, 3] doesn't.

I get:
Error: function `example.Array2D!int.Array2D.opIndexAssign(int f)` is not callable using argument types `(int, int, int)`
expected 1 argument(s), not 3

4. WTF. So basically adding opIndexAssign(E value) disabled ref E opIndex(int i, int j). Shouldn't it consider both?

I'm surprised how convoluted this is. Is this really the way it's supposed to work or is there a bug or something?


So what is the best/clear/concise/D way to do these for a custom type?

I was planning for:
foo[] = bar; // Full copy
foo[] = 0; // Full clear
foo[0 .. 5, 1] = bar[ 0 .. 5]; // Row/Col copy
foo[1, 0 .. 5] = 0; // Row/Col clear
foo[0 .. 5, 2 .. 4] = bar[ 1 .. 6, 0 .. 2 ]; // Box copy
foo[0 .. 5, 2 .. 4] = 0; // Box clear

Anyway, this is not a huge deal breaker for me, I was just surprised and felt like I'm missing something.
I suppose I can manually define every case one by one and not return/use any references etc.
or use alias this to forward to impl[] (which I don't want to do since I don't want to change .length for example)
or just use normal functions and be done with it.

And it's not actually just a regular array I'm making, so that's why it will be mostly custom code, except the very basics.

On Saturday, 4 May 2019 at 15:18:58 UTC, Random D user wrote:
> I wanted to make a 2D array like structure and support D slice like operations,
> but I had surprisingly bad experience.
>
> I quickly copy pasted the example from the docs: https://dlang.org/spec/operatoroverloading.html#array-ops
>
> It's something like this:
> struct Array2D(E)
> {
>     E[] impl;
>     int stride;
>     int width, height;
>
>     this(int width, int height, E[] initialData = [])
>     ref E opIndex(int i, int j)
>     Array2D opIndex(int[2] r1, int[2] r2)
>     auto opIndex(int[2] r1, int j)
>     auto opIndex(int i, int[2] r2)
>     int[2] opSlice(size_t dim)(int start, int end)
>     @property int opDollar(size_t dim : 0)()
>     @property int opDollar(size_t dim : 1)()
> }
>
> So basic indexing works fine:
> Array2D!int foo(4, 4);
> foo[0, 1] = foo[2, 3];
>
> But array copy and setting/clearing doesn't:
> int[] bar = [ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15 ];
> foo[] = bar[];
>
> And I get this very cryptic message:
> (6): Error: template `example.Array2D!int.Array2D.opSlice` cannot deduce function from argument types `!()()`, candidates are:
> (51):        `example.Array2D!int.Array2D.opSlice(ulong dim)(int start, int end) if (dim >= 0 && (dim < 2))`
>
> 1. WTF `!()()` and I haven't even called anything with opSlice i.e. `a .. b`?
>
> Anyway, it doesn't overload [] with opIndex(), so fine, I add that.
> T[] opIndex() { return impl; }
>
> Now I get:
> foo[] = bar[]; // or foo[] = bar;
> Error: `foo[]` is not an lvalue and cannot be modified
>
> Array copying docs say:
> When the slice operator appears as the left-hand side of an assignment expression, it means that the contents of the array are the target of the assignment rather than a reference to the array. Array copying happens when the left-hand side is a slice, and the right-hand side is an array of or pointer to the same type.
>
> 2.WTF I do have slice operator left of assignment.
> So I guess [] is just wonky named getter (and not an operator) for a slice object and that receives the = so it's trying to overwrite/set the slice object itself.
>
> Next I added a ref to the E[] opIndex():
> ref E[] opIndex() { return impl; }
>
> Now foo[] = bar[] works as expected, but then I tried
> foo[] = 0;
> and that fails:
> Error: cannot implicitly convert expression `0` of type `int` to `int[]`
>
> 3. WTF. Didn't I just get reference directly to the slice and array copy works, why doesn't array setting?
>
> The ugly foo[][] = 0 does work, but it's so ugly/confusing that I'd rather just use a normal function.
>
> So I added:
> ref E[] opIndexAssign(E value) { impl[] = value; return impl; }
>
> And now foo[] = 0; works, but foo[0, 1] = foo[2, 3] doesn't.
>
> I get:
> Error: function `example.Array2D!int.Array2D.opIndexAssign(int f)` is not callable using argument types `(int, int, int)`
> expected 1 argument(s), not 3
>
> 4. WTF. So basically adding opIndexAssign(E value) disabled ref E opIndex(int i, int j). Shouldn't it consider both?
>
> I'm surprised how convoluted this is. Is this really the way it's supposed to work or is there a bug or something?
>
>
> So what is the best/clear/concise/D way to do these for a custom type?
>
> I was planning for:
> foo[] = bar; // Full copy
> foo[] = 0; // Full clear
> foo[0 .. 5, 1] = bar[ 0 .. 5]; // Row/Col copy
> foo[1, 0 .. 5] = 0; // Row/Col clear
> foo[0 .. 5, 2 .. 4] = bar[ 1 .. 6, 0 .. 2 ]; // Box copy
> foo[0 .. 5, 2 .. 4] = 0; // Box clear
>
> Anyway, this is not a huge deal breaker for me, I was just surprised and felt like I'm missing something.
> I suppose I can manually define every case one by one and not return/use any references etc.
> or use alias this to forward to impl[] (which I don't want to do since I don't want to change .length for example)
> or just use normal functions and be done with it.
>
> And it's not actually just a regular array I'm making, so that's why it will be mostly custom code, except the very basics.

The de facto multi dimensional array type in D is mir's ndslice

https://github.com/libmir/mir-algorithm/blob/master/source/mir/ndslice/slice.d#L479

On Saturday, 4 May 2019 at 15:18:58 UTC, Random D user wrote:
> But array copy and setting/clearing doesn't:
> int[] bar = [ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15 ];
> foo[] = bar[];
>
> And I get this very cryptic message:
> (6): Error: template `example.Array2D!int.Array2D.opSlice` cannot deduce function from argument types `!()()`, candidates are:
> (51):        `example.Array2D!int.Array2D.opSlice(ulong dim)(int start, int end) if (dim >= 0 && (dim < 2))`
>
> 1. WTF `!()()` and I haven't even called anything with opSlice



This comes from some old history: arr[] used to call opSlice(), and now it is preferred to implement opIndex() instead, but the compiler still supports the old zero-arg opSlice() too.

Since opIndex() didn't work, it moved to trying the older opSlice(), and that didn't work leading it to give up and issue the error.

But yeah, the error should probably mention the newer function name instead of the fallback it is failing on...

> Next I added a ref to the E[] opIndex():
> ref E[] opIndex() { return impl; }

I would avoid ref as much as you can, for one because it conflates getting and assigning (as you saw), but also because it leaks your internal implementation detail to the user; if you didn't use an array internally, that api would break.

> So I added:
> ref E[] opIndexAssign(E value) { impl[] = value; return impl; }
>
> And now foo[] = 0; works, but foo[0, 1] = foo[2, 3] doesn't.
> *snip*
> 4. WTF. So basically adding opIndexAssign(E value) disabled ref E opIndex(int i, int j). Shouldn't it consider both?

Once you implement an opIndexAssign - any opIndexAssign - all uses of `a[...] = c` will go through it instead of normal opIndex.

Generally speaking, opIndex is for getting, opIndexAssign is for setting. Sometimes setting can be done via a getter function (like if ref), but they are mostly two different functions and you should implement them both if you want read/write access.

This is especially important if your underlying data isn't actually in a literal array.

> foo[] = bar; // Full copy

implement:

opIndexAssign(typeof(this) rhs);

In opIndexAssign, generally, the arguments are (value_on_right_hand_side, indexes...)

Since there was no index given here, you don't want an index argument.

> foo[] = 0; // Full clear

implement:

opIndexAssign(int rhs);

Now it will take any int for the whole thing.

> foo[0 .. 5, 1] = bar[ 0 .. 5]; // Row/Col copy

This is translated to:

this.opIndexAssign(bar.opIndex(bar.opSlice!0(0, 5)), this.opSlice!0(0, 5), 1)

Three different functions there. On the left, we see x[...] = y, so we know that is opIndexAssign again. The first argument is what it is set to, other arguments are the slices given.

Any x .. y is translated to opSlice!dim(low, high). The !0 in there is because it was given as the zeroth (first) argument in the slice.

In this example, `foo[1 .. 2, 3 .. 4]`, it would call

opIndex( opSlice!0(1, 2), opSlice!1(3, 4) )

With the !0 and !1 indicating which position the slice was in.

The implementation of these functions would depend on just what your innards are... but the types might be something like this:

struct SliceHelper {
   size_t start;
   size_t end;
   int stride;
}

SliceHelper opSlice(size_t dimension)(size_t start, size_t end) {
    /* return the helper with the appropriate values */
}

now, we can implement opIndex for getting in terms of that and some regular items:

// get a single item at point x, y
int opIndex(size_t x, size_t y) { }

// well not necessarily an array, maybe a range for lazy processing, but meh you get the idea
// this gives a slice in the X dimension with a fixed Y coordinate
// e.g. foo[ 0 .. 5, 3]
int[] opIndex(SliceHelper x, size_t y) {}

// foo[0, 4 .. 6]
int[] opIndex(size_t x, SliceHelper y) {}

// and now a 2d section of it
Array2d!int opIndex(SliceHelper x, SliceHelper y) { }

// and the zero-arg version, for foo[]
// here I return this for an example, but by convention,
// this should actually return a range object that is a
// view into this container... which might be `this` but
// might not be, depending on the details of your code like
// if it has internal references or other stuff you don't want
// to leak to the outside.

typeof(this) opIndex() { return this; }


And then a similar combination of arguments for opIndexAssign
(I return void here but it is also common to return this; )

void opIndexAssign(int rhs) // this[] = rhs
void opIndexAssign(int rhs, size_t x, size_t y) // this[x, y] = rhs
void opIndexAssign(int rhs, size_t x, SliceHelper y) // this[x, y1 .. y2] = rhs


.... you get the idea. The overload for other types of rhs.

This might be dozens of functions! You can minimize that a bit by templating them and using internal static if, loops, etc to narrow it down.


void opIndexAssign(R, Idx1, Idx2)(R rhs, Idx1 x, Idx2 y) {
    // test types of Idx1+Idx2 for figuring out what to change
    // test type of R to see what to change it to
}


> foo[1, 0 .. 5] = 0; // Row/Col clear

Again, notice the x[...] = y, so we know it is opIndexAssign.


opIndexAssign(0 /* rhs */, 1, opSlice!1(0, 5))

> foo[0 .. 5, 2 .. 4] = bar[ 1 .. 6, 0 .. 2 ]; // Box copy

opIndexAssign( // opIndexAssign as a setter
   bar.opIndex(bar.opSlice!0(1, 6), bar.opSlice!1(0, 2)), // rhs value, notice opIndex as a getter
   opSlice!0(0, 5), opSlice!1(2, 4) /* lhs slice args */)

> foo[0 .. 5, 2 .. 4] = 0; // Box clear

foo.opIndexAssign( // setter
   0, // rhs
   foo.opSlice!0(0, 5), foo.opSlice!1(2, 4) // lhs slice args
)


> I suppose I can manually define every case one by one and not return/use any references etc.


Yeah, it can be quite a lot of combinations of arguments. Generally, it is n^n * k, where n is the number of dimensions of your array and k is the number of different types you want to be able to assign to it. Then times two for getters and setters, then plus two for the empty arg ones.

For 1d with one type, it is easy: 1^1 * 1 * 2 + 2:

the 1^1 * 1 is simplified: int func(int), and *2 is Index and IndexAssign:

opIndex(int) and opIndexAssign(int, int)

then the plus two

opIndex() and opIndexAssign(int) for foo[] and foo[] = n.


OK, maybe actually *3 instead of *2 if you want to do

opOpIndexAssign as well, to enable like

foo[] += 4;

that's separate too.


But for 2d and 3d and more arrays, the number of functions explodes really fast. You will probably want to template that into something generic and just implement it that way.

On Saturday, 4 May 2019 at 16:10:36 UTC, Adam D. Ruppe wrote:
> On Saturday, 4 May 2019 at 15:18:58 UTC, Random D user wrote:
>> But array copy and setting/clearing doesn't:
>> int[] bar = [ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15 ];
>> foo[] = bar[];
> Generally speaking, opIndex is for getting, opIndexAssign is for setting.

Thanks a lot for a very detailed answer. Sorry about the late reply.

> But for 2d and 3d and more arrays, the number of functions explodes really fast.

Yeah, tastes like C++, but I guess I'll bite.
I value debuggability and I only have the 2D case, so I think templates are out.

On Saturday, 4 May 2019 at 15:36:51 UTC, Nicholas Wilson wrote:
> On Saturday, 4 May 2019 at 15:18:58 UTC, Random D user wrote:
>> I wanted to make a 2D array like structure and support D slice like operations,
>> but I had surprisingly bad experience.
>
> The de facto multi dimensional array type in D is mir's ndslice
>
> https://github.com/libmir/mir-algorithm/blob/master/source/mir/ndslice/slice.d#L479

Thanks. I'll take a look.