I'm struggling with implementing multidimensional arrays in the library. It definitely is a messy task, trying to get some pretty syntax for the user!

The current intension of my work is something like a case-study, to demonstrate a way, how multidimensional arrays might be defined as a language feature in the future. My initial hope, to create some library that might be usable as such faded very quickly. In C++, I could certainly do it. In D, I'm either far from understanding the language at all, or the language is far from being as powerful as C++ in the current state.

"Powerful" in this context means: The possibility, to write a library that offers an expressive syntax to the user.

Now, I'm already stuck at a simple point of assigning a value to a cell. How should a define something remotely similar to

        a[1,2] = x;

with a being one of my user-defined arrays? opIndex works only for one index-argument. Making that argument an array of two integers makes no sense, since we don't have array literals. opCall cannot return anything that I could assign to. So, affectively, I'm stuck with something like

        a.assign(1,2,x);

(I could probably do a[1][2] = x for my user-defined array, making opIndex return an array of lower dimesionality, but that would be highly inefficient and lead in a completely wrong direction.)

Now, to turn my frustration into productivity, I have an suggestion about a language extension:

* opIndex with more than one index parameter. To avoid ambiguity, the current "opIndex(idx,value)" should probably be renamed to something like "opIndexAssign".

This will be a simple, straightforward extension. If we have multidim-arrays in the language some day in the future, this operator would be reasonable anyway, if not, it is necessary even more, to be able to define these arrays in the library.

Norbert Nemec wrote:

>I'm struggling with implementing multidimensional arrays in the library. It
>definitely is a messy task, trying to get some pretty syntax for the user!
>
>The current intension of my work is something like a case-study, to
>demonstrate a way, how multidimensional arrays might be defined as a
>language feature in the future. My initial hope, to create some library
>that might be usable as such faded very quickly. In C++, I could certainly
>do it. In D, I'm either far from understanding the language at all, or the
>language is far from being as powerful as C++ in the current state.
>
>"Powerful" in this context means: The possibility, to write a library that
>offers an expressive syntax to the user.
>
>Now, I'm already stuck at a simple point of assigning a value to a cell. How
>should a define something remotely similar to
>
>        a[1,2] = x;
>
>with a being one of my user-defined arrays? opIndex works only for one
>index-argument. Making that argument an array of two integers makes no
>sense, since we don't have array literals. opCall cannot return anything
>that I could assign to. So, affectively, I'm stuck with something like
>
>        a.assign(1,2,x);
>
>(I could probably do a[1][2] = x for my user-defined array, making opIndex
>return an array of lower dimesionality, but that would be highly
>inefficient and lead in a completely wrong direction.)
>
>Now, to turn my frustration into productivity, I have an suggestion about a
>language extension:
>
>* opIndex with more than one index parameter. To avoid ambiguity, the
>current "opIndex(idx,value)" should probably be renamed to something like
>"opIndexAssign".
>
>This will be a simple, straightforward extension. If we have multidim-arrays
>in the language some day in the future, this operator would be reasonable
>anyway, if not, it is necessary even more, to be able to define these
>arrays in the library.
>  
>
Sounds reasonable.

-- 
-Anderson: http://badmama.com.au/~anderson/

"Norbert Nemec" <Norbert.Nemec@gmx.de> wrote in message news:c6p0fd$12v8$1@digitaldaemon.com...
> I'm struggling with implementing multidimensional arrays in the library.
It
> definitely is a messy task, trying to get some pretty syntax for the user!
>
> The current intension of my work is something like a case-study, to demonstrate a way, how multidimensional arrays might be defined as a language feature in the future. My initial hope, to create some library that might be usable as such faded very quickly. In C++, I could certainly do it. In D, I'm either far from understanding the language at all, or the language is far from being as powerful as C++ in the current state.
>
> "Powerful" in this context means: The possibility, to write a library that offers an expressive syntax to the user.

It will be a while (if ever) that D has as much syntax overloading as C++. I think one of Walters goals was to make the basic syntax fairly "fixed" in order to have predictable semantics. Where to draw the line is always hard to say. Right now D is somewhere in between Java and C++ leaning more towards C++ than Java.

> Now, I'm already stuck at a simple point of assigning a value to a cell.
How
> should a define something remotely similar to
>
>         a[1,2] = x;
>
> with a being one of my user-defined arrays? opIndex works only for one index-argument. Making that argument an array of two integers makes no sense, since we don't have array literals. opCall cannot return anything that I could assign to. So, affectively, I'm stuck with something like
>
>         a.assign(1,2,x);
>
> (I could probably do a[1][2] = x for my user-defined array, making opIndex return an array of lower dimesionality, but that would be highly inefficient and lead in a completely wrong direction.)

One issue is a[1,2] = x is not allowed for built-in arrays. Well, I tried it and it looks like the comma is treated as the comma operator so that 1,2 evaluates to 2. Indexing in D is 1D only (no pun intended).

> Now, to turn my frustration into productivity, I have an suggestion about
a
> language extension:
>
> * opIndex with more than one index parameter. To avoid ambiguity, the current "opIndex(idx,value)" should probably be renamed to something like "opIndexAssign".
>
> This will be a simple, straightforward extension. If we have
multidim-arrays
> in the language some day in the future, this operator would be reasonable anyway, if not, it is necessary even more, to be able to define these arrays in the library.

Hmm. It would seem odd to me to have an overloadable operator for an operator that doesn't work for the built-in types.

[snip]
>         a[1,2] = x;
> with a being one of my user-defined arrays? opIndex works only for one
> index-argument. Making that argument an array of two integers makes no
> sense, since we don't have array literals. opCall cannot return anything
> that I could assign to.
[snip]

It might not be too bad to have opCall return a pointer and have users explicitly dereference the pointer:

struct test {
  double[9] d;
  double* opCall(int n,int m) {
    return &(x[3*n+m]);
  }
}

int main() {
  test x;
  *x(1,1) = 41.0;
  printf("%g\n",*x(1,1));
  return 0;
}

The opCall in C++ would be replaced with n-d indexing and the pointer would be replaced with a reference.

What about using a proxy object like:


struct Col
{
   private int [] array;
   static Col opCall(int size)
   {
       Col col;
       col.array.length = size;
       return col;
   }
   int opIndex(int i) { return array[i]; }
     int opIndex(int i, int value) { return array[i] = value; }
}

class Array
{
   private Col [] array;
   this(int size, int size2)
   {
       array.length = size;
             foreach ( inout Col col; array)
           col.array.length = size2;
   }
     Col opIndex(int i) { return array[i];  }
     Col opIndex(int i, Col value) { return array[i] = value;  }
}


int main (char[][] args)
{
       Array a = new Array(10,10);

       a[1][0] = 5;
       a[1][1] = 6;
             printf("a[1][0] = %d\n", a[1][0]);
             printf("a[1][1] = %d\n", a[1][1]);

       return 0;
}


Norbert Nemec wrote:

>I'm struggling with implementing multidimensional arrays in the library. It
>definitely is a messy task, trying to get some pretty syntax for the user!
>
>The current intension of my work is something like a case-study, to
>demonstrate a way, how multidimensional arrays might be defined as a
>language feature in the future. My initial hope, to create some library
>that might be usable as such faded very quickly. In C++, I could certainly
>do it. In D, I'm either far from understanding the language at all, or the
>language is far from being as powerful as C++ in the current state.
>
>"Powerful" in this context means: The possibility, to write a library that
>offers an expressive syntax to the user.
>
>Now, I'm already stuck at a simple point of assigning a value to a cell. How
>should a define something remotely similar to
>
>        a[1,2] = x;
>
>with a being one of my user-defined arrays? opIndex works only for one
>index-argument. Making that argument an array of two integers makes no
>sense, since we don't have array literals. opCall cannot return anything
>that I could assign to. So, affectively, I'm stuck with something like
>
>        a.assign(1,2,x);
>
>(I could probably do a[1][2] = x for my user-defined array, making opIndex
>return an array of lower dimesionality, but that would be highly
>inefficient and lead in a completely wrong direction.)
>
>Now, to turn my frustration into productivity, I have an suggestion about a
>language extension:
>
>* opIndex with more than one index parameter. To avoid ambiguity, the
>current "opIndex(idx,value)" should probably be renamed to something like
>"opIndexAssign".
>
>This will be a simple, straightforward extension. If we have multidim-arrays
>in the language some day in the future, this operator would be reasonable
>anyway, if not, it is necessary even more, to be able to define these
>arrays in the library.
>
>
>  
>


-- 
-Anderson: http://badmama.com.au/~anderson/

Ben Hinkle wrote:
> It will be a while (if ever) that D has as much syntax overloading as C++. I think one of Walters goals was to make the basic syntax fairly "fixed" in order to have predictable semantics. Where to draw the line is always hard to say. Right now D is somewhere in between Java and C++ leaning more towards C++ than Java.

Of course, this is one of the decisions one has to make when designing a language. C++ goes the way "Try to allow as much as possible and see what creative ideas people might have in the future." In D it is generally much harder to do something that Walter did not think of. Of course, following the C++ paradigm holds the danger of building up a incredibly complexity as we see it in C++.

> One issue is a[1,2] = x is not allowed for built-in arrays. Well, I tried it and it looks like the comma is treated as the comma operator so that 1,2 evaluates to 2. Indexing in D is 1D only (no pun intended).

True, we don't have rectangular arrays yet. As I said, the ultimate goal would be to have them and would aim at using the notation a[x,y] to distinguishing them from ragged C-style arrays a[x][y].

(The current rectangular arrays with fixed size - like int[3][3] - are something completely different from what I have in mind.)

Now, the intention of my proposal is, to extend the syntax now in a simple, straightforward way, and think about the details of the builtin-array lateron.

> Hmm. It would seem odd to me to have an overloadable operator for an operator that doesn't work for the built-in types.

"yet", that is. Maybe we should wait, but I don't believe that is necessary.

Ben Hinkle wrote:

> [snip]
>>         a[1,2] = x;
>> with a being one of my user-defined arrays? opIndex works only for one
>> index-argument. Making that argument an array of two integers makes no
>> sense, since we don't have array literals. opCall cannot return anything
>> that I could assign to.
> [snip]
> 
> It might not be too bad to have opCall return a pointer and have users explicitly dereference the pointer:
> 
> struct test {
>   double[9] d;
>   double* opCall(int n,int m) {
>     return &(x[3*n+m]);
>   }
> }
> 
> int main() {
>   test x;
>   *x(1,1) = 41.0;
>   printf("%g\n",*x(1,1));
>   return 0;
> }

As a crude hack for the moment - fine. Beyond that: it will be very confusing to the user, who expects x(1,1) to return a value in an expression.

> The opCall in C++ would be replaced with n-d indexing and the pointer would be replaced with a reference.

In C++, there is no n-d indexing either. That suggestion would be something new in D.

I thought about it, but it adds quite some overhead. In your implementation, it does not make a difference, since the array is pointer-to-pointer anyway, but with a true rectangular array, the stepwise-slicing costs more than doing it in one step.


J Anderson wrote:

> What about using a proxy object like:
> 
> 
> struct Col
> {
>     private int [] array;
>     static Col opCall(int size)
>     {
>         Col col;
>         col.array.length = size;
>         return col;
>     }
>     int opIndex(int i) { return array[i]; }
> 
>     int opIndex(int i, int value) { return array[i] = value; }
> }
> 
> class Array
> {
>     private Col [] array;
>     this(int size, int size2)
>     {
>         array.length = size;
> 
>         foreach ( inout Col col; array)
>             col.array.length = size2;
>     }
> 
>     Col opIndex(int i) { return array[i];  }
> 
>     Col opIndex(int i, Col value) { return array[i] = value;  }
> }
> 
> 
> int main (char[][] args)
> {
>         Array a = new Array(10,10);
> 
>         a[1][0] = 5;
>         a[1][1] = 6;
> 
>         printf("a[1][0] = %d\n", a[1][0]);
> 
>         printf("a[1][1] = %d\n", a[1][1]);
> 
>         return 0;
> }
> 
> 
> Norbert Nemec wrote:
> 
>>I'm struggling with implementing multidimensional arrays in the library. It definitely is a messy task, trying to get some pretty syntax for the user!
>>
>>The current intension of my work is something like a case-study, to demonstrate a way, how multidimensional arrays might be defined as a language feature in the future. My initial hope, to create some library that might be usable as such faded very quickly. In C++, I could certainly do it. In D, I'm either far from understanding the language at all, or the language is far from being as powerful as C++ in the current state.
>>
>>"Powerful" in this context means: The possibility, to write a library that offers an expressive syntax to the user.
>>
>>Now, I'm already stuck at a simple point of assigning a value to a cell. How should a define something remotely similar to
>>
>>        a[1,2] = x;
>>
>>with a being one of my user-defined arrays? opIndex works only for one index-argument. Making that argument an array of two integers makes no sense, since we don't have array literals. opCall cannot return anything that I could assign to. So, affectively, I'm stuck with something like
>>
>>        a.assign(1,2,x);
>>
>>(I could probably do a[1][2] = x for my user-defined array, making opIndex return an array of lower dimesionality, but that would be highly inefficient and lead in a completely wrong direction.)
>>
>>Now, to turn my frustration into productivity, I have an suggestion about a language extension:
>>
>>* opIndex with more than one index parameter. To avoid ambiguity, the current "opIndex(idx,value)" should probably be renamed to something like "opIndexAssign".
>>
>>This will be a simple, straightforward extension. If we have multidim-arrays in the language some day in the future, this operator would be reasonable anyway, if not, it is necessary even more, to be able to define these arrays in the library.
>>
>>
>> 
>>
> 
> 

Norbert Nemec wrote:

>I thought about it, but it adds quite some overhead. In your implementation,
>it does not make a difference, since the array is pointer-to-pointer
>anyway, but with a true rectangular array, the stepwise-slicing costs more
>than doing it in one step.
>
>  
>
You could do something like this and I guess (in this example) most of the overhead would be optimised away.

struct Col
{
   private int [][] array;
     private int i;
     static Col opCall(int [][]array, int pos)
   {
       Col col;
       col.array = array;
       col.i = pos;
       return col;
   }
     int opIndex(int j) { return array[i][j]; }
     int opIndex(int j, int value) { return array[i][j] = value; }
     void length(int len) { array.length = len; }
}

class Array
{
   private int [][] array;
     this(int size, int size2)
   {
       array.length = size;
             foreach ( inout  int [] arr; array)
           arr.length = size2;
   }
     Col opIndex(int i)
   {
       return Col(array, i);    }
}


int main (char[][] args)
{
       Array a = new Array(10,10);

       a[1][0] = 5;
       a[1][1] = 6;
             printf("a[1][0] = %d\n", a[1][0]);
             printf("a[1][1] = %d\n", a[1][1]);

       return 0;
}

-- 
-Anderson: http://badmama.com.au/~anderson/

You are using ragged arrays internally. These are inefficient anyway. Working with a single 'private int* array' internally, you will see that the stepwise indexing will take away the advantage.

J Anderson wrote:
> You could do something like this and I guess (in this example) most of the overhead would be optimised away.
> 
> struct Col
> {
>     private int [][] array;
> 
>     private int i;
> 
>     static Col opCall(int [][]array, int pos)
>     {
>         Col col;
>         col.array = array;
>         col.i = pos;
>         return col;
>     }
> 
>     int opIndex(int j) { return array[i][j]; }
> 
>     int opIndex(int j, int value) { return array[i][j] = value; }
> 
>     void length(int len) { array.length = len; }
> }
> 
> class Array
> {
>     private int [][] array;
> 
>     this(int size, int size2)
>     {
>         array.length = size;
> 
>         foreach ( inout  int [] arr; array)
>             arr.length = size2;
>     }
> 
>     Col opIndex(int i)
>     {
>         return Col(array, i);
>     }
> }
> 
> 
> int main (char[][] args)
> {
>         Array a = new Array(10,10);
> 
>         a[1][0] = 5;
>         a[1][1] = 6;
> 
>         printf("a[1][0] = %d\n", a[1][0]);
> 
>         printf("a[1][1] = %d\n", a[1][1]);
> 
>         return 0;
> }
>