Arrays are not passed by value, nor by reference.

lets consider:

void fn(int[] a)
{
  a[0] = 1;
  a.length = 3;
  a[1] = 2;
  printf("a.length = %d, a[0] = %d, a[1] = %d\n", a.length, a[0], a[1]);
}

int main()
{
  int[] t;
  t.length = 2;
  fn(t);
  printf("t.length = %d, t[0] = %d, t[1] = %d\n", t.length, t[0], t[1]);
  return 0;
}

If arrays were passed by value (as an int) this would print:
a.length = 3, a[0] = 1, a[1] = 2
t.length = 2, t[0] = 0, t[0] = 0
(Original array unchanged)

If arrays were passed by reference (as an Object) this would print:
a.length = 3, a[0] = 1, a[1] = 2
t.length = 3, t[0] = 1, t[1] = 2
(Original array is the same)

But actually it prints:

a.length = 3, a[0] = 1, a[1] = 2
t.length = 2, t[0] = 1, t[1] = 0

(if it cannot resize the array in place) *OR* (Undefined Behaviour)

a.length = 3, a[0] = 1, a[1] = 2
t.length = 2, t[0] = 1, t[1] = 2

if there is enough memory to resize the array in place.

So arrays are not passed by value, nor by reference.
Some changes to the array are incorporated into the original array, and some
are not.
They are using "passing by array reference", using which needs a deep
understanding of the low-level implementation of the arrays.
I understand that this is a result of the current (fast) implemetation of
arrays, but the result is *unacceptable*.
One easy solution would be to disallow passing arrays as "in" parameters and
require "inout". (I still like the "ref" keyword better than "inout")
But using "inout" parameters is slower, isn't it? It is "just another level
of indirection". So an effective solution would include redesigning the
low-level array implementation.

Yours,
Sandor

Sandor Hojtsy wrote:
> So arrays are not passed by value, nor by reference.
> Some changes to the array are incorporated into the original array, and some
> are not.
> They are using "passing by array reference", using which needs a deep
> understanding of the low-level implementation of the arrays.
> I understand that this is a result of the current (fast) implemetation of
> arrays, but the result is *unacceptable*.
> One easy solution would be to disallow passing arrays as "in" parameters and
> require "inout". (I still like the "ref" keyword better than "inout")
> But using "inout" parameters is slower, isn't it? It is "just another level
> of indirection". So an effective solution would include redesigning the
> low-level array implementation.

Although I disagree with the 'ref' keyword, as I think that 'inout' fits the  in/out/inout semantics better, I think that it's weird that you can resize an array that's passed as an 'in' parameter at all.  Shouldn't changing the values or resizing be an error?   I thought that this was the point of the 'in' keyword in the first place.

Evan

I would check you D compiler ?
there is still an issue with arrays passed as 'in' because the length is
stored (AFAIK) within the "array type" variable not within the array
"object". so the behaviour is similar to C or Java where arrays are passed
by reference always.

I modified you code .... and use DMD 0.44

void fn(int[] a){
  a[0] = 4;
  a[1] = 4;
  printf("a.length = %d, a[0] = %d, a[1] = %d\n", a.length, a[0], a[1]);
}

void fnex(int[] a){
  a[0] = 1;
  a.length = 3;
  a[1] = 2;
  printf("a.length = %d, a[0] = %d, a[1] = %d\n", a.length, a[0], a[1]);
}

void fnref(inout int[] a){
  a[0] = 4;
  a[1] = 4;
  printf("a.length = %d, a[0] = %d, a[1] = %d\n", a.length, a[0], a[1]);
}

void fnexref(inout int[] a){
  a[0] = 1;
  a.length = 3;
  a[1] = 2;
  printf("a.length = %d, a[0] = %d, a[1] = %d\n", a.length, a[0], a[1]);
}

int main(){
  int[] t;
  t.length = 2;
  t[0] = 0;
  t[1] = 1;
  printf("initial t.length = %d, t[0] = %d, t[1] = %d\n", t.length, t[0],
t[1]);
  fn(t);
  printf("after call fn(int[]a) t.length = %d, t[0] = %d, t[1] = %d\n",
t.length, t[0], t[1]);
  fnex(t);
  printf("after call fnex(int[]a) t.length = %d, t[0] = %d, t[1] = %d\n",
t.length, t[0], t[1]);
  t.length = 2;
  t[0] = 0;
  t[1] = 1;
  printf("reset t t.length = %d, t[0] = %d, t[1] = %d\n", t.length, t[0],
t[1]);
  fnref(t);
  printf("after call fn(inout int[]a) t.length = %d, t[0] = %d, t[1] =
%d\n", t.length, t[0], t[1]);
  fnexref(t);
  printf("after call fnex(inout int[]a) t.length = %d, t[0] = %d, t[1] =
%d\n", t.length, t[0], t[1]);
  return 0;
}

give the following output ...

initial t.length = 2, t[0] = 0, t[1] = 1
a.length = 2, a[0] = 4, a[1] = 4
after call fn(int[]a) t.length = 2, t[0] = 4, t[1] = 4
a.length = 3, a[0] = 1, a[1] = 2
after call fnex(int[]a) t.length = 2, t[0] = 1, t[1] = 2
reset t t.length = 2, t[0] = 0, t[1] = 1
a.length = 2, a[0] = 4, a[1] = 4
after call fn(inout int[]a) t.length = 2, t[0] = 4, t[1] = 4
a.length = 3, a[0] = 1, a[1] = 2
after call fnex(inout int[]a) t.length = 3, t[0] = 1, t[1] = 2

as you see inout does work, it modifies both the array and the length, it is
just 'in' that has odd behaviour
it can modify the array contents but not the length.

IMHO an array passed as 'in' should do copy-on-write (at compile time) so read only access is fast, if there are any writes then the compiler does `a = a.dup;` in the method prolog. which would give both speed and pass by value semantics.

(off to try some more tests)..

Mike.

"Sandor Hojtsy" <hojtsy@index.hu> wrote in message news:anrkjd$boh$1@digitaldaemon.com...
>
> Arrays are not passed by value, nor by reference.
>
> lets consider:
>
> void fn(int[] a)
> {
>   a[0] = 1;
>   a.length = 3;
>   a[1] = 2;
>   printf("a.length = %d, a[0] = %d, a[1] = %d\n", a.length, a[0], a[1]);
> }
>
> int main()
> {
>   int[] t;
>   t.length = 2;
>   fn(t);
>   printf("t.length = %d, t[0] = %d, t[1] = %d\n", t.length, t[0], t[1]);
>   return 0;
> }
>
> If arrays were passed by value (as an int) this would print:
> a.length = 3, a[0] = 1, a[1] = 2
> t.length = 2, t[0] = 0, t[0] = 0
> (Original array unchanged)
>
> If arrays were passed by reference (as an Object) this would print:
> a.length = 3, a[0] = 1, a[1] = 2
> t.length = 3, t[0] = 1, t[1] = 2
> (Original array is the same)
>
> But actually it prints:
>
> a.length = 3, a[0] = 1, a[1] = 2
> t.length = 2, t[0] = 1, t[1] = 0
>
> (if it cannot resize the array in place) *OR* (Undefined Behaviour)
>
> a.length = 3, a[0] = 1, a[1] = 2
> t.length = 2, t[0] = 1, t[1] = 2
>
> if there is enough memory to resize the array in place.
>
> So arrays are not passed by value, nor by reference.
> Some changes to the array are incorporated into the original array, and
some
> are not.
> They are using "passing by array reference", using which needs a deep
> understanding of the low-level implementation of the arrays.
> I understand that this is a result of the current (fast) implemetation of
> arrays, but the result is *unacceptable*.
> One easy solution would be to disallow passing arrays as "in" parameters
and
> require "inout". (I still like the "ref" keyword better than "inout") But using "inout" parameters is slower, isn't it? It is "just another
level
> of indirection". So an effective solution would include redesigning the low-level array implementation.
>
> Yours,
> Sandor
>
>

further to my last post I tried .. (again DMD 0.44)

void fnst(int[] a){ int b[2]; b[0]=1;b[1]=2;
 a = b; printf("a.length = %d, a[0] = %d, a[1] = %d\n", a.length, a[0],
a[1]);
}

void fnstex(int[] a){ int b[3]; b[0]=10;b[1]=20;b[2]=30;
 a = b; printf("a.length = %d, a[0] = %d, a[1] = %d\n", a.length, a[0],
a[1]);
}

void fn(int[] a){ int[]b = new int[2]; b[0]=1;b[1]=2;
 a = b; printf("a.length = %d, a[0] = %d, a[1] = %d\n", a.length, a[0],
a[1]);
}

void fnex(int[] a){ int[]b = new int[3]; b[0]=10;b[1]=20;b[2]=30;
 a = b; printf("a.length = %d, a[0] = %d, a[1] = %d\n", a.length, a[0],
a[1]);
}

void fnstref(inout int[] a){ int b[2]; b[0]=1;b[1]=2;
 a = b; printf("a.length = %d, a[0] = %d, a[1] = %d\n", a.length, a[0],
a[1]);
}

void fnstexref(inout int[] a){ int b[3]; b[0]=10;b[1]=20;b[2]=30;
 a = b; printf("a.length = %d, a[0] = %d, a[1] = %d\n", a.length, a[0],
a[1]);
}

void fnref(inout int[] a){ int[] b = new int[2]; b[0]=1;b[1]=2;
 a = b; printf("a.length = %d, a[0] = %d, a[1] = %d\n", a.length, a[0],
a[1]);
}

void fnexref(inout int[] a){ int[] b = new int[3]; b[0]=10;b[1]=20;b[2]=30;
 a = b; printf("a.length = %d, a[0] = %d, a[1] = %d\n", a.length, a[0],
a[1]);
}

int main(){
  int[] t;
  t.length = 2;
  t[0] = 4;
  t[1] = 4;
  printf("initial t.length = %d, t[0] = %d, t[1] = %d\n", t.length, t[0],
t[1]);
  fn(t);
  printf("after call fn(int[]a) t.length = %d, t[0] = %d, t[1] = %d\n",
t.length, t[0], t[1]);
  fnex(t);
  printf("after call fnex(int[]a) t.length = %d, t[0] = %d, t[1] = %d\n",
t.length, t[0], t[1]);
  fnst(t);
  printf("after call fnst(int[]a) t.length = %d, t[0] = %d, t[1] = %d\n",
t.length, t[0], t[1]);
  fnstex(t);
  printf("after call fnstex(int[]a) t.length = %d, t[0] = %d, t[1] = %d\n",
t.length, t[0], t[1]);
  t.length = 2;
  t[0] = 4;
  t[1] = 4;
  printf("reset t t.length = %d, t[0] = %d, t[1] = %d\n", t.length, t[0],
t[1]);
  fnref(t);
  printf("after call fnref(inout int[]a) t.length = %d, t[0] = %d, t[1] =
%d\n", t.length, t[0], t[1]);
  fnexref(t);
  printf("after call fnexref(inout int[]a) t.length = %d, t[0] = %d, t[1] =
%d\n", t.length, t[0], t[1]);
  fnstref(t);
  printf("after call fnstref(inout int[]a) t.length = %d, t[0] = %d, t[1] =
%d\n", t.length, t[0], t[1]);
  fnstexref(t);
  printf("after call fnstexref(inout int[]a) t.length = %d, t[0] = %d, t[1]
= %d\n", t.length, t[0], t[1]);
  return 0;
}

which gives a very expected .....

initial t.length = 2, t[0] = 4, t[1] = 4
a.length = 2, a[0] = 1, a[1] = 2
after call fn(int[]a) t.length = 2, t[0] = 4, t[1] = 4
a.length = 3, a[0] = 10, a[1] = 20
after call fnex(int[]a) t.length = 2, t[0] = 4, t[1] = 4
a.length = 2, a[0] = 1, a[1] = 2
after call fnst(int[]a) t.length = 2, t[0] = 4, t[1] = 4
a.length = 3, a[0] = 10, a[1] = 20
after call fnstex(int[]a) t.length = 2, t[0] = 4, t[1] = 4
reset t t.length = 2, t[0] = 4, t[1] = 4
a.length = 2, a[0] = 1, a[1] = 2
after call fn(inout int[]a) t.length = 2, t[0] = 1, t[1] = 2
a.length = 3, a[0] = 10, a[1] = 20
after call fnex(inout int[]a) t.length = 3, t[0] = 10, t[1] = 20
a.length = 2, a[0] = 1, a[1] = 2
after call fn(inout int[]a) t.length = 2, t[0] = 1, t[1] = 2
a.length = 3, a[0] = 10, a[1] = 20
after call fnex(inout int[]a) t.length = 3, t[0] = 10, t[1] = 20

BUT AFAIK the code in fnstref/fnstexref is very very dangerious.

void fnstexref(inout int[] a){
    int b[3]; b[0]=10;b[1]=20;b[2]=30;
 a = b; // a is now exactly the same as b
}
 having returned from this function, b which was allocated on the stack is
NOT promoted to a heap object
 so a which hold a reference to the same area of memory which will be
refering to a section of stack which
 has just been removed, this will is not show as a problem UNTIL you call
another function.
 this is true if you put "automatic" arrays passed as dynamics into an
object within the heap you then have
 no way to know when it are accessed (or by which thread).
the assignment a = b; much promote b to be a heap object or be dissallowed
forcing the programmer
to do a = b.dup; which must return a heap version of the static/stack
allocatted array.


"Sandor Hojtsy" <hojtsy@index.hu> wrote in message news:anrkjd$boh$1@digitaldaemon.com...
>
> Arrays are not passed by value, nor by reference.
>
> lets consider:
>
> void fn(int[] a)
> {
>   a[0] = 1;
>   a.length = 3;
>   a[1] = 2;
>   printf("a.length = %d, a[0] = %d, a[1] = %d\n", a.length, a[0], a[1]);
> }
>
> int main()
> {
>   int[] t;
>   t.length = 2;
>   fn(t);
>   printf("t.length = %d, t[0] = %d, t[1] = %d\n", t.length, t[0], t[1]);
>   return 0;
> }
>
> If arrays were passed by value (as an int) this would print:
> a.length = 3, a[0] = 1, a[1] = 2
> t.length = 2, t[0] = 0, t[0] = 0
> (Original array unchanged)
>
> If arrays were passed by reference (as an Object) this would print:
> a.length = 3, a[0] = 1, a[1] = 2
> t.length = 3, t[0] = 1, t[1] = 2
> (Original array is the same)
>
> But actually it prints:
>
> a.length = 3, a[0] = 1, a[1] = 2
> t.length = 2, t[0] = 1, t[1] = 0
>
> (if it cannot resize the array in place) *OR* (Undefined Behaviour)
>
> a.length = 3, a[0] = 1, a[1] = 2
> t.length = 2, t[0] = 1, t[1] = 2
>
> if there is enough memory to resize the array in place.
>
> So arrays are not passed by value, nor by reference.
> Some changes to the array are incorporated into the original array, and
some
> are not.
> They are using "passing by array reference", using which needs a deep
> understanding of the low-level implementation of the arrays.
> I understand that this is a result of the current (fast) implemetation of
> arrays, but the result is *unacceptable*.
> One easy solution would be to disallow passing arrays as "in" parameters
and
> require "inout". (I still like the "ref" keyword better than "inout") But using "inout" parameters is slower, isn't it? It is "just another
level
> of indirection". So an effective solution would include redesigning the low-level array implementation.
>
> Yours,
> Sandor
>
>

"Mike Wynn" <mike.wynn@l8night.co.uk> wrote in message news:anrrdt$iup$1@digitaldaemon.com...
> I would check you D compiler ?

Do you mean version? 0.44

> there is still an issue with arrays passed as 'in' because the length is stored (AFAIK) within the "array type" variable not within the array "object". so the behaviour is similar to C or Java where arrays are passed by reference always.
>
> I modified you code .... and use DMD 0.44

....

> as you see inout does work, it modifies both the array and the length, it
is
> just 'in' that has odd behaviour

Yes. "inout" works, as "inout" should, but it is slower then "in". But the "in" parameter: I would not say "odd", rather "broken".

> it can modify the array contents but not the length.

Not really. If you resize the array, it may or may not be reallocated to an
other position in the memory. So modifications after the resize, may or may
not be done to the original array. That is called Undefined Behaviour.
And is the length not a full-right property of the array? Do you consider
this behaviour logical? If you don't do anything to correct this, at least
disallow passing arrays as "in".

> IMHO an array passed as 'in' should do copy-on-write (at compile time) so read only access is fast, if there are any writes then the compiler does
`a
> = a.dup;` in the method prolog. which would give both speed and pass by value semantics.

Fine with me. But I already hear the arguments about dummy programmers passing all arrays by copy, and wasting CPU time.

> "Sandor Hojtsy" <hojtsy@index.hu> wrote in message news:anrkjd$boh$1@digitaldaemon.com...
> >
> > Arrays are not passed by value, nor by reference.
> >
> > lets consider:
> >
> > void fn(int[] a)
> > {
> >   a[0] = 1;
> >   a.length = 3;
> >   a[1] = 2;
> >   printf("a.length = %d, a[0] = %d, a[1] = %d\n", a.length, a[0], a[1]);
> > }
> >
> > int main()
> > {
> >   int[] t;
> >   t.length = 2;
> >   fn(t);
> >   printf("t.length = %d, t[0] = %d, t[1] = %d\n", t.length, t[0], t[1]);
> >   return 0;
> > }
> >
> > If arrays were passed by value (as an int) this would print:
> > a.length = 3, a[0] = 1, a[1] = 2
> > t.length = 2, t[0] = 0, t[0] = 0
> > (Original array unchanged)
> >
> > If arrays were passed by reference (as an Object) this would print:
> > a.length = 3, a[0] = 1, a[1] = 2
> > t.length = 3, t[0] = 1, t[1] = 2
> > (Original array is the same)
> >
> > But actually it prints:
> >
> > a.length = 3, a[0] = 1, a[1] = 2
> > t.length = 2, t[0] = 1, t[1] = 0
> >
> > (if it cannot resize the array in place) *OR* (Undefined Behaviour)
> >
> > a.length = 3, a[0] = 1, a[1] = 2
> > t.length = 2, t[0] = 1, t[1] = 2
> >
> > if there is enough memory to resize the array in place.
> >
> > So arrays are not passed by value, nor by reference.
> > Some changes to the array are incorporated into the original array, and
> some
> > are not.
> > They are using "passing by array reference", using which needs a deep
> > understanding of the low-level implementation of the arrays.
> > I understand that this is a result of the current (fast) implemetation
of
> > arrays, but the result is *unacceptable*.
> > One easy solution would be to disallow passing arrays as "in" parameters
> and
> > require "inout". (I still like the "ref" keyword better than "inout") But using "inout" parameters is slower, isn't it? It is "just another
> level
> > of indirection". So an effective solution would include redesigning the low-level array implementation.
> >
> > Yours,
> > Sandor
> >
> >
>
>

"Evan McClanahan" <evan@dontSPAMaltarinteractive.com> wrote in message news:anrq9c$hpc$1@digitaldaemon.com...
> Sandor Hojtsy wrote:
> > So arrays are not passed by value, nor by reference.
> > Some changes to the array are incorporated into the original array, and
some
> > are not.
> > They are using "passing by array reference", using which needs a deep
> > understanding of the low-level implementation of the arrays.
> > I understand that this is a result of the current (fast) implemetation
of
> > arrays, but the result is *unacceptable*.
> > One easy solution would be to disallow passing arrays as "in" parameters
and
> > require "inout". (I still like the "ref" keyword better than "inout") But using "inout" parameters is slower, isn't it? It is "just another
level
> > of indirection". So an effective solution would include redesigning the low-level array implementation.
>
> Although I disagree with the 'ref' keyword, as I think that 'inout' fits the  in/out/inout semantics better,

You mean it rhimes? Is that important?

> I think that it's weird that you can
> resize an array that's passed as an 'in' parameter at all.  Shouldn't
> changing the values or resizing be an error?   I thought that this was
> the point of the 'in' keyword in the first place.

It was not. "In" means "pass a copy of the value to the function".
For "primitive types" (Java jargon), this means copy the value, for objects,
it means copy the reference.
For arrays it means some mystic in-between.

void fn(in int a, in Object o)
{
  a = 3;
  o = NULL;
}

int main()
{
  int b = 2;
  Object o = new Object();
  fn(b, o);
  // b and o is unchanged here
  return 0;
}

Sandor

> > IMHO an array passed as 'in' should do copy-on-write (at compile time)
so
> > read only access is fast, if there are any writes then the compiler does
> `a
> > = a.dup;` in the method prolog. which would give both speed and pass by value semantics.
>
> Fine with me. But I already hear the arguments about dummy programmers passing all arrays by copy, and wasting CPU time.
>
I thought D was aimed at experianced programmers.
having to consider the performance effects of an action is part of
programming in any language, as long as
the effects are known then I see no problem.

I agree that arrays passed 'in' are not just odd but broken.

I find the following  nasty too
a dynamic array has a length that I can effect but a capacity that I can
not.
you can't have a zero length array,
null dynamic arrays can be added to.
stack allocated static arrays can be passed as mutable dynamic arrays as a
side effect the memory they reference can end up being referenced from a
heap item or a earlier stack frame.


Mike.

"Mike Wynn" <mike.wynn@l8night.co.uk> wrote in message news:anudkf$1oal$1@digitaldaemon.com...
> I find the following  nasty too
> a dynamic array has a length that I can effect but a capacity that I can
> not.

Correct.

> you can't have a zero length array,

Sure you can.

> null dynamic arrays can be added to.

A null dynamic array is just one with 0 length.

> stack allocated static arrays can be passed as mutable dynamic arrays as a side effect the memory they reference can end up being referenced from a heap item or a earlier stack frame.

Yes, arrays are passed by reference, like class objects are. Perhaps the solution is to simply disallow resizing of an array passed as 'in'.

"Sandor Hojtsy" <hojtsy@index.hu> wrote in message news:anrkjd$boh$1@digitaldaemon.com...
> They are using "passing by array reference", using which needs a deep
> understanding of the low-level implementation of the arrays.
> I understand that this is a result of the current (fast) implemetation of
> arrays, but the result is *unacceptable*.
> One easy solution would be to disallow passing arrays as "in" parameters
and
> require "inout". (I still like the "ref" keyword better than "inout") But using "inout" parameters is slower, isn't it? It is "just another
level
> of indirection". So an effective solution would include redesigning the low-level array implementation.

Actually, I think simply disallowing resizing of 'in' arrays would do the trick.

"Mike Wynn" <mike.wynn@l8night.co.uk> wrote in message news:anrt0b$kfh$1@digitaldaemon.com...
> void fnstexref(inout int[] a){
>     int b[3]; b[0]=10;b[1]=20;b[2]=30;
>  a = b; // a is now exactly the same as b
> }
>  having returned from this function, b which was allocated on the stack is
> NOT promoted to a heap object
>  so a which hold a reference to the same area of memory which will be
> refering to a section of stack which
>  has just been removed, this will is not show as a problem UNTIL you call
> another function.

Yes, that case should generate an error message.