I haven't started reading Andrei's chapter on arrays yet. I hope I won't find out that the following behavior is expected. :)

import std.cstream;

void modify(int[] a)
{
    a[0] = 1;
    a ~= 2;

    dout.writefln("During: ", a);
}

void main()
{
    int[] a = [ 0 ];

    dout.writefln("Before: ", a);
    modify(a);
    dout.writefln("After : ", a);
}

The output with dmd 2.035 is

Before: [0]
During: [1,2]
After : [1]

I don't understand arrays. :D

Ali

== Quote from Ali Cehreli (acehreli@yahoo.com)'s article
> I haven't started reading Andrei's chapter on arrays yet. I hope I won't find
out that the following behavior is expected. :)
> import std.cstream;
> void modify(int[] a)
> {
>     a[0] = 1;
>     a ~= 2;
>     dout.writefln("During: ", a);
> }
> void main()
> {
>     int[] a = [ 0 ];
>     dout.writefln("Before: ", a);
>     modify(a);
>     dout.writefln("After : ", a);
> }
> The output with dmd 2.035 is
> Before: [0]
> During: [1,2]
> After : [1]
> I don't understand arrays. :D
> Ali

This is one of those areas where the low-level details of how arrays are implemented arrays leak out.  This is unfortunate, but in a close-to-the-metal language it's sometimes a necessary evil.

(Dynamic) Arrays are structs that consist of a pointer to the first element and a length.  Essentially, the memory being pointed to by the array is passed by reference, but the pointer to the memory and the length of the array are passed by value.  While this may seem ridiculous at first, it's a tradeoff that allows for the extremely convenient slicing syntax we have to be implemented efficiently.

When you do the a[0] = 1, what you're really doing is:

*(a.ptr) = 1;

When you do the a ~= 2, what you're really doing is:

// Make sure the block of memory pointed to by a.ptr
// has enough capacity to be appended to.
a.length += 1;
*(a.ptr + 1) = 2;

Realistically, the only way to understand D arrays and use them effectively is to understand the basics of how they work under the hood.  If you try to memorize a bunch of abstract rules, it will seem absurdly confusing.

Ali Cehreli schrieb:
> I haven't started reading Andrei's chapter on arrays yet. I hope I won't find out that the following behavior is expected. :)
> 
> import std.cstream;
> 
> void modify(int[] a)
> {
>     a[0] = 1;
>     a ~= 2;
> 
>     dout.writefln("During: ", a);
> }
> 
> void main()
> {
>     int[] a = [ 0 ];
> 
>     dout.writefln("Before: ", a);
>     modify(a);
>     dout.writefln("After : ", a);
> }
> 
> The output with dmd 2.035 is
> 
> Before: [0]
> During: [1,2]
> After : [1]
> 
> I don't understand arrays. :D
> 
> Ali
> 

int[] a;
a is kind of a pointer, one with the extra length information.
When passed to modify(), a is passed by-value, the contained data is
certainly passed by-reference since a points to the data.

This is why the a.length was not updated.

If you change "modify" to :
void modify(ref int[] a){...

it should work as you expected.

dsimcha wrote:
> == Quote from Ali Cehreli (acehreli@yahoo.com)'s article
>> I haven't started reading Andrei's chapter on arrays yet. I hope I won't find
> out that the following behavior is expected. :)
>> import std.cstream;
>> void modify(int[] a)
>> {
>>     a[0] = 1;
>>     a ~= 2;
>>     dout.writefln("During: ", a);
>> }
>> void main()
>> {
>>     int[] a = [ 0 ];
>>     dout.writefln("Before: ", a);
>>     modify(a);
>>     dout.writefln("After : ", a);
>> }
>> The output with dmd 2.035 is
>> Before: [0]
>> During: [1,2]
>> After : [1]
>> I don't understand arrays. :D
>> Ali
> 
> This is one of those areas where the low-level details of how arrays are
> implemented arrays leak out.  This is unfortunate, but in a close-to-the-metal
> language it's sometimes a necessary evil.
> 
> (Dynamic) Arrays are structs that consist of a pointer to the first element and a
> length.  Essentially, the memory being pointed to by the array is passed by
> reference, but the pointer to the memory and the length of the array are passed by
> value.  While this may seem ridiculous at first, it's a tradeoff that allows for
> the extremely convenient slicing syntax we have to be implemented efficiently.
> 
> When you do the a[0] = 1, what you're really doing is:
> 
> *(a.ptr) = 1;
> 
> When you do the a ~= 2, what you're really doing is:
> 
> // Make sure the block of memory pointed to by a.ptr
> // has enough capacity to be appended to.
> a.length += 1;
> *(a.ptr + 1) = 2;
> 
> Realistically, the only way to understand D arrays and use them effectively is to
> understand the basics of how they work under the hood.  If you try to memorize a
> bunch of abstract rules, it will seem absurdly confusing.

main.a starts as:
struct {
  int length = 1;
  int *data = 0x12345; // some address pointing to [ 0 ]
}

inside of modify, a is:
struct { // different then main.a
   int length = 2;
   int *data = 0x12345; // same as main.a data [ 1, 2]
}

back in main:
struct { // same as original main.a
  int length = 1;
  int *data = 0x12345; // hasn't changed address, but data has to [ 1 ]
}


To get the expected results, pass a as a reference:

void modify(ref int[] a);

"Frank Benoit" <keinfarbton@googlemail.com> wrote in message news:hcvff9$9cr$1@digitalmars.com...
> Ali Cehreli schrieb:
>> I haven't started reading Andrei's chapter on arrays yet. I hope I won't find out that the following behavior is expected. :)
>>
>> import std.cstream;
>>
>> void modify(int[] a)
>> {
>>     a[0] = 1;
>>     a ~= 2;
>>
>>     dout.writefln("During: ", a);
>> }
>>
>> void main()
>> {
>>     int[] a = [ 0 ];
>>
>>     dout.writefln("Before: ", a);
>>     modify(a);
>>     dout.writefln("After : ", a);
>> }
>>
>> The output with dmd 2.035 is
>>
>> Before: [0]
>> During: [1,2]
>> After : [1]
>>
>> I don't understand arrays. :D
>>
>> Ali
>>
>
> int[] a;
> a is kind of a pointer, one with the extra length information.
> When passed to modify(), a is passed by-value, the contained data is
> certainly passed by-reference since a points to the data.
>
> This is why the a.length was not updated.
>
> If you change "modify" to :
> void modify(ref int[] a){...
>
> it should work as you expected.

Or you could force totally-by-value semantics with:

void modify(const(int)[] a){
int[] _a = a.dup;
...

(Anyone know if scope can be used on that to allocate it on the stack, or is that just for classes?)

I do agree it can sometimes be a bit weird though. But like others mentioned, it's kind of a necissary evil, and once you understand how the arrays work under-the-hood, it becomes a bit easier.

dsimcha wrote:
> == Quote from Ali Cehreli (acehreli@yahoo.com)'s article
>> I haven't started reading Andrei's chapter on arrays yet. I hope I won't find
> out that the following behavior is expected. :)
>> import std.cstream;
>> void modify(int[] a)
>> {
>>     a[0] = 1;
>>     a ~= 2;
>>     dout.writefln("During: ", a);
>> }
>> void main()
>> {
>>     int[] a = [ 0 ];
>>     dout.writefln("Before: ", a);
>>     modify(a);
>>     dout.writefln("After : ", a);
>> }
>> The output with dmd 2.035 is
>> Before: [0]
>> During: [1,2]
>> After : [1]
>> I don't understand arrays. :D
>> Ali
> 
> This is one of those areas where the low-level details of how arrays are
> implemented arrays leak out.  This is unfortunate, but in a close-to-the-metal
> language it's sometimes a necessary evil.
> 
> (Dynamic) Arrays are structs that consist of a pointer to the first element and a
> length.  Essentially, the memory being pointed to by the array is passed by
> reference, but the pointer to the memory and the length of the array are passed by
> value.  While this may seem ridiculous at first, it's a tradeoff that allows for
> the extremely convenient slicing syntax we have to be implemented efficiently.
> 
> When you do the a[0] = 1, what you're really doing is:
> 
> *(a.ptr) = 1;
> 
> When you do the a ~= 2, what you're really doing is:
> 
> // Make sure the block of memory pointed to by a.ptr
> // has enough capacity to be appended to.
> a.length += 1;
> *(a.ptr + 1) = 2;
> 
> Realistically, the only way to understand D arrays and use them effectively is to
> understand the basics of how they work under the hood.  If you try to memorize a
> bunch of abstract rules, it will seem absurdly confusing.

I don't think it's that bad. Bartosz tried to get me into a diatribe about how array behavior can't be defined formally. Of course it can.

The chunk and the limits of the chunk are part of D's array abstraction. The limits are passed by value. The ~= operation may nondeterministically choose to bind the limits to a different chunk. The right to modify members as they want is a fundamental right of any non-const member, so no confusion there. The decision is encapsulated. User code must write code that works according to that specification.

Could there be a better array specification? No doubt. But much as he tried, Bartosz couldn't come up with one. We couldn't come up with one. So if you could come up with one, speak up or forever use the existing one.


Andrei

Thanks for all the responses.

And yes, I know that 'ref' is what works for me here. I am trying to figure out whether I should develop a guideline like "always pass arrays with 'ref', or you may face surprises."

I understand it very well now and was able to figure out a way to cause some bugs. :)

What can be said about the output of the following program? Will main.a[0] be printed as 1 or 111?

import std.cstream;

void modify(int[] a)
{
    a[0] = 1;

    // ... more operations ...

    a[0] = 111;
}

void main()
{
    int[] a;
    a ~= 0;
    modify(a);

    dout.writefln(a[0]);
}

It depends on the operations in between the two assignments to a[0] in 'modify':

- if we leave the comment in place, main.a[0] is 111

- if we replace the comment with this code

    foreach (i; 0 .. 10) {
        a ~= 2;
    }

then main.a[0] is 1. In a sense, modify.a caused only "some" side effects in main.a. If we shorten the foreach, then main.a[0] is again 111. To me, this is at an unmanagable level. Unless we always pass with 'ref'.

I don't think that this is easy to explain to a learner; and I think that is a good indicator that there is a problem with these semantics.

Ali

Ali Cehreli wrote:
> Thanks for all the responses.
> 
> And yes, I know that 'ref' is what works for me here. I am trying to figure out whether I should develop a guideline like "always pass arrays with 'ref', or you may face surprises."
> 
> I understand it very well now and was able to figure out a way to cause some bugs. :)
> 
> What can be said about the output of the following program? Will main.a[0] be printed as 1 or 111?
> 
> import std.cstream;
> 
> void modify(int[] a)
> {
>     a[0] = 1;
> 
>     // ... more operations ...
> 
>     a[0] = 111;
> }
> 
> void main()
> {
>     int[] a;
>     a ~= 0;
>     modify(a);
> 
>     dout.writefln(a[0]);
> }
> 
> It depends on the operations in between the two assignments to a[0] in 'modify':
> 
> - if we leave the comment in place, main.a[0] is 111
> 
> - if we replace the comment with this code
> 
>     foreach (i; 0 .. 10) {
>         a ~= 2;
>     }
> 
> then main.a[0] is 1. In a sense, modify.a caused only "some" side effects in main.a. If we shorten the foreach, then main.a[0] is again 111. To me, this is at an unmanagable level. Unless we always pass with 'ref'.
> 
> I don't think that this is easy to explain to a learner; and I think that is a good indicator that there is a problem with these semantics.

The ball is in your court to define better semantics.

Andrei

Ali Cehreli wrote...

This helps me with the meaning of in, out & ref. http://bayimg.com/NaeOgaaCC

Andrei Alexandrescu, el  5 de noviembre a las 16:10 me escribiste:
> Ali Cehreli wrote:
> >Thanks for all the responses.
> >
> >And yes, I know that 'ref' is what works for me here. I am trying to figure out whether I should develop a guideline like "always pass arrays with 'ref', or you may face surprises."
> >
> >I understand it very well now and was able to figure out a way to cause some bugs. :)
> >
> >What can be said about the output of the following program? Will main.a[0] be printed as 1 or 111?
> >
> >import std.cstream;
> >
> >void modify(int[] a)
> >{
> >    a[0] = 1;
> >
> >    // ... more operations ...
> >
> >    a[0] = 111;
> >}
> >
> >void main()
> >{
> >    int[] a;
> >    a ~= 0;
> >    modify(a);
> >
> >    dout.writefln(a[0]);
> >}
> >
> >It depends on the operations in between the two assignments to a[0] in 'modify':
> >
> >- if we leave the comment in place, main.a[0] is 111
> >
> >- if we replace the comment with this code
> >
> >    foreach (i; 0 .. 10) {
> >        a ~= 2;
> >    }
> >
> >then main.a[0] is 1. In a sense, modify.a caused only "some" side effects in main.a. If we shorten the foreach, then main.a[0] is again 111. To me, this is at an unmanagable level. Unless we always pass with 'ref'.
> >
> >I don't think that this is easy to explain to a learner; and I think that is a good indicator that there is a problem with these semantics.
> 
> The ball is in your court to define better semantics.

Just make arrays a reference value, like classes!

-- 
Leandro Lucarella (AKA luca)                     http://llucax.com.ar/
----------------------------------------------------------------------
GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145  104C 949E BFB6 5F5A 8D05)
----------------------------------------------------------------------
A lo que Peperino respondióles: aquel que tenga sabañones que se los
moje, aquel que padece calvicie no padece un osito, no es bueno comer
lechón en día de gastritis, no mezcleis el vino con la sandía, sacad la
basura después de las ocho, en caso de emergencia rompa el vidrio con
el martillo, a cien metros desvio por Pavón.
	-- Peperino Pómoro