On Thu, 17 Feb 2005 11:10:13 +0000, Stewart Gordon <smjg_1998@yahoo.com> wrote:
> Regan Heath wrote:
>> On Wed, 16 Feb 2005 12:58:46 +0000, Stewart Gordon <smjg_1998@yahoo.com>  wrote:
> <snip>
>>>      b = a;
>>>      a = a.dup;
>>>      foreach (inout x; a) x--;
>>   Not this, because "b = a" is a copy i.e. "b = a.dup"
>
> What do you mean?  b = a on dynamic arrays is, by definition, a reference assignment.

Gargh, you're totally right.

If "b = a;" is:
 - b reference a

If "a--;" is:
 - create a new array
 - set new[x] = a[x] - 1;

Then "b = a--;" is:
 - b reference a
 - create new array
 - set new[x] = a[x] - 1
 - a reference new

which is exactly what you had above :)

>>> or
>>>
>>>      b = a.dup;
>>>      foreach (inout x; a) x--;
>>  This is probably the most sensible.
> <snip>
>
> And it's also consistent with what can be done with opPostInc and opPostDec on a class.

At the time I was thinking it was consistent with:

int i = 5;
int j;

j = i--;


The result in both cases is however the same, b contains what a did, and a contains a[x]-1, eg.

>>>      b = a;
>>>      a = a.dup;
>>>      foreach (inout x; a) x--;

 - b reference a
 - create new array
 - new[x] = a[x] - 1
 - a reference new

----

>>>      b = a.dup;
>>>      foreach (inout x; a) x--;

 - b reference a.dup
 - a[x] = a[x] - 1

----

I think the top one is more consistent with D's reference semantics, the bottom one is consistent with D's value type semantics i.e. it's what happens with ints.

So I reckon the top one is the 'correct' one.

Regan

On Thu, 17 Feb 2005 05:58:39 +0100, Norbert Nemec <Norbert@Nemec-online.de> wrote:
> Regan Heath schrieb:
>> What about:
>>  test.length    //length of array
>> test[].length  //length of each element in array
>
> Bad idea: 'test[]' is an array by itself, so it has a length property by itself...

Is it? I thought it was a class, with a length property.

# class Foo {
#   int length;
# }

now, if it had an [] operator (opIndex?) then that would be a problem.
I have no bright ideas yet...

If it was an array...

Foo[][] test;

test.length     == length of [][]
test[].length   == length of each [] in []
test[][].length == foo length property.

Regan.

On Fri, 18 Feb 2005 11:55:09 +1300, Regan Heath <regan@netwin.co.nz> wrote:
> On Thu, 17 Feb 2005 11:10:13 +0000, Stewart Gordon <smjg_1998@yahoo.com> wrote:
>> Regan Heath wrote:
>>> On Wed, 16 Feb 2005 12:58:46 +0000, Stewart Gordon <smjg_1998@yahoo.com>  wrote:
>> <snip>
>>>>      b = a;
>>>>      a = a.dup;
>>>>      foreach (inout x; a) x--;
>>>   Not this, because "b = a" is a copy i.e. "b = a.dup"
>>
>> What do you mean?  b = a on dynamic arrays is, by definition, a reference assignment.
>
> Gargh, you're totally right.
>
> If "b = a;" is:
>   - b reference a
>
> If "a--;" is:
>   - create a new array
>   - set new[x] = a[x] - 1;

forgot
    - a reference new

> Then "b = a--;" is:
>   - b reference a
>   - create new array
>   - set new[x] = a[x] - 1
>   - a reference new
>
> which is exactly what you had above :)
>
>>>> or
>>>>
>>>>      b = a.dup;
>>>>      foreach (inout x; a) x--;
>>>  This is probably the most sensible.
>> <snip>
>>
>> And it's also consistent with what can be done with opPostInc and opPostDec on a class.
>
> At the time I was thinking it was consistent with:
>
> int i = 5;
> int j;
>
> j = i--;
>
>
> The result in both cases is however the same, b contains what a did, and a contains a[x]-1, eg.
>
>>>>      b = a;
>>>>      a = a.dup;
>>>>      foreach (inout x; a) x--;
>
>   - b reference a
>   - create new array
>   - new[x] = a[x] - 1
>   - a reference new
>
> ----
>
>>>>      b = a.dup;
>>>>      foreach (inout x; a) x--;
>
>   - b reference a.dup
>   - a[x] = a[x] - 1
>
> ----
>
> I think the top one is more consistent with D's reference semantics, the bottom one is consistent with D's value type semantics i.e. it's what happens with ints.
>
> So I reckon the top one is the 'correct' one.
>
> Regan

xs0 wrote:
>> How about this?
>>
>>     while (...) {
>>         a.length = x.length;
>>         a[] = x + y;
>>     }
> 
> Again, why would this not be the default case (i.e. done by the compiler?) Also consider that if a is int[][][], you'll also need to allocate all the int[][]'s and really really quickly that a=x+y becomes totally invisible, all because you didn't want to change your spec (which is completely fine, aside from the issue we're arguing over).
> 
> And I'd guess that only the outer array gets reused with a[]=x+y? So what would you have the user type then? a[][][]=x+y? That's probably not even legal.. And what if a is of type T as in template(T)?

At the moment, a[][][] is just the same as a[], since it's just slicing the outermost dimension again and again.  Well done, you do have a point there.  Maybe someone else has an idea....

>> Exactly.  So it seems silly to convolute the language semantics for this minority of cases.
> 
> There's nothing convoluted.. both a= and a[]= are defined only for things that are currently supported. x+y on arrays is not defined currently, so you can define = for array ops as you want. = doesn't do the same for structs and classes either, and no one seems to mind (even though they're really similar otherwise).

If you simply mean that structs are contained by value and classes by reference, then the semantics _are_ the same: copy what is represented by the right operand into the piece of memory represented by the left operand, whether this is a value or a reference.

>> <snip>
> 
> you tend to snip my best arguments, methinks :P
> 
>> Where a is a dynamic array, a = ... by definition does a reference assignment.
> 
> I don't agree there's any such definition, but let's not go there. What if it is a static array, you'll force the users to type [] every time? When they (this time really) obviously want it in-place. Or would you have it work differently in each case?

I wouldn't consider it at all obvious that someone who types

    z = x + y;

really meant

    z[] = x + y;

If I wrote the former, I would consider it obvious that it is what I meant.

<snip>
> a+=b acts like in-place for classes,

Like I said, only if the class defines opAddAssign.  Otherwise, it's equivalent to a = a + b.

> is in-place for primitives,

Because primitives are contained by value.

> so why would it be different for arrays?

My thought is that it would be just like a class on which opAddAssign isn't defined.

>>> int[] a=new ...;
>>> int[] b=a;
>>>
>>> a+=5;
>>>
>>> // with my suggestion, a is still the same object and a===b
>>> // according to your spec a!==b
>>>
>>> <snip>
>> 
>> Hmm....
> 
> Hmm what?

Good question....

>> Just think about it.  Suppose
>>
>>     z = x;
>>
>> does a reference assignment, but
>>
>>     z = x + y;
>>
>> does an in-place modification.  
> 
> Maybe, but z+=x certainly indicates in-place modification, so it would seem you can't have it totally consistent anyhow.. If that is the case I'd certainly prefer in-place whenever possible.

Again, there's always the option of using z[] += x.

>> <snip>
> 
> The semantics of the = operator are not always the same even without array ops. x+y is also not defined to be anything in particular (currently). Why do you keep insisting that they are? Considering that your spec itself says that the effect is the same as if you wrote the indicated code, you can just change the expansion in the spec to not reallocate when not necessary and there you have it.

It already does, in the second paragraph of the first heading.  Except that it appears your use of the word "necessary" might be inconsistent with mine.

>>>>     z = x + y;
>>> 
>>> I really fail to see why a new array is necessary here, if it already exists. To me it looks the same, as if:
>> 
>> Because this is what the programmer asked for.
> 
> No, he didn't. I can't see from that statement that the programmer wishes to allocate a new z.

The basic request is that any other references to the data pointed to by z before the assignment will still be pointing to the same data as they were.

<snip>
> To sum up, you'd have stuff defined like this:
> 
> a = b + c; // create a new array (really bad performance-wise,
>            // but still desired behavior like 1% of time)
> a[] = b + c; // fail if a is null/wrong size, otherwise work in-place
>              // useful when the operands are really known to be the same
>              // size all the time (that would be like 20% of time)
> a = (b+c).dup; // same as a=b+c (and as useless)
> 
> And I'd have it defined like this:
> 
> a = b + c; // work in-place when possible (useful like 80% of the time)
> a[] = b + c; // same as above
> a = (b+c).dup; // same as above
> 
> So you'd have two syntaxes for the same crappy-most-of-the-time behavior, and you'd have the programmer write additional code for the most common case (for handling arbitrary lengths with memory reuse).
> 
> Would you agree that is a good summary?

Good at summarising, yes.  Good at convincing me, I'm not sure.

Maybe we should disallow

    a = b + c;

altogether, instead requiring the coder to specify explicitly

    a[] = b + c;

or

    a = (b + c).dup;

Stewart.

-- 
My e-mail is valid but not my primary mailbox.  Please keep replies on the 'group where everyone may benefit.

Regan Heath wrote:
<snip>
> If it was an array...
> 
> Foo[][] test;
> 
> test.length     == length of [][]
> test[].length   == length of each [] in []
> test[][].length == foo length property.

The trouble is that, at the moment, test[][] is equivalent to test[]. It's just the whole array taken a slice of again.

Another catch is that if test is a struct/class, by writing test[].length you could be asking to take the length of the array returned by its opSlice method.  And so either some existing code may well be broken or the semantics wouldn't be context-free....

(OK, so one would be an lvalue, and one an rvalue, but I recall commenting on that PCC before....)

Stewart.

-- 
My e-mail is valid but not my primary mailbox.  Please keep replies on the 'group where everyone may benefit.

> At the moment, a[][][] is just the same as a[], since it's just slicing the outermost dimension again and again.  Well done, you do have a point there.  Maybe someone else has an idea....

Wow .. You actually agreed with something :)


> If you simply mean that structs are contained by value and classes by reference, then the semantics _are_ the same: copy what is represented by the right operand into the piece of memory represented by the left operand, whether this is a value or a reference.

Well, I guess we understand semantics differently.. This is how I see it for the = operator:

int a=b;          // copy value of b to a
struct a=b;       // copy contents of b to a (might seem the same as
                  // the previous case, but struct is a composite
                  // and furthermore is different from a class)
class a=b;        // make a point to b
array a=b;        // make a point to b
array a[]=(int)b; // set all elements of a to b
array a[]=b;      // copy data from b to a
array a=b[c..d];  // create a new slice with same contents as b[c..d]
                  // (memory is reused when possible, so watch out)
array a=b.dup;    // create a copy of b and make a point to it

These explanations are not all equal, so you can't say that the semantics are the same. IMHO, of course :)


> I wouldn't consider it at all obvious that someone who types
> 
>     z = x + y;
> 
> really meant
> 
>     z[] = x + y;
> 
> If I wrote the former, I would consider it obvious that it is what I meant.

If you knew that all the vars are arrays, that z=x+y means that a new array gets created and you wrote that, you would indeed consider it obvious. It's not obvious in itself; if x, y and z are ints, you don't expect a new int to get allocated.. You also don't expect (*a)=2+3 to change the pointer, yet it is just as reference to int, as the above z is reference to array..


> Like I said, only if the class defines opAddAssign.  Otherwise, it's equivalent to a = a + b.

But regardless of whether defined opAddAssign is defined, when you see a+=b, you interpret it as in-place, even though behind the scenes something else happens. The difference is also that you write (or not) opAddAssign yourself, so you can change what is going on, which is not the case for arrays.


> My thought is that it would be just like a class on which opAddAssign isn't defined.

But why would it not be like a class on which opAddAssign _is_ defined? At this point it can go either way and I don't see any real benefits in your case and I do see real benefits in my case.


> Again, there's always the option of using z[] += x.

If you know in advance z is an array (and possibly even an array of dimension exactly 1). What would you write in all other cases?


> It already does, in the second paragraph of the first heading.  Except that it appears your use of the word "necessary" might be inconsistent with mine.

Well, the whole spec could just as easily be defined only with loop expansions of such expressions and there'd be no need to create new arrays and then optimize them away (isn't that kind of pointless?). (a+b*c).dup can be defined as a special case that actually creates a new array and there you have it. The only issue is with what checks are done, where I again suggest:

a=b+c (or any other expression with arrays) means that a is made to be a valid array to hold the result of b+c (the "it just works in 99% of cases" principle); if it already is, nothing is done; if it's null, it gets allocated; if it's the wrong size, its .length is adjusted. this is really useful functionality and saves a lot of typing, errors, CPU time and whatnot, even though it doesn't change a as reference, only a's length/values/nullness..

a[]=b+c, on the other hand, means that a is already expected to be the right size and it is an error, if it is not (like it currently is with a[]=b or a[]=b[]). Just like a=b+c except an added assert(a!=null && a.length=b.length).


>> No, he didn't. I can't see from that statement that the programmer wishes to allocate a new z.
> 
> The basic request is that any other references to the data pointed to by z before the assignment will still be pointing to the same data as they were.

Why is this a basic request? Slices also have perfectly useful behavior, but it's not required that either they point to the same memory or to different memory; it works in a way that's efficient (memory gets reused) yet still user-friendly (if reuse is not possible, it automatically gets reallocated). Why not have the same with array ops? Automatically being the keyword - so it just works.



> Maybe we should disallow
> 
>     a = b + c;
> 
> altogether, instead requiring the coder to specify explicitly
> 
>     a[] = b + c;
> 
> or
> 
>     a = (b + c).dup;

Well, that's an option, but I don't think it's a good one, unless we get a new operator that does what I'm trying to suggest (automatic handling of stuff that really can be automatically handled). Like <== or something, and it can be equal to = in all cases but arrays (so templates are possible). With templates you can then use = if you don't expect arrays (as in you feel that they will not work), and <== if you do expect/support them.


xs0

xs0 wrote:
<snip>
> And I'd guess that only the outer array gets reused with a[]=x+y? So what would you have the user type then? a[][][]=x+y? That's probably not even legal.. And what if a is of type T as in template(T)?

Just thinking about this, at the moment we don't seem to have a means of deep in-place copying of nested dynamic arrays.  If we're going to support deep in-place assignment when doing arithmetic, we should have a corresponding means of simply copying an array in this way.  And so I guess the syntax would correspond.

<snip>
> I don't agree there's any such definition, but let's not go there. What if it is a static array, you'll force the users to type [] every time? When they (this time really) obviously want it in-place. Or would you have it work differently in each case?
<snip>

Oops, I kind of slipped in my last reply.  With static arrays, always assigning by value.  I didn't get round to covering static arrays in my proposal, but figured it would be straightforward.

Indeed, when assigning to a static array, obviously it would be in place, since static arrays are by value.  And if you have

    int[6][] x = new int[6][42];
    x[] = ...;

then the assignment would be fully in place.  But at least one question remains: how should assignment of an expression on static arrays to a dynamic array be handled?  (Is this a reference assignment at the moment?  I should expect so.)

Stewart.

-- 
My e-mail is valid but not my primary mailbox.  Please keep replies on the 'group where everyone may benefit.

xs0 wrote:
<snip>
>> I wouldn't consider it at all obvious that someone who types
>>
>>     z = x + y;
>>
>> really meant
>>
>>     z[] = x + y;
>>
>> If I wrote the former, I would consider it obvious that it is what I meant.
> 
> 
> If you knew that all the vars are arrays, that z=x+y means that a new array gets created and you wrote that, you would indeed consider it obvious. It's not obvious in itself; if x, y and z are ints, you don't expect a new int to get allocated.. You also don't expect (*a)=2+3 to change the pointer, yet it is just as reference to int, as the above z is reference to array..

Of course not, here it's obvious that I'm changing the contents of an already allocated piece of memory.  Just like it is when I use [] to force in-place assignment.

<snip>
>> My thought is that it would be just like a class on which opAddAssign isn't defined.
> 
> But why would it not be like a class on which opAddAssign _is_ defined? At this point it can go either way and I don't see any real benefits in your case and I do see real benefits in my case.
> 
>> Again, there's always the option of using z[] += x.
> 
> If you know in advance z is an array (and possibly even an array of dimension exactly 1). What would you write in all other cases?

Again, you have a point or two.

>> It already does, in the second paragraph of the first heading.  Except that it appears your use of the word "necessary" might be inconsistent with mine.
> 
> Well, the whole spec could just as easily be defined only with loop expansions of such expressions and there'd be no need to create new arrays and then optimize them away (isn't that kind of pointless?).0

This would seem a step back towards the poorly-defined old spec.  But I'm not sure.  Moreover, we should certainly be able to pass an array expression to a function that expects an array.

<snip>
>> The basic request is that any other references to the data pointed to by z before the assignment will still be pointing to the same data as they were.
> 
> Why is this a basic request? Slices also have perfectly useful behavior, but it's not required that either they point to the same memory or to different memory; it works in a way that's efficient (memory gets reused) yet still user-friendly (if reuse is not possible, it automatically gets reallocated). Why not have the same with array ops? Automatically being the keyword - so it just works.

Yes, it should reuse memory where possible.  But only where the compiler can determine that this doesn't affect the behaviour of the program - and hence again this would be part of optimisation rather than language spec.

After all, D is designed with the optimisation technology of modern compilers in mind.

>> Maybe we should disallow
>>
>>     a = b + c;
>>
>> altogether, instead requiring the coder to specify explicitly
>>
>>     a[] = b + c;
>>
>> or
>>
>>     a = (b + c).dup;
> 
> Well, that's an option, but I don't think it's a good one, unless we get a new operator that does what I'm trying to suggest (automatic handling of stuff that really can be automatically handled). Like <== or something, and it can be equal to = in all cases but arrays (so templates are possible). With templates you can then use = if you don't expect arrays (as in you feel that they will not work), and <== if you do expect/support them.

Yes, generic programming implications are something else to consider. Which option are you suggesting that <== should be equivalent to?

The "don't expect arrays" case could also cover instances that don't do arithmetic on the parameter type at all, and as such will work implicitly.

Stewart.

-- 
My e-mail is valid but not my primary mailbox.  Please keep replies on the 'group where everyone may benefit.

> Of course not, here it's obvious that I'm changing the contents of an already allocated piece of memory.  Just like it is when I use [] to force in-place assignment.

Well, I should've written {{ int a; int *b=&a; a=2+3; }} and b still points to a.. But anyhow, we now seem to agree that there are issues, so let's leave = alone :)


> Yes, it should reuse memory where possible.  But only where the compiler can determine that this doesn't affect the behaviour of the program - and hence again this would be part of optimisation rather than language spec.

Well, but this is exactly the problem - the compiler can't determine that (except for inner operations, as in + and * in (a+b*c) and possibly for local variables that are created within the function and don't get passed as parameters and don't get sliced or casted to pointers or whatnot..) But that totally ruins reuse between function calls, for example, and also a bunch of other cases.. (even if no other method touches an object var and it is protected from the outside world, you can still extend the class and write a method that does, for example).

I don't believe it is a problem to reuse even when it could affect the behavior, if it is documented. First, in many cases it will not really affect the behavior, even if the compiler can't determine that. Second, in a lot of other cases, this will actually be desired. In the few cases it will not be desired, the programmer will know to take care of that, as it can be the first thing written in the description of <== operator and assuming that a=b+c is made illegal for arrays, all cases are covered.. And, for the umpteenth time, slices also reuse memory without causing major problems :)


> Yes, generic programming implications are something else to consider. Which option are you suggesting that <== should be equivalent to?

a <== expr means variable a gets allocated (if null) or resized (if non-null but wrong size) so it matches expr, including all potential dimensions. This also means that it will be useful as a replacement of a[]=b (as in a<==b), when you'll want lazy allocation and/or automatic handling of different sizes.. For non-array types, it behaves exactly like =.


> The "don't expect arrays" case could also cover instances that don't do arithmetic on the parameter type at all, and as such will work implicitly.

Well, if all you do is a=b, it will still work with arrays, although if that's really all you do, it shouldn't be an issue?

Of course, there's also op=. Which one should it be ( op= or op<== ), assuming not both get supported (which will probably not happen.. think about <=<== or >>><== or !<>=<== :)? I'd vote for <==, as op= does mean in-place in the case where it is normally used (i.e. primitives).

To further improve templating it might also make sense to extend .dup to value-based types (to handle cases where you do want copies made). Unlike most other properties, it does make sense on all types - it says I want a copy. On primitive types and structs, .dup would just be their value itself (and that is their copy already). Classes can also implement .dup to return their clone. This only leaves a slight inefficiency when you do something like {{ a=(b+c*d).dup }} with classes, but when this becomes an issue, you can handle all class types with a single template specialization for Object.


With all that, .minIdx, .maxIdx, .sum and .count and decent compiler support, some really neat things could be done, effective both in amount of code needed and resulting speed.


xs0

Stewart Gordon schrieb:
> Regan Heath wrote:
> <snip>
> 
>> If it was an array...
>>
>> Foo[][] test;
>>
>> test.length     == length of [][]
>> test[].length   == length of each [] in []
>> test[][].length == foo length property.
> 
> 
> The trouble is that, at the moment, test[][] is equivalent to test[]. It's just the whole array taken a slice of again.

Why "at the moment"? If you want to change this, I believe you are asking for deep trouble. (Not only compatibility, but consistency of the language)

[] simply is the full-slice operator. If a regular slice operator returns an array, so should the full-slice operator.