Sean Kelly wrote:
> Jarrett Billingsley wrote:
>> "Sean Kelly" <sean@f4.ca> wrote in message news:f3uoj5$1b1i$1@digitalmars.com...
>>
>>> Most array algorithms would apply.  But I'm still not sure I see
>>> the point of having an immutable reference, because it's just passed
>>> by value anyway.  Who cares if the size of the array is modified
>>> within a function where it's not passed by reference?  The change is
>>> just local to the function anyway.
>>
>> If that array is pointing into some meaningful area of memory (like
>> in the example, a texture buffer), resizing the array could (probably
>> would) move the array around, which I guess isn't illegal but then
>> the function operating on the array wouldn't be accessing the correct
>> place.  Prevent them from changing the length, it prevents them from
>> accessing anywhere but there.
>
> Well yeah.  I don't personally think this is a problem because it
> doesn't affect the callee in any way, but I can see how others might
> disagree.  Doesn't 'final' do this now though?

There was a lot of talk about this problem today. Final doesn't quite do it, because it's not part of the type signature of the function. But making it a type signature adds a whole raft of complexity, and also inevitably completely screws up the notion of transitivity of const and invariant.

(The problem is if I pass a slice to a function, and then that function reallocates the slice by changing the length or appending to it, it can affect other slices into the same data in very hard-to-explain ways. Essentially, we needed to find a way to disallow it. Options explored and found unacceptable were final (causes more problems), ref counting arrays (too expensive), always duping (too expensive), adding 'slice' bits to the fat pointers (doesn't work), adding ref counting to the memory chunks (too expensive), and data flow analysis hacks (unreliable, inconsistent). Note that other languages solve the problem with ref counting, but we considered that unacceptable for D because it adds 50% to the cost of transferring arrays around, and big runtime costs in manipulating the ref counts. And that doesn't even consider multithreading or exception safety requirements.)

Now, it turns out that it is very rare for a function to legitimately want to resize a buffer passed to it. So we finally hit on the idea of making a resizeable array a different type, say:

   T[n]   a;  // static array
   T[]    b;  // dynamic array
   T[new] c;  // resizeable array

   a.length = 6;   // error, static array
   b.length = 6;   // error, dynamic array is not resizeable
   c.length = 6;   // ok
   b = c;          // ok, implicit conversion of resizeable to dynamic
   c = b;          // error
   c = b[0..n];    // error
   b = c[0..n];    // ok, but watch out if c is later resized
   b = c[0..n].dup; // ok, no worries
   c = cast(T[new])b; // ok, but this will be deliberate

Note that .dup will return a T[new], as will operator new. Slicing will return a T[]. The runtime representation of T[new] will be identical to that of T[], just the type is different.

So, if our function parameter is typed as T[], we know there isn't going to be any monkey business going on in that function with our other slices. If it's T[new], we know we need to take a hard look at it.

I think this will entail only very rare changes to user code, but will be a big improvement in type safety. Note that it will *still* possible for array resizing to affect other slices into the same array, but it's far, far less likely to happen inadvertently. The attractiveness of this solution is it nearly completely solves the problem with zero runtime cost.

I agree with your reasoning for this (honestly, the solution had never occured to me before, although looking back I can think of a few places where it would have been quite useful), although I can't say I'm a fan of the syntax.  The "new" inside the brackets just looks strange; a new what?

A few possible alternatives:

T[new]	// Proposed
T[$]	// Since $ already means "length" for arrays
T[*]	// Indicating the index could be anything
T[..]	// Bit of a stretch...
T[volatile] // Bit long
T[[]]	// Just ugly

Like I said; I'm erring on the side of liking the idea, just not the use of "new" in that particular spot.

	-- Daniel

Walter Bright wrote

> The attractiveness of this
> solution is it nearly completely solves the problem with zero
> runtime cost.

Great thought.
If incorporating this new type, please address also the complication
of having two views on the storage allocated for resizable arrays:
- the "potential" capacity, to which the `length' property currently
can grow without the need of allocating more consecutive space
- the "actual" capacity, which is currently set/get by the `length'
property

It is a pity, that the documentation,
http://www.digitalmars.com/d/arrays.html#resize,
illustrates a pattern that can be easily incorporated into the
language as a default behaviour:
- introduce new `length.pot' r/w-property
- whenever the `length' property exceeds the `length.pot' property
double the `length.pot' property and allocate appropriate space
- issue an error when `length.pot' is decreased below the value of
`length'

-manfred

Daniel Keep wrote:
> The "new" inside the brackets just looks strange; a new what?

I'm with you here ;)


> A few possible alternatives:
> 
> T[new]	// Proposed

Looks awkward, yeah. And it would produce false hits for anyone doing new-hunting (happens when you care about allocations, but prototype using 'new').


> T[$]	// Since $ already means "length" for arrays

I don't like it... Looks like an out of bounds error to my parser ;)


> T[*]	// Indicating the index could be anything

++votes;


-- 
Tomasz Stachowiak
http://h3.team0xf.com/
h3/h3r3tic on #D freenode

Daniel Keep wrote:
> I agree with your reasoning for this (honestly, the solution had never
> occured to me before, although looking back I can think of a few places
> where it would have been quite useful), although I can't say I'm a fan
> of the syntax.  The "new" inside the brackets just looks strange; a new
> what?
> 
> A few possible alternatives:
> 
> T[new]	// Proposed

Like the idea.  Not being able to tell a slice from a "real" array has always made me a bit uncomfortable, though it never occurred to me to make them separate types.  I would have expected it would introduce more headaches than it solves, but sounds like maybe not.  Cool.

T[new], though, doesn't speak to me as a syntax.

> T[$]	// Since $ already means "length" for arrays

Reminding us that '$'/length  can be assigned to ... not bad.

> T[..]	// Bit of a stretch...

I liked it at first till I realized it would mean the *opposite* of the what seems obvious to me -- i.e. I took it to mean that T is slice-like, meaning it can't be resized.  But it's supposed to be the other case.

> T[*]	// Indicating the index could be anything

That's not bad.

> T[volatile] // Bit long
Yeh, too long.

> T[[]]	// Just ugly
Eh.

Here's another:

T[~] // mnemonic: T can be concatenated onto

Or maybe it should just be a different brace type?
T{} = new T[10];

Probably breaks context freeness or something, but I like it if for no other reason than the braces are shift-[ shift-] on the US keyboard. It's also easy to type, and it's a nice mnemonic for array with augmented capabilities.

All the T[punctuation] varieties are going to be pretty annoying to type, I think.


--bb

Bill Baxter wrote:
> T[~] // mnemonic: T can be concatenated onto

Seconded. I like it! :)
 -downs

Tom S wrote:
> Daniel Keep wrote:
>> The "new" inside the brackets just looks strange; a new what?
> 
> I'm with you here ;)
> 
> 
>> A few possible alternatives:
>>
>> T[new]    // Proposed
> 
> Looks awkward, yeah. And it would produce false hits for anyone doing new-hunting (happens when you care about allocations, but prototype using 'new').

Huh. To me, that sounds like an argument *for* using [new], since any array passed as [new] can cause an allocation in that function.

I like the [new] syntax :P

L.

Daniel Keep wrote:

> 
> I agree with your reasoning for this (honestly, the solution had never occured to me before, although looking back I can think of a few places where it would have been quite useful), although I can't say I'm a fan of the syntax.  The "new" inside the brackets just looks strange; a new what?

> T[*]  // Indicating the index could be anything

This one.

-- 
Lars Ivar Igesund
blog at http://larsivi.net
DSource, #d.tango & #D: larsivi
Dancing the Tango

Manfred Nowak wrote:
> Walter Bright wrote
> 
>> The attractiveness of this solution is it nearly completely solves the problem with zero
>> runtime cost. 
> 
> Great thought.
> If incorporating this new type, please address also the complication of having two views on the storage allocated for resizable arrays:
> - the "potential" capacity, to which the `length' property currently can grow without the need of allocating more consecutive space
> - the "actual" capacity, which is currently set/get by the `length' property
> 
> It is a pity, that the documentation, http://www.digitalmars.com/d/arrays.html#resize,
> illustrates a pattern that can be easily incorporated into the language as a default behaviour:
> - introduce new `length.pot' r/w-property
> - whenever the `length' property exceeds the `length.pot' property double the `length.pot' property and allocate appropriate space
> - issue an error when `length.pot' is decreased below the value of `length'

I agree. Although I'd keep the syntax less freaky by adding a property directly to arrays, array.capacity makes the most sense IMHO. It can be write-only for all I care.

Honestly, the array.length=12; array.length=0; -pattern (if you can call it that) is horrendous and should have been taken care of before v1.0 was released, in my not so humble opinion.

L.

Manfred Nowak wrote:
> ...
> If incorporating this new type, please address also the complication
> of having two views on the storage allocated for resizable arrays:
> - the "potential" capacity, to which the `length' property currently
> can grow without the need of allocating more consecutive space
> - the "actual" capacity, which is currently set/get by the `length'
> property
> 
> It is a pity, that the documentation,
> http://www.digitalmars.com/d/arrays.html#resize,
> illustrates a pattern that can be easily incorporated into the
> language as a default behaviour:
> - introduce new `length.pot' r/w-property
> - whenever the `length' property exceeds the `length.pot' property
> double the `length.pot' property and allocate appropriate space
> - issue an error when `length.pot' is decreased below the value of
> `length'
> 
> -manfred

This can be pretty trivially implemented as a user type; if Walter added implicit casts and, say, opDollar, it'd be nigh indistinguishable from regular arrays.

Just threw up a quick and dirty version here: http://www.prowiki.org/wiki4d/wiki.cgi?DanielKeep/snippets (bottom of the page).

	-- Daniel