Wow, there went several hours of debugging.

Increasing the length of a slice, by setting its length, will initialize the new elements and reallocate if necessary.

I did not realize length was "smart", I guess I should have guessed.

Anyway, to work around this, and probably also be more clear, create a new slice from the same pointer.

`array = array.ptr[0..newLength];`

On Sat, Dec 12, 2020 at 12:53:09AM +0000, Jonathan Levi via Digitalmars-d wrote:
> Wow, there went several hours of debugging.
> 
> Increasing the length of a slice, by setting its length, will initialize the new elements and reallocate if necessary.
> 
> I did not realize length was "smart", I guess I should have guessed.
> 
> Anyway, to work around this, and probably also be more clear, create a new slice from the same pointer.
> 
> `array = array.ptr[0..newLength];`

I highly recommend reading the following article if you work with D arrays in any non-trivial way:

	https://dlang.org/articles/d-array-article.html


T

-- 
Fact is stranger than fiction.

Yes, pointers are the only unsafe way to access memory, slices don't allow it.

Am 12.12.2020 um 01:53 schrieb Jonathan Levi:
> Wow, there went several hours of debugging.
> 
> Increasing the length of a slice, by setting its length, will initialize the new elements and reallocate if necessary.
> 
> I did not realize length was "smart", I guess I should have guessed.
> 
> Anyway, to work around this, and probably also be more clear, create a new slice from the same pointer.
> 
> `array = array.ptr[0..newLength];`

One way around this is to call `array.assumeSafeAppend();` before setting the new length. In this case it will reuse the already allocated block as long as it is large enough and only reallocate if necessary.

On Saturday, 12 December 2020 at 00:53:09 UTC, Jonathan Levi wrote:
> Wow, there went several hours of debugging.
>
> Increasing the length of a slice, by setting its length, will initialize the new elements and reallocate if necessary.
>
> I did not realize length was "smart", I guess I should have guessed.
>
> Anyway, to work around this, and probably also be more clear, create a new slice from the same pointer.
>
> `array = array.ptr[0..newLength];`

Hold on -- how does this not corrupt memory? As soon as the length exceeds the allocated capacity (the point at which the slice would be reallocated when setting its length) you will have a silent out of bounds violation, identical to overflowing a C array.

Am I wrong??

If you do not want the expansion to be initialized, I guess you could allocate a new uninitialized slice and copy contents over explicitly.

--Bastiaan.

On Saturday, 12 December 2020 at 00:53:09 UTC, Jonathan Levi wrote:
> Wow, there went several hours of debugging.
>
> Increasing the length of a slice, by setting its length, will initialize the new elements and reallocate if necessary.
>
> I did not realize length was "smart", I guess I should have guessed.
>
> Anyway, to work around this, and probably also be more clear, create a new slice from the same pointer.
>
> `array = array.ptr[0..newLength];`

You're setting yourself up for failure with that. What are you trying to "work around"? The allocation, or the initialization?

On Saturday, 12 December 2020 at 00:53:09 UTC, Jonathan Levi wrote:
> Wow, there went several hours of debugging.
>
> Increasing the length of a slice, by setting its length, will initialize the new elements and reallocate if necessary.
>
> I did not realize length was "smart", I guess I should have guessed.
>
> Anyway, to work around this, and probably also be more clear, create a new slice from the same pointer.
>
> `array = array.ptr[0..newLength];`

There is a big downside in doing that: the array will not check whether it's still referring to valid memory after the resize.

Your way is efficient in machine code, but in most cases it's highly unpractical to skip on memory safety to speed up code like this.

In the general case, this is a better way to resize arrays without reallocating:

```
@safe resizedWithin(T)(T[] arr, T[] within, size_t newSize)
{  if(newSize == 0) return arr[0..0];
   auto startIndex= &arr[0] - &within[0];
   return within[startIndex .. startIndex + newSize];
}

@safe void main()
{  import std;
   auto containerArray = iota(1000).array;
   auto array = containerArray[50 .. 60];
   array = array.resizedWithin(containerArray, 20);
   //[50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65,
   //66, 67, 68, 69]
   writeln(array);
}
```

Here, if you accidently gave too big new size for `array`, or `array` wasn't withing `containerArray` (except if `array.length == 0`), the program would immediately abort instead of making an invalid array.

On Sunday, 13 December 2020 at 13:19:35 UTC, Dukc wrote:
> ```
> @safe resizedWithin(T)(T[] arr, T[] within, size_t newSize)
> {  if(newSize == 0) return arr[0..0];
>    auto startIndex= &arr[0] - &within[0];
>    return within[startIndex .. startIndex + newSize];
> }
> ```

Okay, there is a bug in my code that it won't work if `arr` is originally of length 0. May well contain other bugs, use with care :D. But hey, at least no memory corruption, as it's `@safe`.

On 12/11/20 7:53 PM, Jonathan Levi wrote:
> Wow, there went several hours of debugging.
> 
> Increasing the length of a slice, by setting its length, will initialize the new elements and reallocate if necessary.
> 
> I did not realize length was "smart", I guess I should have guessed.
> 
> Anyway, to work around this, and probably also be more clear, create a new slice from the same pointer.
> 
> `array = array.ptr[0..newLength];`

Lots of good responses to a mostly ambiguous message.

So let's go over some possibilities:

1. You want to *shrink* the array length. array = array[0 .. newLength] works just fine. No reallocation, no initialization.

2. You want to *grow* the array length. array = array.ptr[0 .. newLength] is incredibly wrong and dangerous. You should not do this.

3. You wish to have no allocation for growing an array beyond it's already-allocated block. I only mention this because it could be implied by your message, even though I'm pretty sure you don't mean this. This is fantasy, and you should not do this. Memory corruption is something you don't want to deal with. It's the reason why your chosen solution is incorrect.

4. You wish to have no allocation for growing an array into it's ALREADY allocated block. This is possible, and even possible without reinitializing the new elements. In this context, your code is actually OK, though like Sönke mentions, you should call assumeSafeAppend on the array:

assert(newLength <= array.capacity); // ensure I am not growing beyond the block.
array = array.ptr[0 .. newLength]; // yay, new data that is uninitialized (mostly).
array.assumeSafeAppend(); // now the runtime is aware that I have taken over that data for use.

Why is it important to call assumeSafeAppend? A few reasons:

1. The GC will run destructors on elements in an array only if they are known to be used (in the case that your elements have destructors).
2. If you don't call it, appending to the original slice could overwrite your data
3. If you try to append to the resulting array and there technically would be space to fill inside the current block, the runtime will needlessly reallocate if your array ends outside where it thinks it should end.

Alternative to the assert, you could check for capacity and newLength to be consistent, and if not, reallocate yourself.

-Steve

On Saturday, 12 December 2020 at 14:12:06 UTC, Mike Parker wrote:
> On Saturday, 12 December 2020 at 00:53:09 UTC, Jonathan Levi wrote:
>> Wow, there went several hours of debugging.
>>
>> Increasing the length of a slice, by setting its length, will initialize the new elements and reallocate if necessary.
>>
>> I did not realize length was "smart", I guess I should have guessed.
>>
>> Anyway, to work around this, and probably also be more clear, create a new slice from the same pointer.
>>
>> `array = array.ptr[0..newLength];`
>
> You're setting yourself up for failure with that. What are you trying to "work around"? The allocation, or the initialization?

How is avoiding an expensive potentially memory leaking operation "setting yourself up for failure"?