http://d.puremagic.com/issues/show_bug.cgi?id=10593

           Summary: array's reserve/capacity go haywire if length has been
                    changed prior
           Product: D
           Version: unspecified
          Platform: x86
        OS/Version: Windows
            Status: NEW
          Severity: critical
          Priority: P2
         Component: druntime
        AssignedTo: nobody@puremagic.com
        ReportedBy: monarchdodra@gmail.com


--- Comment #0 from monarchdodra@gmail.com 2013-07-10 03:45:32 PDT ---
On a win32 install on win7 64.

Array size must be bigger than 2047.

Not sure who is failing here (reserve? capacity?) but there is a big discrepancy between the two. It may and/or may not also make subsequent appends fail, or behave erratically.

My tests show capacity failing to report correctly. relocation being indeterminate (I know it is not determinate behavior, but there's stink in my tests), and reserve going out of control...

This is my test program:

//----
import std.array, std.stdio, core.memory;

void main()
{
    enum M = 2047;
    enum N = 4080;
    //Basic case, everything works correctly
    {
        ubyte[] a = new ubyte[](M);
        writefln("a.capacity before: %s", a.capacity);      //4079
        writefln("a.reserve(%s): %s", N, a.reserve(N)); //8175
        writefln("a.reserve(%s): %s", N, a.reserve(N)); //8175
        writefln("a.capacity after:  %s", a.capacity);      //8175
        auto b = a;
        b.length = N;
        writefln("Relocation after append?  %s", a.ptr !is b.ptr); //false
    }
    writeln();

    {
        ubyte[] a = new ubyte[](M);
        a ~= 1; // <= This little bastard here >:(
        writefln("a.capacity before: %s", a.capacity); //4079
        writefln("a.reserve(%s): %s", N, a.reserve(N)); //8175
        writefln("a.capacity after: %s", a.capacity); //4079 !!!
        auto b = a;
        b.length = N;
        writefln("Relocation after append? %s", a.ptr !is b.ptr); //false
    }
    writeln();

    {
        ubyte[] a;
        a.length = M; // <= This little bastard here >:(
        writefln("a.capacity before: %s", a.capacity); //4079
        writefln("a.reserve(%s): %s", N, a.reserve(N)); //8175
        writefln("a.capacity after: %s", a.capacity); //4079 !!!
        auto b = a;
        b.length = N;
        writefln("Relocation after append? %s", a.ptr !is b.ptr); //false
    }
    writeln();

    {
        ubyte[] a;
        a.length = M;
        writefln("a.reserve(%s): %s", N, a.reserve(N)); //8175
        writefln("a.reserve(%s): %s", N, a.reserve(N)); //12271 !!!
        writefln("a.reserve(%s): %s", N, a.reserve(N)); //16367 !!!
        writefln("a.reserve(%s): %s", N, a.reserve(N)); //20463 !!!
        writefln("a.capacity after: %s", a.capacity); //4079 !!!
    }
    writeln();
}
//----

And corresponding output (!!! added manually).

//----
a.capacity before: 4079
a.reserve(4080): 8175
a.reserve(4080): 8175
a.capacity after:  8175
Relocation after append?  false

a.capacity before: 4079
a.reserve(4080): 8175
a.capacity after: 4079
Relocation after append? false

a.capacity before: 4079
a.reserve(4080): 8175
a.capacity after: 4079
Relocation after append? false

a.reserve(4080): 8175
a.reserve(4080): 12271
a.reserve(4080): 16367
a.reserve(4080): 20463
a.capacity after: 4079
//----

To trigger, M must be at least as big as 2047, and N must be enough to require work.

Marking as critical, as this is really a core feature of D, and *really* getting in the way of some of my array/appender fixes.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=10593



--- Comment #1 from monarchdodra@gmail.com 2013-07-10 05:05:14 PDT ---
Note: The same behavior happens with GC.extend too: extend will succeed, but the reported capacity will not be updated.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=10593


monarchdodra@gmail.com changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Platform|x86                         |All
         OS/Version|Windows                     |All
           Severity|critical                    |blocker


--- Comment #2 from monarchdodra@gmail.com 2013-07-15 13:40:35 PDT ---
I'm marking this as blocker.

The fact that a reserve could fail, while still doing something, is highly problematic.

I also believe this is the issue creating the failures here: https://github.com/D-Programming-Language/phobos/pull/1413

Will provide more information as I can.

Also: Confirmed on all platforms.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=10593


Martin Nowak <code@dawg.eu> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |code@dawg.eu


--- Comment #3 from Martin Nowak <code@dawg.eu> 2013-07-20 09:58:49 PDT ---
It seems that reserve doesn't update the capacity.
If bug 6372 is not the reason for this you might want to look at the array
blockinfo cache.

reduced test case
---
cat > bug.d << CODE
void main()
{
    enum M = 4079;
    ubyte[] a = new ubyte[](M - 1);
    a ~= 1;
    immutable u = a.reserve(M+1);
    assert(u == a.capacity);
}
CODE
dmd -run bug
---

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=10593


Rainer Schuetze <r.sagitario@gmx.de> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |r.sagitario@gmx.de


--- Comment #4 from Rainer Schuetze <r.sagitario@gmx.de> 2013-07-21 00:34:21 PDT ---
There is an update missing to the blkinfo cache in the array extending:

https://github.com/D-Programming-Language/druntime/pull/548

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=10593



--- Comment #5 from github-bugzilla@puremagic.com 2013-07-21 00:45:27 PDT ---
Commit pushed to master at https://github.com/D-Programming-Language/druntime

https://github.com/D-Programming-Language/druntime/commit/4f0e73f1fdcd8c8c3898089f3ae1707f77ff857d Merge pull request #548 from rainers/fix10593

fix  Issue 10593 - array's reserve/capacity go haywire if length has been changed prior

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=10593


monarchdodra@gmail.com changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|                            |FIXED


-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------