http://d.puremagic.com/issues/show_bug.cgi?id=10701

           Summary: segfault after GC.realloc
           Product: D
           Version: D2
          Platform: x86_64
        OS/Version: Linux
            Status: NEW
          Severity: normal
          Priority: P2
         Component: druntime
        AssignedTo: nobody@puremagic.com
        ReportedBy: monarchdodra@gmail.com


--- Comment #0 from monarchdodra@gmail.com 2013-07-23 01:21:05 PDT ---
Created an attachment (id=1238)
segfault after realloc

Only reproduced on 64 bit posix systems.

This is kind of complicated. The reduced program needs to iterate a couple of times (probably to corrupt the GC?) a few times before the problem will trigger. The good news, is that the segfault deterministically repeats itselft, so the debug "should" be easy to do.

It would *appear* that the core culprit is creating a dynamic array (or size larger than 4080), and then calling realloc on "array.ptr". Now, I'm not 100% sure this is legal to begin with, since "array.ptr" is actually offset by 16 bytes from the start of the memory block. Is that actually undefined behavior, or does it just reduce the chances of the program working?

I'd simply leave it at that and move on, but there is something that bothers me deeply:

//----
ubyte[] arr = new ubyte[](5000);
GC.realloc(arr.ptr, 0, GC.BlkAttr.NO_SCAN);
//----
This works 100% fine (AFAIK, never segfaulted), but this:

//----
ubyte[] arr;
arr.length = 5000;
GC.realloc(arr.ptr, 0, GC.BlkAttr.NO_SCAN);
//----

Doing this ends up segfaulting later down the line.
*** Why is the behavior different? ***
I believe it is worth trying to investigate this at least a little, we might be
able to unravel a bug somewhere inside the code...

The code is in the attachment. I've reduced it as much as I could.

If somebody with more skills (and is more used to debugging in a *nix
environment) could take a peak?

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=10701


Maxim Fomin <maxim@maxim-fomin.ru> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |maxim@maxim-fomin.ru


--- Comment #1 from Maxim Fomin <maxim@maxim-fomin.ru> 2013-07-23 03:56:52 PDT ---
Reduced:

import core.memory, std.array;

extern(C) int printf(const char*, ...);

void readt()
{
    //ubyte[] result = new ubyte[](5000); //This works
    ubyte[] result; result.length = 5000; //But this fails
    GC.free(result.ptr);
    result = null;
}

string toStr(long src)
{
    auto w = appender!string();
    return "";
}

void main()
{
    foreach(int i; 0 .. 256)
    {
        printf("Step: %d\n", i);
        string corruptme = "./temp";
        foreach(e; 0 .. 256)
        {
            corruptme ~= toStr(1);
        }
        readt();
    }
}


Removing appender makes bug go away.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=10701



--- Comment #2 from monarchdodra@gmail.com 2013-07-23 05:04:48 PDT ---
(In reply to comment #1)
> Reduced:
> [...]
> Removing appender makes bug go away.

Nice.

Appender in itself isn't doing anything much. As long as an allocation occurs, any function will do:

//----
string toStr(long src)
{
    new int;
    return "";
}
//----

EG:
//----
import core.memory;

extern(C) int printf(const char*, ...);

void readt()
{
    //ubyte[] result = new ubyte[](5000); //This works
    ubyte[] result; result.length = 5000; //But this fails
    GC.free(result.ptr);
    result = null;
}

string toStr(long src)
{
    new int;
    return "";
}

void main()
{
    foreach(int i; 0 .. 1024)
    {
        printf("Step: %d\n", i);
        string corruptme = "./temp";
        foreach(e; 0 .. 256)
        {
            corruptme ~= toStr(1);
        }
        readt();
    }
}
//----

This still preserves the "This works/But this fails" issue. Cores on iteration 255 (adding more "new int" will divide that number by the amount of "new")

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=10701


Maxim Fomin <maxim@maxim-fomin.ru> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Summary|segfault after GC.realloc   |segfault in GC
           Severity|normal                      |critical


--- Comment #3 from Maxim Fomin <maxim@maxim-fomin.ru> 2013-07-23 06:07:35 PDT ---
(In reply to comment #2)
> (In reply to comment #1)
> > Reduced:
> > [...]
> > Removing appender makes bug go away.
> 
> Nice.
> 
> Appender in itself isn't doing anything much. As long as an allocation occurs, any function will do:
> 
> //----
> string toStr(long src)
> {
>     new int;
>     return "";
> }
> //----

Then futher reduced:

import core.memory;

extern(C) int printf(const char*, ...);

void readt()
{
    //ubyte[] result = new ubyte[](5000); //This works
    ubyte[] result; result.length = 5000; //But this fails
    GC.free(result.ptr); //works if commented out
    result = null;
}

void main()
{
    foreach(i; 0 .. 1024)
    {
        foreach(e; 0 .. 1024)
        {
            new int;
        }

        readt();
    }
}

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=10701


safety0ff.bugz <safety0ff.bugz@gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|                            |pull
                 CC|                            |safety0ff.bugz@gmail.com


--- Comment #4 from safety0ff.bugz <safety0ff.bugz@gmail.com> 2013-10-25 12:16:48 PDT ---
https://github.com/D-Programming-Language/druntime/pull/642

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=10701


Martin Nowak <code@dawg.eu> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |code@dawg.eu


--- Comment #5 from Martin Nowak <code@dawg.eu> 2013-10-30 09:31:24 PDT ---
cat > bug.d << CODE
import core.memory;
unittest
{
    ubyte[] result; result.length = 4096;
    GC.free(result.ptr);
    GC.collect();
}
CODE

dmd -main -unittest -run bug

----

Happens in Gcx.isMarked which incorrectly handles B_FREE pages.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=10701



--- Comment #6 from github-bugzilla@puremagic.com 2013-10-30 17:32:34 PDT ---
Commits pushed to master at https://github.com/D-Programming-Language/druntime

https://github.com/D-Programming-Language/druntime/commit/b28fc2b8cc1099d6da7ba935c222b435f46de77a regression test for Issue 10701

- The test isn't 100% reliable because it depends
  on the GC and array append cache state.

https://github.com/D-Programming-Language/druntime/commit/525a9b5e0ad0a831d3d392184a8ed9f04c4293f2 Merge pull request #1 from dawgfoto/fix10701

regression test for Issue 10701

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------