View mode: basic / threaded / horizontal-split · Log in · Help
August 24, 2012
[Issue 8583] New: AA ushort[dchar] byValue range is corrupted on x86_64
http://d.puremagic.com/issues/show_bug.cgi?id=8583

          Summary: AA ushort[dchar] byValue range is corrupted on x86_64
          Product: D
          Version: D2
         Platform: x86_64
       OS/Version: All
           Status: NEW
         Severity: critical
         Priority: P2
        Component: DMD
       AssignedTo: nobody@puremagic.com
       ReportedBy: dmitry.olsh@gmail.com


--- Comment #0 from Dmitry Olshansky <dmitry.olsh@gmail.com> 2012-08-24 14:00:21 PDT ---
The following sample outlines the problem:

void main(){
   ushort[dchar] simpleIndices = ['A':437];
   assert(simpleIndices['A'] == 437);
   assert(simpleIndices.byKey.front == 'A');
   assert(simpleIndices.byValue.front == 437); // this fails on x64
   //assert(simpleIndices.byValue.front == 0); //and this passes on x64 WTF??!
}

compiled with -m32 it passes
with -m64 it fails and instead the value is 0. 
Looking through my corrupted data in the wild I assume it may as well be
something else then 0 too. 

Tested both with latest git and vanila 2.060 on OS Linux x64 & Win32.

There is a workaround of using .values which does allocate an array of values.
Still I consider it critical as corrupted value bug is usually hard to spot and
narrow down.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
August 25, 2012
[Issue 8583] AA ushort[dchar] byValue range is corrupted on x86_64
http://d.puremagic.com/issues/show_bug.cgi?id=8583


edmccard@verizon.net changed:

          What    |Removed                     |Added
----------------------------------------------------------------------------
                CC|                            |edmccard@verizon.net


--- Comment #1 from edmccard@verizon.net 2012-08-24 19:40:31 PDT ---
From my tests, the problem occurs whenever keytype.sizeof <= 8, for any value
type. For example, ushort[string], ushort[uint[3]] and ushort[Foo] are ok,
where

 struct Foo { int x; int y; int z; }

but ushort[uint], ushort[uint[2]] and ushort[Bar] fail, where

 struct Bar { int x; int y; }

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
August 25, 2012
[Issue 8583] AA ushort[dchar] byValue range is corrupted on x86_64
http://d.puremagic.com/issues/show_bug.cgi?id=8583



--- Comment #2 from edmccard@verizon.net 2012-08-24 20:05:06 PDT ---
(In reply to comment #1)
> From my tests, the problem occurs whenever keytype.sizeof <= 8, for any value
> type.

Spoke too soon; the problem is not that simple, or does not depend solely on
the size of the key type. For example, string[uint[3]] works but
ushort[uint[3]] fails, ushort[uint[4]] works, but both string[uint[5]] and
ushort[uint[5]] fail. (replace uint[x] with a struct of the same size, and the
same things happen).

Sorry for the noise.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
November 14, 2012
[Issue 8583] [64 bit] AA ushort[dchar] byValue range is corrupted on x86_64
http://d.puremagic.com/issues/show_bug.cgi?id=8583


Don <clugdbug@yahoo.com.au> changed:

          What    |Removed                     |Added
----------------------------------------------------------------------------
                CC|                            |exetoc@gmail.com


--- Comment #3 from Don <clugdbug@yahoo.com.au> 2012-11-14 02:46:21 PST ---
*** Issue 7632 has been marked as a duplicate of this issue. ***

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
January 16, 2013
[Issue 8583] [64 bit] AA ushort[dchar] byValue range is corrupted on x86_64
http://d.puremagic.com/issues/show_bug.cgi?id=8583


Simon Harris <pearfalse@googlemail.com> changed:

          What    |Removed                     |Added
----------------------------------------------------------------------------
                CC|                            |pearfalse@googlemail.com


--- Comment #4 from Simon Harris <pearfalse@googlemail.com> 2013-01-16 07:37:10 PST ---
Something interesting I found in rt/aaA.d, line 106: aligntsize() on 64-bit
compilation returns a value aligned to 16 bytes. _aaGetX uses this function on
key sizes when creating new elements. So for AAs whose key is <= 8 bytes, the
value is still 16 bytes further on in memory.

You can check this by repeating the same alignment trick on the addresses of
byValue.front:

void testAAType(K, V)(V[K] aa)
{
   writefln("testAAType with key %s, value %s", typeid(K), typeid(V));

   for (auto v = aa.byValue; !v.empty; v.popFront()) { // emulated foreach (v
; aa.byValue)
       V* fp = &v.front();
       writef("Unaligned: <%s>", *fp);

       V* fpaligned = cast(V*) (( (cast(ptrdiff_t) fp) + 15) & ~15); // copy
what aligntsize does on x64

       version(D_LP64) {
           writefln("; Aligned: <%s>", *fp.alignto16());
       }
       else {
           writefln(" (Not testing aligned value on a 32-bit executable)");
       }
   }
}

Looking at the key sizes seems to match the pass/fail results in comment #2,
with one exception: ushort[uint[3]]. Not sure why the alignment trick is needed
here, given that uint[3].sizeof > 8. Maybe DMD aligns
AssociativeArray!(uint[3], ushort).Slot to 4 bytes to save space, and since
_aaGetX uses a completely different type set, alignment info gets lost
somewhere.

Tested with DMD 2.060 and 2.061 on OS X 10.8.2.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
January 21, 2013
[Issue 8583] [64 bit] AA ushort[dchar] byValue range is corrupted on x86_64
http://d.puremagic.com/issues/show_bug.cgi?id=8583



--- Comment #5 from github-bugzilla@puremagic.com 2013-01-21 10:28:09 PST ---
Commits pushed to master at https://github.com/D-Programming-Language/druntime

https://github.com/D-Programming-Language/druntime/commit/e9445dcbbfd7dec9275e78fba6633b689ca06c09
Unify AA value alignment to size_t.sizeof (fixes bug 8583)

Sets the alignment of values in both AA declarations (object and aaA)
to size_t.sizeof bytes, which fixes byValue range iteration on x64.

https://github.com/D-Programming-Language/druntime/commit/9eb3f28e299cb0ce0b2fb22798f4086f008a8a87
Add unit tests for bug 8583; restore 16-byte aligment on x64

This adds unit tests for the alignment fix to Slot, ensuring the two
declarations of an AA slot stay in sync. The 16-byte alignment rule for
x64 has been restored.

https://github.com/D-Programming-Language/druntime/commit/105e12f5b933797a4cae84295a3ebe445e8ae909
Merge pull request #386 from pearfalse/master

Explicitly set AA value alignment in slot to size_t.sizeof bytes (fixes bug
8583)

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
January 21, 2013
[Issue 8583] [64 bit] AA ushort[dchar] byValue range is corrupted on x86_64
http://d.puremagic.com/issues/show_bug.cgi?id=8583


Alex Rønne Petersen <alex@lycus.org> changed:

          What    |Removed                     |Added
----------------------------------------------------------------------------
            Status|NEW                         |RESOLVED
                CC|                            |alex@lycus.org
        Resolution|                            |FIXED


-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
February 12, 2013
[Issue 8583] [64 bit] AA ushort[dchar] byValue range is corrupted on x86_64
http://d.puremagic.com/issues/show_bug.cgi?id=8583



--- Comment #6 from github-bugzilla@puremagic.com 2013-02-11 23:12:16 PST ---
Commits pushed to https://github.com/D-Programming-Language/druntime

https://github.com/D-Programming-Language/druntime/commit/e9445dcbbfd7dec9275e78fba6633b689ca06c09
Unify AA value alignment to size_t.sizeof (fixes bug 8583)

https://github.com/D-Programming-Language/druntime/commit/9eb3f28e299cb0ce0b2fb22798f4086f008a8a87
Add unit tests for bug 8583; restore 16-byte aligment on x64

https://github.com/D-Programming-Language/druntime/commit/105e12f5b933797a4cae84295a3ebe445e8ae909
Merge pull request #386 from pearfalse/master

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Top | Discussion index | About this forum | D home