I wrote about this one yesterday in the D forum. I wasn't yet sure it wasn't my fault. Now I managed to isolate it in smaller code... Maybe it could be redused even more, I don't know I've copied stuctures from my source...
When const int c=5; everything goes as expected, but if c becomes larger  number like 50000 the associative array clas.test is damaged. And in my program when I found that I didn't have any char[50000] at all. Just when memory usage increases the array gets damage. And I don't know maybe some other data is corrupted too...

Here is the code:

-------------------cut here------------------------------------------

import std.stdio;

struct akStyleEntry
{
	byte number;
	char[] description;
	char[] font_face;
	byte font_size;
	ushort flags;
	int colour_back;
	int colour_fore;
	byte charset;
}

class akStyle
{
	akStyleEntry[] styles;
}

akStyle[char[]] LoadStyles()
{
	akStyle[char[]] ret;
	ret["gosho"]=new akStyle;
	ret["tosho"]=new akStyle;
	ret["sosho"]=new akStyle;
	ret["mosho"]=new akStyle;
	for(int c=0;c<a;c++)
	{
		akStyleEntry tr;
		tr.font_face="font face"~std.string.toString(c)~"\0";
		ret["gosho"].styles~=tr;
		ret["tosho"].styles~=tr;
		ret["sosho"].styles~=tr;
		ret["mosho"].styles~=tr;
	}
	return ret;
}

class clas
{
	static akStyle[char[]] test;
}

char[] tst;

const int a=5;
const int b=50;
const int c=5;

int main(char[][] argv)
{
	clas.test=LoadStyles();
	for(int c=0;c<a;c++)
	{
		writef("%s\n",clas.test["gosho"].styles[c].font_face);
		writef("%s\n",clas.test["tosho"].styles[c].font_face);
		writef("%s\n",clas.test["sosho"].styles[c].font_face);
		writef("%s\n",clas.test["mosho"].styles[c].font_face);
	}
	char[c] m;
	for(int c=0;c<b;c++)
	{
		tst~=m;
	}
	for(int c=0;c<a;c++)
	{
		writefln(__LINE__);
		writef("%s\n",clas.test["gosho"].styles[c].font_face);
		writefln(__LINE__);
		writef("%s\n",clas.test["tosho"].styles[c].font_face);
		writefln(__LINE__);
		writef("%s\n",clas.test["sosho"].styles[c].font_face);
		writefln(__LINE__);
		writef("%s\n",clas.test["mosho"].styles[c].font_face);
	}
	return 1;
}

Looks like the GC fullCollect is to blame. If in your code I replace
 char[c] m;
 for(int c=0;c<b;c++)
 {
 tst~=m;
 }
with
 gc.fullCollect();
it also fails. Without the fullCollect it is fine. For some odd reason the
GC thinks your strings are garbage. ouch.

"bobef" <bobef@paintballforce.com> wrote in message news:d1rl7b$26bb$1@digitaldaemon.com...
>I wrote about this one yesterday in the D forum. I wasn't yet sure it wasn't my fault. Now I managed to isolate it in smaller code... Maybe it could be redused even more, I don't know I've copied stuctures from my source...
> When const int c=5; everything goes as expected, but if c becomes larger number like 50000 the associative array clas.test is damaged. And in my program when I found that I didn't have any char[50000] at all. Just when memory usage increases the array gets damage. And I don't know maybe some other data is corrupted too...
>
> Here is the code:
>
> -------------------cut here------------------------------------------
>
> import std.stdio;
>
> struct akStyleEntry
> {
> byte number;
> char[] description;
> char[] font_face;
> byte font_size;
> ushort flags;
> int colour_back;
> int colour_fore;
> byte charset;
> }
>
> class akStyle
> {
> akStyleEntry[] styles;
> }
>
> akStyle[char[]] LoadStyles()
> {
> akStyle[char[]] ret;
> ret["gosho"]=new akStyle;
> ret["tosho"]=new akStyle;
> ret["sosho"]=new akStyle;
> ret["mosho"]=new akStyle;
> for(int c=0;c<a;c++)
> {
> akStyleEntry tr;
> tr.font_face="font face"~std.string.toString(c)~"\0";
> ret["gosho"].styles~=tr;
> ret["tosho"].styles~=tr;
> ret["sosho"].styles~=tr;
> ret["mosho"].styles~=tr;
> }
> return ret;
> }
>
> class clas
> {
> static akStyle[char[]] test;
> }
>
> char[] tst;
>
> const int a=5;
> const int b=50;
> const int c=5;
>
> int main(char[][] argv)
> {
> clas.test=LoadStyles();
> for(int c=0;c<a;c++)
> {
> writef("%s\n",clas.test["gosho"].styles[c].font_face);
> writef("%s\n",clas.test["tosho"].styles[c].font_face);
> writef("%s\n",clas.test["sosho"].styles[c].font_face);
> writef("%s\n",clas.test["mosho"].styles[c].font_face);
> }
> char[c] m;
> for(int c=0;c<b;c++)
> {
> tst~=m;
> }
> for(int c=0;c<a;c++)
> {
> writefln(__LINE__);
> writef("%s\n",clas.test["gosho"].styles[c].font_face);
> writefln(__LINE__);
> writef("%s\n",clas.test["tosho"].styles[c].font_face);
> writefln(__LINE__);
> writef("%s\n",clas.test["sosho"].styles[c].font_face);
> writefln(__LINE__);
> writef("%s\n",clas.test["mosho"].styles[c].font_face);
> }
> return 1;
> }

more info. It works if the "byte" fields of the struct are removed. It looks like the GC is goofing up with detecting pointers somehow. It looks like if the size of the struct is an odd number the GC misses pointers.


"bobef" <bobef@paintballforce.com> wrote in message news:d1rl7b$26bb$1@digitaldaemon.com...
>I wrote about this one yesterday in the D forum. I wasn't yet sure it wasn't my fault. Now I managed to isolate it in smaller code... Maybe it could be redused even more, I don't know I've copied stuctures from my source...
> When const int c=5; everything goes as expected, but if c becomes larger number like 50000 the associative array clas.test is damaged. And in my program when I found that I didn't have any char[50000] at all. Just when memory usage increases the array gets damage. And I don't know maybe some other data is corrupted too...
>
> Here is the code:
>
> -------------------cut here------------------------------------------
>
> import std.stdio;
>
> struct akStyleEntry
> {
> byte number;
> char[] description;
> char[] font_face;
> byte font_size;
> ushort flags;
> int colour_back;
> int colour_fore;
> byte charset;
> }
>
> class akStyle
> {
> akStyleEntry[] styles;
> }
>
> akStyle[char[]] LoadStyles()
> {
> akStyle[char[]] ret;
> ret["gosho"]=new akStyle;
> ret["tosho"]=new akStyle;
> ret["sosho"]=new akStyle;
> ret["mosho"]=new akStyle;
> for(int c=0;c<a;c++)
> {
> akStyleEntry tr;
> tr.font_face="font face"~std.string.toString(c)~"\0";
> ret["gosho"].styles~=tr;
> ret["tosho"].styles~=tr;
> ret["sosho"].styles~=tr;
> ret["mosho"].styles~=tr;
> }
> return ret;
> }
>
> class clas
> {
> static akStyle[char[]] test;
> }
>
> char[] tst;
>
> const int a=5;
> const int b=50;
> const int c=5;
>
> int main(char[][] argv)
> {
> clas.test=LoadStyles();
> for(int c=0;c<a;c++)
> {
> writef("%s\n",clas.test["gosho"].styles[c].font_face);
> writef("%s\n",clas.test["tosho"].styles[c].font_face);
> writef("%s\n",clas.test["sosho"].styles[c].font_face);
> writef("%s\n",clas.test["mosho"].styles[c].font_face);
> }
> char[c] m;
> for(int c=0;c<b;c++)
> {
> tst~=m;
> }
> for(int c=0;c<a;c++)
> {
> writefln(__LINE__);
> writef("%s\n",clas.test["gosho"].styles[c].font_face);
> writefln(__LINE__);
> writef("%s\n",clas.test["tosho"].styles[c].font_face);
> writefln(__LINE__);
> writef("%s\n",clas.test["sosho"].styles[c].font_face);
> writefln(__LINE__);
> writef("%s\n",clas.test["mosho"].styles[c].font_face);
> }
> return 1;
> }

I don't know... It is strange to me... I am not competent enough to deal  with garbage collectors...
I just hope Walter will fix it, because it makes my program not working...

Ben Hinkle wrote:
> more info. It works if the "byte" fields of the struct are removed. It looks like the GC is goofing up with detecting pointers somehow. It looks like if the size of the struct is an odd number the GC misses pointers.
> 
> 
> "bobef" <bobef@paintballforce.com> wrote in message news:d1rl7b$26bb$1@digitaldaemon.com...
> 
>>I wrote about this one yesterday in the D forum. I wasn't yet sure it wasn't my fault. Now I managed to isolate it in smaller code... Maybe it could be redused even more, I don't know I've copied stuctures from my source...
>>When const int c=5; everything goes as expected, but if c becomes larger number like 50000 the associative array clas.test is damaged. And in my program when I found that I didn't have any char[50000] at all. Just when memory usage increases the array gets damage. And I don't know maybe some other data is corrupted too...
>>
>>Here is the code:
>>
>>-------------------cut here------------------------------------------
>>
>>import std.stdio;
>>
>>struct akStyleEntry
>>{
>>byte number;
>>char[] description;
>>char[] font_face;
>>byte font_size;
>>ushort flags;
>>int colour_back;
>>int colour_fore;
>>byte charset;
>>}
>>
>>class akStyle
>>{
>>akStyleEntry[] styles;
>>}
>>
>>akStyle[char[]] LoadStyles()
>>{
>>akStyle[char[]] ret;
>>ret["gosho"]=new akStyle;
>>ret["tosho"]=new akStyle;
>>ret["sosho"]=new akStyle;
>>ret["mosho"]=new akStyle;
>>for(int c=0;c<a;c++)
>>{
>>akStyleEntry tr;
>>tr.font_face="font face"~std.string.toString(c)~"\0";
>>ret["gosho"].styles~=tr;
>>ret["tosho"].styles~=tr;
>>ret["sosho"].styles~=tr;
>>ret["mosho"].styles~=tr;
>>}
>>return ret;
>>}
>>
>>class clas
>>{
>>static akStyle[char[]] test;
>>}
>>
>>char[] tst;
>>
>>const int a=5;
>>const int b=50;
>>const int c=5;
>>
>>int main(char[][] argv)
>>{
>>clas.test=LoadStyles();
>>for(int c=0;c<a;c++)
>>{
>>writef("%s\n",clas.test["gosho"].styles[c].font_face);
>>writef("%s\n",clas.test["tosho"].styles[c].font_face);
>>writef("%s\n",clas.test["sosho"].styles[c].font_face);
>>writef("%s\n",clas.test["mosho"].styles[c].font_face);
>>}
>>char[c] m;
>>for(int c=0;c<b;c++)
>>{
>>tst~=m;
>>}
>>for(int c=0;c<a;c++)
>>{
>>writefln(__LINE__);
>>writef("%s\n",clas.test["gosho"].styles[c].font_face);
>>writefln(__LINE__);
>>writef("%s\n",clas.test["tosho"].styles[c].font_face);
>>writefln(__LINE__);
>>writef("%s\n",clas.test["sosho"].styles[c].font_face);
>>writefln(__LINE__);
>>writef("%s\n",clas.test["mosho"].styles[c].font_face);
>>}
>>return 1;
>>} 
> 
> 
> 

"bobef" <bobef@paintballforce.com> wrote in message news:d1s370$2lqf$1@digitaldaemon.com...
>I don't know... It is strange to me... I am not competent enough to deal with garbage collectors...
> I just hope Walter will fix it, because it makes my program not working...

Here are shorter reproduction steps:
import std.stdio;
import std.gc;
struct S
{
    byte number;
    char[] description;
    char[] font_face;
    byte font_size;
    ushort flags;
    int colour_back;
    int colour_fore;
    byte charset;
}
int main(char[][] argv)
{
    S[] x;
    writefln("size %d",S.sizeof);

    for (int i=0;i<3;i++) {
        S s;
        s.font_face="font face".dup;
        x ~= s;
    }

/* works fine
    S s;
    s.font_face="font face".dup;
    x ~= s;
    s.font_face="font face".dup;
    x ~= s;
    s.font_face="font face".dup;
    x ~= s;
    s.font_face="font face".dup;
    x ~= s;
*/
    fullCollect();
    writefln("%s",x[0].font_face);
    writefln("%s",x[1].font_face);
    return 0;
}

In article <d1rl7b$26bb$1@digitaldaemon.com>, bobef says...
>
>I wrote about this one yesterday in the D forum. I wasn't yet sure it
>wasn't my fault. Now I managed to isolate it in smaller code... Maybe it
>could be redused even more, I don't know I've copied stuctures from my
>source...
>When const int c=5; everything goes as expected, but if c becomes larger
>  number like 50000 the associative array clas.test is damaged. And in
>my program when I found that I didn't have any char[50000] at all. Just when memory usage increases the array gets damage. And I don't know maybe some other data is corrupted too...

This may be related to an implementation detail in the aa code.  Basically, the GC can be called while the aa is being modified (not surprisingly), but when the GC is called the aa pointer is pointing to garbage memory.  In the file aaA.d, replace these lines (at around line 190):

#    *aa = new pa[prime_list[0] + 1];
#    (*aa)[0] = cast(aaA *) cast(void*) new byte[aaA.sizeof];

with these:

#    aaA*[] tmp = new pa[prime_list[0] + 1];
#    tmp[0] = cast(aaA *) cast(void*) new byte[aaA.sizeof];
#    *aa = tmp;

I won't guarantee that this is the cause of your problems, but it's worth a shot.  FWIW, I've had this fixed in the Ares lib since the first release.


Sean

"Ben Hinkle" <bhinkle@mathworks.com> wrote in message news:d1s5gs$2odc$1@digitaldaemon.com...
> "bobef" <bobef@paintballforce.com> wrote in message news:d1s370$2lqf$1@digitaldaemon.com...
> >I don't know... It is strange to me... I am not competent enough to deal with garbage collectors...
> > I just hope Walter will fix it, because it makes my program not
working...
>
> Here are shorter reproduction steps:

Thanks. The problem is that the gc only looks for pointers that are aligned on a 4 byte boundary. What needs to happen is the size of S needs to be rounded up to the next alignment boundary.

In article <d1sm8b$946$1@digitaldaemon.com>, Walter says...
>
>Thanks. The problem is that the gc only looks for pointers that are aligned on a 4 byte boundary. What needs to happen is the size of S needs to be rounded up to the next alignment boundary.

Just to be clear, does this mean that DMD will generate properly aligned structs in a future release?  Either way, would:

align(4) struct S {}

be sufficient?


Sean

"Sean Kelly" <sean@f4.ca> wrote in message news:d1t1bu$ku6$1@digitaldaemon.com...
> In article <d1sm8b$946$1@digitaldaemon.com>, Walter says...
> >
> >Thanks. The problem is that the gc only looks for pointers that are
aligned
> >on a 4 byte boundary. What needs to happen is the size of S needs to be rounded up to the next alignment boundary.
>
> Just to be clear, does this mean that DMD will generate properly aligned
structs
> in a future release?  Either way, would:
>
> align(4) struct S {}
>
> be sufficient?

I don't understand the question.

In article <d1t21o$lhv$1@digitaldaemon.com>, Walter says...
>
>"Sean Kelly" <sean@f4.ca> wrote in message news:d1t1bu$ku6$1@digitaldaemon.com...
>> In article <d1sm8b$946$1@digitaldaemon.com>, Walter says...
>> >
>> >Thanks. The problem is that the gc only looks for pointers that are
>aligned
>> >on a 4 byte boundary. What needs to happen is the size of S needs to be rounded up to the next alignment boundary.
>>
>> Just to be clear, does this mean that DMD will generate properly aligned
>structs
>> in a future release?  Either way, would:
>>
>> align(4) struct S {}
>>
>> be sufficient?
>
>I don't understand the question.

From what you said the problem is that the struct S isn't aligned in a way that's compatible with the current GC implementation.  Since the sample code was pretty straightforward (ie. it contained no alignment instructions or odd pointer gymnastics) this sounds like a bug to me, but I wasn't clear from your response whether you considered it to be.  Basically, I was wondering whether Ben's sample will work correctly in a future D release or if the code will have to be modified.  Also, I was curious whether adding an alignment qualifier to the struct declaration would be sufficient to fix the problem, thus the "align(4) struct S" question.


Sean