December 09, 2014
On 12/09/2014 11:56 AM, Steven Schveighoffer wrote:

>> i checked attributes for GC block holding this array:
>>
>> ```
>> FINALIZE NO_SCAN NO_MOVE APPENDABLE NO_INTERIOR
>> ```
>>
>
> That does not sound right at all. No block should ever have both
> FINALIZE (reserved for objects only) and APPENDABLE (reserved for arrays
> only).

FINALIZE and APPENDABLE together sounds like "an array that holds class objects."

I think I get it as I write this: Do we mean that the array should always hold class references and the class objects should live on other blocks? If so, the memory block for the objects can be marked as FINALIZE? What block should be APPENDABLE?

Of course, this may be all in the documentation but I can't understand it. ;) Here is what is says for FINALIZE: "Finalize the data in this block on collect." (I will study that part a little more. :p)

  http://dlang.org/phobos/core_memory.html#.GC.BlkAttr.FINALIZE

Ali

December 09, 2014
On Tuesday, 9 December 2014 at 19:56:30 UTC, Steven Schveighoffer wrote:
> On 12/9/14 12:40 PM, Ruslan Mullakhmetov wrote:
>> On Tuesday, 9 December 2014 at 16:13:25 UTC, Dicebot wrote:
>> i checked attributes for GC block holding this array:
>>
>> FINALIZE NO_SCAN NO_MOVE APPENDABLE NO_INTERIOR
>>
> That does not sound right at all. No block should ever have both FINALIZE (reserved for objects only) and APPENDABLE (reserved for arrays only).
>
>> also i found that block size is quite small
>>
>> <pre>
>>                 array: [100A2FD00, 100A2F700, 100A33B80, 100A33500,
>> 100A3FE80, 100A3F980, 100A3F400, 100A72600, 100A7DF80, 100A7DA80,
>> 100A7D500]
>>         array ptr: 100A72580 root: 100A72580:128 attr: FINALIZE NO_SCAN
>> NO_MOVE APPENDABLE NO_INTERIOR
>>         [100985A00] keys: [1] as: 1 au: 100A2FD00
>>         [100985A00] keys: [1] as: 1 au: 100A2F700
>>         [100985A00] keys: [1] as: 1 au: 100A33B80
>> </pre>
>>
>> array holds 11 64bit pointers but it's block size is only 128 bytes < 11
>> * 64 = 704 bytes. what's wrong with this arithmetics?
>>
>
> I think there is something you are missing, or something is very corrupt. Can you show the code that prints this?
>
> -Steve

here the piece of code i used to output this value

http://pastebin.com/cQf9Nghp

StreamIndex is ubyte
AccessUnit is some class
December 09, 2014
On 12/9/14 3:18 PM, Ali Çehreli wrote:
> On 12/09/2014 11:56 AM, Steven Schveighoffer wrote:
>
>  >> i checked attributes for GC block holding this array:
>  >>
>  >> ```
>  >> FINALIZE NO_SCAN NO_MOVE APPENDABLE NO_INTERIOR
>  >> ```
>  >>
>  >
>  > That does not sound right at all. No block should ever have both
>  > FINALIZE (reserved for objects only) and APPENDABLE (reserved for arrays
>  > only).
>
> FINALIZE and APPENDABLE together sounds like "an array that holds class
> objects."
>
> I think I get it as I write this: Do we mean that the array should
> always hold class references and the class objects should live on other
> blocks? If so, the memory block for the objects can be marked as
> FINALIZE?

Yes, that's exactly right. A class is never allocated "inline" inside another object or an array.

> What block should be APPENDABLE?

The array of class references can be APPENDABLE.

> Of course, this may be all in the documentation but I can't understand
> it. ;) Here is what is says for FINALIZE: "Finalize the data in this
> block on collect." (I will study that part a little more. :p)
>
>    http://dlang.org/phobos/core_memory.html#.GC.BlkAttr.FINALIZE

In truth, the code expects the block then to have a ClassInfo pointer at the beginning of the block.

See here:

https://github.com/D-Programming-Language/druntime/blob/master/src/rt/lifetime.d#L1225

-Steve
December 09, 2014
On 12/9/14 3:24 PM, Ruslan Mullakhmetov wrote:
> On Tuesday, 9 December 2014 at 19:56:30 UTC, Steven Schveighoffer wrote:
>> On 12/9/14 12:40 PM, Ruslan Mullakhmetov wrote:
>>> On Tuesday, 9 December 2014 at 16:13:25 UTC, Dicebot wrote:
>>> i checked attributes for GC block holding this array:
>>>
>>> FINALIZE NO_SCAN NO_MOVE APPENDABLE NO_INTERIOR
>>>
>> That does not sound right at all. No block should ever have both
>> FINALIZE (reserved for objects only) and APPENDABLE (reserved for
>> arrays only).
>>
> here the piece of code i used to output this value
>
> http://pastebin.com/cQf9Nghp

I literally had to compile this for myself before I saw the error:

if(bi && k) => if(bi & k)

Though that doesn't explain all the issues you reported. I'm curious what the output is after that though...

-Steve
December 09, 2014
On 12/9/14 2:56 PM, Steven Schveighoffer wrote:
> On 12/9/14 12:40 PM, Ruslan Mullakhmetov wrote:

>> array holds 11 64bit pointers but it's block size is only 128 bytes < 11
>> * 64 = 704 bytes. what's wrong with this arithmetics?

Hah, just realized what's wrong. It's not 64 *bytes* per pointer, it's 64 *bits*. So 8 bytes.

11 * 8 == 88.

Starting to sound more and more normal...

-Steve
December 10, 2014
On Tue, 09 Dec 2014 17:18:44 +0000
Ruslan Mullakhmetov via Digitalmars-d-learn
<digitalmars-d-learn@puremagic.com> wrote:

> thanks, I got it: either C++ or D dtors are minefield =)
and in D this is filed *made* of mines. ;-)

> but i still have no clue how to overcome GC =(
why do you want to fight with GC? most of the time GC is your friend.

are you trying to have predictable finalization? you don't have to fight with GC in this case too, there are alot of other methods. wrapper structs, `scoped!`, `RefCounted` and so on. you can have weak references too (this is a hack, but it should work until we got compacting (or precise?) GC ;-).


December 10, 2014
On Wednesday, 10 December 2014 at 02:43:19 UTC, ketmar via Digitalmars-d-learn wrote:
> On Tue, 09 Dec 2014 17:18:44 +0000
> Ruslan Mullakhmetov via Digitalmars-d-learn
> <digitalmars-d-learn@puremagic.com> wrote:
>> but i still have no clue how to overcome GC =(
> why do you want to fight with GC? most of the time GC is your friend.
>

see the topic: i got corruption when dereferencing object.
December 10, 2014
On Wed, 10 Dec 2014 08:32:12 +0000
Ruslan Mullakhmetov via Digitalmars-d-learn
<digitalmars-d-learn@puremagic.com> wrote:

> On Wednesday, 10 December 2014 at 02:43:19 UTC, ketmar via Digitalmars-d-learn wrote:
> > On Tue, 09 Dec 2014 17:18:44 +0000
> > Ruslan Mullakhmetov via Digitalmars-d-learn
> > <digitalmars-d-learn@puremagic.com> wrote:
> >> but i still have no clue how to overcome GC =(
> > why do you want to fight with GC? most of the time GC is your friend.
> >
> 
> see the topic: i got corruption when dereferencing object.
that is easily fixable: just stop dereferencing it! ;-)


December 10, 2014
On Tuesday, 9 December 2014 at 21:38:57 UTC, Steven Schveighoffer wrote:
> On 12/9/14 2:56 PM, Steven Schveighoffer wrote:
>> On 12/9/14 12:40 PM, Ruslan Mullakhmetov wrote:
>
>>> array holds 11 64bit pointers but it's block size is only 128 bytes < 11
>>> * 64 = 704 bytes. what's wrong with this arithmetics?
>
> Hah, just realized what's wrong. It's not 64 *bytes* per pointer, it's 64 *bits*. So 8 bytes.
>
> 11 * 8 == 88.
>
> Starting to sound more and more normal...
>
> -Steve


yes. that was the mistake. also after fixing bug in Blk Attributes printing i got more reasonable attrs

for object blk: FINALIZE
for array of objects blk: NO_SCAN APPENDABLE

this is sound good except for NO_SCAN.


I did simple test file in which allocate array of Foo objects (http://dpaste.dzfl.pl/89ab00a897f6)

there i see blk attrs only APPENDABLE without NO_SCAN.

as far as i understand GC will not scan this array for references and those if the only reference to object is stored in this array will not see it, those assume this object as **not** referenced and collects it, am i right?

the other question why this happens... try to debug more.
December 10, 2014
On Wednesday, 10 December 2014 at 08:46:12 UTC, Ruslan Mullakhmetov wrote:
> yes. that was the mistake. also after fixing bug in Blk Attributes printing i got more reasonable attrs
>
> for object blk: FINALIZE
> for array of objects blk: NO_SCAN APPENDABLE
>
> this is sound good except for NO_SCAN.
>
> ...
> the other question why this happens... try to debug more.

I've done more dubugging.

what i've found:

initially array blk has only attrs APPENDABLE, but after some time this blk is shrinked and reallocated (moved) and then NO_SCAN attr appears.


here the output of my extended logs:

--------
before tag: 1 len: 2 ptr: 103DD9058 root: 103DD8000:8192 attr: APPENDABLE
after tag: 1 len: 3 ptr: 103A21DD0 root: 103A21DC0:64 attr: NO_SCAN APPENDABLE
--------

this is produced by the following code

http://dpaste.dzfl.pl/0c6dc16270a1

so in a nutshell after appending to array via ~= operator blk attrs changed from APPENDABLE to NO_SCAN APPENDABLE which cause the problem.

why and how this happens? can anybody explain it to me?