Thread overview
GC and arrays (D and OS allocated)
Mar 09, 2003
Mike Wynn
Mar 24, 2003
Walter
Mar 24, 2003
Mike Wynn
March 09, 2003
Walter,

if I have the following code ...

 void * ptr;
  ptr = MapViewOfFile( mh, FILE_MAP_READ, 0, 0, 0 );
  return (cast(byte*)ptr)[0..length];  // length is the whole file length

do I need to inform the GC about the memory region I've mapped into my
memory space ?
my consern is that the gc will try to delete the array that the returned
slice points to.

I'm a little unsure how the GC and D interact with respect to arrays

T[] foo = new T[N];

I assume this is;
struct _array : _gc_obj { int size; T first }
struct _slice { int length; T * head; }
allocate some space (at least T.sizeof * N bytes) and tell the GC its an
array, either of values or of objects.
_array _ar = (_array)malloc( enough_data );
_ar.size = enough_data;
tell the gc _ar is an array of type T
_slice foo = { length: N, head : &(_ar.first) }

and foo ~= item;
is

// again using _slice to be the interal reprosentation of foo
if  _slice.head is null { allocate as before; _slice.head[0] = item;
_slice.length = 1; }
else {
    if _slice.head is not the first item in a gc aware array { shallow copy
(into bigger array) and append item }
    else {
        _array _ar;
        get _ar that is the base array of _slice;
        if _ar.size < _slice.length + 1 { shallow copy etc as before }
        else {
            _slice.head[_slice.length] = item;
            _slice.length++;
        }
    }
}

this is based on my observations of the postfixed code.
I believe that the structs should be
struct _array : _gc_obj { int size; _slice * owner; T first }
struct _slice { int length; T * head; }

// again using _slice to be the interal reprosentation of foo
if  _slice.head is null { allocate as before; _slice.head[0] = item;
_slice.length = 1; }
else {
    if _slice.head is not the first item in a gc aware array { shallow copy
(into bigger array) and append item }
    else {
        _array _ar;
        get _ar that is the base array of _slice;
        if _ar.size < _slice.length + 1 { shallow copy etc as before }
        else {
            if _ar.owner != _slice { shallow copy }
            else {
            _slice.head[_slice.length] = item;
            _slice.length++;
            }
        }
    }
}

import c.stdio;

int[] nums = [1,2,3,4];

void print_ar( char[] str, int[] ar ){
 printf("%.*s [", str );
 for( int i = 0; i < ar.length; i++ ){
  if ( i!=0 ) printf( ", " );
  printf( "%i", ar[i] );
 }
 printf( "]\n" );
}

int main( char[][] argv ){
 int[] foo = new int[4];
 int[] tmp;
 foo[0..4] = nums[0..4];
 print_ar( "original foo", foo );
 tmp = foo[1..2];
 print_ar( "foo[1..2] slice tmp", tmp );
 print_ar( "from foo", foo );
 printf( "-appending to tmp" );
 tmp ~= 9;
 print_ar( "foo slice tmp", tmp );
 print_ar( "from foo", foo );
 printf( " trying from start" );
 tmp = foo[0..2];
 print_ar( "foo[0..2] slice tmp", tmp );
 print_ar( "from foo", foo );
 printf( "-appending to tmp" );
 tmp ~= 9;
 print_ar( "foo slice tmp", tmp );
 print_ar( "from foo", foo );
 printf( " trying from start" );
 return 0;
}

outputs

original foo [1, 2, 3, 4]
foo[1..2] slice tmp [2]
from foo [1, 2, 3, 4]
-appending to tmpfoo slice tmp [2, 9]
from foo [1, 2, 3, 4]
 trying from startfoo[0..2] slice tmp [1, 2]
from foo [1, 2, 3, 4]
-appending to tmpfoo slice tmp [1, 2, 9]
from foo [1, 2, 9, 4]
 trying from start

Digital Mars D Compiler Beta v0.59




March 24, 2003
"Mike Wynn" <mike.wynn@l8night.co.uk> wrote in message news:b4e2eq$ttj$1@digitaldaemon.com...
> if I have the following code ...
>
>  void * ptr;
>   ptr = MapViewOfFile( mh, FILE_MAP_READ, 0, 0, 0 );
>   return (cast(byte*)ptr)[0..length];  // length is the whole file length
>
> do I need to inform the GC about the memory region I've mapped into my memory space ?

Yes, *if* you are going to store into it any references to any GC'd objects.

> my consern is that the gc will try to delete the array that the returned slice points to.

No, it won't, as it wasn't allocated by the GC.

> I'm a little unsure how the GC and D interact with respect to arrays
>
> T[] foo = new T[N];
>
> I assume this is;
> struct _array : _gc_obj { int size; T first }
> struct _slice { int length; T * head; }
> allocate some space (at least T.sizeof * N bytes) and tell the GC its an
> array, either of values or of objects.
> _array _ar = (_array)malloc( enough_data );
> _ar.size = enough_data;
> tell the gc _ar is an array of type T
> _slice foo = { length: N, head : &(_ar.first) }
>
> and foo ~= item;
> is
>
> // again using _slice to be the interal reprosentation of foo
> if  _slice.head is null { allocate as before; _slice.head[0] = item;
> _slice.length = 1; }
> else {
>     if _slice.head is not the first item in a gc aware array { shallow
copy
> (into bigger array) and append item }
>     else {
>         _array _ar;
>         get _ar that is the base array of _slice;
>         if _ar.size < _slice.length + 1 { shallow copy etc as before }
>         else {
>             _slice.head[_slice.length] = item;
>             _slice.length++;
>         }
>     }
> }
>
> this is based on my observations of the postfixed code.
> I believe that the structs should be
> struct _array : _gc_obj { int size; _slice * owner; T first }
> struct _slice { int length; T * head; }
>
> // again using _slice to be the interal reprosentation of foo
> if  _slice.head is null { allocate as before; _slice.head[0] = item;
> _slice.length = 1; }
> else {
>     if _slice.head is not the first item in a gc aware array { shallow
copy
> (into bigger array) and append item }
>     else {
>         _array _ar;
>         get _ar that is the base array of _slice;
>         if _ar.size < _slice.length + 1 { shallow copy etc as before }
>         else {
>             if _ar.owner != _slice { shallow copy }
>             else {
>             _slice.head[_slice.length] = item;
>             _slice.length++;
>             }
>         }
>     }
> }
>
> import c.stdio;
>
> int[] nums = [1,2,3,4];
>
> void print_ar( char[] str, int[] ar ){
>  printf("%.*s [", str );
>  for( int i = 0; i < ar.length; i++ ){
>   if ( i!=0 ) printf( ", " );
>   printf( "%i", ar[i] );
>  }
>  printf( "]\n" );
> }
>
> int main( char[][] argv ){
>  int[] foo = new int[4];
>  int[] tmp;
>  foo[0..4] = nums[0..4];
>  print_ar( "original foo", foo );
>  tmp = foo[1..2];
>  print_ar( "foo[1..2] slice tmp", tmp );
>  print_ar( "from foo", foo );
>  printf( "-appending to tmp" );
>  tmp ~= 9;
>  print_ar( "foo slice tmp", tmp );
>  print_ar( "from foo", foo );
>  printf( " trying from start" );
>  tmp = foo[0..2];
>  print_ar( "foo[0..2] slice tmp", tmp );
>  print_ar( "from foo", foo );
>  printf( "-appending to tmp" );
>  tmp ~= 9;
>  print_ar( "foo slice tmp", tmp );
>  print_ar( "from foo", foo );
>  printf( " trying from start" );
>  return 0;
> }
>
> outputs
>
> original foo [1, 2, 3, 4]
> foo[1..2] slice tmp [2]
> from foo [1, 2, 3, 4]
> -appending to tmpfoo slice tmp [2, 9]
> from foo [1, 2, 3, 4]
>  trying from startfoo[0..2] slice tmp [1, 2]
> from foo [1, 2, 3, 4]
> -appending to tmpfoo slice tmp [1, 2, 9]
> from foo [1, 2, 9, 4]
>  trying from start
>
> Digital Mars D Compiler Beta v0.59

Slicing does NOT allocate new data, it just is a reference to existing data. Resizing an array (appending to it counts as resizing) may or may not result in a copy being made, depending on if the gc determines if there is enough room in the chunk the array is allocated in.


March 24, 2003
> > import c.stdio;
> >
> > int[] nums = [1,2,3,4];
> >
> > void print_ar( char[] str, int[] ar ){
> >  printf("%.*s [", str );
> >  for( int i = 0; i < ar.length; i++ ){
> >   if ( i!=0 ) printf( ", " );
> >   printf( "%i", ar[i] );
> >  }
> >  printf( "]\n" );
> > }
> >
> > int main( char[][] argv ){
> >  int[] foo = new int[4];
> >  int[] tmp;
> >  foo[0..4] = nums[0..4];
> >  print_ar( "original foo", foo );
> >  tmp = foo[1..2];
> >  print_ar( "foo[1..2] slice tmp", tmp );
> >  print_ar( "from foo", foo );
> >  printf( "-appending to tmp" );
> >  tmp ~= 9;
> >  print_ar( "foo slice tmp", tmp );
> >  print_ar( "from foo", foo );
> >  printf( " trying from start" );
> >  tmp = foo[0..2];
> >  print_ar( "foo[0..2] slice tmp", tmp );
> >  print_ar( "from foo", foo );
> >  printf( "-appending to tmp" );
> >  tmp ~= 9;
> >  print_ar( "foo slice tmp", tmp );
> >  print_ar( "from foo", foo );
> >  printf( " trying from start" );
> >  return 0;
> > }
> >
> > outputs
> >
> > original foo [1, 2, 3, 4]
> > foo[1..2] slice tmp [2]
> > from foo [1, 2, 3, 4]
> > -appending to tmpfoo slice tmp [2, 9]
> > from foo [1, 2, 3, 4]
> >  trying from startfoo[0..2] slice tmp [1, 2]
> > from foo [1, 2, 3, 4]
> > -appending to tmpfoo slice tmp [1, 2, 9]
> > from foo [1, 2, 9, 4]
> >  trying from start
> >
> > Digital Mars D Compiler Beta v0.59
>
> Slicing does NOT allocate new data, it just is a reference to existing
data.
> Resizing an array (appending to it counts as resizing) may or may not
result
> in a copy being made, depending on if the gc determines if there is enough room in the chunk the array is allocated in.
>
the problem is inconsistancy, if your slice is cut from the head then it can
be expanded in situ
if not i.e. ar[1..3]; then ~= creates a copy.
meaning to return an append safe array slice you have to dup if the array
slice is [0..n] but not if [n..m] where (n!=0)
this causes problems trying to do efficient lazy copy on append.
you either have to dup b4 if sliced [0..n], or always dup b4 append incase
you are passed 0..n slice

to me this is another case of implementation v semantics;
to me   `b = a[5..7]` is a slice of the array and changing b[0] would change
a[5]'
but b ~= foo; I would not expect to change a[7] ever;

I thing dynamic arrays should be a separate class, a[5..7] is a slice I don't think you should be able to append to a slice.