The issue is that the GC does not do any kind of initialization of allocated length, which is a key thing for safe appending to work.  That is done strictly in the array allocation routine.  So appending to anything you allocated with malloc or calloc, regardless of typeinfo, would be using an uninitialized allocated length, and would be bad.

-Steve


----- Original Message ----
> From: Sean Kelly <sean at invisibleduck.org>
> To: Discuss the phobos library for D <phobos at puremagic.com>
> Sent: Wed, July 14, 2010 11:44:20 AM
> Subject: Re: [phobos] custom BlkAttr flags
> 
> But in both cases, TypeInfo could be supplied (either at the point of
>allocation  or set later).  A bit would be more efficient however.  Regarding how  this would all work, I was thinking of redefining calloc as:
> 
> void*  calloc(TypeInfo ti, size_t count);
> 
> So the TypeInfo would replace the  'size' argument from C.  This makes calloc a
>lot more useful than just  "malloc with bzero" as it is now.
> 
> On Jul 14, 2010, at 3:26 AM, Steve  Schveighoffer wrote:
> 
> > I added the fix for the comma, sorry for  that.  All the other places had an

> > ALL_BITS enum member, so the  NO_MOVE line that I copied and changed had a comma.  I thought I  compiled before I checked in, but I guess I did not.
> > 
> > Regarding  the bit being flipped, yes I want the default meaning to be "no append"  because anyone can allocate memory via GC.malloc for any purpose,
>and
>
> >  GC.malloc does not initialize the "allocated length" field inside the block,
>so
>
> > it is not appendable.  It does not mean you cannot append to the  block, it
>just
>
> > means the first append will reallocate into an  appendable block.
> > 
> > Regarding using the typeinfo, that doesn't  work.  Arrays can easily be
>created
>
> > out of blocks that weren't  allocated via the array creation routines.  Think
>of
>
> > void[], and  GC.malloc above.
> > 
> > -Steve
> > 
> > 
> > 
> > 
> > ----- Original Message ----
> >> From: Sean Kelly <sean at invisibleduck.org>
> >>  To: Discuss the phobos library for D <phobos at puremagic.com>
> >>  Sent: Wed, July 14, 2010 12:25:09 AM
> >> Subject: Re: [phobos] custom  BlkAttr flags
> >> 
> >> On Jul 13, 2010, at 8:09 AM, Steve  Schveighoffer wrote:
> >>> 
> >>> What I  propose to  fix this is to allocate one of the block attribute
>flags
>
> >> to
> >> 
> >>> designate that a block is appendable.  The  odd part about it, is that the
>
> >> GC is
> >> 
> >>> not aware of appending, so it is somewhat of a  "user-defined"  flag.
> >>> 
> >>> My question is, how  should we declare the flag?  I  currently only defined
>it
>
> >>  in
> >> 
> >>> lifetime.d, outside the gc, but I'm   thinking it should be defined in the
>GC
>
> >> as
> >> 
> >>> well.  Also, should I  allocate the next bit, or one  of the higher order
> >> bits?
> >>> 
> >>>  I'm  going to check in my code that defines it in lifetime.d only, and
> >> depending
> >> 
> >>> on the responses to this,  I'll add it to the GC definition  too.
> >> 
> >> Your  checkin doesn't compile.  I added the requisite comma for  now to fix

> >> this.  That aside, assuming this design were to be kept I  think  the
>meaning of
>
> >> the bit has to be flipped.  The bit  defaults to unset, so  that should be
>the
>
> >> default  meaning.  You really don't want all blocks to be  appendable by
>default,
>
> >> do you?  More generally, if we start holding   TypeInfo references in the GC
>then
>
> >> we should be able to determine  appendabiility  based on whether the
>TypeInfo is
>
> >> an array  type.  This may not make the next  release, but I'd like to
>address it
>
> >> once I've polished std.concurrency a bit  more.
> >>  _______________________________________________
> >> phobos mailing   list
> >> phobos at puremagic.com
> >>  http://lists.puremagic.com/mailman/listinfo/phobos
> >> 
> > 
> > 
> > 
> > _______________________________________________
> > phobos  mailing list
> > phobos at puremagic.com
> > http://lists.puremagic.com/mailman/listinfo/phobos
> 
> _______________________________________________
> phobos  mailing list
> phobos at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/phobos
> 

Erm... I a potential issue is that this is moving some logic that could be considered runtime stuff into the GC.  Should the GC simply accept a pointer to a custom finalization routine instead?

For Andrei's suggestion, I think it's a good one, but I don't think it would work for arrays of structs.

On Jul 14, 2010, at 8:44 AM, Sean Kelly wrote:

> But in both cases, TypeInfo could be supplied (either at the point of allocation or set later).  A bit would be more efficient however.  Regarding how this would all work, I was thinking of redefining calloc as:
> 
> void* calloc(TypeInfo ti, size_t count);
> 
> So the TypeInfo would replace the 'size' argument from C.  This makes calloc a lot more useful than just "malloc with bzero" as it is now.
> 
> On Jul 14, 2010, at 3:26 AM, Steve Schveighoffer wrote:
> 
>> I added the fix for the comma, sorry for that.  All the other places had an ALL_BITS enum member, so the NO_MOVE line that I copied and changed had a comma.  I thought I compiled before I checked in, but I guess I did not.
>> 
>> Regarding the bit being flipped, yes I want the default meaning to be "no append" because anyone can allocate memory via GC.malloc for any purpose, and GC.malloc does not initialize the "allocated length" field inside the block, so it is not appendable.  It does not mean you cannot append to the block, it just means the first append will reallocate into an appendable block.
>> 
>> Regarding using the typeinfo, that doesn't work.  Arrays can easily be created out of blocks that weren't allocated via the array creation routines.  Think of void[], and GC.malloc above.
>> 
>> -Steve
>> 
>> 
>> 
>> 
>> ----- Original Message ----
>>> From: Sean Kelly <sean at invisibleduck.org>
>>> To: Discuss the phobos library for D <phobos at puremagic.com>
>>> Sent: Wed, July 14, 2010 12:25:09 AM
>>> Subject: Re: [phobos] custom BlkAttr flags
>>> 
>>> On Jul 13, 2010, at 8:09 AM, Steve Schveighoffer wrote:
>>>> 
>>>> What I  propose to fix this is to allocate one of the block attribute flags
>>> to
>>> 
>>>> designate that a block is appendable.  The odd part about it, is that the
>>> GC is
>>> 
>>>> not aware of appending, so it is somewhat of a "user-defined"  flag.
>>>> 
>>>> My question is, how should we declare the flag?  I  currently only defined it
>>> in
>>> 
>>>> lifetime.d, outside the gc, but I'm  thinking it should be defined in the GC
>>> as
>>> 
>>>> well.  Also, should I  allocate the next bit, or one of the higher order
>>> bits?
>>>> 
>>>> I'm  going to check in my code that defines it in lifetime.d only, and
>>> depending
>>> 
>>>> on the responses to this, I'll add it to the GC definition  too.
>>> 
>>> Your checkin doesn't compile.  I added the requisite comma for  now to fix
>>> this.  That aside, assuming this design were to be kept I think  the meaning of
>>> the bit has to be flipped.  The bit defaults to unset, so  that should be the
>>> default meaning.  You really don't want all blocks to be  appendable by default,
>>> do you?  More generally, if we start holding  TypeInfo references in the GC then
>>> we should be able to determine appendabiility  based on whether the TypeInfo is
>>> an array type.  This may not make the next  release, but I'd like to address it
>>> once I've polished std.concurrency a bit  more.
>>> _______________________________________________
>>> phobos mailing  list
>>> phobos at puremagic.com
>>> http://lists.puremagic.com/mailman/listinfo/phobos
>>> 
>> 
>> 
>> 
>> _______________________________________________
>> phobos mailing list
>> phobos at puremagic.com
>> http://lists.puremagic.com/mailman/listinfo/phobos
> 
> _______________________________________________
> phobos mailing list
> phobos at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/phobos

Changed my mind.  The array append doesn't really belong in the GC either, so it would simply be continuing a trend :-)

On Jul 14, 2010, at 8:51 AM, Sean Kelly wrote:

> Erm... I a potential issue is that this is moving some logic that could be considered runtime stuff into the GC.  Should the GC simply accept a pointer to a custom finalization routine instead?
> 
> For Andrei's suggestion, I think it's a good one, but I don't think it would work for arrays of structs.
> 
> On Jul 14, 2010, at 8:44 AM, Sean Kelly wrote:
> 
>> But in both cases, TypeInfo could be supplied (either at the point of allocation or set later).  A bit would be more efficient however.  Regarding how this would all work, I was thinking of redefining calloc as:
>> 
>> void* calloc(TypeInfo ti, size_t count);
>> 
>> So the TypeInfo would replace the 'size' argument from C.  This makes calloc a lot more useful than just "malloc with bzero" as it is now.
>> 
>> On Jul 14, 2010, at 3:26 AM, Steve Schveighoffer wrote:
>> 
>>> I added the fix for the comma, sorry for that.  All the other places had an ALL_BITS enum member, so the NO_MOVE line that I copied and changed had a comma.  I thought I compiled before I checked in, but I guess I did not.
>>> 
>>> Regarding the bit being flipped, yes I want the default meaning to be "no append" because anyone can allocate memory via GC.malloc for any purpose, and GC.malloc does not initialize the "allocated length" field inside the block, so it is not appendable.  It does not mean you cannot append to the block, it just means the first append will reallocate into an appendable block.
>>> 
>>> Regarding using the typeinfo, that doesn't work.  Arrays can easily be created out of blocks that weren't allocated via the array creation routines.  Think of void[], and GC.malloc above.
>>> 
>>> -Steve
>>> 
>>> 
>>> 
>>> 
>>> ----- Original Message ----
>>>> From: Sean Kelly <sean at invisibleduck.org>
>>>> To: Discuss the phobos library for D <phobos at puremagic.com>
>>>> Sent: Wed, July 14, 2010 12:25:09 AM
>>>> Subject: Re: [phobos] custom BlkAttr flags
>>>> 
>>>> On Jul 13, 2010, at 8:09 AM, Steve Schveighoffer wrote:
>>>>> 
>>>>> What I  propose to fix this is to allocate one of the block attribute flags
>>>> to
>>>> 
>>>>> designate that a block is appendable.  The odd part about it, is that the
>>>> GC is
>>>> 
>>>>> not aware of appending, so it is somewhat of a "user-defined"  flag.
>>>>> 
>>>>> My question is, how should we declare the flag?  I  currently only defined it
>>>> in
>>>> 
>>>>> lifetime.d, outside the gc, but I'm  thinking it should be defined in the GC
>>>> as
>>>> 
>>>>> well.  Also, should I  allocate the next bit, or one of the higher order
>>>> bits?
>>>>> 
>>>>> I'm  going to check in my code that defines it in lifetime.d only, and
>>>> depending
>>>> 
>>>>> on the responses to this, I'll add it to the GC definition  too.
>>>> 
>>>> Your checkin doesn't compile.  I added the requisite comma for  now to fix
>>>> this.  That aside, assuming this design were to be kept I think  the meaning of
>>>> the bit has to be flipped.  The bit defaults to unset, so  that should be the
>>>> default meaning.  You really don't want all blocks to be  appendable by default,
>>>> do you?  More generally, if we start holding  TypeInfo references in the GC then
>>>> we should be able to determine appendabiility  based on whether the TypeInfo is
>>>> an array type.  This may not make the next  release, but I'd like to address it
>>>> once I've polished std.concurrency a bit  more.
>>>> _______________________________________________
>>>> phobos mailing  list
>>>> phobos at puremagic.com
>>>> http://lists.puremagic.com/mailman/listinfo/phobos
>>>> 
>>> 
>>> 
>>> 
>>> _______________________________________________
>>> phobos mailing list
>>> phobos at puremagic.com
>>> http://lists.puremagic.com/mailman/listinfo/phobos
>> 
>> _______________________________________________
>> phobos mailing list
>> phobos at puremagic.com
>> http://lists.puremagic.com/mailman/listinfo/phobos
> 
> _______________________________________________
> phobos mailing list
> phobos at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/phobos