== Quote from Denis Koroskin (2korden@gmail.com)'s article
> On Fri, 20 Nov 2009 19:24:05 +0300, dsimcha <dsimcha@yahoo.com> wrote:
> > == Quote from Travis Boucher (boucher.travis@gmail.com)'s article
> >> dsimcha wrote:
> >> > == Quote from Denis Koroskin (2korden@gmail.com)'s article
> >> >> On Fri, 20 Nov 2009 17:28:07 +0300, dsimcha <dsimcha@yahoo.com>
> >> wrote:
> >> >>> == Quote from Travis Boucher (boucher.travis@gmail.com)'s article
> >> >>>> dsimcha wrote:
> >> >>>>> == Quote from Travis Boucher (boucher.travis@gmail.com)'s article
> >> >>>>>> Sean Kelly wrote:
> >> >>>>>>  Its harder
> >> >>>>>> to create a memory leak in D then it is to prevent one in C.
> >> >>>>> void doStuff() {
> >> >>>>>     uint[] foo = new uint[100_000_000];
> >> >>>>> }
> >> >>>>>
> >> >>>>> void main() {
> >> >>>>>     while(true) {
> >> >>>>>         doStuff();
> >> >>>>>     }
> >> >>>>> }
> >> >>>>>
> >> >>>> Hmm, that seems like that should be an implementation bug.
> >> Shouldn't
> >> >>>> foo be marked for GC once it scope?  (I have never used new on a
> >> >>>> primitive type, so I don't know)
> >> >>> It's conservative GC.  D's GC, along with the Hans Boehm GC and
> >> probably
> >> >>> most GCs
> >> >>> for close to the metal languages, can't perfectly identify what's a
> >> >>> pointer and
> >> >>> what's not.  Therefore, for sufficiently large allocations there's
> >> a high
> >> >>> probability that some bit pattern that looks like a pointer but
> >> isn't
> >> >>> one will
> >> >>> keep the allocation alive long after there are no "real" references
> >> to
> >> >>> it left.
> >> >> Aren't uint array allocations have hasPointers flag set off? I always thought they aren't scanned for pointers (unlike, say, void[]).
> >> >
> >> > Right, but they can still be the target of false pointers.  In this
> >> case, false
> >> > pointers keep each instance of foo[] alive, leading to severe memory
> >> leaks.
> >> But the issue is more of a GC implementation issue then a language
> >> issue, correct?
> >
> > Yes.
> >
> >> Or is this an issue of all lower level language garbage collectors?
> >
> > Kinda sorta.  It's possible, but not easy, to implement fully precise GC
> > (except
> > for the extreme corner case of unions of reference and non-reference
> > types) in a
> > close to the metal, statically compiled language.
> Unions could be deprecated in favor of tagged unions (see an example in Cyclone http://cyclone.thelanguage.org/wiki/Tagged%20Unions). Would that help?

It would be negligible.  The idea is that unions of reference and non-reference types are such a corner case that they could be handled conservatively as a special case, and then it's possible, at least in principle, to deal with the other 99.99999% of cases precisely and being conservative in 0.00001% of cases is really of no practical significance.

Keep in mind that we would need to have the ability to pin and scan conservatively anyhow, since a systems language must allow allocation of untyped blocks of memory.

I guess what I should have said is that the SafeD subset can be made 100% precise and D as a whole can be made about 99+% precise.

Denis Koroskin wrote:
> On Fri, 20 Nov 2009 19:24:05 +0300, dsimcha <dsimcha@yahoo.com> wrote:
> 
>> == Quote from Travis Boucher (boucher.travis@gmail.com)'s article
>>> dsimcha wrote:
>>> > == Quote from Denis Koroskin (2korden@gmail.com)'s article
>>> >> On Fri, 20 Nov 2009 17:28:07 +0300, dsimcha <dsimcha@yahoo.com> 
>>> wrote:
>>> >>> == Quote from Travis Boucher (boucher.travis@gmail.com)'s article
>>> >>>> dsimcha wrote:
>>> >>>>> == Quote from Travis Boucher (boucher.travis@gmail.com)'s article
>>> >>>>>> Sean Kelly wrote:
>>> >>>>>>  Its harder
>>> >>>>>> to create a memory leak in D then it is to prevent one in C.
>>> >>>>> void doStuff() {
>>> >>>>>     uint[] foo = new uint[100_000_000];
>>> >>>>> }
>>> >>>>>
>>> >>>>> void main() {
>>> >>>>>     while(true) {
>>> >>>>>         doStuff();
>>> >>>>>     }
>>> >>>>> }
>>> >>>>>
>>> >>>> Hmm, that seems like that should be an implementation bug.  
>>> Shouldn't
>>> >>>> foo be marked for GC once it scope?  (I have never used new on a
>>> >>>> primitive type, so I don't know)
>>> >>> It's conservative GC.  D's GC, along with the Hans Boehm GC and 
>>> probably
>>> >>> most GCs
>>> >>> for close to the metal languages, can't perfectly identify what's a
>>> >>> pointer and
>>> >>> what's not.  Therefore, for sufficiently large allocations 
>>> there's a high
>>> >>> probability that some bit pattern that looks like a pointer but 
>>> isn't
>>> >>> one will
>>> >>> keep the allocation alive long after there are no "real" 
>>> references to
>>> >>> it left.
>>> >> Aren't uint array allocations have hasPointers flag set off? I always
>>> >> thought they aren't scanned for pointers (unlike, say, void[]).
>>> >
>>> > Right, but they can still be the target of false pointers.  In this 
>>> case, false
>>> > pointers keep each instance of foo[] alive, leading to severe 
>>> memory leaks.
>>> But the issue is more of a GC implementation issue then a language
>>> issue, correct?
>>
>> Yes.
>>
>>> Or is this an issue of all lower level language garbage
>>> collectors?
>>
>> Kinda sorta.  It's possible, but not easy, to implement fully precise GC (except
>> for the extreme corner case of unions of reference and non-reference types) in a
>> close to the metal, statically compiled language.
> 
> Unions could be deprecated in favor of tagged unions (see an example in Cyclone http://cyclone.thelanguage.org/wiki/Tagged%20Unions). Would that help?

Probably not since the bit pattern of int i could still match a valid pointer.

Foo.i = cast(int)&Foo;   // for bad practice ugliness

or Foo.i = (some expression that happens to equal &Foo)


Adding extra information to a union could also have the bad side effect of killing performance as writes would include an extra write, and additional memory would be required (which would cause another set of issues on how to handle alignment).

dsimcha wrote:
> == Quote from Denis Koroskin (2korden@gmail.com)'s article
> It would be negligible.  The idea is that unions of reference and non-reference
> types are such a corner case that they could be handled conservatively as a
> special case, and then it's possible, at least in principle, to deal with the
> other 99.99999% of cases precisely and being conservative in 0.00001% of cases is
> really of no practical significance.

Yes, but a moving GC needs to be 100% precise, not 99.99999%.


-- 
Rainer Deyke - rainerd@eldwood.com

== Quote from Rainer Deyke (rainerd@eldwood.com)'s article
> dsimcha wrote:
> > == Quote from Denis Koroskin (2korden@gmail.com)'s article
> > It would be negligible.  The idea is that unions of reference and non-reference
> > types are such a corner case that they could be handled conservatively as a
> > special case, and then it's possible, at least in principle, to deal with the
> > other 99.99999% of cases precisely and being conservative in 0.00001% of cases is
> > really of no practical significance.
> Yes, but a moving GC needs to be 100% precise, not 99.99999%.

Not if you allow pinning, which we'd need anyhow for untyped, conservatively scanned memory blocks.

dsimcha, el 20 de noviembre a las 16:24 me escribiste:
> > > Right, but they can still be the target of false pointers.  In this case, false pointers keep each instance of foo[] alive, leading to severe memory leaks.
> > But the issue is more of a GC implementation issue then a language issue, correct?
> 
> Yes.
> 
> > Or is this an issue of all lower level language garbage collectors?
> 
> Kinda sorta.  It's possible, but not easy, to implement fully precise GC (except for the extreme corner case of unions of reference and non-reference types) in a close to the metal, statically compiled language.

I don't think so if you want to be able to link to C code, unless I'm missing something...

-- 
Leandro Lucarella (AKA luca)                     http://llucax.com.ar/
----------------------------------------------------------------------
GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145  104C 949E BFB6 5F5A 8D05)
----------------------------------------------------------------------
"Lidiar" no es lo mismo que "holguear"; ya que "lidiar" es relativo a
"lidia" y "holguear" es relativo a "olga".
	-- Ricardo Vaporeso

Leandro Lucarella wrote:
> dsimcha, el 20 de noviembre a las 16:24 me escribiste:
>>>> Right, but they can still be the target of false pointers.  In this case, false
>>>> pointers keep each instance of foo[] alive, leading to severe memory leaks.
>>> But the issue is more of a GC implementation issue then a language
>>> issue, correct?
>> Yes.
>>
>>> Or is this an issue of all lower level language garbage
>>> collectors?
>> Kinda sorta.  It's possible, but not easy, to implement fully precise GC (except
>> for the extreme corner case of unions of reference and non-reference types) in a
>> close to the metal, statically compiled language.
> 
> I don't think so if you want to be able to link to C code, unless I'm
> missing something...
> 

The extern (C) stuff and malloc allocated memory isn't garbage collected.

Travis Boucher, el 20 de noviembre a las 16:45 me escribiste:
> Leandro Lucarella wrote:
> >dsimcha, el 20 de noviembre a las 16:24 me escribiste:
> >>>>Right, but they can still be the target of false pointers.  In this case, false pointers keep each instance of foo[] alive, leading to severe memory leaks.
> >>>But the issue is more of a GC implementation issue then a language issue, correct?
> >>Yes.
> >>
> >>>Or is this an issue of all lower level language garbage collectors?
> >>Kinda sorta.  It's possible, but not easy, to implement fully precise GC (except for the extreme corner case of unions of reference and non-reference types) in a close to the metal, statically compiled language.
> >
> >I don't think so if you want to be able to link to C code, unless I'm missing something...
> >
> 
> The extern (C) stuff and malloc allocated memory isn't garbage collected.

I know, but the stack is used as a root of the live data, and if you use C code, you will have frames without type information. So you will never get full a full precise root set, unless you find some way to separate the D stack from the C stack and completely ignore the C stack. Again, unless I'm missing something :)

-- 
Leandro Lucarella (AKA luca)                     http://llucax.com.ar/
----------------------------------------------------------------------
GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145  104C 949E BFB6 5F5A 8D05)
----------------------------------------------------------------------
- Mire, don Inodoro! Una paloma con un anillo en la pata! Debe ser
  mensajera y cayó aquí!
- Y... si no es mensajera es coqueta... o casada.
	-- Mendieta e Inodoro Pereyra

dsimcha wrote:
> == Quote from Rainer Deyke (rainerd@eldwood.com)'s article
>> Yes, but a moving GC needs to be 100% precise, not 99.99999%.
> 
> Not if you allow pinning, which we'd need anyhow for untyped, conservatively scanned memory blocks.

If you allow pinning then you no longer get the full benefits of a moving gc.  It would be nice to be able to trade untyped, conservatively scanned memory blocks for a better gc.


-- 
Rainer Deyke - rainerd@eldwood.com

On Sat, 21 Nov 2009 05:57:48 +0300, Rainer Deyke <rainerd@eldwood.com> wrote:

> dsimcha wrote:
>> == Quote from Rainer Deyke (rainerd@eldwood.com)'s article
>>> Yes, but a moving GC needs to be 100% precise, not 99.99999%.
>>
>> Not if you allow pinning, which we'd need anyhow for untyped, conservatively
>> scanned memory blocks.
>
> If you allow pinning then you no longer get the full benefits of a
> moving gc.  It would be nice to be able to trade untyped, conservatively
> scanned memory blocks for a better gc.
>
>

Pinning is a must-have if you want communicate with code written in other languages (C, for example).
Casting from Object to void* would be forbidden, use void* pinnedAddress = GC.pin(obj); (or similar) instead.

"dsimcha" <dsimcha@yahoo.com> wrote in message news:he6aah$4d6$1@digitalmars.com...
> == Quote from Denis Koroskin (2korden@gmail.com)'s article
>> Aren't uint array allocations have hasPointers flag set off? I always thought they aren't scanned for pointers (unlike, say, void[]).
>
> Right, but they can still be the target of false pointers.  In this case,
> false
> pointers keep each instance of foo[] alive, leading to severe memory
> leaks.

I don't suppose there's a way to lookup the pointers the GC believes it has found to a given piece of GC-ed memory? Sounds like that would be very useful, if not essential, for debugging/optimizing memory usage.