On Tuesday, 14 January 2014 at 11:02:38 UTC, Benjamin Thaut wrote:
> Am 14.01.2014 11:05, schrieb Walter Bright:
>> On 1/14/2014 1:18 AM, Benjamin Thaut wrote:
>>> Current D does not allow a moving collector because of the lack of
>>> compiler
>>> support. It is still not possible to identify all pointers percicesly,
>>> especially those on the stack. Also when you want to implement a
>>> semi-space GC
>>> everything _must_ be moveable. Pinning is not an option for a
>>> semi-space GC.
>>> There for current D does not allow the implementation of semi-space GC
>>> without
>>> some changes to the spec. (E.g. structs / classes containing unions
>>> _must_
>>> provide a custom scanning function).
>>
>>
>> This is not true, I've implemented one for Java.
>
> So how do you implement a semi-space GC with pinned objects?

You manage the pinned object in a different list that the semi-space list?

It's quite often that GCs maintain object inventory with several methods, for example using semi-space for young object but not for old objects..

On 01/14/2014 10:20 AM, Benjamin Thaut wrote:
> Am 14.01.2014 09:15, schrieb Jacob Carlborg:
>> On 2014-01-13 22:23, Timon Gehr wrote:
>>
>>> Yes, there is a cost. The requirement to pin arbitrary objects at
>>> arbitrary times, without the possibility to move at pinning time,
>>> invalidates GC algorithms that depend on being able to move _all_
>>> objects within some pool.
>>
>> Can't these object be pinned somewhere else?
>>
>
> Yes, thats the usual approach. But that means that you have to copy them
> to a other space before pinning, or allocate them in a space that allows
> pinning in the first place.

Once it becomes known that an object should be pinned, it is too late for moving. (There will then exist a potential pointer to the object.)

On 01/14/2014 10:11 AM, Paulo Pinto wrote:
> On Monday, 13 January 2014 at 23:42:50 UTC, Timon Gehr wrote:
>> On 01/13/2014 11:05 PM, Paulo Pinto wrote:
>>> Am 13.01.2014 22:31, schrieb Timon Gehr:
>>>> On 01/13/2014 10:11 PM, Walter Bright wrote:
>>>>>>
>>>>>> Not to say there aren't other ways of doing things, but with random
>>>>>> objects
>>>>>> becoming pinnable puts a big damper on things unless you can identify
>>>>>> all the
>>>>>> objects that might be pinned and isolate them.  But I doubt that's
>>>>>> really
>>>>>> knowable up front.
>>>>>
>>>>> The reason pinning doesn't particularly impede this is because pinning
>>>>> is rare.
>>>>
>>>> In Java or in D? Eg. there are no unions in Java.
>>>
>>> C# has unions.
>>>
>>> http://msdn.microsoft.com/en-us/library/system.runtime.interopservices.structlayoutattribute%28v=vs.110%29.aspx
>>>
>>>
>>
>> "The common language runtime controls the physical layout of the data
>> fields of a class or structure in managed memory. However, ..."
>
>
> Did you also read the remaining part of the page,

Obviously.

> or just looked for something to paste?
> ...

Look, you hadn't done anything besides pasting for backing up your point or making it precise. I'll be proven wrong if you are able to show us how you do the following in C#:

union U{
    int x;
    Object y;
}

> You can control the layout, that is what matters.  Not at what level you
> are expressing it.
> ...

Obviously.

> Ignoring what such runtimes offer, only puts D at disadvantage when
> comparing feature lists, which many in the industry do.
>...

Every resource I have encountered indicates that C# does not offer this functionality because it is detrimental to GC, eg:

http://stackoverflow.com/questions/17771902/struct-memory-hack-to-overlap-object-reference-is-it-possible

14-Jan-2014 01:24, "Ola Fosheim Grøstad" <ola.fosheim.grostad+dlang@gmail.com>" пишет:
> On Monday, 13 January 2014 at 21:08:50 UTC, Dmitry Olshansky wrote:
>> So what? It can't move it and as such there is nothing special about
>> it, it doesn't _cost_ anything to pin object.
>
> Well, the advantage of compaction is that you (in theory) can relocate
> objects so that you:
>
> 1. get temporal and spatial coherency (sitting on the same page, to
> avoid paging)

Bleh. If anything you may as well lose it - manual memory management is the only option to guarantee you something to the extent of having related object together in the same page. That is if you can make it so by hand.

Segregated heaps also get fine locality for similarly-sized objects.

> 2. get rid of fragmentation (less paging, requires smaller address space)
> 3. can have a faster and simpler allocator
>
> If you start using the GC heap for "malloc" with pinning, you loose a
> lot of that? That's the cost.
>

This is a missed potential not an extra cost.
On the contrary write barriers to track precisely every pointer write do *cost* something. This cost is then regained (with interest) by applying simpler and efficient algorithms such as semi-space compaction.


-- 
Dmitry Olshansky

 If Java with all it that time and money can not create a performant GC, what hope has D?

 Full program tracing GC is a dead horse, it does not scale, all of the better Java GC's require 2x or memory of the working set to actually get any speed


 isolated task or allocator based memory that can use GC as a last resort is what you need, but D lacks in this regard

Am 14.01.2014 14:42, schrieb Timon Gehr:
> On 01/14/2014 10:20 AM, Benjamin Thaut wrote:
>> Am 14.01.2014 09:15, schrieb Jacob Carlborg:
>>> On 2014-01-13 22:23, Timon Gehr wrote:
>>>
>>>> Yes, there is a cost. The requirement to pin arbitrary objects at
>>>> arbitrary times, without the possibility to move at pinning time,
>>>> invalidates GC algorithms that depend on being able to move _all_
>>>> objects within some pool.
>>>
>>> Can't these object be pinned somewhere else?
>>>
>>
>> Yes, thats the usual approach. But that means that you have to copy them
>> to a other space before pinning, or allocate them in a space that allows
>> pinning in the first place.
>
> Once it becomes known that an object should be pinned, it is too late
> for moving. (There will then exist a potential pointer to the object.)

Well no, usually you have to pin objects _before_ passing them to C-land. Pinning is not a automatic process its done manually (or by the compiler).

Am 14.01.2014 13:53, schrieb renoX:
> On Tuesday, 14 January 2014 at 11:02:38 UTC, Benjamin Thaut wrote:
>> Am 14.01.2014 11:05, schrieb Walter Bright:
>>> On 1/14/2014 1:18 AM, Benjamin Thaut wrote:
>>>> Current D does not allow a moving collector because of the lack of
>>>> compiler
>>>> support. It is still not possible to identify all pointers percicesly,
>>>> especially those on the stack. Also when you want to implement a
>>>> semi-space GC
>>>> everything _must_ be moveable. Pinning is not an option for a
>>>> semi-space GC.
>>>> There for current D does not allow the implementation of semi-space GC
>>>> without
>>>> some changes to the spec. (E.g. structs / classes containing unions
>>>> _must_
>>>> provide a custom scanning function).
>>>
>>>
>>> This is not true, I've implemented one for Java.
>>
>> So how do you implement a semi-space GC with pinned objects?
>
> You manage the pinned object in a different list that the semi-space list?
>
> It's quite often that GCs maintain object inventory with several
> methods, for example using semi-space for young object but not for old
> objects..
>

I meant a pure semi-space collector. A pure semi-space collector can not pin objects. And I didn't make that up, it even says so in "The Garbage Collection Handbook".

Pinning Unions is a bad approach anyway in my eyes. Because you will have to allocate them in seperate space (which supports pinning) you loose the advantge of fast collections for short living objects which a semi-space collector provides.

On 01/14/2014 06:56 PM, Benjamin Thaut wrote:
> Am 14.01.2014 14:42, schrieb Timon Gehr:
>> On 01/14/2014 10:20 AM, Benjamin Thaut wrote:
>>> Am 14.01.2014 09:15, schrieb Jacob Carlborg:
>>>> On 2014-01-13 22:23, Timon Gehr wrote:
>>>>
>>>>> Yes, there is a cost. The requirement to pin arbitrary objects at
>>>>> arbitrary times, without the possibility to move at pinning time,
>>>>> invalidates GC algorithms that depend on being able to move _all_
>>>>> objects within some pool.
>>>>
>>>> Can't these object be pinned somewhere else?
>>>>
>>>
>>> Yes, thats the usual approach. But that means that you have to copy them
>>> to a other space before pinning, or allocate them in a space that allows
>>> pinning in the first place.
>>
>> Once it becomes known that an object should be pinned, it is too late
>> for moving. (There will then exist a potential pointer to the object.)
>
> Well no, usually you have to pin objects _before_ passing them to
> C-land. Pinning is not a automatic process its done manually (or by the
> compiler).

Well, there is no such manual mechanism in place, so only strategies implemented fully within the compiler work.

I think it is not very sensible to let the compiler insert call-backs to eg. pin objects pointed to from unions. It might as well track occupied pointer fields explicitly then.

In any case, I think we should just require manual marking of unions and call it a day.

Am 14.01.2014 19:26, schrieb Timon Gehr:
> On 01/14/2014 06:56 PM, Benjamin Thaut wrote:
>> Am 14.01.2014 14:42, schrieb Timon Gehr:
>>> On 01/14/2014 10:20 AM, Benjamin Thaut wrote:
>>>> Am 14.01.2014 09:15, schrieb Jacob Carlborg:
>>>>> On 2014-01-13 22:23, Timon Gehr wrote:
>>>>>
>>>>>> Yes, there is a cost. The requirement to pin arbitrary objects at
>>>>>> arbitrary times, without the possibility to move at pinning time,
>>>>>> invalidates GC algorithms that depend on being able to move _all_
>>>>>> objects within some pool.
>>>>>
>>>>> Can't these object be pinned somewhere else?
>>>>>
>>>>
>>>> Yes, thats the usual approach. But that means that you have to copy
>>>> them
>>>> to a other space before pinning, or allocate them in a space that
>>>> allows
>>>> pinning in the first place.
>>>
>>> Once it becomes known that an object should be pinned, it is too late
>>> for moving. (There will then exist a potential pointer to the object.)
>>
>> Well no, usually you have to pin objects _before_ passing them to
>> C-land. Pinning is not a automatic process its done manually (or by the
>> compiler).
>
> Well, there is no such manual mechanism in place, so only strategies
> implemented fully within the compiler work.
>
> I think it is not very sensible to let the compiler insert call-backs to
> eg. pin objects pointed to from unions. It might as well track occupied
> pointer fields explicitly then.
>
> In any case, I think we should just require manual marking of unions and
> call it a day.

Yes, but pinning will still be necessary for passing data to C/C++ functions.

On 01/14/2014 06:56 PM, Benjamin Thaut wrote:
>> Once it becomes known that an object should be pinned, it is too late
>> for moving. (There will then exist a potential pointer to the object.)
>
> Well no, usually you have to pin objects _before_ passing them to
> C-land. Pinning is not a automatic process its done manually (or by the
> compiler).

BTW, our misunderstandings are rooted here:
http://dlang.org/garbage.html

"Object Pinning and a Moving Garbage Collector

Although D does not currently use a moving garbage collector, by following the rules listed above one can be implemented. No special action is required to pin objects. A moving collector will only move objects for which there are no ambiguous references, and for which it can update those references. All other objects will be automatically pinned."