Am 14.01.2014 23:10, schrieb Walter Bright:
>
> I already agreed that there are better ways.
>

Then this was a big misunderstanding.

On Friday, 10 January 2014 at 08:03:03 UTC, Andrei Alexandrescu
wrote:
> The design at http://goo.gl/ZVCJeB seems to be a win. It works well, comprehends all major allocation tropes, someone implemented a subset of it in C++ and measured good results, and a coworker is considering adopting the design for a C++ project as well.

An additional feature that I would like to suggest is the concept
of time-limited collection.

The goal would be to be able to write a main loop in my
application where I say:

const auto FRAME_DUR_MS = 25;
while (keepPlaying) {
   auto start = Clock.currSystemTick()
   update();
   render();
   GC.collectMsecs( FRAME_DUR_MS - (Clock.currSystemTick() -
start).msecs() );
}

The idea is that the collector, conceptually, has a routine that
it runs. I.e.

1. Point to bottom of stack, scan through.
2. Pop registers from each thread, check values.
3. Determine unreachable values amongst everything allocated.
4. Call destructors on objects.
5. Free memory.

Since "unreachability" is a one-way street, I can pause anywhere
in this process and resume from that location at a later time. I
don't have to go through all of the steps every time I am called.
So while there's no guarantee that any one call to collectMsecs()
actually releases any memory, if the caller continues to call it
reliably, they should be able to keep on top of their memory
usage (or know that they need to allocate less frequently if they
see their memory continue to grow, even under this scheme).

On Tuesday, 14 January 2014 at 22:36:12 UTC, Benjamin Thaut wrote:
> Am 14.01.2014 23:10, schrieb Walter Bright:
>>
>> I already agreed that there are better ways.
>>
>
> Then this was a big misunderstanding.

So, the issue seems to be that everyone is treating the GC as the Thors hammer and seeing everything as a nails?

The GC does and can not solve all problems. Why force people to use it when it doesn't work well?

I would think having a multi-layered approach would be best. Allow the GC to do everything it can but also allow it to be configured from 0 to 60. For noobs, non-realtime apps, etc the GC can be set to do it all. One can trade speed for memory(less memory, more scanning etc...). One can disable to in specific cases(such as on unions), have separate heaps for GC memory and non-GC allocated memory, etc.

It seems that a more robust GC with many features(with the feature of not using it and either manually allocating or using ARC).

The the only real issue becomes how to keep the GC in sync with the non-GC allocated memory... and this only happens if one chooses to use non-GC features(which must be intentional so hopefully the programming knows what he's doing if he got that far).

I doubt this will ever be possible to solve efficiently so why bother? Leave it up to the programmer to deal with it. Make some resources available to solve these problems but don't try to dictate what must be done. If the programmer is going to bypass the GC he already must have good reasons so why put up roadblocks? Cause *you* think you know better than him what he needs to do?


For example, suppose we have a non-GC based pointer. Allow the programmer to mark it as a pointer with certain contextual parameters so that either the GC ca n "deal with it" better or at least for debugging purposes help find memory leaks.

I'm way more interested in having a GC that doesn't stop the world and a compiler that doesn't require the GC for simple stuff as slices(could it not use a buffer and/or ARC instead/alternatively?).

On Wednesday, 15 January 2014 at 00:24:48 UTC, Frustrated wrote:
> On Tuesday, 14 January 2014 at 22:36:12 UTC, Benjamin Thaut wrote:
>> Am 14.01.2014 23:10, schrieb Walter Bright:
>>>
>>> I already agreed that there are better ways.
>>>
>>
>> Then this was a big misunderstanding.
>
> So, the issue seems to be that everyone is treating the GC as the Thors hammer and seeing everything as a nails?
>
> The GC does and can not solve all problems. Why force people to use it when it doesn't work well?
>
> I would think having a multi-layered approach would be best. Allow the GC to do everything it can but also allow it to be configured from 0 to 60. For noobs, non-realtime apps, etc the GC can be set to do it all. One can trade speed for memory(less memory, more scanning etc...). One can disable to in specific cases(such as on unions), have separate heaps for GC memory and non-GC allocated memory, etc.
>
> It seems that a more robust GC with many features(with the feature of not using it and either manually allocating or using ARC).
>
> The the only real issue becomes how to keep the GC in sync with the non-GC allocated memory... and this only happens if one chooses to use non-GC features(which must be intentional so hopefully the programming knows what he's doing if he got that far).
>
> I doubt this will ever be possible to solve efficiently so why bother? Leave it up to the programmer to deal with it. Make some resources available to solve these problems but don't try to dictate what must be done. If the programmer is going to bypass the GC he already must have good reasons so why put up roadblocks? Cause *you* think you know better than him what he needs to do?
>
>
> For example, suppose we have a non-GC based pointer. Allow the programmer to mark it as a pointer with certain contextual parameters so that either the GC ca n "deal with it" better or at least for debugging purposes help find memory leaks.
>
> I'm way more interested in having a GC that doesn't stop the world and a compiler that doesn't require the GC for simple stuff as slices(could it not use a buffer and/or ARC instead/alternatively?).

Try to do the pull request that goes with that, and we will be
able to discuss the real problems.

On Tuesday, 14 January 2014 at 23:48:33 UTC, Chris Williams wrote:
> On Friday, 10 January 2014 at 08:03:03 UTC, Andrei Alexandrescu
> wrote:
>> The design at http://goo.gl/ZVCJeB seems to be a win. It works well, comprehends all major allocation tropes, someone implemented a subset of it in C++ and measured good results, and a coworker is considering adopting the design for a C++ project as well.
>
> An additional feature that I would like to suggest is the concept
> of time-limited collection.
>
> The goal would be to be able to write a main loop in my
> application where I say:
>
> const auto FRAME_DUR_MS = 25;
> while (keepPlaying) {
>    auto start = Clock.currSystemTick()
>    update();
>    render();
>    GC.collectMsecs( FRAME_DUR_MS - (Clock.currSystemTick() -
> start).msecs() );
> }
>
> The idea is that the collector, conceptually, has a routine that
> it runs. I.e.
>
> 1. Point to bottom of stack, scan through.
> 2. Pop registers from each thread, check values.
> 3. Determine unreachable values amongst everything allocated.
> 4. Call destructors on objects.
> 5. Free memory.
>
> Since "unreachability" is a one-way street, I can pause anywhere
> in this process and resume from that location at a later time. I
> don't have to go through all of the steps every time I am called.
> So while there's no guarantee that any one call to collectMsecs()
> actually releases any memory, if the caller continues to call it
> reliably, they should be able to keep on top of their memory
> usage (or know that they need to allocate less frequently if they
> see their memory continue to grow, even under this scheme).

I really love this idea. However I'm not sure how kernels would like no sleeping to occur at least by my understanding.
It would definitely be a big plus to anything related to gui's.

On Monday, 13 January 2014 at 17:45:26 UTC, Dmitry Olshansky
wrote:
> 13-Jan-2014 13:20, Rainer Schuetze пишет:
>> On Sunday, 12 January 2014 at 10:40:50 UTC, Benjamin Thaut wrote:
>> Having to explicitely pin every pointer passed to C functions would be
>> very expensive.
>
> How would it be expensive? I don't see a need to do anything to "pin" a memory block, at the time of scanning there will be a potential pointer to it (in the stack space of C function or registers).

The stack reference can be moved outside the memory visible to
the GC:

// D
extern(C) void funC(const char* s);
void main()
{
      funC("Hello".dup.ptr);
}

// C
void funC(const char* s)
{
      SomeStruct* struc = malloc(sizeof(SomeStruct));
      struc->ptr = s;
      s = 0;
      struc->doSomething();
      free(struc);
}

The "s = 0;" will remove the last reference to the passed string
visible to the GC. The duped string might get collected during
execution of "doSomething". This is a problem with the current GC
already, though it is pretty unlikely that both this kind of
operation is used in the C function (or similar) and there are no
other references in the D program.

With a moving GC, references in the D code no longer pin the
object, so the problem appears if the C function just works as
above.

Another bad use case in the C function: the passed stack
parameter is used to iterate to the end of an array and then back
later. The pointer to the end might not be within the GC
allocated memory block anymore, but pointing to the next.

On Monday, 13 January 2014 at 22:48:37 UTC, evansl wrote:
> On 01/11/14 03:30, Rainer Schuetze wrote:
>>
>>
>> On 10.01.2014 22:42, Andrei Alexandrescu wrote:
> [snip]
>>> std.emplace will continue to work as a way to build an object at a
>>> specified address. I suspect that allocating and manipulating objects on
>>> the GC heap in particular may have certain restrictions. One possibility
>>> to avoid such restrictions is to have a function typify(T)(void* p)
>>> which ascribes type T to heap location p.
>>
>> That sounds similar to my gc_emplace function. The problematic part is
>> how to save that information in the GC.
>>
> [snip]
> Couldn't you store a pointer to the typeinfo within the
> allocated memory? IOW, the first part of the allocated memory
> would be the typeinfo* followed by the actual memory used to store the
> value of the allocated T object.
>
> -regards,
> Larry

std.emplace can be used on a partial memory block (e.g. as part
of a struct), so you will have to add the emplaced type info in
addition to the outer struct's type info. There can be multiple
areas with emplaced dta within the same memory allocation, too.
So you'll need to store a list of type infos paired with the
offsets within the meory block. How do you do this efficiently
without extra cost for the usual scanning?

On Wednesday, 15 January 2014 at 08:54:51 UTC, Rainer Schuetze wrote:
> std.emplace can be used on a partial memory block (e.g. as part
> of a struct), so you will have to add the emplaced type info in
> addition to the outer struct's type info. There can be multiple
> areas with emplaced dta within the same memory allocation, too.

This is a good point. It is a mistake to mix manual memory layout and byte-level control  with an obligatory GC regime. Those optimizations should either be done automatically by a high level optimizer, triggered by compiler hits, or be non-GC. Basically, one has to decide whether to focus on high level or low level for heap allocations.

D is by design a high-level effort with low-level access bolted on everywhere. That undermines the premises for efficient high-level implementation. I think you either have to restrict the low level to the non-GC-heap-allocations and inner loops at the leaves of the call-tree, or start with a low level language design and very carefully add optional high level constructs. High level as the default with low level control interspersed everywhere makes high level optimization intractable.

On Wednesday, 15 January 2014 at 01:55:13 UTC, Rikki Cattermole
wrote:
> I really love this idea. However I'm not sure how kernels would like no sleeping to occur at least by my understanding.
> It would definitely be a big plus to anything related to gui's.

Sleep allows a thread to give way to another thread within the
same process. If an app only has one thread, then that should end
up ceding time to other processes, but the OS is still able to
perform process switching regardless of whether an application
ever has all of its threads asleep at once or not.

On Wednesday, 15 January 2014 at 21:54:18 UTC, Chris Williams wrote:
> On Wednesday, 15 January 2014 at 01:55:13 UTC, Rikki Cattermole
> wrote:
>> I really love this idea. However I'm not sure how kernels would like no sleeping to occur at least by my understanding.
>> It would definitely be a big plus to anything related to gui's.
>
> Sleep allows a thread to give way to another thread within the
> same process. If an app only has one thread, then that should end
> up ceding time to other processes, but the OS is still able to
> perform process switching regardless of whether an application
> ever has all of its threads asleep at once or not.

I'm referring to cpu prioritization. For the process. It'll depend heavily on the kernel/config of it however.