I am not sure about the title...

All debates about memory management are interesting even if I am not sure to be able to help a lot on it, but I have some concerns about the time will elapse to get a valid answer.

My concerns as Dlang user are :
 - Even if GC is the solution, how long I need suffer with destructor's issues (calls order)?
 - When we will able to see a performant GC implementation can satisfy someone like Manu :) ? Months, years, a decade?
 - Same question if D migrate to ARC?

Why theses questions :
 - Memory management seems to be one of last (or the last) critical point for a vastly adoption in production.
 - Some serious alternatives of D appears or will appears if it took too much time for D to convince the industry. Despite some strong features will certainly never be implemented in alternatives.

Dlang seems to be a lab technology for the moment, maybe it's due to the fact their is no major behind company, which is a good point, but I'd love so much use D in my day job... For DQuick we completely loose our motivation cause of the lack of clear plans on decisions in the language having a deep impact on this project.
IMO bottleneck points of languages have to be solved even if in a decade those points will be revised. Maybe things must know many iterations before being at the expected level. Don't forget the ecosystem around the language will also take years to grow before coming really interesting.

In my sens C++ fails to evolve cause of conservative decisions, Apple with Clang work on include removal. I'd prefer to see the C++ community took this decision to make it standard even if it break the language.
I would be sad to see D, making the same.

If phobos comes allocation free or use allocators, then it will be easier to migrate from one memory management to an other one.


In conclusion would be possible to migrate to ARC if the time gap in comparison of an ideal implementation of GC is of the order of many years or a decade?


Tell me if my questions are irrelevant, but please to put me in a jail :/.


PS : Maybe cause I am young, I am hasty :)

On Sat, May 10, 2014 at 09:16:54PM +0200, Xavier Bigand via Digitalmars-d wrote: [...]
> My concerns as Dlang user are :
>  - Even if GC is the solution, how long I need suffer with
>  destructor's issues (calls order)?

Dtor calling order and GC are fundamentally incompatible. I don't think this will ever be changed. The problem is, how do you guarantee that the GC will only clean up garbage in the order of reference? You can't do this without killing GC performance.


>  - When we will able to see a performant GC implementation can satisfy
>  someone like Manu :) ? Months, years, a decade?

I think somebody is working on porting a D1 concurrent GC to D2, so hopefully that will be done sometime in the near future... But I don't know if that's enough to satisfy Manu. His requirements are pretty high. :)


>  - Same question if D migrate to ARC?

I highly doubt D will migrate to ARC. ARC will probably become *possible*, but some language features fundamentally rely on the GC, and I can't see how that will ever be changed.



> Why theses questions :
>  - Memory management seems to be one of last (or the last) critical
>  point for a vastly adoption in production.
[...]

Nah, it's just the thing that gets complained about the most. There are other big issues that need to be fixed. Like compatibility with handheld architectures. Completing the implementation of @safe. Fixing the holes in the type system (esp. w.r.t. const/immutable). Issues with 'shared'. AA implementation.

I'm sure people can come up with many other big items that need to be addressed.


T

-- 
Mediocrity has been pushed to extremes.

I think the reason people ask about improving the GC so frequently is that it's not clear when any potential future iprovements will come around. I think perhaps things may become more clear to those concerned if they were told who was working on GC, roughly when they can expect to see certain improvements, and so on.

I think the recent work Walter has been doing on the @nogc attribute is valuable, and will be a major contributor to reducing the number of undue allocations in the standard library. While this will not address quality of implementation issues with the garbage collector, this will certainly reduce the impact of performance problems caused by pause times. Culprits will be easier to track down and eliminate with the new attribute and its semantics.

On Saturday, 10 May 2014 at 19:41:15 UTC, H. S. Teoh via Digitalmars-d wrote:
> On Sat, May 10, 2014 at 09:16:54PM +0200, Xavier Bigand via Digitalmars-d wrote:
> [...]
>> My concerns as Dlang user are :
>>  - Even if GC is the solution, how long I need suffer with
>>  destructor's issues (calls order)?
>
> Dtor calling order and GC are fundamentally incompatible. I don't think
> this will ever be changed. The problem is, how do you guarantee that the
> GC will only clean up garbage in the order of reference? You can't do
> this without killing GC performance.

You can build a queue of root nodes in terms of parent-child ownership if you have parent backpointers. That allows you to separate scanning from releasing. You can then release when idle using a priority queue.

You can optimize scanning by tracing parent-pointers first then mark parent-children trees as live when hitting roots using extra datastructures and meta information. (assuming the tree has no external pointers below the root)

It has language and runtime consequences, but I doubt it will kill performance.

(I don't think it belongs in a system level language though)

On Saturday, 10 May 2014 at 19:17:02 UTC, Xavier Bigand wrote:
>  - When we will able to see a performant GC implementation can satisfy someone like Manu :) ? Months, years, a decade?

Never, Manu wants to do fine granularity allocations.

>  - Same question if D migrate to ARC?

You probably also need to use an autorelease pool for realtime callbacks and add whole program analysis to avoid exessive ref counting.

> For DQuick we completely loose our motivation cause of the lack of clear plans on decisions in the language having a deep impact on this project.

Yes, evolutionary drift is a key problem. Not having worked out the core memory model before adding language features is also a problem

> Don't forget the ecosystem around the language will also take years to grow before coming really interesting.

For applications, yes.

But I don't think that is true for a system level language...  A solid language and compiler with minimal runtime requirements would cut it.

> In my sens C++ fails to evolve cause of conservative decisions, Apple with Clang work on include removal. I'd prefer to see the C++ community took this decision to make it standard even if it break the language.
> I would be sad to see D, making the same.

D is limiting itself by:

1. Requiring C/C++ compatible runtime for all threads. You could do better by having some threads with no FFI.

2. Limiting breaking changes for a development branch...

3. Having 3 different backends. Language design does affect IR/backend if you eant performance. Rewriting 3 backends is... Not realistic,

> If phobos comes allocation free or use allocators, then it will be easier to migrate from one memory management to an other one.

Phobos is a non-issue, language constructs and runtime is the primary issue. (you can have a real time library)

> In conclusion would be possible to migrate to ARC if the time gap in comparison of an ideal implementation of GC is of the order of many years or a decade?

No, because it still won't perform well with multi threading and by that time CPUs will support transactional memory. RC kills transactions by doing writes.

Isolates are easy to do, but they are not performant.

On 11 May 2014 05:39, H. S. Teoh via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
> On Sat, May 10, 2014 at 09:16:54PM +0200, Xavier Bigand via Digitalmars-d wrote:
>>  - Same question if D migrate to ARC?
>
> I highly doubt D will migrate to ARC. ARC will probably become *possible*, but some language features fundamentally rely on the GC, and I can't see how that will ever be changed.

Which ones are incompatible with ARC?

Am 11.05.2014 03:31, schrieb "Ola Fosheim Grøstad" <ola.fosheim.grostad+dlang@gmail.com>":
> On Saturday, 10 May 2014 at 19:41:15 UTC, H. S. Teoh via Digitalmars-d
> wrote:
>> On Sat, May 10, 2014 at 09:16:54PM +0200, Xavier Bigand via
>> Digitalmars-d wrote:
>> [...]
>>> My concerns as Dlang user are :
>>>  - Even if GC is the solution, how long I need suffer with
>>>  destructor's issues (calls order)?
>>
>> Dtor calling order and GC are fundamentally incompatible. I don't think
>> this will ever be changed. The problem is, how do you guarantee that the
>> GC will only clean up garbage in the order of reference? You can't do
>> this without killing GC performance.
>
> You can build a queue of root nodes in terms of parent-child ownership
> if you have parent backpointers. That allows you to separate scanning
> from releasing. You can then release when idle using a priority queue.
>

This is what java.lang.ref.ReferenceQueue are for in Java, but one needs to be a GC expert on how to use it, otherwise it will hinder the GCs work.

--
Paulo

On Saturday, 10 May 2014 at 19:41:15 UTC, H. S. Teoh  wrote:
> On Sat, May 10, 2014 at 09:16:54PM +0200, Xavier Bigand wrote:
> 
>> Why theses questions :
>>  - Memory management seems to be one of last (or the last) critical
>>  point for a vastly adoption in production.
> [...]
>
> Nah, it's just the thing that gets complained about the most. There are
> other big issues that need to be fixed. Like compatibility with handheld
> architectures. Completing the implementation of @safe. Fixing the holes
> in the type system (esp. w.r.t. const/immutable). Issues with 'shared'.
> AA implementation.

+1000! :-P

---
Paolo

Am Sun, 11 May 2014 14:52:50 +1000
schrieb Manu via Digitalmars-d <digitalmars-d@puremagic.com>:

> On 11 May 2014 05:39, H. S. Teoh via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
> > On Sat, May 10, 2014 at 09:16:54PM +0200, Xavier Bigand via Digitalmars-d wrote:
> >>  - Same question if D migrate to ARC?
> >
> > I highly doubt D will migrate to ARC. ARC will probably become *possible*, but some language features fundamentally rely on the GC, and I can't see how that will ever be changed.
> 
> Which ones are incompatible with ARC?

Pass-by-value slices as 2 machine words

-- 
Marco

On 12 May 2014 02:38, Marco Leise via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
> Am Sun, 11 May 2014 14:52:50 +1000
> schrieb Manu via Digitalmars-d <digitalmars-d@puremagic.com>:
>
>> On 11 May 2014 05:39, H. S. Teoh via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>> > On Sat, May 10, 2014 at 09:16:54PM +0200, Xavier Bigand via Digitalmars-d wrote:
>> >>  - Same question if D migrate to ARC?
>> >
>> > I highly doubt D will migrate to ARC. ARC will probably become *possible*, but some language features fundamentally rely on the GC, and I can't see how that will ever be changed.
>>
>> Which ones are incompatible with ARC?
>
> Pass-by-value slices as 2 machine words

64bit pointers are only 40-48 bits, so there's 32bits waste for an
offset... and if the base pointer is 32byte aligned (all allocated
memory is aligned), then you can reclaim another 5 bits there... I
think saving an arg register would probably be worth a shift.
32bit pointers... not so luck :/
video games consoles though have bugger all memory, so heaps of spare
bits in the pointers! :P