Hello everyone! Being a C/C++ programmer I don't understand, why such language as D (system programming language) implemented garbage collector as a core feature, not as additional optional module or library. I and many other C/C++ programmers prefer to control things manually and use flexible allocation schemes that suitable for concrete situations. When every nanosecond matters, this is the only option, at least nowadays.

So this particular thing stops many of us from using D. When you can abandon performance, you usually choose Java (Scala) or C# because of their rich support and libraries. And the opposite matter is when you need high performance. In this case there is almost nothing to choose from. C/C++11 now looks not so bad.

And I think of idea of complete extraction of GC from D. For this to achieve, I suppose, I have to dig deeply into D compiler and also correct/re-implement many things in modules, so this ends up with almost new version of D.

I would like to hear your suggestions and some basic instructions. Maybe this is not a good idea at all or it will be very hard to realize.

Thank you for your attention!

The current idea is to add @nogc attribute to functions in phobos, which can live without GC, so that you could use them from other @nogc functions.

You can also help with allocators design: http://forum.dlang.org/thread/lji5db$30j3$1@digitalmars.com

On Thursday, 11 September 2014 at 12:38:54 UTC, Andrey Lifanov wrote:
> Hello everyone! Being a C/C++ programmer I don't understand, why such language as D (system programming language) implemented garbage collector as a core feature, not as additional optional module or library.

I can enlighten you ;-) The reason is safety. Past experience (especially with C & C++) has shown that manual memory management is easy to get wrong. Besides, certain features would not easily be possible without it (dynamic arrays, closures).

> I and many other C/C++ programmers prefer to control things manually and use flexible allocation schemes that suitable for concrete situations. When every nanosecond matters, this is the only option, at least nowadays.
>
> So this particular thing stops many of us from using D. When you can abandon performance, you usually choose Java (Scala) or C# because of their rich support and libraries. And the opposite matter is when you need high performance. In this case there is almost nothing to choose from. C/C++11 now looks not so bad.
>
> And I think of idea of complete extraction of GC from D. For this to achieve, I suppose, I have to dig deeply into D compiler and also correct/re-implement many things in modules, so this ends up with almost new version of D.
>
> I would like to hear your suggestions and some basic instructions. Maybe this is not a good idea at all or it will be very hard to realize.

I don't think it is necessary to remove the GC completely. It can only interfere with your program in three situations:

1) When you allocate and run out of memory, the GC will first try to release some unneeded memory before requesting more from the OS. If you don't allocate, anything in a performance critical section of your program, the GC will never run.
2) (Actually a consequence of 1) When the GC runs, it stops all threads, including those that never allocate.
3) When you call it manually.

For 1), there is the @nogc attribute. Any function marked with this attribute is guaranteed to never allocate on the GC heap, including via any other functions it calls. If you write `void main() @nogc`, your entire program will be GC-less. This attribute has only been introduced in the latest release, and some parts of the standard library cannot be used with it yet.

For 2), you can call `GC.disable()` and `GC.enable()` to switch the GC on/off temporarily. You can still allocate memory, but the GC will not run.

For 3): Do it when you can afford the latency, for example between frames, or when you are not in a performance critical section of your program. Right now, it is not anytime capable, which means you cannot give it a deadline until that it either has to finish, or abort the current operation. This would be an interesting enhancement.

As for manual memory management, Andrei is currently working on an allocator library: http://erdani.com/d/phobos-prerelease/std_allocator.html

On Thursday, 11 September 2014 at 12:38:54 UTC, Andrey Lifanov wrote:
> And I think of idea of complete extraction of GC from D.

You could also recompile the runtime library without the GC. Heck, with the new @nogc on your main, the compiler (rather than the linker) should even give you nicish error messages if you try to use it, but I've done it before that was an option.

Generally though, GC fear is overblown. Use it in most places and just don't use it where it makes things worse.

Thank you for quick response!

I guess I need further investigation and write good tests to compare C++ and D solutions. The main advantage over GC-language is that in case of manual memory management I know completely when and what will be freed or allocated (with the help of smart pointers/reference counting, of course). You can say that this has no importance to programmer, but it has, because you don't have performance spikes and don't have to waste the processor and slow memory time for scanning what things need to be collected. So, the big advantage (with the price of greater responsibility) is much greater predictability of how your program will perform.

The main problem of heap-intensive programs with huge amount of objects is heap fragmentation. During the program work there can be "holes" of memory chunks which complicate further allocations, specially for big continuous arrays. Also it ruins cache performance, because similar objects, belonging to one array, can be stationed far from each other, divided by such "holes". Fairly speaking, C/C++ do not have the built-in solution for such problem, but you can program it manually there.

So maybe instead of getting rid of GC I will consider the implementation of optimized moving GC.

On Thu, 11 Sep 2014 15:23:53 +0000
Andrey Lifanov via Digitalmars-d <digitalmars-d@puremagic.com> wrote:

> is that in case of manual memory management I know completely when and what will be freed or allocated (with the help of smart pointers/reference counting, of course).
but you don't. you can only estimate, that's all. what if you passing refcounted object to another function which stores it somewhere? oops.

and in D you are free to use structs, `scoped!`, and malloc()/free()
(see std.typecons for "scoped" template source to see how it's done).

you can write your own array implementations too. slices are hard to do without proper GC though, and closures requres GC, AFAIK.

> So maybe instead of getting rid of GC I will consider the implementation of optimized moving GC.
copying GC requires support from the compiler side, and it is not that easy at all (imagine malloc()ed blocks which holds references to GC-alloced objects, for example). current D GC is conservative, so you can just register malloc()ed block as root, but for copying GC you'll need either to inform GC about exact block structure, or provide your own scan/copy/fix callbacks.

and if copying GC will not do 'stop-the-world', some threads can hold pointers in registers... and inserting read/write barriers will hurt performance...

I have recently found: http://en.wikibooks.org/wiki/D_Programming/Garbage_collector/Thoughts_about_better_GC_implementations

Good stuff there.

Am 11.09.2014 14:38, schrieb Andrey Lifanov:
> Hello everyone! Being a C/C++ programmer I don't understand, why such
> language as D (system programming language) implemented garbage
> collector as a core feature, not as additional optional module or
> library. I and many other C/C++ programmers prefer to control things
> manually and use flexible allocation schemes that suitable for concrete
> situations. When every nanosecond matters, this is the only option, at
> least nowadays.
>
> ...

Since the mid-70's there are system programming languages with GC.

Namely Algol 68(IFIP), Mesa/Cedar (Xerox), Modula-3 (Olivetti), Oberon (ETHZ) and a few others.

They just happened to be married to OSes that weren't as successful as UNIX jumping out of the research labs into the industry.

It is about time systems programming catches up with the 70's innovations outside of the PDP-11 world.

--
Paulo

On Thursday, 11 September 2014 at 15:39:26 UTC, ketmar via
Digitalmars-d wrote:
> On Thu, 11 Sep 2014 15:23:53 +0000
> Andrey Lifanov via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>
>> is that in case of manual memory management I know completely when and what will be freed or allocated (with the help of smart pointers/reference counting, of course).
> but you don't. you can only estimate, that's all. what if you passing
> refcounted object to another function which stores it somewhere? oops.

What do you mean? Some sort of memory leak? I guess, you can
always write programs, no matter with or without GC, that will
steal or hide stuff. As usual countermeasures, you just have to
carefully plan single storage/multiple users code.