Hello dear D community.

This is my very first post here. I've been intrigued with D for quite a while, and I would love to have a project to work on in D. I'm working as a C++ software engineer, and should be moderately up-to-date with best practices of C++ programming.

I've read a good part of "The D Programming Language" and must say that I *love* almost everything about D. Metaprogramming, which quickly gets very convoluted in C++ (to say the least), is just beautiful in D. The approach to multi-threaded programming, and the adoption of concepts of functional programming, are all pure gold.

The one thing that I am unsure about in D is garbage collection. I guess this is some sort of FAQ and may have been answered dozens of times. I apologise in advance.

The main reason I dislike garbage collection for is that destructors of objects on the heap are not called at defined times. Techniques like RAII (http://en.wikipedia.org/wiki/Resource_Acquisition_Is_Initialization) have become an often used pattern in my programming, and works nicely for short lived objects (on the stack), but by means of smart pointers (i.e. reference counting) also for long lived objects (on the heap).

Also, garbage collection often tends to increase the memory footprint of your software. You may be able to avoid this with some expert knowledge about the garbage collector, but this of course invalidates one of the main arguments in favour of GC, which is less complexity. I'd say that if you want to write memory- and CPU-efficient code, then you'll need a certain amount of understanding of your memory management. Be it your manual memory management, or the inner workings of your garbage collector.

To cut things short, even after giving it a lot of thought I still feel uncomfortable about garbage collection. I'm wondering whether it is viable to use the same smart pointer techniques as in C++: is there an implementation already of such smart pointers? Can I switch off GC in my D programs altogether? How much does the standard library rely on GC? I.e. would I have to write a alternative standard library to do this?

There is one blog post I found interesting to read in this context, which also underlines my worries: http://3d.benjamin-thaut.de/?p=20

On Tuesday, 25 December 2012 at 14:13:21 UTC, Sven Over wrote:
> To cut things short, even after giving it a lot of thought I still feel uncomfortable about garbage collection. I'm wondering whether it is viable to use the same smart pointer techniques as in C++: is there an implementation already of such smart pointers? Can I switch off GC in my D programs altogether? How much does the standard library rely on GC? I.e. would I have to write a alternative standard library to do this?

std.typecons.RefCounted!T

core.memory.GC.disable();

Phobos does rely on the GC to some extent. Most algorithms and ranges do not though.

> std.typecons.RefCounted!T
>
> core.memory.GC.disable();

Wow. That was easy.

I see, D's claim of being a multi-paradigm language is not false.

> Phobos does rely on the GC to some extent. Most algorithms and ranges do not though.

Running (library) code that was written with GC in mind and turning GC off doesn't sound ideal.

But maybe this allows me to familiarise myself more with D. Who knows, maybe I can learn to stop worrying and love garbage collection.

Thanks for your help!

On Tuesday, 25 December 2012 at 14:48:48 UTC, Sven Over wrote:
>> Phobos does rely on the GC to some extent. Most algorithms and ranges do not though.
>
> Running (library) code that was written with GC in mind and turning GC off doesn't sound ideal.

It isn't if you care about memory leaks. And also bare in mind that some language constructs also relies on the GC, like some dynamic arrays operations (concatenation and appending for example). Right now is pretty hard to write D code that doesn't really use the GC at all.

On Tuesday, December 25, 2012 16:33:27 Leandro Lucarella wrote:
> On Tuesday, 25 December 2012 at 14:48:48 UTC, Sven Over wrote:
> >> Phobos does rely on the GC to some extent. Most algorithms and ranges do not though.
> > 
> > Running (library) code that was written with GC in mind and
> > turning GC off doesn't sound ideal.
> 
> It isn't if you care about memory leaks. And also bare in mind that some language constructs also relies on the GC, like some dynamic arrays operations (concatenation and appending for example). Right now is pretty hard to write D code that doesn't really use the GC at all.

There's also often no reason not to have the GC on and use it for certain stuff but use ref-counted objects as your main type of memory-management. Then, you avoid whatever issues the GC has in most cases but don't have to worry about what it takes to be able to not use it at all. For instance, arrays would probably be GC-allocated in general, since then you can use slices and whatnot, but maybe all of your user-defined objects on the heap could be malloced and freed with reference counts (though that sort of thing should be much easier once we have custom allocators - it's a bit more of a pain right now than it should be).

- Jonathan M Davis

On Tuesday, 25 December 2012 at 19:23:59 UTC, Jonathan M Davis wrote:
> On Tuesday, December 25, 2012 16:33:27 Leandro Lucarella wrote:
>> On Tuesday, 25 December 2012 at 14:48:48 UTC, Sven Over wrote:
>> >> Phobos does rely on the GC to some extent. Most algorithms and
>> >> ranges do not though.
>> > 
>> > Running (library) code that was written with GC in mind and
>> > turning GC off doesn't sound ideal.
>> 
>> It isn't if you care about memory leaks. And also bare in mind
>> that some language constructs also relies on the GC, like some
>> dynamic arrays operations (concatenation and appending for
>> example). Right now is pretty hard to write D code that doesn't
>> really use the GC at all.
>
> There's also often no reason not to have the GC on and use it for certain stuff
> but use ref-counted objects as your main type of memory-management. Then, you
> avoid whatever issues the GC has in most cases but don't have to worry about
> what it takes to be able to not use it at all. For instance, arrays would
> probably be GC-allocated in general, since then you can use slices and
> whatnot, but maybe all of your user-defined objects on the heap could be
> malloced and freed with reference counts (though that sort of thing should be
> much easier once we have custom allocators - it's a bit more of a pain right
> now than it should be).
>
> - Jonathan M Davis
In my experience its quite some work to remove the GC entierly, but it is possible. Also it makes writing Code also easier ans more effektive in some places because:
-You don't have to register manually allocated blocks with the gc
-You control the exact order of destruction
-You control the time of destruction
-You control in which thread a object gets destructed
-You will be encouraged to not write code that does do 100 allocations just to format a string (because it will leak)

But obviously it has an impact on productivity. But I'm still more productive writing manual memory managed code in D then in C++.

Kind Regards
Benjamin Thaut

On Tuesday, 25 December 2012 at 19:23:59 UTC, Jonathan M Davis wrote:
> There's also often no reason not to have the GC on and use it for certain stuff

One thing that really freaks me out is the fact that the garbage collector pauses the whole process, i.e. all threads.

In my job I'm writing backend services that power a big web site. Perfomance is key, as the response time of the data service in most cases directly adds to the page load time. The bare possibility that the whole service pauses for, say, 100ms is making me feel very uncomfortable.

We easily achieve the performance and reliability we need in C++, but I would love to give D a chance, as it solves many inconveniences of C++ in an elegant way. Metaprogramming and the threading model, just to name two.

> For instance, arrays would probably be GC-allocated in general, since
> then you can use slices and whatnot,

A smart-pointer type for arrays can easily provide slices. It keeps a reference to the full array (which gets destructed when the last reference is dropped), but addresses a subrange.

Thanks everyone for all the replies!

On Monday, 31 December 2012 at 12:14:22 UTC, Sven Over wrote:
> On Tuesday, 25 December 2012 at 19:23:59 UTC, Jonathan M Davis wrote:
>> There's also often no reason not to have the GC on and use it for certain stuff
>
> One thing that really freaks me out is the fact that the garbage collector pauses the whole process, i.e. all threads.
>
> In my job I'm writing backend services that power a big web site. Perfomance is key, as the response time of the data service in most cases directly adds to the page load time. The bare possibility that the whole service pauses for, say, 100ms is making me feel very uncomfortable.
>
> We easily achieve the performance and reliability we need in C++, but I would love to give D a chance, as it solves many inconveniences of C++ in an elegant way. Metaprogramming and the threading model, just to name two.
>
>> For instance, arrays would probably be GC-allocated in general, since
>> then you can use slices and whatnot,
>
> A smart-pointer type for arrays can easily provide slices. It keeps a reference to the full array (which gets destructed when the last reference is dropped), but addresses a subrange.
>
> Thanks everyone for all the replies!

I think the problem would be if you try to append to the slice (~): If the underlying array is not GC allocated, then it will append to a new GC allocated array (AFAIK).

As long as you don't append, I'd say you are fine.

On Wednesday, 26 December 2012 at 11:43:55 UTC, Benjamin Thaut wrote:
> In my experience its quite some work to remove the GC entierly, but it is possible.

Hi Benjamin! I've seen your druntime and thBase repos on GitHub. Very interesting stuff. As I have little (or to be honest: no) experience in D it's difficult for me to judge how much work needs to be done. Is the idea to replace Phobos completely?

> Also it makes writing Code also easier ans more effektive in some places
> because:
> -You don't have to register manually allocated blocks with the gc
> -You control the exact order of destruction
> -You control the time of destruction
> -You control in which thread a object gets destructed
> -You will be encouraged to not write code that does do 100 allocations just to format a string (because it will leak)

I personally agree with all your points, but I must admit that their relevance depends on the programming patterns you are using. Experienced GC programmers certainly favour patterns that work well with GC. I personally prefer a style of programming that makes use of destructors and relies on them being executed at the right time.

> But obviously it has an impact on productivity.

I'm not sure about this at all. I write software that is designed to offer a certain performance, and latency is a big issue. I don't believe that I can achieve that with a "just allocate memory and don't ever care" attitude. I would need some background knowledge about how the GC works, and write code that doesn't screw up badly. Manual memory management also requires me to have an understanding of the implications behind it. But there are well-tested helpers (e.g. smart pointers) that simplify the task a lot.

I do admit that, when performance (both in the sense of being fast and offering reliable performance) is not such a critical issue, then GC won't hurt and "just allocating" is fine and of course reduces development time. However, I tend to just use Python for this class problems.

It comes down to a matter of choice and personal preference. I'm not saying that GC is bad and shouldn't be used. Many people are very happy with it and it does a great job for them. But my personal preference is manual memory management (then again greatly automated by use of reference counting and smart pointers).

On Monday, 31 December 2012 at 12:36:06 UTC, monarch_dodra wrote:
> On Monday, 31 December 2012 at 12:14:22 UTC, Sven Over wrote:

>> A smart-pointer type for arrays can easily provide slices. It keeps a reference to the full array (which gets destructed when the last reference is dropped), but addresses a subrange.

> I think the problem would be if you try to append to the slice (~): If the underlying array is not GC allocated, then it will append to a new GC allocated array (AFAIK).
>
> As long as you don't append, I'd say you are fine.

Of course your smart-array-pointer type needs to implement the ~ operator and create a copy of the array in that case. I guess the full implementation would include something that would resemble that of std::vector in C++.

I should get started writing those types. Then I could either demonstrate that it does work, or understand that it doesn't (and why).