View mode: basic / threaded / horizontal-split · Log in · Help
December 25, 2012
Smart pointers instead of GC?
Hello dear D community.

This is my very first post here. I've been intrigued with D for 
quite a while, and I would love to have a project to work on in 
D. I'm working as a C++ software engineer, and should be 
moderately up-to-date with best practices of C++ programming.

I've read a good part of "The D Programming Language" and must 
say that I *love* almost everything about D. Metaprogramming, 
which quickly gets very convoluted in C++ (to say the least), is 
just beautiful in D. The approach to multi-threaded programming, 
and the adoption of concepts of functional programming, are all 
pure gold.

The one thing that I am unsure about in D is garbage collection. 
I guess this is some sort of FAQ and may have been answered 
dozens of times. I apologise in advance.

The main reason I dislike garbage collection for is that 
destructors of objects on the heap are not called at defined 
times. Techniques like RAII 
(http://en.wikipedia.org/wiki/Resource_Acquisition_Is_Initialization) 
have become an often used pattern in my programming, and works 
nicely for short lived objects (on the stack), but by means of 
smart pointers (i.e. reference counting) also for long lived 
objects (on the heap).

Also, garbage collection often tends to increase the memory 
footprint of your software. You may be able to avoid this with 
some expert knowledge about the garbage collector, but this of 
course invalidates one of the main arguments in favour of GC, 
which is less complexity. I'd say that if you want to write 
memory- and CPU-efficient code, then you'll need a certain amount 
of understanding of your memory management. Be it your manual 
memory management, or the inner workings of your garbage 
collector.

To cut things short, even after giving it a lot of thought I 
still feel uncomfortable about garbage collection. I'm wondering 
whether it is viable to use the same smart pointer techniques as 
in C++: is there an implementation already of such smart 
pointers? Can I switch off GC in my D programs altogether? How 
much does the standard library rely on GC? I.e. would I have to 
write a alternative standard library to do this?

There is one blog post I found interesting to read in this 
context, which also underlines my worries: 
http://3d.benjamin-thaut.de/?p=20
December 25, 2012
Re: Smart pointers instead of GC?
On Tuesday, 25 December 2012 at 14:13:21 UTC, Sven Over wrote:
> To cut things short, even after giving it a lot of thought I 
> still feel uncomfortable about garbage collection. I'm 
> wondering whether it is viable to use the same smart pointer 
> techniques as in C++: is there an implementation already of 
> such smart pointers? Can I switch off GC in my D programs 
> altogether? How much does the standard library rely on GC? I.e. 
> would I have to write a alternative standard library to do this?

std.typecons.RefCounted!T

core.memory.GC.disable();

Phobos does rely on the GC to some extent. Most algorithms and 
ranges do not though.
December 25, 2012
Re: Smart pointers instead of GC?
> std.typecons.RefCounted!T
>
> core.memory.GC.disable();

Wow. That was easy.

I see, D's claim of being a multi-paradigm language is not false.

> Phobos does rely on the GC to some extent. Most algorithms and 
> ranges do not though.

Running (library) code that was written with GC in mind and 
turning GC off doesn't sound ideal.

But maybe this allows me to familiarise myself more with D. Who 
knows, maybe I can learn to stop worrying and love garbage 
collection.

Thanks for your help!
December 25, 2012
Re: Smart pointers instead of GC?
On Tuesday, 25 December 2012 at 14:48:48 UTC, Sven Over wrote:
>> Phobos does rely on the GC to some extent. Most algorithms and 
>> ranges do not though.
>
> Running (library) code that was written with GC in mind and 
> turning GC off doesn't sound ideal.

It isn't if you care about memory leaks. And also bare in mind 
that some language constructs also relies on the GC, like some 
dynamic arrays operations (concatenation and appending for 
example). Right now is pretty hard to write D code that doesn't 
really use the GC at all.
December 25, 2012
Re: Smart pointers instead of GC?
On Tuesday, December 25, 2012 16:33:27 Leandro Lucarella wrote:
> On Tuesday, 25 December 2012 at 14:48:48 UTC, Sven Over wrote:
> >> Phobos does rely on the GC to some extent. Most algorithms and
> >> ranges do not though.
> > 
> > Running (library) code that was written with GC in mind and
> > turning GC off doesn't sound ideal.
> 
> It isn't if you care about memory leaks. And also bare in mind
> that some language constructs also relies on the GC, like some
> dynamic arrays operations (concatenation and appending for
> example). Right now is pretty hard to write D code that doesn't
> really use the GC at all.

There's also often no reason not to have the GC on and use it for certain stuff 
but use ref-counted objects as your main type of memory-management. Then, you 
avoid whatever issues the GC has in most cases but don't have to worry about 
what it takes to be able to not use it at all. For instance, arrays would 
probably be GC-allocated in general, since then you can use slices and 
whatnot, but maybe all of your user-defined objects on the heap could be 
malloced and freed with reference counts (though that sort of thing should be 
much easier once we have custom allocators - it's a bit more of a pain right 
now than it should be).

- Jonathan M Davis
December 26, 2012
Re: Smart pointers instead of GC?
On Tuesday, 25 December 2012 at 19:23:59 UTC, Jonathan M Davis 
wrote:
> On Tuesday, December 25, 2012 16:33:27 Leandro Lucarella wrote:
>> On Tuesday, 25 December 2012 at 14:48:48 UTC, Sven Over wrote:
>> >> Phobos does rely on the GC to some extent. Most algorithms 
>> >> and
>> >> ranges do not though.
>> > 
>> > Running (library) code that was written with GC in mind and
>> > turning GC off doesn't sound ideal.
>> 
>> It isn't if you care about memory leaks. And also bare in mind
>> that some language constructs also relies on the GC, like some
>> dynamic arrays operations (concatenation and appending for
>> example). Right now is pretty hard to write D code that doesn't
>> really use the GC at all.
>
> There's also often no reason not to have the GC on and use it 
> for certain stuff
> but use ref-counted objects as your main type of 
> memory-management. Then, you
> avoid whatever issues the GC has in most cases but don't have 
> to worry about
> what it takes to be able to not use it at all. For instance, 
> arrays would
> probably be GC-allocated in general, since then you can use 
> slices and
> whatnot, but maybe all of your user-defined objects on the heap 
> could be
> malloced and freed with reference counts (though that sort of 
> thing should be
> much easier once we have custom allocators - it's a bit more of 
> a pain right
> now than it should be).
>
> - Jonathan M Davis
In my experience its quite some work to remove the GC entierly, 
but it is possible. Also it makes writing Code also easier ans 
more effektive in some places because:
-You don't have to register manually allocated blocks with the gc
-You control the exact order of destruction
-You control the time of destruction
-You control in which thread a object gets destructed
-You will be encouraged to not write code that does do 100 
allocations just to format a string (because it will leak)

But obviously it has an impact on productivity. But I'm still 
more productive writing manual memory managed code in D then in 
C++.

Kind Regards
Benjamin Thaut
December 31, 2012
Re: Smart pointers instead of GC?
On Tuesday, 25 December 2012 at 19:23:59 UTC, Jonathan M Davis 
wrote:
> There's also often no reason not to have the GC on and use it 
> for certain stuff

One thing that really freaks me out is the fact that the garbage 
collector pauses the whole process, i.e. all threads.

In my job I'm writing backend services that power a big web site. 
Perfomance is key, as the response time of the data service in 
most cases directly adds to the page load time. The bare 
possibility that the whole service pauses for, say, 100ms is 
making me feel very uncomfortable.

We easily achieve the performance and reliability we need in C++, 
but I would love to give D a chance, as it solves many 
inconveniences of C++ in an elegant way. Metaprogramming and the 
threading model, just to name two.

> For instance, arrays would probably be GC-allocated in general, 
> since
> then you can use slices and whatnot,

A smart-pointer type for arrays can easily provide slices. It 
keeps a reference to the full array (which gets destructed when 
the last reference is dropped), but addresses a subrange.

Thanks everyone for all the replies!
December 31, 2012
Re: Smart pointers instead of GC?
On Monday, 31 December 2012 at 12:14:22 UTC, Sven Over wrote:
> On Tuesday, 25 December 2012 at 19:23:59 UTC, Jonathan M Davis 
> wrote:
>> There's also often no reason not to have the GC on and use it 
>> for certain stuff
>
> One thing that really freaks me out is the fact that the 
> garbage collector pauses the whole process, i.e. all threads.
>
> In my job I'm writing backend services that power a big web 
> site. Perfomance is key, as the response time of the data 
> service in most cases directly adds to the page load time. The 
> bare possibility that the whole service pauses for, say, 100ms 
> is making me feel very uncomfortable.
>
> We easily achieve the performance and reliability we need in 
> C++, but I would love to give D a chance, as it solves many 
> inconveniences of C++ in an elegant way. Metaprogramming and 
> the threading model, just to name two.
>
>> For instance, arrays would probably be GC-allocated in 
>> general, since
>> then you can use slices and whatnot,
>
> A smart-pointer type for arrays can easily provide slices. It 
> keeps a reference to the full array (which gets destructed when 
> the last reference is dropped), but addresses a subrange.
>
> Thanks everyone for all the replies!

I think the problem would be if you try to append to the slice 
(~): If the underlying array is not GC allocated, then it will 
append to a new GC allocated array (AFAIK).

As long as you don't append, I'd say you are fine.
December 31, 2012
Re: Smart pointers instead of GC?
On Wednesday, 26 December 2012 at 11:43:55 UTC, Benjamin Thaut 
wrote:
> In my experience its quite some work to remove the GC entierly, 
> but it is possible.

Hi Benjamin! I've seen your druntime and thBase repos on GitHub. 
Very interesting stuff. As I have little (or to be honest: no) 
experience in D it's difficult for me to judge how much work 
needs to be done. Is the idea to replace Phobos completely?

> Also it makes writing Code also easier ans more effektive in 
> some places
> because:
> -You don't have to register manually allocated blocks with the 
> gc
> -You control the exact order of destruction
> -You control the time of destruction
> -You control in which thread a object gets destructed
> -You will be encouraged to not write code that does do 100 
> allocations just to format a string (because it will leak)

I personally agree with all your points, but I must admit that 
their relevance depends on the programming patterns you are 
using. Experienced GC programmers certainly favour patterns that 
work well with GC. I personally prefer a style of programming 
that makes use of destructors and relies on them being executed 
at the right time.

> But obviously it has an impact on productivity.

I'm not sure about this at all. I write software that is designed 
to offer a certain performance, and latency is a big issue. I 
don't believe that I can achieve that with a "just allocate 
memory and don't ever care" attitude. I would need some 
background knowledge about how the GC works, and write code that 
doesn't screw up badly. Manual memory management also requires me 
to have an understanding of the implications behind it. But there 
are well-tested helpers (e.g. smart pointers) that simplify the 
task a lot.

I do admit that, when performance (both in the sense of being 
fast and offering reliable performance) is not such a critical 
issue, then GC won't hurt and "just allocating" is fine and of 
course reduces development time. However, I tend to just use 
Python for this class problems.

It comes down to a matter of choice and personal preference. I'm 
not saying that GC is bad and shouldn't be used. Many people are 
very happy with it and it does a great job for them. But my 
personal preference is manual memory management (then again 
greatly automated by use of reference counting and smart 
pointers).
December 31, 2012
Re: Smart pointers instead of GC?
On Monday, 31 December 2012 at 12:36:06 UTC, monarch_dodra wrote:
> On Monday, 31 December 2012 at 12:14:22 UTC, Sven Over wrote:

>> A smart-pointer type for arrays can easily provide slices. It 
>> keeps a reference to the full array (which gets destructed 
>> when the last reference is dropped), but addresses a subrange.

> I think the problem would be if you try to append to the slice 
> (~): If the underlying array is not GC allocated, then it will 
> append to a new GC allocated array (AFAIK).
>
> As long as you don't append, I'd say you are fine.

Of course your smart-array-pointer type needs to implement the ~ 
operator and create a copy of the array in that case. I guess the 
full implementation would include something that would resemble 
that of std::vector in C++.

I should get started writing those types. Then I could either 
demonstrate that it does work, or understand that it doesn't (and 
why).
« First   ‹ Prev
1 2 3 4 5
Top | Discussion index | About this forum | D home