View mode: basic / threaded / horizontal-split · Log in · Help
January 09, 2013
Re: manual memory management
On 01/09/2013 09:54 AM, Mehrdad wrote:
> You (or Walter I guess) are the first person I've seen who calls C++ garbage
> collected.

Maybe not GC in the truest sense, but one of the first realizations I had when I 
started teaching myself C++ (having originally learned C) was, "Oh, gosh, I 
don't have to manually handle memory allocation/deallocation any more."

Of course, that's an overstatement in reality, but I still find that for almost 
all of the C++ code I've ever had to write, I've been able to avoid manual 
memory management.
January 09, 2013
Re: manual memory management
On Wednesday, 9 January 2013 at 02:49:34 UTC, Rob T wrote:
>
> I'm very surprised that not too many people have been screaming 
> for dynamic linking and runtime loading. It's very hard for me 
> to imagine not having the feature because it's so darn useful 
> and an essential feature if your strategy is to allow 3rd 
> parties to extend an application without hacking at the source 
> code.
>
> If there's another better way, I'd sure like to know about it!
>
> --rt

There were many people screaming about it.
Just there is nobody who could make it work.

Walter claimed that compiler is shared-lib ready, it is just 
druntime that is lacking. And he hasn't got knowledge to make it 
work on his own.

Sean Kelly is out - he was Walter's bet to make it work.

My hope was Martin Nowak, he was working on it but seems that he 
also got busy with other stuff
January 09, 2013
Re: manual memory management
Am 09.01.2013 12:39, schrieb nazriel:
> On Wednesday, 9 January 2013 at 02:49:34 UTC, Rob T wrote:
>>
>> I'm very surprised that not too many people have been screaming for
>> dynamic linking and runtime loading. It's very hard for me to imagine
>> not having the feature because it's so darn useful and an essential
>> feature if your strategy is to allow 3rd parties to extend an
>> application without hacking at the source code.
>>
>> If there's another better way, I'd sure like to know about it!
>>
>> --rt
>
> There were many people screaming about it.
> Just there is nobody who could make it work.
>
> Walter claimed that compiler is shared-lib ready, it is just druntime
> that is lacking. And he hasn't got knowledge to make it work on his own.
>
> Sean Kelly is out - he was Walter's bet to make it work.
>
> My hope was Martin Nowak, he was working on it but seems that he also
> got busy with other stuff

The compiler is not shared-lib ready. At least not on windows. It does 
not support exporting data symbols. E.g.

export uint g_myGlobal;

This is mostly a problem for "hidden" data symbols, like vtables, module 
info objects, type info objects and other stuff D relies on.

Druntime on windows does already handle everything else pefectly (e.g. 
threads and TLS)

Kind Regards
Benjamin Thaut
January 09, 2013
Re: manual memory management
On Wednesday, 9 January 2013 at 09:17:35 UTC, Dmitry Olshansky 
wrote:
> 09-Jan-2013 12:54, Mehrdad пишет:
>> On Wednesday, 9 January 2013 at 08:51:47 UTC, Paulo Pinto 
>> wrote:
>>> On Wednesday, 9 January 2013 at 08:28:44 UTC, Mehrdad wrote:
>>>> On Wednesday, 9 January 2013 at 08:14:35 UTC, Walter Bright 
>>>> wrote:
>>>>> On 1/8/2013 11:42 PM, Mehrdad wrote:
>>>>>> (True, it wouldn't give you the power of a systems 
>>>>>> language, but
>>>>>> that's quite
>>>>>> obviouly not my point -- the point is that it's a 
>>>>>> _perfectly possible_
>>>>>> memory-safe language which we made, so I don't understand 
>>>>>> Walter's
>>>>>> comment about
>>>>>> a GC being "required" for a memory-safe language.)
>>>>>
>>>>>
>>>>> The misunderstanding is you are not considering reference 
>>>>> counting
>>>>> as a form of GC. It is.
>>>>
>>>> So you would say that C++ code (which uses reference 
>>>> counting) uses
>>>> garbage collection?
>>>
>>> Yes.
>>
>> You (or Walter I guess) are the first person I've seen who 
>> calls C++
>> garbage collected.
>>
>
> That's a stretch - IMHO I'd call the language garbage collected 
> iff it is the default way language-wise (e.g. if C++ new was 
> ref-counted I'd call C++ garbage collected).
>
> This way D is garbage collected language because the language 
> default is GC.

I was being provocative on purpose.

Having said that, a C++ application where *_ptr<> + STL types are 
used everywhere can make C++ almost a safe language.

Unfortunately most C++ libraries make use of naked pointers.

..
Paulo
January 09, 2013
Re: manual memory management
On Wednesday, 9 January 2013 at 10:21:29 UTC, Mehrdad wrote:
> On Wednesday, 9 January 2013 at 10:14:07 UTC, dennis luehring 
> wrote:
>> Am 09.01.2013 11:09, schrieb Mehrdad:
>>> On Wednesday, 9 January 2013 at 10:07:42 UTC, deadalnix wrote:
>>>> Reference counting tend to create big pauses when 
>>>> deallocating
>>>> as objects tends to dies in group.
>>>
>>>
>>>
>>>
>>> I don't get it... it's slower to deallocate a bunch of objects
>>> together with refcounting than to deallocate all of them
>>> individually over a longer period of time?
>>>
>>
>> could be - think of an large hierarchy of objects which tends 
>> to take some time to deconstruct ... a background gc could be 
>> better for your application speed in this situation - by the 
>> cost of smaller pause and more resource usage
>
>
> Come to think of it, C++ allocators are meant for exactly this: 
> throwing away an entire batch of objects in 1 go. Beats GCs any 
> day.
>
>
>>
>> or better - what is the real reason for Java and C# for using 
>> garbage collectors if ref-counting will be always better 
>> (except cyclic stuff)?
>
> Pretty sure the only reason C#/Java use a GC _is_ for cyclic 
> stuff, and that's it.
>
>
> If you have any other reasons please show me a benchmark that 
> shows a GC being faster than the equivalent refcounted code 
> (I've seen lots of talks in theory about how it _COULD_ be 
> different but never seen any examples in practice; would love 
> to see one).

Reference counting always implies extra booking code per memory 
access, I fail to see how it can be made faster than any parallel 
GC.

The Aonix VM for example is used in military scenarios like 
missile radar systems and battleship gun's control systems, beats 
any game timing requirements, I would say.

--
Paulo
January 09, 2013
Re: manual memory management
On 1/9/13 4:25 AM, Benjamin Thaut wrote:
> The compiler is not shared-lib ready. At least not on windows. It does
> not support exporting data symbols. E.g.
>
> export uint g_myGlobal;
>
> This is mostly a problem for "hidden" data symbols, like vtables, module
> info objects, type info objects and other stuff D relies on.

Are there bugzilla entries for this?

Andrei
January 09, 2013
Re: manual memory management
On Wednesday, 9 January 2013 at 08:59:01 UTC, Jonathan M Davis 
wrote:
> [...] whereas all dynamic linking really does is save disk 
> space.

Saving on disk space is a minor advantage. The main advantage is 
allowing shared libs to be distributed without having to re-link 
then in manually. For example, if a bug is fixed in a shared lib, 
all applications automatically get the bug fix, but with 
statically linked libs, you have to re-link all the apps that use 
the lib to gain access to the bug fix. With static linking you 
also have no easy way to ensure that your apps are all using the 
most up-to-date version of a shared lib. Effectively, without 
dynamic linking, collections of applications, such as operating 
systems would be very difficult to deploy and maintain to the 
point of being impractical.

D is simply a whole lot less useful without full dynamic runtime 
linking.

--rt
January 09, 2013
Re: manual memory management
On Wednesday, 9 January 2013 at 11:24:32 UTC, Jonathan M Davis 
wrote:
> Walter wasn't arguing that there wasn't a place for manual 
> memory management.
> He was arguing that you can't guarantee memory safety if you're 
> using manual
> memory management - hence why malloc and free are @system. 
> @system code is not
> memory safe like @safe code is, but it still very much has it's 
> place.
>
> - Jonathan M Davis

You cannot guarantee memory safety with a GC either, depending on 
the definition of "memory safety".

For example, you can still access deallocated memory by mistake, 
run out of memory due to accumulating persistent pointers left 
around by mistake, or free memory that was not supposed to be 
freed by mistake. The GC implementation may fail due to bugs, 
deallocating live memory or failing to deallocate inactive memory.

The only thing a GC can do for you, is free up the programmer 
from the tedium of managing memory. It also allows constructs 
that otherwise would be very difficult or impractical to 
implement. The effect can be very positive, but there are no 
guarantees of memory safety.

I also doubt that a "one size fits" all approach to garbage 
collection will ever satisfy everyone. What's needed is the 
ability to both manage memory manually and automatically (D 
allows this which is good), but also allow for the automated 
methods to be easily replaced (plug-ins come to mind), and if 
possible allow sections of code to be managed by completely 
different garbage collector implementations that are designed to 
serve different purposes (this may or may not be practical to do).

--rt
January 09, 2013
Re: manual memory management
On Wed, Jan 09, 2013 at 08:33:23AM +0100, Rob T wrote:
[...]
> There is a point being made here that is perfectly valid. There is a
> form of memory leak that a GC can never catch, such as when when
> memory is allocated and simply never deallocated by mistake due to a
> persistent "in use" pointer that should have been nulled but wasn't.

No form of memory management will be immune to programmer error. That's
plain impossible. You can write broken code in any language, under any
kind of memory management system. Short of going heapless, nothing is
going to help you here.

(Actually, even going heapless won't help you -- think of what happens
if you have a fixed-sized stack and forget to pop used-up elements. Yes
such a bug is way more obvious than a pointer that didn't get nulled,
but both are still just bugs.)


> In addition, the GC itself may fail to deallocated freed memory or
> even free live memory by mistake. I've seen bugs described to that
> effect.

I thought the idea was to make it so that SafeD is free from this kind
of problem (short of a bug in the GC itself) -- AFAIK, the GC freeing
live memory is caused by having XOR'ed pointers or other such tricks
that cause the GC to fail to detect the pointer to the memory. In
theory, SafeD should prevent this by not allowing unsafe operations on
pointers like XOR'ing. SafeD still has a long ways to go, though.


> There simply is no panacea to the memory leak problem. What
> a GC does do, is free the programmer from a ton of tedium, and even
> allow for constructs that would normally not be practical to
> implement, but it can never guarantee anything more than that.
[...]

Yes. I don't think there are many things in the programming world that
can be guaranteed. The compiler would have to be clairvoyant (not merely
pass the Turing test) to catch these sorts of errors.


T

-- 
To provoke is to call someone stupid; to argue is to call each other stupid.
January 09, 2013
Re: manual memory management
On Wed, Jan 09, 2013 at 06:26:55PM +0100, Rob T wrote:
> On Wednesday, 9 January 2013 at 11:24:32 UTC, Jonathan M Davis
> wrote:
> >Walter wasn't arguing that there wasn't a place for manual memory
> >management.  He was arguing that you can't guarantee memory safety if
> >you're using manual memory management - hence why malloc and free are
> >@system. @system code is not memory safe like @safe code is, but it
> >still very much has it's place.
> >
> >- Jonathan M Davis
> 
> You cannot guarantee memory safety with a GC either, depending on
> the definition of "memory safety".
> 
> For example, you can still access deallocated memory by mistake,

Are you sure about this? Short of doing pointer arithmetic (which is
unsafe by definition), I don't see how you can do this. Where would you
get the pointer to that memory from? It has to be stored somewhere,
meaning the GC will not deallocate the memory pointed to.


> run out of memory due to accumulating persistent pointers left around
> by mistake,

That is not unsafe. That just means you run out of memory and get an
OutOfMemory exception or the OS kills your process. No safety issue
there.


> or free memory that was not supposed to be freed by mistake.

How? The GC, by definition, doesn't free the memory unless nothing is
pointing to it.


> The GC implementation may fail due to bugs, deallocating live memory
> or failing to deallocate inactive memory.

You're conflating the theory of GCs and its implementation, which is
prone to bugs. By that argument, you might as well say that there is no
such thing as memory safety, because implementations are always prone to
bugs. Your RAM could catch fire, for example. There's nothing you can do
in software to prevent that. That's not a helpful argument, though. The
question at hand is, given a GC, which presumably has been thoroughly
debugged, does it guarantee memory safety?

I think it does, because as long as there's a pointer to the memory
left, the GC will not collect it, and so you can't use the pointer to
access invalid memory. If there are no more pointers to that memory,
then by definition you have nothing to access that memory with. Like I
said, you have to get the pointer from *somewhere*. As long as the
pointer is somewhere, the GC will find it, and mark the memory as used,
so it won't be collected. If there are no pointers left, you also have
no way to access that memory anymore. So you can never access invalid
memory. It's very straightforward.

This memory safety can be broken only if you allow pointer arithmetic
and casts to/from pointers. But everyone knows that pointer arithmetic
is unsafe by definition; I don't think we're talking about that here.

[...]
> I also doubt that a "one size fits" all approach to garbage
> collection will ever satisfy everyone.

Of course not. That's why the game devs on this list have been doing
GC-less D programming. :-)

But that also means they have to either preallocate everything, or they
have to manually manage the memory (with malloc/free or equivalent),
which is unsafe, because it's very easy to free some memory while there
are still pointers to it left.


> What's needed is the ability to both manage memory manually and
> automatically (D allows this which is good), but also allow for the
> automated methods to be easily replaced (plug-ins come to mind), and
> if possible allow sections of code to be managed by completely
> different garbage collector implementations that are designed to serve
> different purposes (this may or may not be practical to do).
[...]

I don't know about the practicality of using multiple GCs in a single
app, but certainly the ability to choose from a number of alternative GC
implementations at compile-time would be very welcome. You could have a
GC that has high throughput but introduces intermittent long pauses, and
another GC that has lower throughput but has no pauses longer than, say,
0.1 seconds. Then at compile-time you choose one, depending on what you
need. An interactive app will choose the second, but a batch processing
app will benefit from the first.


T

-- 
Why waste time learning, when ignorance is instantaneous? -- Hobbes, from Calvin & Hobbes
6 7 8 9 10 11 12 13 14
Top | Discussion index | About this forum | D home