On Monday, 27 February 2012 at 09:33:32 UTC, so wrote:
> On Monday, 27 February 2012 at 08:39:54 UTC, Paulo Pinto wrote:
>> I keep bringing this issues, because I am a firm believer that when
>> people that fight against a GC are just fighting a lost battle.
>>
>> Like back in the 80's people were fighting against Pascal or C versus
>> Assembly. Or in the 90' were fighting against C++ versus C.
>>
>> Now C++ is even used for operating systems, BeOS, Mac OS X drivers,
>> COM/WinRT.
>
> It is not a fair analogy. Unlike MMM and GC, C++ can do everything C can do and has more sugar. What they are argue i think whether or not OO is a solution to everything and troubles with its implementation in C++.
>
>> Sure a systems programming language needs some form of manual memory
>> management for "exceptional situations", but 90% of the time you will
>> be allocating either referenced counted or GCed memory.
>>
>> What will you do when the major OS use a systems programming language like forces GC or reference counting on you do? Which is already slowly happening with GC and ARC on Mac OS X, WinRT on Windows 8, mainstream OS, as well as the Oberon, Spin, Mirage, Home, Inferno and Singularity research OSs.
>>
>> Create your own language to allow you to live in the past?
>>
>> People that refuse to adapt to times stay behind, those who adapt, find ways to profit from the new reality.
>>
>> But as I said before, that is my opinion and as a simple human is also prone to errors. Maybe my ideas regarding memory management in systems languages are plain wrong, the future will tell.
>>
>> --
>> Paulo
>
> As i said in many threads regarding GC and MMM, it is not about this vs that.
> There should be no religious stances. Both have their strengths and failures.
> What every single discussion on this boils down to is some people downplay the failures of their religion :)
>
> And that staying behind thing is something i never understand!
> It is a hype, it is marketing! To sell you a product that doesn't deserve half its price!
>
> Religion/Irrationality has no place in what we do. Show me a better tool, "convince me" it is better and i will be using that tool. I don't give a damn if it is D or vim i am leaving behind.

I also don't have any problem with tools, what matters is what the customer wants and with what I can make him happy, not what the tool flavor of the month is.

Personally, even thought it might come differently in my posts, I also believe there are situations where MMM is much better than GC. Or where GC is not even feasible.

The thing is, regardless of what anyone might think, the major desktop OS are integrating reference counting and GC at the kernel level. Now when the systems programming languages that are part of these OS, offer these types of memory management, there is no way to choose otherwise, even if one so wishes.

--
Paulo

On Mon, 27 Feb 2012 09:32:27 +0100, Timon Gehr <timon.gehr@gmx.ch> wrote:

> On 02/27/2012 02:29 AM, Martin Nowak wrote:
>> On Sun, 26 Feb 2012 20:26:41 +0100, Walter Bright
>> <newshound2@digitalmars.com> wrote:
>>
>>> On 2/26/2012 7:04 AM, Simon wrote:
>>>> On 26/02/2012 03:22, Walter Bright wrote:
>>>>> On 2/25/2012 4:01 PM, Simon wrote:
>>>>>> On 25/02/2012 22:55, Walter Bright wrote:
>>>>>>> Enter C++'s shared_ptr. But that works by, for each object,
>>>>>>> allocating a
>>>>>>> *second* chunk of memory to hold the reference count. Right off
>>>>>>> the bat,
>>>>>>> you've got twice as many allocations & frees with shared_ptr than
>>>>>>> a GC
>>>>>>> would have.
>>>>>>
>>>>>> http://www.boost.org/doc/libs/1_43_0/libs/smart_ptr/make_shared.html
>>>>>>
>>>>>> so you don't have to have twice as many allocations.
>>>>>
>>>>> There are many ways to do shared pointers, including one where the
>>>>> reference count is part of the object being allocated. But the C++11
>>>>> standard share_ptr does an extra allocation.
>>>>
>>>> The stl one is based on boost, so it has make_shared as well:
>>>>
>>>> http://en.cppreference.com/w/cpp/memory/shared_ptr
>>>>
>>>> and it's in vs 2010
>>>>
>>>> http://msdn.microsoft.com/en-us/library/ee410595.aspx
>>>>
>>>> Not that I'm claiming shared pointers are superior to GC.
>>>
>>> At the GoingNative C++ conference, the guy who is in charge of STL for
>>> VS said that it did an extra allocation for the reference count.
>>
>> It's actually quite nice to combine unique_ptr and shared_ptr.
>> One can lazily create the refcount only when the pointers are shared.
>> Often one can get away with unique ownership.
>
> Ok. Btw, if the utility is in charge of allocation, then the refcount can be allocated together with the storage.
>
Yeah, I used the pointer for the deleter in the unshared case.
Assuming that the allocator interface will require a delegate
callback and not a function it might make more sense to directly
stuff it onto the heap.
There is one caveat with make_shared, weak_ptrs will keep your object
alive because they need the control block. One could do a realloc though.

>>
>> https://gist.github.com/1920202
>
> Neat. Possible improvement (if I understand your code correctly): Don't add the GC range if all possible aliasing is through Ptr.

I hope I did that. Or do you mean when holding a class. I think it's needed
for classes because of the monitor, not sure though.

On 02/27/2012 03:42 PM, Martin Nowak wrote:
...
>>> https://gist.github.com/1920202
>>
>> Neat. Possible improvement (if I understand your code correctly):
>> Don't add the GC range if all possible aliasing is through Ptr.
>
> I hope I did that.

import smart_ptr;

struct S{
    Ptr!int a;
    Ptr!double b;
}

static assert(hasAliasing!S);

Therefore, I think your code will add the storage of Ptr!S to the GC, even though it manages all its pointers manually.


> Or do you mean when holding a class. I think it's needed
> for classes because of the monitor, not sure though.

Is it needed for unshared class instances?

On Tue, 28 Feb 2012 09:30:12 +0100, Timon Gehr <timon.gehr@gmx.ch> wrote:

> On 02/27/2012 03:42 PM, Martin Nowak wrote:
> ...
>>>> https://gist.github.com/1920202
>>>
>>> Neat. Possible improvement (if I understand your code correctly):
>>> Don't add the GC range if all possible aliasing is through Ptr.
>>
>> I hope I did that.
>
> import smart_ptr;
>
> struct S{
>      Ptr!int a;
>      Ptr!double b;
> }
>
> static assert(hasAliasing!S);
>
All right, I would need a Ptr aware hasAliasing, that's true.
That's one of the few places where C++'s lookup rules for template
specializations are useful. std.traits.hasAliasing already has Rebindable hacks.

Either way I had an idea to split this into two parts.
- unique_ptr which handles ownership and destruction, could be Unique!(Scoped!T) as well.
- A move-on-shared wrapper, that has value semantics unless it gets shared, in which
  case the value is moved to a refcounted heap store.

As a (not as experienced as other) game developer, I'd like to share my
point of view (again).

1. smart pointer vs GC war isn't the point, it's a knowledge &
implementation war and isn't related to languages that allow both (as C++ &
D).
2. C++ base code is just there and ready to be used. Only
begining-from-scratch projects can afford to use D
3. D isn't available directly to console hardware. If it is in some ways,
it isn't from the point of view of the vendor.
4. Learning a language have a cost. Any game development company will
totally dodge this kind of cost as much as it can, even if it's more
expensive to stay in the current language. (also D benefit over C++ isn't
really clear at this very moment)
5. STL isn't a problem anymore, most of the time, see:
http://gamedev.stackexchange.com/questions/268/stl-for-games-yea-or-nay
    My experience with STL in console (NDS) games: it's fine if you know
what you're doing (as someone else already mentionned)


I would totally use D for new (home or indie) game project but any company with history or that wants to be considered serious from console vendor perspective would use languages proposed by console vendors.

So, to help:

1. make cross-platform game development easier at least for accessible
platforms like PC & mobile (that one is "easy" I think)
2. convince game platform vendors to say "you can use D" - at least
3. find a way to "show" what we call a "killer app". That's the only way to
convince game company developers of considering another technology.

Joël Lamotte - Klaim

On Saturday, 25 February 2012 at 22:55:37 UTC, Walter Bright
wrote:
> Most straight up GC vs malloc/free benchmarks miss something crucial. A GC allows one to do substantially *fewer* allocations. It's a lot faster to not allocate than to allocate.

The trouble is you don't have to look far to find managed apps
that visibly pause at random due to garbage collection. I've
personally never come across this behaviour with a program that
manages its own memory.

> Consider C strings. You need to keep track of ownership of it. That often means creating extra copies, rather than sharing a single copy.
>
> Enter C++'s shared_ptr. But that works by, for each object, allocating a *second* chunk of memory to hold the reference count. Right off the bat, you've got twice as many allocations & frees with shared_ptr than a GC would have.

Or, you could use std::make_shared().

Given that even C++ struggles to be accepted as a systems
programming language, I can't understand why garbage collection
is used even in the phobos library. If systems programmers think
that C++ has too much overhead, they sure as hell aren't going to
be happy with garbage collection in the low level libraries.

It's an interesting point that hard real-time programs allocate
all their memory up front, mitigating the entire issue, but
obviously this is at the extreme end of the scale.

Regards,

Ben

On 26 February 2012 00:55, Walter Bright <newshound2@digitalmars.com> wrote:

> On 2/25/2012 2:08 PM, Paulo Pinto wrote:
>
>> Most standard compiler malloc()/free() implementations are actually
>> slower than
>> most advanced GC algorithms.
>>
>
> Most straight up GC vs malloc/free benchmarks miss something crucial. A GC allows one to do substantially *fewer* allocations. It's a lot faster to not allocate than to allocate.
>

Do you really think that's true? Are there any statistics to support that? I'm extremely sceptical of this claim.

I would have surely thought using a GC leads to a significant *increase* in
allocations for a few reasons:
  It's easy to allocate, ie, nothing to discourage you
  It's easy to clean up - you don't have to worry about cleanup problems,
makes it simpler to use in many situations
  Dynamic arrays are easy - many C++ users will avoid dynamic arrays
because the explicit allocation/clean up implies complexity, one will
always use the stack, or a fixed array where they can get away with it
  Slicing, concatenation, etc performs bucket loads of implicit GC
allocations
  Strings... - C coders who reject the stl will almost always have a
separate string heap with very particular allocation patterns, and almost
always refcounted
  Phobos/druntine allocate liberally - the CRT almost never allocates

This is my single biggest fear in D. I have explicit control within my own
code, but I wonder if many D libraries will be sloppy and over-allocate all
over the place, and be generally unusable in many applications.
If D is another language like C where the majority of libraries (including
the standard libraries I fear) are unusable in various contexts, then that
kinda defeats the purpose. D's module system is one of its biggest selling
points.

I think there should be strict phobos allocation policies, and ideally, druntime should NEVER allocate if it can help it.


Consider C strings. You need to keep track of ownership of it. That often
> means creating extra copies, rather than sharing a single copy.
>

Rubbish, strings are almost always either refcounted or on the stack for dynamic strings, or have fixed memory allocated within structures. I don't think I've ever seen someone duplicating strings into separate allocations liberally.



> Enter C++'s shared_ptr. But that works by, for each object, allocating a *second* chunk of memory to hold the reference count. Right off the bat, you've got twice as many allocations & frees with shared_ptr than a GC would have.
>

Who actually uses shared_ptr? Talking about the stl is misleading... an overwhelming number of C/C++ programmers avoid the stl like the plague (for these exact reasons). Performance oriented programmers rarely use STL out of the box, and that's what we're talking about here right? If you're not performance oriented, then who cares about the GC either?

Manu:

>   It's easy to allocate, ie, nothing to discourage you

This is partially a cultural thing, and it's partially caused by what the language offers you. I have seen Ada code where most things are stack-allocated, because doing this is handy in Ada.
In D stack-allocated variable-length arrays will help move some heap allocations to the stack. And the usage of scoped class instances has to improve some more.


>   Strings... - C coders who reject the stl will almost always have a
> separate string heap with very particular allocation patterns, and almost
> always refcounted

I think in future D will be free to have a special heap for strings. I think that in D source code there is already enough semantics to do this. There is one person that is working on the D GC, so maybe he's interested about this.


> an overwhelming number of C/C++ programmers avoid the stl like the plague (for
> these exact reasons). Performance oriented programmers rarely use STL out
> of the box,

Often I have heard the opposite claims too. Like in the recent GoingNative2012 conference.

Bye,
bearophile

On 03/06/2012 01:27 PM, Manu wrote:
> On 26 February 2012 00:55, Walter Bright <newshound2@digitalmars.com
> <mailto:newshound2@digitalmars.com>> wrote:
>
>     On 2/25/2012 2:08 PM, Paulo Pinto wrote:
>
>         Most standard compiler malloc()/free() implementations are
>         actually slower than
>         most advanced GC algorithms.
>
>
>     Most straight up GC vs malloc/free benchmarks miss something
>     crucial. A GC allows one to do substantially *fewer* allocations.
>     It's a lot faster to not allocate than to allocate.
>
>
> Do you really think that's true? Are there any statistics to support that?
> I'm extremely sceptical of this claim.
>
> I would have surely thought using a GC leads to a significant *increase*
> in allocations for a few reasons:
>    It's easy to allocate, ie, nothing to discourage you

If you believe this, why do you raise this issue?

>    It's easy to clean up - you don't have to worry about cleanup
> problems, makes it simpler to use in many situations

GC does not prevent memory leaks, it does not support deterministic cleanup, and most implementations perform poorly on certain workloads. You were saying?

>    Dynamic arrays are easy - many C++ users will avoid dynamic arrays
> because the explicit allocation/clean up implies complexity, one will
> always use the stack, or a fixed array where they can get away with it
>    Slicing,

Slicing does never allocate.

> concatenation, etc performs bucket loads of implicit GC
> allocations

a~b

Nothing implicit about that.

The only case where memory allocation is implicit is for closures.


>    Strings... - C coders who reject the stl will almost always have a
> separate string heap with very particular allocation patterns, and
> almost always refcounted
>    Phobos/druntine allocate liberally - the CRT almost never allocates
>
> This is my single biggest fear in D. I have explicit control within my
> own code, but I wonder if many D libraries will be sloppy and
> over-allocate all over the place, and be generally unusable in many
> applications.

IMHO this fear is unjustified. If the library developers are that sloppy, chances are that the library is not worth using, even when leaving all memory allocation concerns away. (It is likely that you aren't the only programmer familiar with some of the issues.)

> If D is another language like C where the majority of libraries
> (including the standard libraries I fear) are unusable in various
> contexts, then that kinda defeats the purpose. D's module system is one
> of its biggest selling points.
>
> I think there should be strict phobos allocation policies,

Yes, a function that does not obviously need to allocate shouldn't, and if possible there should be alternatives that do not allocate. Do you have any particular examples where such a policy would be violated in Phobos?

> and ideally, druntime should NEVER allocate if it can help it.
>

+1. What are examples of unnecessary allocations in druntime?

>
>     Consider C strings. You need to keep track of ownership of it. That
>     often means creating extra copies, rather than sharing a single copy.
>
>
> Rubbish, strings are almost always either refcounted

Technically, refcounting is a form of GC.

> or on the stack for
> dynamic strings, or have fixed memory allocated within structures. I
> don't think I've ever seen someone duplicating strings into separate
> allocations liberally.

It is impossible to slice a zero-terminated string without copying it in the general case and refcounting slices is not trivial.


>
>     Enter C++'s shared_ptr. But that works by, for each object,
>     allocating a *second* chunk of memory to hold the reference count.
>     Right off the bat, you've got twice as many allocations & frees with
>     shared_ptr than a GC would have.
>
>
> Who actually uses shared_ptr? Talking about the stl is misleading... an
> overwhelming number of C/C++ programmers avoid the stl like the plague
> (for these exact reasons). Performance oriented programmers rarely use
> STL out of the box, and that's what we're talking about here right?

Possibly now you are the one who is to provide supporting statistics.

> If you're not performance oriented, then who cares about the GC either?

There is a difference between not performance oriented and performance agnostic. Probably everyone cares about performance to some extent.

On 06.03.2012 16:27, Manu wrote:
> On 26 February 2012 00:55, Walter Bright <newshound2@digitalmars.com
> <mailto:newshound2@digitalmars.com>> wrote:
>
>     On 2/25/2012 2:08 PM, Paulo Pinto wrote:
>
>         Most standard compiler malloc()/free() implementations are
>         actually slower than
>         most advanced GC algorithms.
>
>
>     Most straight up GC vs malloc/free benchmarks miss something
>     crucial. A GC allows one to do substantially *fewer* allocations.
>     It's a lot faster to not allocate than to allocate.
>
>
> Do you really think that's true? Are there any statistics to support that?
> I'm extremely sceptical of this claim.
>
> I would have surely thought using a GC leads to a significant *increase*
> in allocations for a few reasons:
>    It's easy to allocate, ie, nothing to discourage you
>    It's easy to clean up - you don't have to worry about cleanup
> problems, makes it simpler to use in many situations
>    Dynamic arrays are easy - many C++ users will avoid dynamic arrays
> because

you mean like new[] ? That's just gloried allocation. STL vector is a dynamic array of C++ and it's being used a lot.

the explicit allocation/clean up implies complexity, one will
> always use the stack,
 or a fixed array where they can get away with it

Still possible

>    Slicing,
Nope

concatenation, etc performs bucket loads of implicit GC
> allocations

It sure does, like it does without GC with reallocs and refcounting.

>    Strings... - C coders who reject the stl will almost always have a
> separate string heap with very particular allocation patterns, and
> almost always refcounted

>    Phobos/druntine allocate liberally - the CRT almost never allocates

It's just most of CRT has incredibly bad usability, partly because it lack _any_ notion of allocators. And policy of using statically allocated shared data like in localtime, srand etc. shown remarkably bad M-T scalability.

> This is my single biggest fear in D. I have explicit control within my
> own code, but I wonder if many D libraries will be sloppy and
> over-allocate all over the place, and be generally unusable in many
> applications.
> If D is another language like C where the majority of libraries
> (including the standard libraries I fear) are unusable in various
> contexts, then that kinda defeats the purpose. D's module system is one
> of its biggest selling points.
>
> I think there should be strict phobos allocation policies, and ideally,
> druntime should NEVER allocate if it can help it.
>

>
>     Consider C strings. You need to keep track of ownership of it. That
>     often means creating extra copies, rather than sharing a single copy.
>
>
> Rubbish, strings are almost always either refcounted

like COW?

or on the stack for
> dynamic strings, or have fixed memory allocated within structures. I
> don't think I've ever seen someone duplicating strings into separate
> allocations liberally.

I've seen some, and sometimes memory corruption when people hesitated to copy even when they *do* need to pass a copy.

>
>     Enter C++'s shared_ptr. But that works by, for each object,
>     allocating a *second* chunk of memory to hold the reference count.
>     Right off the bat, you've got twice as many allocations & frees with
>     shared_ptr than a GC would have.
>
>
> Who actually uses shared_ptr?

Like everybody? Though with c++11 move semantics a unique_ptr is going to lessen it's widespread use. And there are ways to spend less then 2 proper memory allocations per shared_ptr, like keeping special block allocator for ref-counters.
More importantly smart pointers are here to stay in C++.

Talking about the stl is misleading... an
> overwhelming number of C/C++ programmers avoid the stl like the plague
> (for these exact reasons).

Instead of using it properly. It has a fair share of failures but it's not that bad.

Performance oriented programmers rarely use
> STL out of the box, and that's what we're talking about here right? If
> you're not performance oriented, then who cares about the GC either?


-- 
Dmitry Olshansky