View mode: basic / threaded / horizontal-split · Log in · Help
February 27, 2012
Re: John Carmack applauds D's pure attribute
On Monday, 27 February 2012 at 09:33:32 UTC, so wrote:
> On Monday, 27 February 2012 at 08:39:54 UTC, Paulo Pinto wrote:
>> I keep bringing this issues, because I am a firm believer that 
>> when
>> people that fight against a GC are just fighting a lost battle.
>>
>> Like back in the 80's people were fighting against Pascal or C 
>> versus
>> Assembly. Or in the 90' were fighting against C++ versus C.
>>
>> Now C++ is even used for operating systems, BeOS, Mac OS X 
>> drivers,
>> COM/WinRT.
>
> It is not a fair analogy. Unlike MMM and GC, C++ can do 
> everything C can do and has more sugar. What they are argue i 
> think whether or not OO is a solution to everything and 
> troubles with its implementation in C++.
>
>> Sure a systems programming language needs some form of manual 
>> memory
>> management for "exceptional situations", but 90% of the time 
>> you will
>> be allocating either referenced counted or GCed memory.
>>
>> What will you do when the major OS use a systems programming 
>> language like forces GC or reference counting on you do? Which 
>> is already slowly happening with GC and ARC on Mac OS X, WinRT 
>> on Windows 8, mainstream OS, as well as the Oberon, Spin, 
>> Mirage, Home, Inferno and Singularity research OSs.
>>
>> Create your own language to allow you to live in the past?
>>
>> People that refuse to adapt to times stay behind, those who 
>> adapt, find ways to profit from the new reality.
>>
>> But as I said before, that is my opinion and as a simple human 
>> is also prone to errors. Maybe my ideas regarding memory 
>> management in systems languages are plain wrong, the future 
>> will tell.
>>
>> --
>> Paulo
>
> As i said in many threads regarding GC and MMM, it is not about 
> this vs that.
> There should be no religious stances. Both have their strengths 
> and failures.
> What every single discussion on this boils down to is some 
> people downplay the failures of their religion :)
>
> And that staying behind thing is something i never understand!
> It is a hype, it is marketing! To sell you a product that 
> doesn't deserve half its price!
>
> Religion/Irrationality has no place in what we do. Show me a 
> better tool, "convince me" it is better and i will be using 
> that tool. I don't give a damn if it is D or vim i am leaving 
> behind.

I also don't have any problem with tools, what matters is what 
the customer wants and with what I can make him happy, not what 
the tool flavor of the month is.

Personally, even thought it might come differently in my posts, I 
also believe there are situations where MMM is much better than 
GC. Or where GC is not even feasible.

The thing is, regardless of what anyone might think, the major 
desktop OS are integrating reference counting and GC at the 
kernel level. Now when the systems programming languages that are 
part of these OS, offer these types of memory management, there 
is no way to choose otherwise, even if one so wishes.

--
Paulo
February 27, 2012
Re: John Carmack applauds D's pure attribute
On Mon, 27 Feb 2012 09:32:27 +0100, Timon Gehr <timon.gehr@gmx.ch> wrote:

> On 02/27/2012 02:29 AM, Martin Nowak wrote:
>> On Sun, 26 Feb 2012 20:26:41 +0100, Walter Bright
>> <newshound2@digitalmars.com> wrote:
>>
>>> On 2/26/2012 7:04 AM, Simon wrote:
>>>> On 26/02/2012 03:22, Walter Bright wrote:
>>>>> On 2/25/2012 4:01 PM, Simon wrote:
>>>>>> On 25/02/2012 22:55, Walter Bright wrote:
>>>>>>> Enter C++'s shared_ptr. But that works by, for each object,
>>>>>>> allocating a
>>>>>>> *second* chunk of memory to hold the reference count. Right off
>>>>>>> the bat,
>>>>>>> you've got twice as many allocations & frees with shared_ptr than
>>>>>>> a GC
>>>>>>> would have.
>>>>>>
>>>>>> http://www.boost.org/doc/libs/1_43_0/libs/smart_ptr/make_shared.html
>>>>>>
>>>>>> so you don't have to have twice as many allocations.
>>>>>
>>>>> There are many ways to do shared pointers, including one where the
>>>>> reference count is part of the object being allocated. But the C++11
>>>>> standard share_ptr does an extra allocation.
>>>>
>>>> The stl one is based on boost, so it has make_shared as well:
>>>>
>>>> http://en.cppreference.com/w/cpp/memory/shared_ptr
>>>>
>>>> and it's in vs 2010
>>>>
>>>> http://msdn.microsoft.com/en-us/library/ee410595.aspx
>>>>
>>>> Not that I'm claiming shared pointers are superior to GC.
>>>
>>> At the GoingNative C++ conference, the guy who is in charge of STL for
>>> VS said that it did an extra allocation for the reference count.
>>
>> It's actually quite nice to combine unique_ptr and shared_ptr.
>> One can lazily create the refcount only when the pointers are shared.
>> Often one can get away with unique ownership.
>
> Ok. Btw, if the utility is in charge of allocation, then the refcount  
> can be allocated together with the storage.
>
Yeah, I used the pointer for the deleter in the unshared case.
Assuming that the allocator interface will require a delegate
callback and not a function it might make more sense to directly
stuff it onto the heap.
There is one caveat with make_shared, weak_ptrs will keep your object
alive because they need the control block. One could do a realloc though.

>>
>> https://gist.github.com/1920202
>
> Neat. Possible improvement (if I understand your code correctly): Don't  
> add the GC range if all possible aliasing is through Ptr.

I hope I did that. Or do you mean when holding a class. I think it's needed
for classes because of the monitor, not sure though.
February 28, 2012
Re: John Carmack applauds D's pure attribute
On 02/27/2012 03:42 PM, Martin Nowak wrote:
...
>>> https://gist.github.com/1920202
>>
>> Neat. Possible improvement (if I understand your code correctly):
>> Don't add the GC range if all possible aliasing is through Ptr.
>
> I hope I did that.

import smart_ptr;

struct S{
    Ptr!int a;
    Ptr!double b;
}

static assert(hasAliasing!S);

Therefore, I think your code will add the storage of Ptr!S to the GC, 
even though it manages all its pointers manually.


> Or do you mean when holding a class. I think it's needed
> for classes because of the monitor, not sure though.

Is it needed for unshared class instances?
February 28, 2012
Re: John Carmack applauds D's pure attribute
On Tue, 28 Feb 2012 09:30:12 +0100, Timon Gehr <timon.gehr@gmx.ch> wrote:

> On 02/27/2012 03:42 PM, Martin Nowak wrote:
> ...
>>>> https://gist.github.com/1920202
>>>
>>> Neat. Possible improvement (if I understand your code correctly):
>>> Don't add the GC range if all possible aliasing is through Ptr.
>>
>> I hope I did that.
>
> import smart_ptr;
>
> struct S{
>      Ptr!int a;
>      Ptr!double b;
> }
>
> static assert(hasAliasing!S);
>
All right, I would need a Ptr aware hasAliasing, that's true.
That's one of the few places where C++'s lookup rules for template
specializations are useful. std.traits.hasAliasing already has Rebindable  
hacks.

Either way I had an idea to split this into two parts.
- unique_ptr which handles ownership and destruction, could be  
Unique!(Scoped!T) as well.
- A move-on-shared wrapper, that has value semantics unless it gets  
shared, in which
  case the value is moved to a refcounted heap store.
February 28, 2012
Re: John Carmack applauds D's pure attribute
As a (not as experienced as other) game developer, I'd like to share my
point of view (again).

1. smart pointer vs GC war isn't the point, it's a knowledge &
implementation war and isn't related to languages that allow both (as C++ &
D).
2. C++ base code is just there and ready to be used. Only
begining-from-scratch projects can afford to use D
3. D isn't available directly to console hardware. If it is in some ways,
it isn't from the point of view of the vendor.
4. Learning a language have a cost. Any game development company will
totally dodge this kind of cost as much as it can, even if it's more
expensive to stay in the current language. (also D benefit over C++ isn't
really clear at this very moment)
5. STL isn't a problem anymore, most of the time, see:
http://gamedev.stackexchange.com/questions/268/stl-for-games-yea-or-nay
   My experience with STL in console (NDS) games: it's fine if you know
what you're doing (as someone else already mentionned)


I would totally use D for new (home or indie) game project but any company
with history or that wants to be considered serious from console vendor
perspective would use languages proposed by console vendors.

So, to help:

1. make cross-platform game development easier at least for accessible
platforms like PC & mobile (that one is "easy" I think)
2. convince game platform vendors to say "you can use D" - at least
3. find a way to "show" what we call a "killer app". That's the only way to
convince game company developers of considering another technology.

Joël Lamotte - Klaim
March 06, 2012
Re: John Carmack applauds D's pure attribute
On Saturday, 25 February 2012 at 22:55:37 UTC, Walter Bright
wrote:
> Most straight up GC vs malloc/free benchmarks miss something 
> crucial. A GC allows one to do substantially *fewer* 
> allocations. It's a lot faster to not allocate than to allocate.

The trouble is you don't have to look far to find managed apps
that visibly pause at random due to garbage collection. I've
personally never come across this behaviour with a program that
manages its own memory.

> Consider C strings. You need to keep track of ownership of it. 
> That often means creating extra copies, rather than sharing a 
> single copy.
>
> Enter C++'s shared_ptr. But that works by, for each object, 
> allocating a *second* chunk of memory to hold the reference 
> count. Right off the bat, you've got twice as many allocations 
> & frees with shared_ptr than a GC would have.

Or, you could use std::make_shared().

Given that even C++ struggles to be accepted as a systems
programming language, I can't understand why garbage collection
is used even in the phobos library. If systems programmers think
that C++ has too much overhead, they sure as hell aren't going to
be happy with garbage collection in the low level libraries.

It's an interesting point that hard real-time programs allocate
all their memory up front, mitigating the entire issue, but
obviously this is at the extreme end of the scale.

Regards,

Ben
March 06, 2012
Re: John Carmack applauds D's pure attribute
On 26 February 2012 00:55, Walter Bright <newshound2@digitalmars.com> wrote:

> On 2/25/2012 2:08 PM, Paulo Pinto wrote:
>
>> Most standard compiler malloc()/free() implementations are actually
>> slower than
>> most advanced GC algorithms.
>>
>
> Most straight up GC vs malloc/free benchmarks miss something crucial. A GC
> allows one to do substantially *fewer* allocations. It's a lot faster to
> not allocate than to allocate.
>

Do you really think that's true? Are there any statistics to support that?
I'm extremely sceptical of this claim.

I would have surely thought using a GC leads to a significant *increase* in
allocations for a few reasons:
 It's easy to allocate, ie, nothing to discourage you
 It's easy to clean up - you don't have to worry about cleanup problems,
makes it simpler to use in many situations
 Dynamic arrays are easy - many C++ users will avoid dynamic arrays
because the explicit allocation/clean up implies complexity, one will
always use the stack, or a fixed array where they can get away with it
 Slicing, concatenation, etc performs bucket loads of implicit GC
allocations
 Strings... - C coders who reject the stl will almost always have a
separate string heap with very particular allocation patterns, and almost
always refcounted
 Phobos/druntine allocate liberally - the CRT almost never allocates

This is my single biggest fear in D. I have explicit control within my own
code, but I wonder if many D libraries will be sloppy and over-allocate all
over the place, and be generally unusable in many applications.
If D is another language like C where the majority of libraries (including
the standard libraries I fear) are unusable in various contexts, then that
kinda defeats the purpose. D's module system is one of its biggest selling
points.

I think there should be strict phobos allocation policies, and ideally,
druntime should NEVER allocate if it can help it.


Consider C strings. You need to keep track of ownership of it. That often
> means creating extra copies, rather than sharing a single copy.
>

Rubbish, strings are almost always either refcounted or on the stack for
dynamic strings, or have fixed memory allocated within structures. I don't
think I've ever seen someone duplicating strings into separate allocations
liberally.



> Enter C++'s shared_ptr. But that works by, for each object, allocating a
> *second* chunk of memory to hold the reference count. Right off the bat,
> you've got twice as many allocations & frees with shared_ptr than a GC
> would have.
>

Who actually uses shared_ptr? Talking about the stl is misleading... an
overwhelming number of C/C++ programmers avoid the stl like the plague (for
these exact reasons). Performance oriented programmers rarely use STL out
of the box, and that's what we're talking about here right? If you're not
performance oriented, then who cares about the GC either?
March 06, 2012
Re: John Carmack applauds D's pure attribute
Manu:

>   It's easy to allocate, ie, nothing to discourage you

This is partially a cultural thing, and it's partially caused by what the language offers you. I have seen Ada code where most things are stack-allocated, because doing this is handy in Ada.
In D stack-allocated variable-length arrays will help move some heap allocations to the stack. And the usage of scoped class instances has to improve some more.


>   Strings... - C coders who reject the stl will almost always have a
> separate string heap with very particular allocation patterns, and almost
> always refcounted

I think in future D will be free to have a special heap for strings. I think that in D source code there is already enough semantics to do this. There is one person that is working on the D GC, so maybe he's interested about this.


> an overwhelming number of C/C++ programmers avoid the stl like the plague (for
> these exact reasons). Performance oriented programmers rarely use STL out
> of the box,

Often I have heard the opposite claims too. Like in the recent GoingNative2012 conference.

Bye,
bearophile
March 06, 2012
Re: John Carmack applauds D's pure attribute
On 03/06/2012 01:27 PM, Manu wrote:
> On 26 February 2012 00:55, Walter Bright <newshound2@digitalmars.com
> <mailto:newshound2@digitalmars.com>> wrote:
>
>     On 2/25/2012 2:08 PM, Paulo Pinto wrote:
>
>         Most standard compiler malloc()/free() implementations are
>         actually slower than
>         most advanced GC algorithms.
>
>
>     Most straight up GC vs malloc/free benchmarks miss something
>     crucial. A GC allows one to do substantially *fewer* allocations.
>     It's a lot faster to not allocate than to allocate.
>
>
> Do you really think that's true? Are there any statistics to support that?
> I'm extremely sceptical of this claim.
>
> I would have surely thought using a GC leads to a significant *increase*
> in allocations for a few reasons:
>    It's easy to allocate, ie, nothing to discourage you

If you believe this, why do you raise this issue?

>    It's easy to clean up - you don't have to worry about cleanup
> problems, makes it simpler to use in many situations

GC does not prevent memory leaks, it does not support deterministic 
cleanup, and most implementations perform poorly on certain workloads. 
You were saying?

>    Dynamic arrays are easy - many C++ users will avoid dynamic arrays
> because the explicit allocation/clean up implies complexity, one will
> always use the stack, or a fixed array where they can get away with it
>    Slicing,

Slicing does never allocate.

> concatenation, etc performs bucket loads of implicit GC
> allocations

a~b

Nothing implicit about that.

The only case where memory allocation is implicit is for closures.


>    Strings... - C coders who reject the stl will almost always have a
> separate string heap with very particular allocation patterns, and
> almost always refcounted
>    Phobos/druntine allocate liberally - the CRT almost never allocates
>
> This is my single biggest fear in D. I have explicit control within my
> own code, but I wonder if many D libraries will be sloppy and
> over-allocate all over the place, and be generally unusable in many
> applications.

IMHO this fear is unjustified. If the library developers are that 
sloppy, chances are that the library is not worth using, even when 
leaving all memory allocation concerns away. (It is likely that you 
aren't the only programmer familiar with some of the issues.)

> If D is another language like C where the majority of libraries
> (including the standard libraries I fear) are unusable in various
> contexts, then that kinda defeats the purpose. D's module system is one
> of its biggest selling points.
>
> I think there should be strict phobos allocation policies,

Yes, a function that does not obviously need to allocate shouldn't, and 
if possible there should be alternatives that do not allocate. Do you 
have any particular examples where such a policy would be violated in 
Phobos?

> and ideally, druntime should NEVER allocate if it can help it.
>

+1. What are examples of unnecessary allocations in druntime?

>
>     Consider C strings. You need to keep track of ownership of it. That
>     often means creating extra copies, rather than sharing a single copy.
>
>
> Rubbish, strings are almost always either refcounted

Technically, refcounting is a form of GC.

> or on the stack for
> dynamic strings, or have fixed memory allocated within structures. I
> don't think I've ever seen someone duplicating strings into separate
> allocations liberally.

It is impossible to slice a zero-terminated string without copying it in 
the general case and refcounting slices is not trivial.


>
>     Enter C++'s shared_ptr. But that works by, for each object,
>     allocating a *second* chunk of memory to hold the reference count.
>     Right off the bat, you've got twice as many allocations & frees with
>     shared_ptr than a GC would have.
>
>
> Who actually uses shared_ptr? Talking about the stl is misleading... an
> overwhelming number of C/C++ programmers avoid the stl like the plague
> (for these exact reasons). Performance oriented programmers rarely use
> STL out of the box, and that's what we're talking about here right?

Possibly now you are the one who is to provide supporting statistics.

> If you're not performance oriented, then who cares about the GC either?

There is a difference between not performance oriented and performance 
agnostic. Probably everyone cares about performance to some extent.
March 06, 2012
Re: John Carmack applauds D's pure attribute
On 06.03.2012 16:27, Manu wrote:
> On 26 February 2012 00:55, Walter Bright <newshound2@digitalmars.com
> <mailto:newshound2@digitalmars.com>> wrote:
>
>     On 2/25/2012 2:08 PM, Paulo Pinto wrote:
>
>         Most standard compiler malloc()/free() implementations are
>         actually slower than
>         most advanced GC algorithms.
>
>
>     Most straight up GC vs malloc/free benchmarks miss something
>     crucial. A GC allows one to do substantially *fewer* allocations.
>     It's a lot faster to not allocate than to allocate.
>
>
> Do you really think that's true? Are there any statistics to support that?
> I'm extremely sceptical of this claim.
>
> I would have surely thought using a GC leads to a significant *increase*
> in allocations for a few reasons:
>    It's easy to allocate, ie, nothing to discourage you
>    It's easy to clean up - you don't have to worry about cleanup
> problems, makes it simpler to use in many situations
>    Dynamic arrays are easy - many C++ users will avoid dynamic arrays
> because

you mean like new[] ? That's just gloried allocation. STL vector is a 
dynamic array of C++ and it's being used a lot.

the explicit allocation/clean up implies complexity, one will
> always use the stack,
 or a fixed array where they can get away with it

Still possible

>    Slicing,
Nope

concatenation, etc performs bucket loads of implicit GC
> allocations

It sure does, like it does without GC with reallocs and refcounting.

>    Strings... - C coders who reject the stl will almost always have a
> separate string heap with very particular allocation patterns, and
> almost always refcounted

>    Phobos/druntine allocate liberally - the CRT almost never allocates

It's just most of CRT has incredibly bad usability, partly because it 
lack _any_ notion of allocators. And policy of using statically 
allocated shared data like in localtime, srand etc. shown remarkably bad 
M-T scalability.

> This is my single biggest fear in D. I have explicit control within my
> own code, but I wonder if many D libraries will be sloppy and
> over-allocate all over the place, and be generally unusable in many
> applications.
> If D is another language like C where the majority of libraries
> (including the standard libraries I fear) are unusable in various
> contexts, then that kinda defeats the purpose. D's module system is one
> of its biggest selling points.
>
> I think there should be strict phobos allocation policies, and ideally,
> druntime should NEVER allocate if it can help it.
>

>
>     Consider C strings. You need to keep track of ownership of it. That
>     often means creating extra copies, rather than sharing a single copy.
>
>
> Rubbish, strings are almost always either refcounted

like COW?

or on the stack for
> dynamic strings, or have fixed memory allocated within structures. I
> don't think I've ever seen someone duplicating strings into separate
> allocations liberally.

I've seen some, and sometimes memory corruption when people hesitated to 
copy even when they *do* need to pass a copy.

>
>     Enter C++'s shared_ptr. But that works by, for each object,
>     allocating a *second* chunk of memory to hold the reference count.
>     Right off the bat, you've got twice as many allocations & frees with
>     shared_ptr than a GC would have.
>
>
> Who actually uses shared_ptr?

Like everybody? Though with c++11 move semantics a unique_ptr is going 
to lessen it's widespread use. And there are ways to spend less then 2 
proper memory allocations per shared_ptr, like keeping special block 
allocator for ref-counters.
More importantly smart pointers are here to stay in C++.

Talking about the stl is misleading... an
> overwhelming number of C/C++ programmers avoid the stl like the plague
> (for these exact reasons).

Instead of using it properly. It has a fair share of failures but it's 
not that bad.

Performance oriented programmers rarely use
> STL out of the box, and that's what we're talking about here right? If
> you're not performance oriented, then who cares about the GC either?


-- 
Dmitry Olshansky
2 3 4 5 6 7 8
Top | Discussion index | About this forum | D home