On Tuesday, 4 February 2014 at 13:03:24 UTC, Araq wrote:
> 1. D doesn't restrict pointers, interior pointers abound especially thanks to the design of slices. Interior pointers are a problem for every high performance GC algorithm I'm aware of.

Interior pointers can always be replaced with a pointer + offset, so in the case of slices it can be resolved at the implementation level at some memory trade off? In the case of internal pointers elsewhere you would have to introduce a new type "pointer+offset", but that would break C-interop.

> 2. D's design doesn't easily allow for read or write barriers in general. You simply cannot make a GC behave without barriers.

It is difficult without affecting the C-compatible semantics. There are some interesting options on 64 bit CPUs where you in theory can do compaction to a fresh region and put hardware traps on the old region, rewriting pointers trying to access the old region or converting misses to exceptions on the fly (assuming you can have a barrier/isolate that prevents comparison of old/new pointers).

Another obvious solution is thread local GC for thread local memory or segmented GC (collect for only a specific pool, refuse pointers to escape the pool, and put a lock on that pool during GC).

> That's just bullshit. The opposite is true: Of the big players D is easily the language that is most hostile to an efficient GC implementation.

Because C is hostile to GC. So you either forsake C-compatibility or stay inefficient?

On Tuesday, 4 February 2014 at 13:18:51 UTC, Dicebot wrote:
> For me perfect solution would have been to use an allocator concept as language basis instead and let you chose any conformant allocator for built-in language features. With both GC and ARC available in Phobos / druntime.

That is exactly what Rust did succsessfully.

http://pcwalton.github.io/blog/2013/06/02/removing-garbage-collection-from-the-rust-language/

got pulled.

They deprecated the @ pointer, which is a managed poiter equivalent to what D's 'new' produces, to a library solution.

They now have basically only ~T and &T left in the language. ~T being an owned pointer that frees at the end of its scope and &T, a borrowed pointer (or reference, but *not* RC) with no direct effect on memory management.

There is Rc<T> and Gc<T> in the library for those who wish to use them.

To quote from the above link:

>>
Programmers don’t know which to use, since some operations are available with ~ and some operations are available with @. Actually, we were confused on this point for a long time as well—it wasn’t clear whether ~ or @ would become dominant. We debated for a long time which to present first, ~ or @. However, as the language and community evolved, and coding standards became more settled, a clear winner emerged: the owning pointer ~. In practice, the rule has been that programmers should use ~ to allocate in all circumstances except when they have no way of knowing precisely when the object in question should be freed.
<<

This is what happens if you give users choice.

One of Rust's priorities is to have a runtime footprint as small as possible. I would give priority to not break existing D code. So keep GC'd 'new' in the language (the runtime will never get as small as Rust's; don't care) and see if it is possible to add owned and ARC pointers from there.

On Tuesday, 4 February 2014 at 14:19:36 UTC, Frank Bauer wrote:
> On Tuesday, 4 February 2014 at 13:18:51 UTC, Dicebot wrote:
>> For me perfect solution would have been to use an allocator concept as language basis instead and let you chose any conformant allocator for built-in language features. With both GC and ARC available in Phobos / druntime.
>
> That is exactly what Rust did succsessfully.
>
> http://pcwalton.github.io/blog/2013/06/02/removing-garbage-collection-from-the-rust-language/
>
> got pulled.
>
> They deprecated the @ pointer, which is a managed poiter equivalent to what D's 'new' produces, to a library solution.
>
> They now have basically only ~T and &T left in the language. ~T being an owned pointer that frees at the end of its scope and &T, a borrowed pointer (or reference, but *not* RC) with no direct effect on memory management.
>
> There is Rc<T> and Gc<T> in the library for those who wish to use them.
>
> To quote from the above link:
>
>>>
> Programmers don’t know which to use, since some operations are available with ~ and some operations are available with @. Actually, we were confused on this point for a long time as well—it wasn’t clear whether ~ or @ would become dominant. We debated for a long time which to present first, ~ or @. However, as the language and community evolved, and coding standards became more settled, a clear winner emerged: the owning pointer ~. In practice, the rule has been that programmers should use ~ to allocate in all circumstances except when they have no way of knowing precisely when the object in question should be freed.
> <<
>
> This is what happens if you give users choice.
>
> One of Rust's priorities is to have a runtime footprint as small as possible. I would give priority to not break existing D code. So keep GC'd 'new' in the language (the runtime will never get as small as Rust's; don't care) and see if it is possible to add owned and ARC pointers from there.

Rust still remains to prove itself on the mainstream market.

While D already has commercial users.

On Tuesday, 4 February 2014 at 06:47:03 UTC, Walter Bright wrote:
> On 2/3/2014 7:03 PM, Adam Wilson wrote:
>> Note that ObjC has special syntax to handle weak pointers. It's not well
>> understood by many.
>
> Sounds like explicitly managed memory is hardly worse.
>
> On 2/3/2014 3:13 PM, woh wrote:
>>
>>  Any system that forces a single way of handling memory as the only viable
>> method, be it GC( as D currently does)
>
> This is incorrect. You can use malloc/free in D, as well as write & use your own allocators, and even write your own ref counted types.
>
> On 2/3/2014 1:42 PM, Shammah Chancellor wrote:
>> It's also probably
>> possible to create a drop-in replacement for the GC to do something else.
>
> It certainly is possible. There's nothing magic about the current GC, it's just library code.


Andrei Alexandrescu wrote:
> 2. Work on Phobos to see what can be done about avoiding unnecessary allocation. Most likely we'll need to also add a @nogc flag.
> ...
> 4. Work on the core language and druntime to see how to seamlessly accommodate alternate GC mechanisms such as reference counting.
> ...
> I thought I made it clear that GC avoidance (which includes considering built-in reference counting) is a major focus of 2014.
>
> Andrei

Im totally confused: Walter, do you back what Andrei says or do we have a good cop / bad cop situation here?

On Tuesday, 4 February 2014 at 11:57:25 UTC, Paulo Pinto wrote:
> On Tuesday, 4 February 2014 at 11:13:03 UTC, Don wrote:
>> On Tuesday, 4 February 2014 at 10:32:26 UTC, ed wrote:
>>>>> Realistically D as a systems language isn't even at the hobby stage.
>>>>
>>>> We're using D as a systems language on a global commercial scale. And no, we don't use malloc/free. We just don't use Phobos.
>>>
>>> Maybe I'm thinking too much embedded, which I admit isn't fair on D and at this stage maybe not a realistic comparison.
>>
>> Yeah, I dunno what "systems language" means really.
>
> For me it means you can write an OS with it, even if some tiny parts require the use of Assembly glue.

I know that's the *origin* of the term, but I don't think that's the *meaning*.

I mean, that's a really obscure task. People almost never write a new OS, and languages have claimed to be systems languages without ever having done it.

AFAICT what it really means is "like C" in some vaguely-defined sense.

On Tuesday, 4 February 2014 at 14:22:41 UTC, Paulo Pinto wrote:
> Rust still remains to prove itself on the mainstream market.
>
> While D already has commercial users.

It is dangerous position. D is not that far ahead to appeal to own authority.

"..it takes all the running you can do, to keep in the same place. If you want to get somewhere else, you must run at least twice as fast as that!"

On Tuesday, 4 February 2014 at 13:52:38 UTC, Paulo Pinto wrote:
> - No implicit allocations outside NEW
> - No closures

True, on the OS level, in drivers and in real time call backs you want everything to be explicit.

> - You can use untraced pointers in unsafe modules.

I assume it would be possible to add @notrace as an attribute to pointers/references in D so that you can avoid tracing objects that are known to be in the pool of traced objects?

The best would be to have "notrace-pointer-types" and programmer specified "trace functions" that are more optimal. E.g. keeping a list of all active Nodes and have your own function for scanning it for outgoing pointers, then keep all pointers to Nodes non-traceable.

unique is best, should be default, down with GC!!


Tuesday, 4 February 2014 at 14:19:36 UTC, Frank Bauer wrote:
> On Tuesday, 4 February 2014 at 13:18:51 UTC, Dicebot wrote:
>> For me perfect solution would have been to use an allocator concept as language basis instead and let you chose any conformant allocator for built-in language features. With both GC and ARC available in Phobos / druntime.
>
> That is exactly what Rust did succsessfully.
>
> http://pcwalton.github.io/blog/2013/06/02/removing-garbage-collection-from-the-rust-language/
>
> got pulled.
>
> They deprecated the @ pointer, which is a managed poiter equivalent to what D's 'new' produces, to a library solution.
>
> They now have basically only ~T and &T left in the language. ~T being an owned pointer that frees at the end of its scope and &T, a borrowed pointer (or reference, but *not* RC) with no direct effect on memory management.
>
> There is Rc<T> and Gc<T> in the library for those who wish to use them.
>
> To quote from the above link:
>
>>>
> Programmers don’t know which to use, since some operations are available with ~ and some operations are available with @. Actually, we were confused on this point for a long time as well—it wasn’t clear whether ~ or @ would become dominant. We debated for a long time which to present first, ~ or @. However, as the language and community evolved, and coding standards became more settled, a clear winner emerged: the owning pointer ~. In practice, the rule has been that programmers should use ~ to allocate in all circumstances except when they have no way of knowing precisely when the object in question should be freed.
> <<
>
> This is what happens if you give users choice.
>
> One of Rust's priorities is to have a runtime footprint as small as possible. I would give priority to not break existing D code. So keep GC'd 'new' in the language (the runtime will never get as small as Rust's; don't care) and see if it is possible to add owned and ARC pointers from there.

On Tuesday, 4 February 2014 at 02:51:21 UTC, Andrei Alexandrescu wrote:
>>   Rust seems to have found a very nice model, and even cpp with
>> value/unique/rc/weak is IMO superior to what D currently offers
>
> I think Rust expended too much language complexity on that one issue.
>
> Andrei

By the way, while this statement was true for initial design, they have recently moved to much more simple model, replacing most of more complicated pointer types with library solutions. I think those who refer to Rust example are more likely to have in mind that new model and your judgement seems to be based on previous one.

On Tue, Feb 04, 2014 at 01:11:23PM +1000, Manu wrote:
> On 4 February 2014 12:05, Nick Sabalausky < SeeWebsiteToContactMe@semitwist.com> wrote:
> 
> > On 2/3/2014 4:13 PM, H. S. Teoh wrote:
> >
> >> I've seen real-life examples of ARCs gone horribly, horribly wrong, whereas had a GC been used in the first place things wouldn't have gone down that route.
> >>
> >>
> > I'm curious to hear more about this.
> >
> 
> Me too.

Well, first I have to apologize for my misleading message, I meant *reference counting systems*, not ARC specifically, since the case I had in mind is a reference-counting system implemented in C, so obviously it's not *A*RC.  :-(

But anyway, what happened is that in the past, we used to do just straight malloc/free, but over time discovered that it was an unending source of embarrassing pointer bugs and memory leaks. So at some point, the team lead decided to implement a reference-counting system instead.

Which worked *better*, I suppose, but it was hardly the panacea that people might think. It was still prone to pointer bugs and memory leaks caused by not updating the reference counts properly -- something that an ARC wouldn't have, I guess.

But then people started doing things with dtors that they shouldn't do... like this one especially egregious abuse that I had in mind, where there was a container pointing to a bunch of subobjects, and whenever one of the subobjects got deleted, the rest must be deleted as well. This was implemented by having the dtor of the subobject destruct the container, which, according to refcount thinking, should "automatically" clean up everything. Except that it didn't, because this implementation was wrong on so many levels... the first being that the subobject can't have its dtor invoked unless the container didn't contribute to the reference count, and of course, once the dtor is invoked, it tries to destruct the container by decreasing *its* reference count, which makes one wonder who's keeping the reference to the container, and what happens with the container's reference to the current subobject.  This was the case that really needed a GC, because there are just too many cross-references going on. Had we used a GC to begin with, we'd never have gone down this slide of increasingly wrong code. And this was just the one instance we discovered; who knows how many other such abuses are still lurking in the code somewhere.

Now you may argue that an ARC wouldn't permit this sort of abuses, but the fact of the matter is, the same people who can't write bug-free malloc/free code are the same people who can't write bug-free reference counting code, and since we have very little say in who we work with, we're going to have to continue dealing with stupid bugs of this sort. Even with an ARC, we're going to get people writing stupid code that create memory leaks with cycles and what-not.  To fully address this issue at its root, what we really need is a GC, since it eliminates an entire class of memory bugs. Except that the decision makers are hardcore C coders who have irrational GC phobia (and I used to be one of them), so this will probably never happen in *this* job.  (Maybe I should consider working for Sociomantic or something...)


T

-- 
Truth, Sir, is a cow which will give [skeptics] no more milk, and so they are gone to milk the bull. -- Sam. Johnson