February 04, 2014
On 4 February 2014 14:27, Manu <turkeyman@gmail.com> wrote:

> On 4 February 2014 14:19, Michel Fortin <michel.fortin@michelf.ca> wrote:
>
>> On 2014-02-04 03:45:33 +0000, Manu <turkeyman@gmail.com> said:
>>
>>  The majority of trivial allocations don't produce cycles; closures,
>>> strings, temporary arrays and working sets.
>>>
>>
>> Oh, no. Beware of closures. That's a frequent problem in Objective-C ARC, even for those who understand ARC well. You have to be very careful of closures creating cycles if you use them as callbacks. For instance, you have a view that has a pointer to the model and sets a callback for the model to call when something change to update the view; that's a cycle and you need to use a weak ref to the view within the closure. Pretty common pattern.
>
>
> Ah right. Interesting.
> It sounds like this could be addressed easily though by the API that
> manages the 'event' handler. If you use a raw delegate, maybe it's a
> problem, but it sounds like an event-programming construct, and that
> usually requires an event class that can handle multiple subscribers, as
> soon as that exists, it's easy to hide the weak reference behind the event
> api...
>

I can say that all the closures that I've observed in my D apps so far have been entirely trivial.


February 04, 2014
On 2/3/2014 10:30 PM, Manu wrote:
> On 4 February 2014 12:59, Andrei Alexandrescu <SeeWebsiteForEmail@erdani.org
>> wrote:
>>
>> Kinda difficult to explain the market success of Android.

Not really. With PalmOS long dead (RIP) and MS famously just flailing around on anything battery-powered, Android has been the only realistic option for those who want a handheld computer, but don't want to sell all their freedoms to Apple. Plus it's Google.

>
> 2. It's free for product vendors.

No.

I mean, on the surface yea it appears free, but realistically it's on the decline: Device manufacturers (partly under pressure from asswhores like Verizon, no doubt) have been moving towards a system where normal, affordable, contract-subsidized devices (not that I'm a fan of the contracts) are locked-down as much as possible [1], even using eFuses now, and thereby rendered impractical for average developers (since you can't freely test on multiple OS versions).

But, GOLLY GEE!! FOR *ONLY SEVERAL HUNDREDS* OF PURELY OUT-OF-POCKET DOLLARS, these douchebag corporations like Google or Samsung will happily sell you a "developer" phone that restricts FEWER of your freedoms! Isn't that great? And best of all, Google gets to pretend being an Android developer is still free! Get it? Because the extra $$$ developers pay is for the psuedo-optional *wink wink nudge nudge* "developer phone" instead of a "developer account" like mean-ol'-Apple.

[1] Not only the usual locked bootloaders, but see also Knox-secured bootloader. And no, I'm not being tin-foil-hat here, I've actually been bit by that and had a device bricked because of that forced-update Knox-secured bullshit, just because I needed to switch between 4.3 and 4.1.2 which Samsung decided I shouldn't be allowed to do (Knox eFuse). Luckily it was still under warranty and pretty much all salespeople and support reps are about as intelligent as your average house pet, so I was able to get a replacement...at least after they finally got tired of screwing up the replacement orders (hmmm, a DOA device...a wrong device from a completely different manufacturer...).

February 04, 2014
Manu <turkeyman@gmail.com> writes:

> These are the problems:
>  * GC stalls for long periods time at completely un-predictable moments.
>  * GC stalls become longer *and* more frequent as memory becomes less
> available, and the working pool becomes larger (what a coincidence).
>  * Memory footprint is unknowable, what if you don't have a virtual memory
> manager? What if your total memory is measured in megabytes?
>  * It's not possible to know when destruction of an object will happen, which
> has known workarounds (like in C#) but is also annoying in many cases, and
> supports the prior point.
>
> Conclusion:
>   GC is unfit for embedded systems. One of the most significant remaining and
> compelling uses for a native systems language.

Hi Manu,

Doing a bit of searching, I came across the Metronome GC that IBM came up with.  It appears to try to address the problems you raise:

http://researcher.watson.ibm.com/researcher/view_project_subpage.php?id=175

Any thoughts?

Jerry
February 04, 2014
On Mon, 03 Feb 2014 18:57:00 -0800, Andrei Alexandrescu <SeeWebsiteForEmail@erdani.org> wrote:

> On 2/3/14, 5:36 PM, Adam Wilson wrote:
>> You still haven't dealt with the cyclic reference problem in ARC. There
>> is absolutely no way ARC can handle that without programmer input,
>> therefore, it is simply not possible to switch D to ARC without adding
>> some language support to deal with cyclic-refs. Ergo, it is simply not
>> possible to seamlessly switch D to ARC without creating all kinds of
>> havoc as people now how memory leaks where they didn't before. In order
>> to support ARC the D language will necessarily have to grow/change to
>> accommodate it. Apple devs constantly have trouble with cyclic-refs to
>> this day.
>
> The stock response: weak pointers. But I think the best solution is to allow some form of automatic reference counting backed up by the GC, which will lift cycles.
>
> Andrei
>

The immediate problem that I can see here is you're now paying for TWO GC algorithms. There is no traditional GC without a Mark phase (unless it's a copying collector, which will scare off the Embedded guys), and the mark phase is actually typically the longer portion of the pause. If you have ARC backed up by a GC you'll still have to mark+collect which means the GC still has to track ARC memory and then when a collection is needed, mark and collect. This means that you might reduce the total number of pauses, but you won't eliminate them. That in turn makes it an invalid tool for RT/Embedded purposes. And of course we still have the costs of ARC. Manu still can't rely on pause-free (although ARC isn't either) memory management, and the embedded guys still have to pay the costs in heap size to support the GC.

Going the other way, GC is default with ARC support on the side, is not as troublesome from an implementation standpoint because the GC does not have to be taught about the ARC memory. This means that ARC memory is free of being tracked by the GC and the GC has less overall memory to track which makes collection cycles faster. However, I don't think that the RT/Embedded guys will like this either, because it means you are still paying for the GC at some point, and they'll never know for sure if a library they are using is going to GC-allocate (and collect) when they don't expect it.

The only way I can see to make the ARC crowd happy is to completely replace the GC entirely, along with the attendant language changes (new keywords, etc) that are probably along the lines of Rust. I strongly believe that the reason we've never seen a GC backed ARC system is because in practice it doesn't completely solve any of the problems with either system but costs quite a bit more than either system on it's own.

-- 
Adam Wilson
GitHub/IRC: LightBender
Aurora Project Coordinator
February 04, 2014
On 4 February 2014 15:22, Jerry <jlquinn@optonline.net> wrote:

> Manu <turkeyman@gmail.com> writes:
>
> > These are the problems:
> >  * GC stalls for long periods time at completely un-predictable moments.
> >  * GC stalls become longer *and* more frequent as memory becomes less
> > available, and the working pool becomes larger (what a coincidence).
> >  * Memory footprint is unknowable, what if you don't have a virtual
> memory
> > manager? What if your total memory is measured in megabytes?
> >  * It's not possible to know when destruction of an object will happen,
> which
> > has known workarounds (like in C#) but is also annoying in many cases,
> and
> > supports the prior point.
> >
> > Conclusion:
> >   GC is unfit for embedded systems. One of the most significant
> remaining and
> > compelling uses for a native systems language.
>
> Hi Manu,
>
> Doing a bit of searching, I came across the Metronome GC that IBM came up with.  It appears to try to address the problems you raise:
>
> http://researcher.watson.ibm.com/researcher/view_project_subpage.php?id=175
>
> Any thoughts?
>

Interesting. I'll read :)
I know nothing about it, so I can't really comment. And I'm also not an
authority on what can/can't be done in terms of GC implementation in the
context of D.


February 04, 2014
On 4 February 2014 15:23, Adam Wilson <flyboynw@gmail.com> wrote:

> On Mon, 03 Feb 2014 18:57:00 -0800, Andrei Alexandrescu < SeeWebsiteForEmail@erdani.org> wrote:
>
>  On 2/3/14, 5:36 PM, Adam Wilson wrote:
>>
>>> You still haven't dealt with the cyclic reference problem in ARC. There is absolutely no way ARC can handle that without programmer input, therefore, it is simply not possible to switch D to ARC without adding some language support to deal with cyclic-refs. Ergo, it is simply not possible to seamlessly switch D to ARC without creating all kinds of havoc as people now how memory leaks where they didn't before. In order to support ARC the D language will necessarily have to grow/change to accommodate it. Apple devs constantly have trouble with cyclic-refs to this day.
>>>
>>
>> The stock response: weak pointers. But I think the best solution is to allow some form of automatic reference counting backed up by the GC, which will lift cycles.
>>
>> Andrei
>>
>>
> The immediate problem that I can see here is you're now paying for TWO GC algorithms. There is no traditional GC without a Mark phase (unless it's a copying collector, which will scare off the Embedded guys), and the mark phase is actually typically the longer portion of the pause. If you have ARC backed up by a GC you'll still have to mark+collect which means the GC still has to track ARC memory and then when a collection is needed, mark and collect. This means that you might reduce the total number of pauses, but you won't eliminate them. That in turn makes it an invalid tool for RT/Embedded purposes. And of course we still have the costs of ARC. Manu still can't rely on pause-free (although ARC isn't either) memory management, and the embedded guys still have to pay the costs in heap size to support the GC.
>

So, the way I see this working in general, is that because in the majority
case, ARC would release memory immediately freeing up memory regularly, an
alloc that would have usually triggered a collect will happen far, far less
often.
Practically, this means that the mark phase, which you say is the longest
phase, would be performed far less often.

For me and my kind, I think the typical approach would be to turn off the
backing GC, and rely on marking weak references correctly.
This satisfies my requirements, and I also lose nothing in terms of
facilities in Phobos or other libraries (assuming that those libraries have
also marked weak references correctly, which I expect phobos would
absolutely be required to do).

This serves both worlds nicely, I retain access to libraries since they use the same allocator, the GC remains (and is run less often) for those that want care-free memory management, and for RT/embedded users, they can *practically* disable the GC, and take responsibility for weak references themselves, which I'm happy to do.


Going the other way, GC is default with ARC support on the side, is not as
> troublesome from an implementation standpoint because the GC does not have to be taught about the ARC memory. This means that ARC memory is free of being tracked by the GC and the GC has less overall memory to track which makes collection cycles faster. However, I don't think that the RT/Embedded guys will like this either, because it means you are still paying for the GC at some point, and they'll never know for sure if a library they are using is going to GC-allocate (and collect) when they don't expect it.
>

It also means that phobos and other libraries will use the GC because it's
the default. Correct, I don't see this as a valid solution. In fact, I
don't see it as a solution at all.
Where would implicit allocations like strings, concatenations, closures be
allocated?
I might as well just use RefCounted, I don't see this offering anything
much more than that.

The only way I can see to make the ARC crowd happy is to completely replace
> the GC entirely, along with the attendant language changes (new keywords, etc) that are probably along the lines of Rust. I strongly believe that the reason we've never seen a GC backed ARC system is because in practice it doesn't completely solve any of the problems with either system but costs quite a bit more than either system on it's own.


Really? [refer to my first paragraph in the reply]
It seems to me like ARC in front of a GC would result in the GC running far
less collect cycles. And the ARC opposition would be absolved of having to
tediously mark weak references. Also, the GC opposition can turn the GC
off, and everything will still work (assuming they take care of their
cycles).
I don't really see the disadvantage here, except that the
only-GC-at-all-costs-I-won't-even-consider-ARC crowd would gain a
ref-count, but they would also gain the advantage where the GC would run
less collect cycles. That would probably balance out.

I'm certainly it would be better than what we have, and in theory, everyone would be satisfied.


February 04, 2014
On 2/3/2014 7:03 PM, Adam Wilson wrote:
> Note that ObjC has special syntax to handle weak pointers. It's not well
> understood by many.

Sounds like explicitly managed memory is hardly worse.

February 04, 2014
On 2/3/2014 3:13 PM, woh wrote:
>
>   Any system that forces a single way of handling memory as the only viable
> method, be it GC( as D currently does)

This is incorrect. You can use malloc/free in D, as well as write & use your own allocators, and even write your own ref counted types.

February 04, 2014
On 2/3/2014 1:42 PM, Shammah Chancellor wrote:
> It's also probably
> possible to create a drop-in replacement for the GC to do something else.

It certainly is possible. There's nothing magic about the current GC, it's just library code.

February 04, 2014
On Mon, 03 Feb 2014 22:39:02 -0800, Walter Bright <newshound2@digitalmars.com> wrote:

> On 2/3/2014 7:03 PM, Adam Wilson wrote:
>> Note that ObjC has special syntax to handle weak pointers. It's not well
>> understood by many.
>
> Sounds like explicitly managed memory is hardly worse.
>

Well special syntax is needed to make ARC work because of the requirement to explicitly mark a pointer as a weak-reference. It would also mean that all existing D code would subtly break (leak memory) in any case where a weak-reference is required and was previously handled transparently by the GC. I won't say that manually managed memory is worse, because it does automatically clean up most of the trash, but ARC still allows to shot yourself in the foot easily enough.

-- 
Adam Wilson
GitHub/IRC: LightBender
Aurora Project Coordinator