On 23 February 2015 at 07:47, Walter Bright via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
> On 2/22/2015 8:36 AM, Manu via Digitalmars-d wrote:
>>
>> I have no idea where to start.
>
>
> Start by making a ref counted type and see what the pain points are.

All my ref counting types fiddle with the ref in every assignment, or every function call and return. Unless the language has some sort of support for ref counting, I don't know how we can do anything about that.

On Monday, 23 February 2015 at 01:38:35 UTC, Manu wrote:
> All my ref counting types fiddle with the ref in every assignment, or every function call and return.

Hmm, the optimizer could potentially tell "inc X; dec X;" is useless and remove it without knowing what it is for.

On 23 February 2015 at 11:41, Adam D. Ruppe via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
> On Monday, 23 February 2015 at 01:38:35 UTC, Manu wrote:
>>
>> All my ref counting types fiddle with the ref in every assignment, or every function call and return.
>
>
> Hmm, the optimizer could potentially tell "inc X; dec X;" is useless and remove it without knowing what it is for.

Yeah, except we're talking about libraries, and in that context I often have:
extern(C) IncRef(T*);
extern(C) DecRef(T*);

Optimiser can't offer anything.

On 23 February 2015 at 03:13, Andrei Alexandrescu via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
> On 2/22/15 8:36 AM, Manu via Digitalmars-d wrote:
>>
>> On 22 February 2015 at 13:53, Daniel Murphy via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>>>
>>> "Manu via Digitalmars-d"  wrote in message news:mailman.7037.1424565826.9932.digitalmars-d@puremagic.com...
>>>
>>>> I personally think ARC in D is the only way forwards. That is an unpopular opinion however... although I think I'm just being realistic ;)
>>>
>>>
>>>
>>> A big part of why it's unpopular is that nobody, including you, wants to implement it to see if it's viable.
>>
>>
>> I have no idea where to start.
>
>
> Simple approaches to reference counting are accessible to any software engineer. The right starting point is "I used reference counting in this project, and here are my findings".
>
> A position such as the following makes the dialog very difficult:
>
> 1. One solution is deemed the only viable.

Propose how GC will ever be a success?
I honestly don't care, I just want a solution that's acceptable. I
tried to convince myself that GC would be fine for half a decade now,
but I've run out of patience.
I've been sitting around waiting for years for someone to say how GC
will ever be an acceptable solution. How long do I have to wait?

Evidence suggests, there IS only one viable solution. Granted, that's
not proven viable; it has barely been explored. (on account of
borderline religious opposition)
I can see why GC will never work in D, I can not see why ARC will never work.

> 2. Details and difficulties are unknown to the proposer.

I'm proposing that the *conversation* needs to be taken seriously. Every time I've raised it in the past it's been immediately dismissed and swiped off the table.

I'm not an expert on garbage collection, I have practically nothing to
add. I'm also not particularly interested in garbage collection (of
any form); I just want it to work.
But it doesn't take an expert to recognise that in 6 years, nobody has
presented any forward momentum on the GC front, no matter how
fantastical.
I can easily visualise a way forward with RC. There's plenty of room
for exploration. Sure it's not trivial, but maybe it's *possible*, as
it certainly seems that GC is not.

> 3. It must be implemented by others, not the proposer.

It must be discussed before we even think about implementing it. And that is predicated by not being dismissed on impact.

> 4. It must be part of the language; any experimentation outside the language is considered an unnecessary waste of time.

RC performance in a lib depends a lot on scope overloads of
constructor/destructor/postblit to eliminate ref fiddling code.
Scope proposals were butchered. I'm disappointed with where that went.

On 2/22/15 5:57 PM, Manu via Digitalmars-d wrote:
> I can easily visualise a way forward with RC.

Then do it. Frankly it seems to me you're doing anything you possibly can to talk yourself out of doing work.

Andrei

On Monday, 23 February 2015 at 01:41:17 UTC, Adam D. Ruppe wrote:
> On Monday, 23 February 2015 at 01:38:35 UTC, Manu wrote:
>> All my ref counting types fiddle with the ref in every assignment, or every function call and return.
>
> Hmm, the optimizer could potentially tell "inc X; dec X;" is useless and remove it without knowing what it is for.

It is not that easy. First you need to increment/decrement in an atomic manner (unless we finally decide to fix holes in the type system) so the optimizer is mostly blind.

But even if it could (we are not far from being able to do it), in most scenarios it is still an issue as you get potential exception unwinding. The unwind path must find the right reference count in there.

On 23 February 2015 at 14:11, Andrei Alexandrescu via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
> On 2/22/15 5:57 PM, Manu via Digitalmars-d wrote:
>>
>> I can easily visualise a way forward with RC.
>
>
> Then do it. Frankly it seems to me you're doing anything you possibly can to talk yourself out of doing work.

Excellent technique to immediately invalidate everything someone says.
Thanks for that.
You dismissed absolutely everything I said, and then imply that I have
no right to comment unless I do it myself.
It's got nothing to do with doing work. ARC (or something like it) is
almost religiously opposed. We can't even have a reasonable
conversation about it, or really explore it's implications before
someone (that ideally know's what they're doing) thinks about writing
code. There's no room for progress in this environment.

What do you want me to do? I use manual RC throughout my code, but the
experience in D today is identical to C++. I can confirm that it's
equally terrible to any C++ implementation I've used. We have no tools
to improve on it.
And whatever, if it's the same as C++, I can live with it. I've had
that my whole career (although it's sad, because we could do much
better).
The problem, as always, is implicit allocations, and allocations from
3rd party libraries. While the default allocator remains incompatible
with workloads I care about, that isolates us from virtually every
library that's not explicitly written to care about my use cases.
That's the situation in C++ forever. We're familiar with it, and it's
shit. I have long hoped we would move beyond that situation in D
(truly the #1 fantasy I had when I first dove in to D 6 years back),
but it's a massive up-hill battle to even gain mindshare, regardless
of actual implementation. There's no shared vision on this matter, the
word is basically "GC is great, everyone loves it, use an RC lib".

How long do we wait for someone to invent a fantastical GC that solves
our problems and works in D? I think it's practically agreed that no
known design can work, but nobody wants to bite the bullet and accept
the fact.
We need to admit we have an impassable problem, and then maybe we can
consider alternatives openly. Obviously, it would be disruptive, but
it might actually work... which is better than where we seem to be
heading (with a velocity of zero).

The fact is, people who are capable of approaching this problem in terms of actual code will never even attempt it until there's resounding consensus that it's worth exploring.

On 2/22/2015 9:53 PM, Manu via Digitalmars-d wrote:
> It's got nothing to do with doing work. ARC (or something like it) is
> almost religiously opposed. We can't even have a reasonable
> conversation about it, or really explore it's implications before
> someone (that ideally know's what they're doing) thinks about writing
> code.

I participated in a very technical thread here on implementing ARC in D. I wanted to make it work - nothing was off the table, language changes, special features, etc.

http://www.digitalmars.com/d/archives/digitalmars/D/draft_proposal_for_ref_counting_in_D_211885.html

It was just unworkable, and nobody who participated in that thread had workable ideas on moving forward with it. Nothing since has come up that changes that. If you've got some ideas, please present them taking into account the issues brought up in that thread.

Also, please take into account; proposals will not get much of a reception if they ignore these points:

1. Increment and decrement, ESPECIALLY DECREMENT, is EXPENSIVE in time and bloat because of exceptions. Swift does it by NOT HAVING EXCEPTIONS. This is not an option for D.

2. As far as I can tell, the idea of flipping a compiler switch and the GC switches to ref counting is a pipe dream fantasy. You can probably make such a scheme work with a very limited language like Javascript, but it is never going to work with D's support for low level programming. The way RC and GC work is different enough that different user coding techniques will be used for them.

3. Memory safety is a requirement for any ARC proposal for D. Swift ignores memory safety concerns.

4. DIP25, now implemented, is a way to address memory safety in D while using reference counting. Any proposal for ARC needs to, at least, understand that proposal.

http://wiki.dlang.org/DIP25

On Monday, 23 February 2015 at 05:54:06 UTC, Manu wrote:
> On 23 February 2015 at 14:11, Andrei Alexandrescu via Digitalmars-d
> <digitalmars-d@puremagic.com> wrote:
>> On 2/22/15 5:57 PM, Manu via Digitalmars-d wrote:
>>>
>>> I can easily visualise a way forward with RC.
>>
>>
>> Then do it. Frankly it seems to me you're doing anything you possibly can to
>> talk yourself out of doing work.
>
> Excellent technique to immediately invalidate everything someone says.
> Thanks for that.
> You dismissed absolutely everything I said, and then imply that I have
> no right to comment unless I do it myself.
> It's got nothing to do with doing work. ARC (or something like it) is
> almost religiously opposed. We can't even have a reasonable
> conversation about it, or really explore it's implications before
> someone (that ideally know's what they're doing) thinks about writing
> code. There's no room for progress in this environment.
>
> What do you want me to do? I use manual RC throughout my code, but the
> experience in D today is identical to C++. I can confirm that it's
> equally terrible to any C++ implementation I've used. We have no tools
> to improve on it.
> And whatever, if it's the same as C++, I can live with it. I've had
> that my whole career (although it's sad, because we could do much
> better).
> The problem, as always, is implicit allocations, and allocations from
> 3rd party libraries. While the default allocator remains incompatible
> with workloads I care about, that isolates us from virtually every
> library that's not explicitly written to care about my use cases.
> That's the situation in C++ forever. We're familiar with it, and it's
> shit. I have long hoped we would move beyond that situation in D
> (truly the #1 fantasy I had when I first dove in to D 6 years back),
> but it's a massive up-hill battle to even gain mindshare, regardless
> of actual implementation. There's no shared vision on this matter, the
> word is basically "GC is great, everyone loves it, use an RC lib".
>
> How long do we wait for someone to invent a fantastical GC that solves
> our problems and works in D? I think it's practically agreed that no
> known design can work, but nobody wants to bite the bullet and accept
> the fact.
> We need to admit we have an impassable problem, and then maybe we can
> consider alternatives openly. Obviously, it would be disruptive, but
> it might actually work... which is better than where we seem to be
> heading (with a velocity of zero).
>
> The fact is, people who are capable of approaching this problem in
> terms of actual code will never even attempt it until there's
> resounding consensus that it's worth exploring.

Personally I think what matters is getting D's situation regarding memory management sorted out, regardless out it will look like in the end.

If I am a bit too quick jumping the gun about GC, is that I have embraced GC languages in my line of work, so I tend to be aware that not all GCs are made alike and some like the ones from e.g. Aonix are good enough for real time situations, the ones someone dies if the GC runs on the wrong moment.

Maybe such GC quality is impossible to achieve in D, I don't know.

What I can say is that I cannot use D on my type of work and keeping up with everything that happens on the JVM, .NET and mobile space already keeps me busy enough.

On Monday, 23 February 2015 at 01:41:17 UTC, Adam D. Ruppe wrote:
> On Monday, 23 February 2015 at 01:38:35 UTC, Manu wrote:
>> All my ref counting types fiddle with the ref in every assignment, or every function call and return.
>
> Hmm, the optimizer could potentially tell "inc X; dec X;" is useless and remove it without knowing what it is for.

INPUT:

try{
nonsharedobj._rc++;
…
}
finally {
nonsharedobj._rc--;
if(nonsharedobj._rc==0) destroy…
}

OPTIMIZED:

try{
…
}
finally {
if(nonsharedobj._rc==0) destroy…
}

----
Thanks to the messed up modular arithmetics that D has chosen you cannot assume the a non-shared live object does not have a rc==0 due to wrapping integers, in the general case.