On Thursday, 17 April 2014 at 15:02:27 UTC, Dicebot wrote:
> void foo(OutputRange buffer)
> {
>     buffer.put(42);
> }
>
> `foo` can't be @nogc here if OutputRange uses GC as backing allocator. However I'd really like to use it to verify that no

Can't you write foo as a template? Then if "buffer" is a ring buffer the memory might be allocated by GC, which is ok if put() does not call the GC and is marked as such.

Where this falls apart is when you introduce a compacting GC and the @nogc code is run in a real time priority thread. Then you need both @nogc_function_calls and @nogc_memory .

Of course, resorting to templates requires some thinking-ahead, and makes reuse more difficult.

You'll probably end up with the @nogc crowd creating their own NoGCOutputRange… :-P

Ola.

Ola Fosheim Grøstad:

> Where this falls apart is when you introduce a compacting GC and the @nogc code is run in a real time priority thread. Then you need both @nogc_function_calls and @nogc_memory .

Perhaps the @nogc proposal is not flexible enough. So probably the problem needs to be looked from a higher distance to find a smarter and more flexible solution. Koka and other ideas appeared in this thread can be seeds for ideas.

Bye,
bearophile

On Thursday, 17 April 2014 at 15:39:38 UTC, Ola Fosheim Grøstad wrote:
> On Thursday, 17 April 2014 at 15:02:27 UTC, Dicebot wrote:
>> void foo(OutputRange buffer)
>> {
>>    buffer.put(42);
>> }
>>
>> `foo` can't be @nogc here if OutputRange uses GC as backing allocator. However I'd really like to use it to verify that no
>
> Can't you write foo as a template? Then if "buffer" is a ring buffer the memory might be allocated by GC, which is ok if put() does not call the GC and is marked as such.

put() may call GC to grow the buffer, this is the very point. What is desired is to check if anything _else_ does call GC, thus the "weak @nogc" parallel.

> Where this falls apart is when you introduce a compacting GC and the @nogc code is run in a real time priority thread. Then you need both @nogc_function_calls and @nogc_memory .

True hard real-time is always special, I am speaking about "softer" but still performance-demanding code (like one that is used in Sociomantic).

> Of course, resorting to templates requires some thinking-ahead, and makes reuse more difficult.

I don't see how templates can help here right now.

> You'll probably end up with the @nogc crowd creating their own NoGCOutputRange… :-P
>
> Ola.

On Thursday, 17 April 2014 at 15:48:29 UTC, bearophile wrote:
> Ola Fosheim Grøstad:
>
>> Where this falls apart is when you introduce a compacting GC and the @nogc code is run in a real time priority thread. Then you need both @nogc_function_calls and @nogc_memory .
>
> Perhaps the @nogc proposal is not flexible enough. So probably the problem needs to be looked from a higher distance to find a smarter and more flexible solution. Koka and other ideas appeared in this thread can be seeds for ideas.
>
> Bye,
> bearophile

Reason why @nogc is desired in general is because it is relatively simple and can be done right now. That alone buts it above all ideas with alternate GC implementation and/or major type system tweaks.

It only needs some tweaks to make it actually useful for common-enoough practical cases.

>
> @nogc
> module mymodule;
>

This is precisely what I had in mind.

On Thursday, 17 April 2014 at 15:02:27 UTC, Dicebot wrote:
> === Problem #1 ===
>
> First problem is that, by an analogy with `pure`, there is no such thing as "weakly @nogc@". A common pattern for performance intensive code is to use output buffers of some sort:
>
> void foo(OutputRange buffer)
> {
>     buffer.put(42);
> }
>
> `foo` can't be @nogc here if OutputRange uses GC as backing allocator. However I'd really like to use it to verify that no hidden allocations happen other than those explicitly coming from user-supplied arguments. In fact, if such "weakly @nogc" thing would have been available, it could be used to clean up Phobos reliably.

I don't really see how this is really any different than safe, nothrow or pure attributes.

Either your code is templated, and the attributes get inferred.

Or it's not templated, and you have to rely on `put`'s base-class signature. If it's not marked @nogc (or safe, pure, or nothrow), then that's that.

--------

That said, your proposal could be applied for all attributes in general. Not just @nogc in particular. In practice though, a simple unittest should cover all your needs. simply create a @nogc (pure, nothrow, safe, ctfe-able) unitest, and call it with a trivial argument. If it doesn't pass, then it probably means you made a gc-related (or impure, throwing, unsafe) call that's unrelated to the passed parameters.

In any case, that's how we've been doing it in phobos since we've started actually caring about attributes.

On 4/16/2014 8:13 PM, Manu via Digitalmars-d wrote:
> On 17 April 2014 03:37, Walter Bright via Digitalmars-d
> <digitalmars-d@puremagic.com <mailto:digitalmars-d@puremagic.com>> wrote:
>     ARC has very serious problems with bloat and performance.
> This is the first I've heard of it, and I've been going on about it for ages.

Consider two points:

1. I can't think of any performant ARC systems.

2. Java would be a relatively easy language to implement ARC in. There's probably a billion dollars invested in Java's GC. Why not ARC?


> Obviously, a critical part of ARC is the compilers ability to reduce redundant
> inc/dec sequences. At which point your 'every time' assertion is false. C++
> can't do ARC, so it's not comparable.

C++ has shared_ptr, with all kinds of escapes.


> With proper elimination, transferring ownership results in no cost, only
> duplication/destruction, and those are moments where I've deliberately committed
> to creation/destruction of an instance of something, at which point I'm happy to
> pay for an inc/dec; creation/destruction are rarely high-frequency operations.

inc/dec isn't as cheap as you imply. The dec usually requires the creation of an exception handling unwinder to do it.


> Have you measured the impact?

No. I don't really know how I could, as I haven't seen an ARC system.


> I've never heard of Obj-C users complaining about the inc/dec costs.

Obj-C only uses ARC for a minority of the objects.


> How often does ref fiddling occur in reality? My guess is that with redundancy
> elimination, it would be surprisingly rare, and insignificant.

Yes, I would be surprised.

>     Further problems with ARC are inability to mix ARC references with non-ARC
>     references, seriously hampering generic code.
> That's why the only workable solution is that all references are ARC references.
> The obvious complication is reconciling malloc pointers, but I'm sure this can
> be addressed with some creativity.
>
> I imagine it would look something like:
> By default, pointers are fat: struct ref { void* ptr, ref_t* rc; }

First off, now pointers are 24 bytes in size. Secondly, every pointer dereference becomes two dereferences (not so good for cache performance).


> malloc pointers could conceivably just have a null entry for 'rc' and therefore
> interact comfortably with rc pointers.
> I imagine that a 'raw-pointer' type would be required to refer to a thin
> pointer. Raw pointers would implicitly cast to fat pointers, and a fat->thin
> casts may throw if the fat pointer's rc is non-null, or compile error if it can
> be known at compile time.

Now we throw in a null check and branch for pointer operations.


> Perhaps a solution is possible where an explicit rc record is not required (such
> that all pointers remain 'thin' pointers)...
> A clever hash of the pointer itself can look up the rc?
> Perhaps the rc can be found at ptr[-1]? But then how do you know if the pointer
> is rc allocated or not? An unlikely sentinel value at ptr[-1]? Perhaps the
> virtual memory page can imply whether pointers allocated in that region are ref
> counted or not? Some clever method of assigning the virtual address space so
> that recognition of rc memory can amount to testing a couple of bits in pointers?
>
> I'm just making things up,

Yes.

> but my point is, there are lots of creative
> possibilities, and I have never seen any work to properly explore the options.

ARC has been known about for many decades. If you haven't seen it "properly explored", perhaps it isn't as simple and cost-effective as it may appear at first blush.


> So then consider ARC seriously. If it can't work, articulate why. I still don't
> know, nobody has told me.
> It works well in other languages, and as far as I can tell, it has the potential
> to produce acceptable results for _all_ D users.

What other languages?


> iOS is a competent realtime platform, Apple are well known for their commitment
> to silky-smooth, jitter-free UI and general feel.

A UI is a good use case for ARC. A UI doesn't require high performance.


> Okay. Where can I read about that? It doesn't seem to have surfaced, at least,
> it was never presented in response to my many instances of raising the topic.
> What are the impasses?

I'd have to go look to find the thread. The impasses were as I pointed out here.


> I'm very worried about this. ARC is the only imaginary solution I have left. In
> lieu of that, we make a long-term commitment to a total fracturing of memory
> allocation techniques, just like C++ today where interaction between libraries
> is always a massive pain in the arse. It's one of the most painful things about
> C/C++, and perhaps one of the primary causes of incompatibility between
> libraries and frameworks. This will transfer into D, but it's much worse in D
> because of the relatively high number of implicit allocations ('~', closures, etc).

There are only about 3 cases of implicit allocation in D, all easily avoided, and with @nogc they'll be trivial to avoid. It is not "much worse".


> Frameworks and libraries become incompatible with each other, which is a problem
> in C/C++ that modern languages (java, C#, python, etc) typically don't suffer.

A GC makes libraries compatible with each other, which is one reason why GCs are very popular.

On 4/17/2014 8:02 AM, Dicebot wrote:
> === Problem #1 ===
>
> First problem is that, by an analogy with `pure`, there is no such thing as
> "weakly @nogc@". A common pattern for performance intensive code is to use
> output buffers of some sort:
>
> void foo(OutputRange buffer)
> {
>      buffer.put(42);
> }
>
> `foo` can't be @nogc here if OutputRange uses GC as backing allocator. However
> I'd really like to use it to verify that no hidden allocations happen other than
> those explicitly coming from user-supplied arguments. In fact, if such "weakly
> @nogc" thing would have been available, it could be used to clean up Phobos
> reliably.
>
> With current limitations @nogc is only useful to verify that embedded code which
> does not have GC at all does not use any GC-triggering language features before
> it comes to weird linker errors / rt-asserts. But that does not work good either
> because of next problem:

Remember that @nogc will be inferred for template functions. That means that whether it is @nogc or not will depend on its arguments being @nogc, which is just what is needed.


> === Problem #2 ===
>
> The point where "I told ya" statement is extremely tempting :) bearophile has
> already pointed this out - for some of language features like array literals you
> can't be sure about possible usage of GC at compile-time as it depends on
> optimizations in backend. And making @nogc conservative in that regard and
> marking all literals as @nogc-prohibited will cripple the language beyond reason.
>
> I can see only one fix for that - defining clear set of array literal use cases
> where optimizing GC away is guaranteed by spec and relying on it.

I know that you bring up the array literal issue and gc a lot, but this is simply not a major issue with @nogc. The @nogc will tell you if it will allocate on the gc or not, on a case by case basis, and you can use easy workarounds as necessary.

On 4/17/2014 9:42 AM, monarch_dodra wrote:
> That said, your proposal could be applied for all attributes in general. Not
> just @nogc in particular. In practice though, a simple unittest should cover all
> your needs. simply create a @nogc (pure, nothrow, safe, ctfe-able) unitest, and
> call it with a trivial argument. If it doesn't pass, then it probably means you
> made a gc-related (or impure, throwing, unsafe) call that's unrelated to the
> passed parameters.

Yup, that should work fine.

On Thu, 17 Apr 2014 04:35:34 -0400, Walter Bright <newshound2@digitalmars.com> wrote:

> On 4/16/2014 8:13 PM, Manu via Digitalmars-d wrote:

>
>> I've never heard of Obj-C users complaining about the inc/dec costs.
>
> Obj-C only uses ARC for a minority of the objects.

Really? Every Obj-C API I've seen uses Objective-C objects, which all use RC.

>> iOS is a competent realtime platform, Apple are well known for their commitment
>> to silky-smooth, jitter-free UI and general feel.
>
> A UI is a good use case for ARC. A UI doesn't require high performance.

I've written video processing/players on iOS, they all use blocks and reference counting, including to do date/time processing per frame. All while using RC network buffers. And it works quite smoothly.

-Steve