On 2014-09-19 17:32, Andrei Alexandrescu wrote:

> Whenever a reference to a Throwable is copied about, passed to
> functions, the compiler inserts appropriately calls to e.g. incRef and
> decRef. (Compiler may assume they cancel each other for optimization
> purposes.

Assuming this would eventually be implemented for regular classes, it would be nice if it could be made compatible with Objective-C ARC [1].

[1] http://clang.llvm.org/docs/AutomaticReferenceCounting.html

-- 
/Jacob Carlborg

On Saturday, 20 September 2014 at 11:50:28 UTC, Marc Schütz wrote:
> On Saturday, 20 September 2014 at 09:19:21 UTC, Dicebot wrote:
>> Yeah but implicitly convertible to what? If you convert it to `Throwable` your reference counting facilities are circumvented resulting in dangling exception reference at point where `Throwable` gets caught.
>
> As I said, throw/catch is at its core a moving operation. For classes this isn't important (they are references), but for an RC wrapper it would be, so we could specify that.
>
> Move-constructing from an lvalue in the context of D can be seen as a three-step process:
>
> 1) create a temporary, initialize with T.init
> 2) bit-swap the variable with the temporary
> 3) destroy the variable
>
> It can be seen that the value that is to be moved (the RC wrapper) must be non-const (only at the head, it may still be tail-const).
>
> Now, any generic RC type that wants to be implicitly convertible to its payload type must do this via borrowing in order to be safe (see my proposal at [1]). Using const-borrowing, we can guarantee that the wrapper will not be thrown (or moved, in general) as long as borrowed references to its payload exist:
>
>     struct RC(T) {
>         // ...
>         T _payload;
>         scope!(const this) borrow() {
>             return _payload;
>         }
>         alias borrow this;
>     }
>
> This already solves avoiding dangling references to the exception: No references can be left behind when the exception is thrown, and the wrapper will not be destroyed, but moved, thus not releasing the exception's memory.
>
> The second part is really "just" some way to transport the wrapper via the exception mechanism, including support for catching wrappers of derived exception types.
>
>>
>> Special casing catching such wrappers to still preserve original ref-counted type while pretending to be Throwable at call site sounds like a terrible hack, much worse than any sort of ARC complexity.
>
> IMO it would only a hack _if_ they were indeed special cased. I'm sure we can find a more generic mechanism that would allow this to be implemented cleanly. Note that the other requirements I described above (borrowing, move semantics) are also things that just happen to be usable here: they are desirable in general, exceptions are just one possible application.
>
> [1] http://wiki.dlang.org/User:Schuetzm/scope

The mechanism might simply be to allow anything to be thrown that
a) supports moving, e.g. provides a method `release()` that returns a unique expression (this requires moving and uniqueness to be specified first, which is a good idea anyway), and
b) is implicitly convertible to Throwable.

On Saturday, 20 September 2014 at 08:42:51 UTC, Uranuz wrote:
> I'm quite a noobie in memory models but from position of user of D language I have some ideas about syntax of switching between memory models. I think that having everywhere declarations using wrapper structs (like RefCounted) is not very user-friendly. But still we need some way to say to compiler how we want to do memory management. I have read tutorial about Rust language. It looks not very clear for me, but I like the idea that type of memory management is included in variable declaration.
>
> Intead of wrapper struct like scoped!A or RefCounted!A we could declare variables with annotations (attributes in D) to say what memory model we want to use. Compiler should understand this annotation and create some code for memory management. I thing it's possible to declare some interface (maybe struct with compile-time duck typing) to support some user-defined memory models or modifications of basic (implemented in a language) memory models.
>
> Why I am saying about annotations? Because we can annotate some function, class or just block at the module scope that should use some sort of memory management and compiler will create corresponding code. In that case we don't need to put wrapper struct around all variables that use ref counting. We just annotate some function or class declaration as ref-counted and that's it!
>
> It's just a concept of what I like to see in the language design of D in future)) Of course there are a lot of problems in a practice.
>
> Also I think that we shouldn't impose some way of memory model to programmer and let him choose his approach but syntax should be simple and clear as much as it could be. Also some defaults should be for most common cases and for *novice* users of D.
>
> Waiting for critic or thoughts!!!))

You are right that there is an intricate relationship between the various types of memory (better: ownership) management, allocators, ref-counting, uniqueness, moving etc, as you write in your other post. They interact in very specific ways. For this reason I don't think it is feasible or desirable to set the type of memory management "from the outside", the types in question need to know how their innards work. In the same vein, if your code uses a specific memory management strategy, it has to be written in a way conforming to it. Walter has already stated that: you cannot simply slap an attribute onto your code saying "please do reference counting", and expect it to work, especially not efficiently.

I think we can get a lot further if we work out said relationships, and create good general purpose wrapper types that implement the different strategies accordingly. I believe, borrowing is crucial to this, which is why I made a proposal about it [1], which also deals with the other related topics.

You do have a point about a simple, accessible syntax being important. Still, I don't see a big difference between @annotations and RC!T wrappers in this regard (except that the former could apply to entire sections of code, which is not a good idea IMO). And I think with auto/scope/const type deduction, code is quite pleasant to read.

[1] http://wiki.dlang.org/User:Schuetzm/scope

On Saturday, 20 September 2014 at 12:27:23 UTC, Jacob Carlborg wrote:
> Assuming this would eventually be implemented for regular classes, it would be nice if it could be made compatible with Objective-C ARC [1].
>
> [1] http://clang.llvm.org/docs/AutomaticReferenceCounting.html

Isn't Objective-C ARC known to be terribly efficient?

How often do you store an exception reference anyway that escapes a catch block? I think all this talk is overkill to solve a non-problem in 99% of practice.


Correct me if I'm wrong, but aren't *all* exceptions in a particular thread generally unreferenced at the end of a catch() block, unless the programmer explicitly escaped that reference?

If so, we don't need refcounting! Here's my solution:

1) Throwables are allocated in a separate thread-local memory pool than most other objects. The @nogc version just aborts the program if this pool ever runs out of memory. The gc version can do a collection cycle and grow the size if necessary. (Though if you need a lot of Throwable objects alive at once, I question wtf is up with your code...)

2) class Throwable adds a method, gcClone, which copies it to the regular GC heap for those cases when you do want to store it or pass it between threads or whatever.

3) A function, @system void clearExceptions() is added. You may manually call this when you are done handling exceptions in this thread.

4) Might also have @safe void prepareExceptions() which pre-allocates the pool. This could also be done automatically or on demand or whatever.



Advantages:

* Throwables are still GC managed as far as most code is concerned. The only time you need caution is if you call the new @system function to free the pool... and even then, you just make sure you don't escape those references before you do.

* If you want to avoid the GC, just manually clear your exceptions from time to time.

* Allocating/deallocating from this specialized memory pool is prolly faster than any other scheme anyway.

* No language changes required, this is all library stuff.

Disadvantages:

* There's a few manual steps to get all the benefit. (We could insert a call to clearExceptions at the end of catch blocks automatically, but without a static check to ensure the objects actually haven't escaped, this would break memory safety. But a manual call isn't that big of a deal)

* If you do it wrong, you'll be annoyed.

* ???



I really think this is a win. I've done a proof of concept before by hacking _d_newclass to use the pool based on the TypeInfo passed in, gives a speedup in all the simple cases I tried. But since that hack still uses new, it wouldn't pass the @nogc test. However, a library function like emplace could be @nogc and do the same thing. Since @nogc is an addition to the function, you'll be modifying it anyway, so changing "throw new" to whatever the new lib function is called can be done at the same time.

It is the catch code that have to worry about freeing the pool.... and even then, only if you want to avoid the GC. Normal code can just let the pool be collected when it is collected.

I think I'll write a little module we can play with.

On Saturday, 20 September 2014 at 14:31:36 UTC, Adam D. Ruppe wrote:
> How often do you store an exception reference anyway that escapes a catch block? I think all this talk is overkill to solve a non-problem in 99% of practice.

Pretty much any time you do fibers + async I/O : to emulate blocking API one needs to catch and store exceptions from I/O routines so that later those can be re-thrown from resumed fiber context.

Am 20.09.2014 15:40, schrieb Ola Fosheim Grostad:
> On Saturday, 20 September 2014 at 12:27:23 UTC, Jacob Carlborg wrote:
>> Assuming this would eventually be implemented for regular classes, it
>> would be nice if it could be made compatible with Objective-C ARC [1].
>>
>> [1] http://clang.llvm.org/docs/AutomaticReferenceCounting.html
>
> Isn't Objective-C ARC known to be terribly efficient?
>

It requires compiler support, though.

- Place retain/release at each place expected by Cocoa programming conventions, as if manually written

- Do a second pass with data-flow analysis to remove retain/release pairs that don't escape current scope.

If you have an Apple account search for:

"Transitioning to ARC Release Notes", ARC
"Memory Management Programming Guide for Core Foundation" - Cocoa conventions and allocators
"WWDC 2012: Adopting Automatic Reference Counting" - Overral description
"WWDC 2012: What's New in LLVM" - Some slides about ARC Optimizer

--
Paulo

On Saturday, 20 September 2014 at 14:33:40 UTC, Paulo Pinto wrote:
> Am 20.09.2014 15:40, schrieb Ola Fosheim Grostad:
>> On Saturday, 20 September 2014 at 12:27:23 UTC, Jacob Carlborg wrote:
>>> Assuming this would eventually be implemented for regular classes, it
>>> would be nice if it could be made compatible with Objective-C ARC [1].
>>>
>>> [1] http://clang.llvm.org/docs/AutomaticReferenceCounting.html
>>
>> Isn't Objective-C ARC known to be terribly efficient?

I'm really sorry, but that was a very confusing typo on my part.

I meant "INefficient"! :-)

> It requires compiler support, though.

:-) Yes, but I was more talking about the representation in memory based on reading the source code that can be found in the repository. I read it the last time ARC was discussed…

But, I am not 100% sure what code Apple actually ships.

On Saturday, 20 September 2014 at 14:33:21 UTC, Dicebot wrote:
> Pretty much any time you do fibers + async I/O : to emulate blocking API one needs to catch and store exceptions from I/O routines so that later those can be re-thrown from resumed fiber context.

Ah, indeed, and that could have a great many of them alive at once. Blargh, it'd need a real allocator to handle freeing them out of order.

Nevertheless though, I still think the lifetime management there is simple enough for the user-programmer that freeing it manually isn't a big hassle and not worth making major changes to the language over.

On 2014-09-20 16:33, Paulo Pinto wrote:

> It requires compiler support, though.

The first thing I asked in this thread was "Are you suggesting we implement ARC?" and the answer was "Yes" [1]. So it looks like Andrei already wants to implement ARC. My definition of ARC is that the compiler inserts the calls to retain/release (or whatever you call them).

[1] http://forum.dlang.org/thread/lvhiam$1pno$1@digitalmars.com#post-lvi0ve:2429il:241:40digitalmars.com

-- 
/Jacob Carlborg