On Saturday, 1 April 2017 at 22:08:27 UTC, Walter Bright wrote:
> On 4/1/2017 7:54 AM, deadalnix wrote:
>> It doesn't need any kind of throw new scope Exception, and was proposed,
>> literally, years ago during discussion around DIP25 and alike.
>
> A link to that proposal would be appreciated.

The forum search isn't returning anything useful so I'm not sure how to get that link. However, it goes roughly as follow. Note that it's a solution to solve DIP25+DIP1000+RC+nogc exception and a sludge of other issues, and that comparing it to any of these independently will yield the obvious it is more complex. But that wouldn't be a fair comparison, as one should compare it to the sum of all these proposals, not to any of them independently.

The compiler already has the nothing of "unique" and use it to some extent, for instance to optimize pure functions and allow to do some magic with new allocations. The proposal relax and extend the concept of unique and make it explicit, via the type qualifier 'owned'. Going into the details here would take too long, so i'll just reference this paper ( https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/msr-tr-2012-79.pdf ) for detailed work in C# about this. This is easily applicable to D.

The GC heap is currently composed of several islands. One island per thread + one shared island + one immutable island. The owned qualifier essentially enable to have more island, and each indirection tagged owned is a jump to another island. What's currently understood as "unique" by the compiler can be considered owned, which includes values returned from strongly pure functions and fresh new allocations.

Let's see how that apply to exceptions and the GC:
 - If a T is thrown, the runtime assumes GC ownership of the exception.
 - If a owned(T) is thrown, the runtime takes ownership of the exception (and of the graph of objects reachable from the exception.

When catching:
 - If the catch isn't scope or owned, then is is a "consume" operation. If the runtime had ownership of the exception, it transfers it to the GC.
 - If the catch is scope, then the runtime keeps ownership of the Exception. Exiting the catch block is going to destroy the Exception and the whole island associated with it.
 - If the catch is owned, then the ownership of the Exception is transferred to the catch block. It is then either transferred back to the runtime in case of rethrow, or consumed/destroyed depending on what the catch block is doing with it.

The only operations that need to be disallowed in nogc code are consuming owned such as their ownership is transferred to the GC, in this case, catch blocks which aren't owned or scope.

This mechanism solves numerous other issues. Notably and non exhaustively:
 - General reduction in the amount of garbage created.
 - Ability to transfers ownership of data between thread safely (without cast to/from shared).
 - Safe std.parralelism .
 - Elaborate construction of shared and immutable objects.
 - Safe reference counting.
 - Safe "arena" style reference counting such as: https://www.youtube.com/watch?v=JfmTagWcqoE
 - Solves problems with collection ownership and alike.

On 4/2/2017 2:05 PM, ketmar wrote:
> Walter Bright wrote:
>
>> 1. If the thrown object was not allocated with the GC (such as if it was
>> 'emplaced'), then doing a GC free on it at the catch site will corrupt memory.
>
> no, it won't. it is completely safe to free non-GC-owned memory with GC[0].
>
> [0] http://dpldocs.info/experimental-docs/core.memory.GC.free.html

Yes, you're right.

On Saturday, 1 April 2017 at 13:34:58 UTC, Andrei Alexandrescu wrote:
> Walter and I discussed the following promising setup:
>
> Use "throw new scope Exception" from @nogc code. That will cause the exception to be allocated in a special stack-like region.
>
> If the catching code uses "catch (scope Exception obj)", then a reference to the exception thus created will be passed to catch. At the end of the catch block there's no outstanding reference to "obj" so it will be freed. All @nogc code must use this form of catch.
>
> If the catching code uses "catch (Exception obj)", the exception is cloned on the gc heap and then freed.
>
> Finally, if an exception is thrown with "throw new Exception" it can be caught with "catch (scope Exception obj)" by copying the exception from the heap into the special region, and then freeing the exception on the heap.
>
> Such a scheme preserves backward compatibility and leverages the work done on "scope".
>
>
> Andrei

How would you deal with the Exception payload, e.g. the message string ? If I have to place it on the heap, I lose @nogc, if I place it in the local scope, it will be destroyed before the Exception handler is called and if I restrict the mechanism to fixed payloads, I could have used static Exception objects in the first place.

It may be possible to copy the message string from local scope to the special stack on throw, but in the general case the payload could be an arbitrary tree of Objects allocated on the heap or the stack, or even on the special exception stack itself.

A method I used to pass scope data up the stack is by using "thrower delegates", i.e. the @nogc throwing function doesn't directly throw an Exception, but it calls a delegate that is passed all the payloads as arguments and which throws a static Exception.

This thrower delegate is provided by the Exception catching function, and has the opportunity to read/copy the parts of the payload it is interested in before the stack is unwound.

On Monday, 3 April 2017 at 08:22:41 UTC, Matthias Bentrup wrote:
> How would you deal with the Exception payload, e.g. the message string ?

Yes current proposal are unable to handle properly multiple levels of indirections.

On 4/2/17 4:21 PM, Guillaume Piolat wrote:
> On Sunday, 2 April 2017 at 17:22:11 UTC, Andrei Alexandrescu wrote:
>> On 4/1/17 2:56 PM, Guillaume Piolat wrote:
>>> The other @nogc blocker is .destroy
>>
>> How do you mean that? -- Andrei
>
> https://github.com/dlang/druntime/blob/master/src/object.d#L2732
>
> destroy() infers it's "@nogc"-ness from rt_finalize which is not nothrow
> and not @nogc:
> https://github.com/dlang/druntime/blob/5a94816c8f1d5c225e560151cebe0a09949896a5/src/object.d#L16
>
>
> I guess the rationale is that rt_finalize call Object.~this() and that
> may GC allocate, and throw.
>
> In turn this cause every wrapper using emplace+destroy to not be @nogc.

Got it. Could you please create an issue about this? -- Andrei

On Sunday, 2 April 2017 at 21:27:07 UTC, deadalnix wrote:
> On Saturday, 1 April 2017 at 22:08:27 UTC, Walter Bright wrote:
>> On 4/1/2017 7:54 AM, deadalnix wrote:
>>> It doesn't need any kind of throw new scope Exception, and was proposed,
>>> literally, years ago during discussion around DIP25 and alike.
>>
>> A link to that proposal would be appreciated.
>
> The forum search isn't returning anything useful so I'm not sure how to get that link. However, it goes roughly as follow. Note that it's a solution to solve DIP25+DIP1000+RC+nogc exception and a sludge of other issues, and that comparing it to any of these independently will yield the obvious it is more complex. But that wouldn't be a fair comparison, as one should compare it to the sum of all these proposals, not to any of them independently.
>
[snip]
>
> This mechanism solves numerous other issues. Notably and non exhaustively:
>  - General reduction in the amount of garbage created.
>  - Ability to transfers ownership of data between thread safely (without cast to/from shared).
>  - Safe std.parralelism.
>  - Elaborate construction of shared and immutable objects.
>  - Safe reference counting.
>  - Safe "arena" style reference counting such as: https://www.youtube.com/watch?v=JfmTagWcqoE
>  - Solves problems with collection ownership and alike.

This silence is killing me!

Can one assume that Walter is thinking about deadalnix's detailed proposal above, and that he will give a formal response, once he has given it serious thought, and discussed it with Andrei ?

cheers Nick

Thank you for posting this.

Figuring all this out thoroughly will take some time, so this is just first impressions.

1. we already have some of the benefits of the proposal because D has transitive immutability

2. I'm looking for a solution where exceptions don't rely on the GC to the point where the GC code doesn't even need to be linked in. This proposal appears to maintain a dependence on the GC.

3. It requires annotation of catch declarations with one of "", "scope", or "owned". I expect this would be a significant problem

4. Since the catch block is not type checked against the corresponding throw, the object will have to have noted in it who owns it.

5. The normal case is:

    throw new Exception("message");
    ...
    catch (Exception e) {
         writeln(e.msg);
    }

which would ideally work without involving the GC at all.

6. reducing the amount of GC garbage created is good, but does not solve the problem of "I don't want to use D because of the GC".

This proposal looks promising for making a better garbage collected language, but people want a language with an optional GC.

On Monday, 3 April 2017 at 21:28:39 UTC, Andrei Alexandrescu wrote:
>
> Got it. Could you please create an issue about this? -- Andrei

Sure.
https://issues.dlang.org/show_bug.cgi?id=17297

On Tuesday, 4 April 2017 at 09:45:14 UTC, Walter Bright wrote:
> 6. reducing the amount of GC garbage created is good, but does not solve the problem of "I don't want to use D because of the GC".

I betcha D would be a better language with more users if you just told those people "sorry, it isn't for you then" and focused on improving what we already have.

I've never had a problem using D without a GC as it is now BTW. But that's because I'm more interested in getting work done than in endlessly whining about a language I don't even actually use or understand in Reddit comments.

On Tuesday, 4 April 2017 at 09:45:14 UTC, Walter Bright wrote:
> 1. we already have some of the benefits of the proposal because D has transitive immutability
>

This works hand in hand with D type qualifier system.

> 2. I'm looking for a solution where exceptions don't rely on the GC to the point where the GC code doesn't even need to be linked in. This proposal appears to maintain a dependence on the GC.
>

Then just do:
auto ePtr = malloc(...);
auto e = *(cast(Exception*) &ePtr);
throw e;

Problem solved, you did not used the GC.

This proposal has nothing to do with Exceptions. It just happens to solve the Exception problem, just as it does for many others.

The problem people have with the GC isn't that it need to be linked in, it is that collection cycles create latency that doesn't work for their use case. If allocation are freed, then there is no GC problem.

Nobody is complaining that they cannot use C++ without its runtime. And for the 0.1% who actually need it, they just write custom allocation, and jump through a few hoops. This is a just good engineering. You are trying to solve the wrong problem.

really, in the process you are also murdering another legit use case that is orders of magnitude more common: library writers. They want to write code that'll work when the GC is used or not.

> 3. It requires annotation of catch declarations with one of "", "scope", or "owned". I expect this would be a significant problem
>

This proposal just add the owned type qualifier. scope already exists. It solves many problem for which new syntax have been introduced, so it's rather rich.

"I don't want this proposal that add one new syntax, I'd rather have 4 smaller proposals that each add a new syntax."

> 4. Since the catch block is not type checked against the corresponding throw, the object will have to have noted in it who owns it.
>

throw never leaks, so there is no point in checking the throw. You already throw something that is owned by the GC, in which case you leaked earlier, or you transfer the ownership to the runtime and you haven't leaked yet.

Because throw never leaks, there is no point in butchering throw to make it compatible with nogc.

Back to the catch block, the question is the same as for a function call or anything else really. It either borrow the exception (scope) take ownership of it (owned) or just delegate the work to the GC.

> 5. The normal case is:
>
>     throw new Exception("message");
>     ...
>     catch (Exception e) {
>          writeln(e.msg);
>     }
>
> which would ideally work without involving the GC at all.
>

This cannot be nogc in the general case because e can be reassigned to anything that is owned by the GC. In that specific case, scope can be inferred.

> 6. reducing the amount of GC garbage created is good, but does not solve the problem of "I don't want to use D because of the GC".
>
> This proposal looks promising for making a better garbage collected language, but people want a language with an optional GC.

If you have guarantee that you won't leak, and so will never run collection cycle, you don't have a GC, you effectively have malloc and free. There is no need to butcher the language because there are morons on the internet. really that's hardly newsworthy.