My previous version did not survive implementation. Here's the revised version. I have submitted it as a DIP, and there's a trial implementation up:

    https://github.com/dlang/dmd/pull/6681

---------------------------------------------
	Exceptions and @nogc version 2

Problem
=======

Exceptions are assumed to be GC collected by the EH design, in that no
attempt is made to control copies or lifetimes. This dependency is not
a performance issue, as exceptions are presumed to be slow.

The issue is it impairs use of @nogc on any code that throws exceptions,
and prevents building D programs that do not link in the GC runtime.

To fix this, the allocation and destruction of the exception objects
must be completely controlled.

Solution
========

Make Throwable optionally ref counted. Add a field `_refcount` which
is !=0 when it is a ref counted instance. The number of parents of
a refcounted Throwable is _refcount+1. This member is private and only
accessible via the @system member function refcount(), to prevent
its use in @safe code.

The only place a refcounted Throwable is ever created is when the following
statement is in the user code:

    throw new E(string);

where E is Throwable or derived from Throwable. Instead of calling the usual
_d_newclass() to allocate E on the GC heap, it will call the new function
_d_newThrowable() which will allocate E and intialize it for refcounting.

When the exception is thrown, either _d_throwc() or _d_throwdwarf() gets called,
which is where druntime takes over. The refcount is incremented in these functions,
usually from 1 to 2.

The thrown object will then wind up either in a catch statement, or in the `next`
linked list of thrown exceptions in flight.

A catch variable `e` as in:

    catch (E e)
    {
        ...
    }

becomes an RAII object, meaning that it is destroyed at the closing }. Such
destruction is done by calling:

    _d_delThrowable(e);

which will test e._refcount to see if it is a ref counted object. If not,
it does nothing. If it is, the refcount is decremented, and if it hits
one then e's destructor is called, followed by free'ing the memory used
by `e`.

In catch blocks, `e` is regarded as `scope`, so that it cannot escape the
catch block. As a special case, if `e` is thrown in the catch block as:

    throw e;

then `e` can escape, and this works because as mentioned before `_d_throwc()` or
`_d_throwdwarf()` will increment the reference count.

The destructor for `Throwable` will, if the refcount is 1, call _d_delThrowable(e.next),
i.e. on the head of the chained list of exceptions. This means that the chained
list can be a mixed collection of refcounted and not-refcounted Throwables, although
the refcounted ones can only be free'd when their antecedent gets reaped by whatever
means.

The chained list must be protected so it cannot be altered or escaped by user code.

Legacy Code Breakage
--------------------

This will break an unknown amount of existing code.

Breakage will come in the form of:

1. leaking Exception objects from catch clauses (caught by
the compiler)

2. Disallowing Exception objects with postblit fields.

3. Catch objects being 'scope' will cause problems in that
everything done with those objects will also have to be 'scope'.
The most likely problem will be printing the objects which
relies on Object.toString() which is not 'scope'. One possible
solution is to force Throwable.toString() to be 'scope', which
will likely cause minimal disruption. Of course, compiling
with -dip1000 will disable such checking and can work in
the interim. Code that needs to leak the thrown exception object
can clone the object.

Conclusion
----------

The result of this should be no leaking memory, no need to link
in the GC, and memory safety.

GC, stack, and refcounted exception objects can coexist in the same
program.

References
----------

http://www.digitalmars.com/d/archives/digitalmars/D/Exceptions_in_nogc_code_299261.html

On Sunday, 9 April 2017 at 03:26:14 UTC, Walter Bright wrote:
> My previous version did not survive implementation. Here's the revised version. I have submitted it as a DIP, and there's a trial implementation up:
>
>     https://github.com/dlang/dmd/pull/6681

Two small procedural notes

- I don't see a DIP proposal in this list:

https://github.com/dlang/DIPs/pulls

- I suggest you not use -dip1006 as the flag to enable this, as the DIP numbering used to be assigned after the DIP proposal was merged and there's currently another proposal in that list that makes the same mistake of claiming that number.  Better to use some other flag for now, until you're sure of the DIP number, or the name clashes will be confusing.

On Sunday, 9 April 2017 at 03:26:14 UTC, Walter Bright wrote:
> My previous version did not survive implementation. Here's the revised version. I have submitted it as a DIP, and there's a trial implementation up:
>
>     https://github.com/dlang/dmd/pull/6681
>
> [ . . . ]

Pardon my ignorance but it does not seem to be clear how preallocated exceptions would work. What about exceptions allocated ultimately by the GC but actually via an intermediate allocator (e.g. FallBack)?

On 4/8/2017 9:30 PM, Nicholas Wilson wrote:
> Pardon my ignorance but it does not seem to be clear how preallocated exceptions
> would work. What about exceptions allocated ultimately by the GC but actually
> via an intermediate allocator (e.g. FallBack)?

You can allocate them any way you like - the runtime won't recognize those as its own refcounted objects, and will not try to free them.

On Sunday, 9 April 2017 at 03:26:14 UTC, Walter Bright wrote:
> Solution
> ========
>
> Make Throwable optionally ref counted.

I like this new direction...

> The only place a refcounted Throwable is ever created is when the following
> statement is in the user code:
>
>     throw new E(string);
>

... but why not go all the way, making it "always" refcounted? (for any "new E", not emplace).

On Sunday, 9 April 2017 at 03:26:14 UTC, Walter Bright wrote:
> The only place a refcounted Throwable is ever created is when the following
> statement is in the user code:
>
>     throw new E(string);

object aMemoryLeak;

void someFunc()
{   throw (aMemoryLeak = new Exception("hello world!"));
}

Would the compiler warn about this or make the exception normally garbage collected?

I'm not sure, but perhaps an entirely different syntax would be in place for throwing a refcounted object, so there's no special casing. For example, could a throw statement without the new keyword mean a refcounted throw?

On Sunday, 9 April 2017 at 03:26:14 UTC, Walter Bright wrote:
> My previous version did not survive implementation. Here's the revised version. I have submitted it as a DIP, and there's a trial implementation up:
>
> [...]

Just a quick note to reduce confusion for reviewers:

> The number of parents of a refcounted Throwable is _refcount+1.

I believe what you mean here is "_refcount-1".
Later you say that the exception is reclaimed when the refcount goes to 1.

On 4/9/2017 1:16 AM, Daniel N wrote:
> ... but why not go all the way, making it "always" refcounted? (for any "new E",
> not emplace).

Backwards compatibility, for one. For another, a general mechanism for safe refcounting of classes has eluded us.

On 4/9/2017 1:35 AM, Dukc wrote:
> object aMemoryLeak;
>
> void someFunc()
> {   throw (aMemoryLeak = new Exception("hello world!"));
> }
>
> Would the compiler warn about this or make the exception normally garbage
> collected?

That would be a regular gc allocated Exception.


> I'm not sure, but perhaps an entirely different syntax would be in place for
> throwing a refcounted object, so there's no special casing. For example, could a
> throw statement without the new keyword mean a refcounted throw?

Didn't want to do a new syntax.

On 4/9/2017 12:05 PM, Andrew Godfrey wrote:
> Just a quick note to reduce confusion for reviewers:
>
>> The number of parents of a refcounted Throwable is _refcount+1.
>
> I believe what you mean here is "_refcount-1".
> Later you say that the exception is reclaimed when the refcount goes to 1.

Thanks for the correction.