On Tuesday, 13 September 2016 at 22:19:54 UTC, Jonathan M Davis wrote:
> The big problem with exceptions being allocated by the GC isn't really the GC but @nogc.

No the problem IS @nogc . Allocating with the GC is absolutely not a problem is you deallocate properly. What is a problem is when you leak (ie, when the ownership is transferred to the GC). If you don't leak, GC do not kicks in.

On Tue, Sep 13, 2016 at 03:19:54PM -0700, Jonathan M Davis via Digitalmars-d wrote: [...]
> But none of the code that's marked @nogc can throw an exception unless you're either dealing with pre-allocated exceptions (in which case, they're less informative),

I don't see why pre-allocated exceptions would be less informative. You can always modify the exception object before throwing it, after all. In fact, I've always wondered about the feasibility of a @nogc exception handling system where the exception is emplaced onto a fixed static buffer, so that no allocation (except at the start of the program) is actually necessary. Of course, chained exceptions throw(!) a monkey wrench into the works, but assuming we forego chained exceptions, wouldn't this work around the problem of being unable to allocate exceptions in @nogc code? (Albeit with its own limitations, obviously. But it would be better than being unable to use exceptions at all in @nogc code.)


[...]
> So, I really think that we need to find a way to make it so that exceptions aren't GC allocated normally anymore - or at least have a way to reasonably and easily not be GC allocated - but the problem is @nogc, not the actual memory management or its cost.
[...]

There's nothing about the 'throw' keyword that requires GC allocation. It's just that `throw new Exception(...)` has become a standard incantation. The exception object itself can, for example, be emplaced onto a static buffer as I propose above.


T

-- 
I think Debian's doing something wrong, `apt-get install pesticide', doesn't seem to remove the bugs on my system! -- Mike Dresser

On Tuesday, 13 September 2016 at 22:19:54 UTC, Jonathan M Davis wrote:
> So, I really think that we need to find a way to make it so that exceptions aren't GC allocated normally anymore - or at least have a way to reasonably and easily not be GC allocated - but the problem is @nogc, not the actual memory management or its cost.
>
> - Jonathan M Davis

Introduce RefCountedException?

Also that the pattern almost always is "throw new" and *very* rarely it is "Exception e = new ...; throw e;". I think we might be able to take advantage of that (e.g. "throw new E" could be a special case of "new" that allocates on some sort of "Exception" heap that is manually managed, or recognize RefCountedException).

On Tuesday, 13 September 2016 at 19:30:16 UTC, Marco Leise wrote:
> Am Tue, 13 Sep 2016 18:16:27 +0000
> schrieb Laeeth Isharc <laeethnospam@nospamlaeeth.com>:
>
>> Thanks you for the clear explanation.   So if you don't have GC allocations within RC structures and pick one or the other,
>>  then the concern does not apply?
>
> That's right. Often such structures contain collections of
> things, not just plain fields. And a list or a hash map working
> in a @nogc environment typically checks its contained type for
> any pointers with hasIndirections!T and if so adds its storage
> area to the GC scanned memory to be on the safe side.
> That means every collection needs a way to exempt its contents
> from GC scanning and the user needs to remember to tell it so.
>
> A practical example of that are the EMSI containers, but other containers, i.e. in my own private code look similar.
>
> https://github.com/economicmodeling/containers
>
>   struct DynamicArray(T, Allocator = Mallocator, bool supportGC = shouldAddGCRange!T)a certain
>   {
>       ...
>   }
>
> Here, when you use a dynamic array you need to specify the type and allocator before you get to the point of opting out of GC scanning. Many will prefer concise code, go with GC scanning to be "better safe than sorry" or don't want to fiddle with the options as long as the program works. This is no complaint, I'm just trying to draw a picture of how people end up with more GC scanned memory than necessary. :)

Thanks,  Marco.

So to a certain extent it's a problem of perception and of cognitive load when one is coming to the language - none of the steps taken together is necessarily difficult,  but cumulatively it's a lot to take in or figure out yourself - then, as you say people do what's easy or manageable, and then those habits come to constitute ones sense of what's implied by the language and implementation when that's not necessarily right.

There's a great blog post to be written on getting along with and without the GC for fun and profit - showing how to do the things you discussed,comparing the amount of GC D generates with eg Java to make vivid and concrete just how the situation is different,  illustrating how one can use the allocator for significant allocations alongside the GC for trivial ones,  illustrating how to pre allocate buffers,  and finally demonstrating how in practice to use the GC profiling instrumentation we have.

Or perhaps rather a series of blog posts.   It would be helpful there to get across how the situation has improved.   Because the topic is almost guaranteed to come up in social media discussions (which matter as when people Google for it that is often what they will find), and we live in an age where the complexity means people use heuristic thinking,  and you can hardly blame them when there is no one place to point them too (as we have for ranges,  slices etc).

I would write it myself if I had time and understood better not just garbage collection techniques,  but also how other languages are in practice.   But it's not something for me at this point,  and so someone else will have to do so.   I will see when it is a bit quieter in a year or two if someone that works with me wants to do it,  but in the meantime it's a great opportunity to improve the messaging.

The wiki could also be a bit clearer last I looked on low-GC solutions.   Eg you won't easily find EMSI containers from a casual browse.

Walter has mentioned the value as a technology professional from blogging and having a personal page and I think that is right.


Laeeth

On 9/13/2016 4:13 PM, H. S. Teoh via Digitalmars-d wrote:
> There's nothing about the 'throw' keyword that requires GC allocation.
> It's just that `throw new Exception(...)` has become a standard
> incantation. The exception object itself can, for example, be emplaced
> onto a static buffer as I propose above.

There's currently nothing that would prevent any handler code from saving a reference to the thrown exception. Statically allocating them would break that.

On 9/13/2016 4:59 AM, Shachar Shemesh wrote:
> Here's my worries about the hybrid approach. The GC run time is proportional not
> to the amount of memory you manage with the GC, but to the amount of memory that
> might hold a pointer to a GC managed memory. In other words, if most of my
> memory is RC managed, but some of it is GC, I pay the price of both memory
> manager on most of my memory.

Memory allocated with malloc() is unknown to the GC. This works fine unless a reference to the GC memory is inserted into malloc'd data, which is why there's an API to the GC to let it know about such things.

On 9/13/2016 11:24 AM, deadalnix wrote:
> No you don't, as how often the GC kicks in depend of the rate at which you
> produce garbage, which is going to be very low with an hybrid approach.


Also, if you only use GC for exceptions, there isn't going to be much memory it needs to scan.

On 09/13/2016 05:43 PM, Walter Bright wrote:
> The all-or-nothing approach to using the GC is as wrong as any
> programming methodology is.

That's a bit much.

> There's currently nothing that would prevent any handler code from
> saving a reference to the thrown exception. Statically allocating
> them would break that.

Didn't we reach the agreement to closely investigate a reference counted exceptions hierarchy?


Andrei

On 9/13/2016 6:09 PM, Andrei Alexandrescu wrote:
> On 09/13/2016 05:43 PM, Walter Bright wrote:
>> There's currently nothing that would prevent any handler code from
>> saving a reference to the thrown exception. Statically allocating
>> them would break that.
>
> Didn't we reach the agreement to closely investigate a reference counted
> exceptions hierarchy?

Yes we did. The design of Phobos would be better and more flexible if exceptions didn't rely on the GC.

On 14/09/16 02:59, Walter Bright wrote:
>
> Memory allocated with malloc() is unknown to the GC. This works fine
> unless a reference to the GC memory is inserted into malloc'd data,
> which is why there's an API to the GC to let it know about such things.
>

But if you do want to allow it, then my original problem comes back. You have to scan the malloced memory because you are not sure where that memory might contain pointers to GC managed memory.

And the hybrid hybrid approach (i.e. - only some of the memory allocated by malloc is scanned) is a wasp nest of potential bugs and problems.

Shachar