On Sunday, 9 February 2014 at 18:50:57 UTC, Adam D. Ruppe wrote:
> On Sunday, 9 February 2014 at 18:05:42 UTC, Jakob Ovrum wrote:
>> This isn't accurate. GC is inherent to the `new` operator, not to classes.
>
> The new operator doesn't necessarily have to GC. It isn't hard to hack it to use another method, even for a particular type of object: http://arsdnet.net/dcode/except.d

The `new` operator is intended to imply the infinite lifetime model as of the deprecation of overloading `new` and `delete`.

On Sunday, 9 February 2014 at 20:32:54 UTC, Dmitry Olshansky wrote:
> Might be a good idea but compiler is pretty conservative with what can be created at compile-time and emplace may not play nice with CTFE.

I didn't intend to imply compile-time construction, just statically allocate `ubyte[__traits(classInstanceSize, Ex)]` and lazily emplace `Ex` there instead of GC-allocating with `new`. It would be doable with `std.typecons.Scoped` if that type wasn't naively anonymized a couple of releases ago.

On Sunday, 9 February 2014 at 20:26:20 UTC, Dmitry Olshansky wrote:
> I'm saying that basically classes imply infinite lifetime model. Then you may work extra hard and do things like emplace and manual allocation.

Infinite lifetime is also only with `new`. The "extra work" with emplace and manual allocation is the domain of library code (e.g. `C c = alloc!C(ctorArgs);`).

The only issues with using library code instead of `new` are details like allocation of non-static nested classes and allocation of classes using private constructors etc.

It is true that classes rely on *uniqueness* to some extent, but uniqueness is not particular to GC memory.

On Sunday, February 09, 2014 21:57:13 Dmitry Olshansky wrote:
> Split out of "List of Phobos functions that allocate memory?".
> 
> To reiterate, here is some critique, compiled:
> 
> 1. Exceptions are class instances, hence (by default) are allocated on GC heap. This is wrong default, GC is no place for temporaries.

The more I think about it, the less I'm convinced that this is a big deal. Sure, it would be nice if they were malloc-ed and reference-counted so that they'd go away immediately after they were used and didn't risk triggering a collection, but for exceptions to work right, they need inheritance, so they have to be classes, and since exceptions are generally rare, having the extra overhead of the memory allocation usually isn't a big deal, and if it triggers a collection or sticks around for a little while after it's used (because no collection gets run), most code won't care.

The code that really cares is code that's having to throw exceptions more frequently and/or in situations where those exceptions really need to be fast. Personally, I've only run into that in unit tests, and I don't think that the GC has much do with that slowness (and Adam Ruppe's current work on that seems to support that it's not the GC that's an issue). So, if the stack traces (or whatever it is that's making them slow) can be fixed to be faster, then as far as unit tests go, I would consider the matter taken care of and the fact that the GC is used for exceptions to be a non-issue.

The place where this becomes an issue then is code that needs exceptions to be really fast (e.g. it sounds like vibe.d falls in that camp). And in that case, it doesn't really matter whether the exceptions are allocated on the GC heap or malloc's heap. If memory allocation is slowing them down, then they need to get rid of the memory allocation entirely, in which case, doing something like having a pool of pre-allocated exception objects to reuse would make a lot more sense. And in that case, it would probably be better if they weren't on the GC heap, but the exception-throwing code wouldn't really care either way. That would be up to the pool. The same goes if only a single, static exception were used. It might be marginally better if it weren't on the GC heap, because it would avoid being scanned, but in those cases where you want speed, you _want_ long lifetimes for the exceptions, not short lifetimes like you're suggesting, because you want to reuse the exceptions in order to avoid needing to allocate new ones. The only way that short lifetimes would work is if we weren't dealing with classes and the exceptions were on the stack, but that negates our ability to have an exception hierarchy - which is critical to how exceptions work.

And if some code is getting exceptions frequently enough that the memory allocation is the bottleneck, then maybe exceptions aren't the best choice either. I agree that exceptions need to be much, much faster than they are, but they're still intended for the error case, which should be relatively infrequent.

> 2. Stack trace is constructed on throw. User pays no matter if the trace is needed or not. This is in the works, thankfully.

Yes, which should be a significant improvement and likely a much larger gain than any memory allocation issues.

> 3. Turns out message is expected to be a string, formatted apriori: https://github.com/D-Programming-Language/druntime/blob/master/src/object_.d #L1306 Formatting a string in such setting inevitably allocates and it happens at the throw site, even if nobody is using that message down the line. At least one can override toString...

Ideally, creating the string that toString returns would be put off until toString is called (particularly since that includes the stack trace), but I would hope that creating the message string to pass to the exception's constructor would be cheap enough (particularly in light of the fact that the exception is heap-allocated anyway) that it wouldn't be a big deal. So, if we can find a way to make this more efficient without getting messy, that's great, but I wouldn't expect that to be a bottleneck just so long as the actual string that the message gets put into for toString to return (which then includes the file and line and stacktrace and whatnot) isn't created until toString is called.

- Jonathan M Davis

On Monday, 10 February 2014 at 01:16:27 UTC, Jonathan M Davis
wrote:
> On Sunday, February 09, 2014 21:57:13 Dmitry Olshansky wrote:
>> Split out of "List of Phobos functions that allocate memory?".
>> 
>> To reiterate, here is some critique, compiled:
>> 
>> 1. Exceptions are class instances, hence (by default) are allocated on
>> GC heap. This is wrong default, GC is no place for temporaries.
>
> The more I think about it, the less I'm convinced that this is a big deal.
> Sure, it would be nice if they were malloc-ed and reference-counted so that
> they'd go away immediately after they were used and didn't risk triggering a
> collection, but for exceptions to work right, they need inheritance, so they
> have to be classes, and since exceptions are generally rare, having the extra
> overhead of the memory allocation usually isn't a big deal, and if it triggers
> a collection or sticks around for a little while after it's used (because no
> collection gets run), most code won't care.
>
> The code that really cares is code that's having to throw exceptions more
> frequently and/or in situations where those exceptions really need to be fast.
> Personally, I've only run into that in unit tests, and I don't think that the
> GC has much do with that slowness (and Adam Ruppe's current work on that seems
> to support that it's not the GC that's an issue). So, if the stack traces (or
> whatever it is that's making them slow) can be fixed to be faster, then as far
> as unit tests go, I would consider the matter taken care of and the fact that
> the GC is used for exceptions to be a non-issue.
>
> The place where this becomes an issue then is code that needs exceptions to be
> really fast (e.g. it sounds like vibe.d falls in that camp). And in that case,
> it doesn't really matter whether the exceptions are allocated on the GC heap
> or malloc's heap. If memory allocation is slowing them down, then they need to
> get rid of the memory allocation entirely, in which case, doing something like
> having a pool of pre-allocated exception objects to reuse would make a lot
> more sense. And in that case, it would probably be better if they weren't on
> the GC heap, but the exception-throwing code wouldn't really care either way.
> That would be up to the pool. The same goes if only a single, static exception
> were used. It might be marginally better if it weren't on the GC heap, because
> it would avoid being scanned, but in those cases where you want speed, you
> _want_ long lifetimes for the exceptions, not short lifetimes like you're
> suggesting, because you want to reuse the exceptions in order to avoid needing
> to allocate new ones. The only way that short lifetimes would work is if we
> weren't dealing with classes and the exceptions were on the stack, but that
> negates our ability to have an exception hierarchy - which is critical to how
> exceptions work.
>
> And if some code is getting exceptions frequently enough that the memory
> allocation is the bottleneck, then maybe exceptions aren't the best choice
> either. I agree that exceptions need to be much, much faster than they are,
> but they're still intended for the error case, which should be relatively
> infrequent.
>
>> 2. Stack trace is constructed on throw. User pays no matter if the trace
>> is needed or not. This is in the works, thankfully.
>
> Yes, which should be a significant improvement and likely a much larger gain
> than any memory allocation issues.
>
>> 3. Turns out message is expected to be a string, formatted apriori:
>> https://github.com/D-Programming-Language/druntime/blob/master/src/object_.d
>> #L1306 Formatting a string in such setting inevitably allocates and it
>> happens at the throw site, even if nobody is using that message down the
>> line. At least one can override toString...
>
> Ideally, creating the string that toString returns would be put off until
> toString is called (particularly since that includes the stack trace), but I
> would hope that creating the message string to pass to the exception's
> constructor would be cheap enough (particularly in light of the fact that the
> exception is heap-allocated anyway) that it wouldn't be a big deal. So, if we
> can find a way to make this more efficient without getting messy, that's
> great, but I wouldn't expect that to be a bottleneck just so long as the
> actual string that the message gets put into for toString to return (which
> then includes the file and line and stacktrace and whatnot) isn't created
> until toString is called.
>
> - Jonathan M Davis

All that wouldn't be a problem if D had ARC for everything,
including Exception`s.

On Monday, February 10, 2014 01:49:40 inout wrote:
> All that wouldn't be a problem if D had ARC for everything, including Exception`s.

Well, yes and no. ARC would make it so that your typical exception would be freed right after you're done with it, which would be great, but given that those should be fairly rare, and they usually aren't super performance sensitive, having them be managed by the GC isn't all that big a deal IMHO. Where it matters is when you would want to avoid allocating when throwing, in which case, you wouldn't want ARC - or at least you wouldn't care if it was ARC or managed by the GC - because in that case, you'd be looking to avoid the allocation cost by pre-allocating the exceptions and reusing them, and at that point, ARC buys you nothing over the GC except that it avoids having that memory scanned by the GC on a collection (and ARC could even incur some minor overhead when it has to do the ref-counting). So, I don't really see ARC as a solution here - not to the code that's performance-sensitive enough to be worried about the heap allocation anyway.

- Jonathan M Davis

10-Feb-2014 05:16, Jonathan M Davis пишет:
> On Sunday, February 09, 2014 21:57:13 Dmitry Olshansky wrote:
>> Split out of "List of Phobos functions that allocate memory?".
>>
>> To reiterate, here is some critique, compiled:
>>
>> 1. Exceptions are class instances, hence (by default) are allocated on
>> GC heap. This is wrong default, GC is no place for temporaries.
>


> The place where this becomes an issue then is code that needs exceptions to be
> really fast (e.g. it sounds like vibe.d falls in that camp). And in that case,
> it doesn't really matter whether the exceptions are allocated on the GC heap
> or malloc's heap. If memory allocation is slowing them down, then they need to
> get rid of the memory allocation entirely, in which case, doing something like
> having a pool of pre-allocated exception objects to reuse would make a lot
> more sense.

Yes, but this is more of reuse of  the memory and allocating it cheaply. Pools are especially good for objects with short lifetime.

> And in that case, it would probably be better if they weren't on
> the GC heap, but the exception-throwing code wouldn't really care either way.
> That would be up to the pool. The same goes if only a single, static exception
> were used. It might be marginally better if it weren't on the GC heap, because
> it would avoid being scanned, but in those cases where you want speed, you
> _want_ long lifetimes for the exceptions, not short lifetimes like you're
> suggesting, because you want to reuse the exceptions in order to avoid needing
> to allocate new ones.

Exceptions have short lifetimes, that's a simple fact that can be inferred poking at code bases. Their lifespan in most cases is from throw statement to the next catch.
Now if memory is preallocated or not has no bearing to lifetime, as each time an exception is thrown with different data means this is a "new" exception regardless of what memory is being reused.

> The only way that short lifetimes would work is if we
> weren't dealing with classes and the exceptions were on the stack, but that
> negates our ability to have an exception hierarchy - which is critical to how
> exceptions work.

I see you imply that class hierarchy and polymorphism is achieved only with classes. This is a good point. I wonder if virtual functions mechanism can be generalized beyond classes.

>
> And if some code is getting exceptions frequently enough that the memory
> allocation is the bottleneck, then maybe exceptions aren't the best choice
> either.

If the allocation is the bottleneck - cure the allocator.

> I agree that exceptions need to be much, much faster than they are,
> but they're still intended for the error case, which should be relatively
> infrequent.

Exceptions are a mechanism of error handling that has an advantage of cleanly propagating information up across stack frames to the actual point where it could be deal with. Yes, stack unwind implies that it shouldn't probably be 99% of cases.

>> 3. Turns out message is expected to be a string, formatted apriori:
>> https://github.com/D-Programming-Language/druntime/blob/master/src/object_.d
>> #L1306 Formatting a string in such setting inevitably allocates and it
>> happens at the throw site, even if nobody is using that message down the
>> line. At least one can override toString...
>
> Ideally, creating the string that toString returns would be put off until
> toString is called (particularly since that includes the stack trace), but I
> would hope that creating the message string to pass to the exception's
> constructor would be cheap enough (particularly in light of the fact that the
> exception is heap-allocated anyway)

Here it goes - bah if we allocate why not allocate twice. I'm out of words on this point.


-- 
Dmitry Olshansky

10-Feb-2014 07:09, Jonathan M Davis пишет:
> but given that
> those should be fairly rare, and they usually aren't super performance
> sensitive, having them be managed by the GC isn't all that big a deal IMHO.

Let's please stop this trend. A built-in feature should be made as fast as possible, no amount of good excuses will let us avoid that.

If we all agree that having some sort of pool for exceptions is what it takes why putting the burden on all our users to reinvent it?
Moreover the user in question must also redo a bit of homework to understand that allocating exceptions is what slows his/her code down.

-- 
Dmitry Olshansky

10-Feb-2014 02:25, Jakob Ovrum пишет:
> On Sunday, 9 February 2014 at 20:26:20 UTC, Dmitry Olshansky wrote:
>> I'm saying that basically classes imply infinite lifetime model. Then
>> you may work extra hard and do things like emplace and manual allocation.
>
> Infinite lifetime is also only with `new`. The "extra work" with emplace
> and manual allocation is the domain of library code (e.g. `C c =
> alloc!C(ctorArgs);`).

I thought of this for a while and I think lazly dynamic allocation is still better. First things first - there is still lazy initialization both ways.

The only potential gain of statically allocating memory here is on the first exception being thrown, which doesn't gain anything for our prime case of "many exceptions".

Lastly if the said exception is never thrown, static allocation would waste more memory of each thread (TLS). This is especially true for cases where not every thread runs the same code (which is not a small part of the landscape).

>
> The only issues with using library code instead of `new` are details
> like allocation of non-static nested classes and allocation of classes
> using private constructors etc.
>
> It is true that classes rely on *uniqueness* to some extent, but
> uniqueness is not particular to GC memory.

I'm thinking that I probably should compile this discussion to some implementable enhancement request for Phobos.

`cachedException` or some such sounds like something suitable for std.exception.

Thoughts?

-- 
Dmitry Olshansky

Oops, I've meant to reply to this post:

>
> I didn't intend to imply compile-time construction, just statically
> allocate `ubyte[__traits(classInstanceSize, Ex)]` and lazily emplace
> `Ex` there instead of GC-allocating with `new`. It would be doable with
> `std.typecons.Scoped` if that type wasn't naively anonymized a couple of
> releases ago.

I thought of this for a while and I think lazly dynamic allocation is still better. First things first - there is still lazy initialization both ways.

The only potential gain of statically allocating memory here is on the first exception being thrown, which doesn't gain anything for our prime case of "many exceptions".

Lastly if the said exception is never thrown, static allocation would waste more memory of each thread (TLS). This is especially true for cases where not every thread runs the same code (which is not a small part of the landscape).



-- 
Dmitry Olshansky