On Tuesday, 8 October 2013 at 16:18:50 UTC, Adam D. Ruppe wrote:
> One is I can implement a runtime check pretty easily so it'd just be the first step because it would go quickly.

Runtime check is almost useless for this.

On Tuesday, 8 October 2013 at 16:22:25 UTC, Dicebot wrote:
> It is not overblown. It is simply "@nogc" which is lacking but absolutely mandatory. Amount of hidden language allocations makes manually cleaning code of those via runtime asserts completely unreasonable for real project.

Hidden language allocations:
- concatenation operator   ~
- homogeneous arguments   void (T[]... args)
- "real" closures that escapes
- array literals
- some phobos calls

What else am I missing?
I don't see the big problem, and a small fraction of projects will require a complete ban on GC allocation, right?

On Tuesday, 8 October 2013 at 16:29:38 UTC, ponce wrote:
> Hidden language allocations:
> - concatenation operator   ~
> - homogeneous arguments   void (T[]... args)
> - "real" closures that escapes
> - array literals
> - some phobos calls
>
> What else am I missing?
> I don't see the big problem, and a small fraction of projects will require a complete ban on GC allocation, right?

Should be all I am aware of (though closures sometimes do allocate even without escaping AFAIK). This is more than enough.

Imagine stuff like vibe.d - for proper performance you don't want to make any allocations during request handling. Neither GC, nor malloc. It is still perfectly fine to run GC in background (well, assuming we will get concurrent GC one day) for some persistent tasks but how are you going to verify your request handling is clean? By tracking mentioning of array literals in random places by hand? During every single pull review?

I have said on this topic several times - it does not matter what is _possible_ to do with D memory model. It does matter what is _convenient_ to do. If something is possible but needs more attention than in C++ it will be considered by crowd as impossible and no blog posts will change that.

(loud shouting "@noheap, @noheap, @noheap !")

On Tuesday, 8 October 2013 at 16:24:05 UTC, Dicebot wrote:
> Runtime check is almost useless for this.

They're how I do most the allocation checks now and could also be put in unit tests.

But I agree the ideal would be a compile time check. The way I want it to work is to define a new thing in the language, maybe via __traits, that can recursively check the call graph of a function for the presence of any UDA.

Then we attach @gc to the druntime functions that use them, starting at gc_malloc and right up into aaDup, d_newclass, whatever concat is called.

Ensure that the compiler considers these implicit function calls the same as any others and propagates the druntime UDA through the recursive trait.

Then we define @nogc to just be static assert(!hasAttribute!(gc, __traits(getAttributesRecursively, FUNCTION)). Maybe do a compiler or druntime change to @something can be expanded to a template instantation too so this doesn't even have to be a specific builtin. (Though I've actually been able to do this by hacking druntime before.) But whatever that specific impl doesn't matter.



Bottom line: @nogc works after just marking the core functions in druntime that may allocate, and the very same functionality can be used for other things too, like @noheap. Mark malloc with a uda, then define noheap as static assert(!gc && !malloc)

Then we have compile time checking, at least on the function level.

On Tuesday, 8 October 2013 at 17:00:35 UTC, Dicebot wrote:
> Imagine stuff like vibe.d - for proper performance you don't want to make any allocations during request handling.

That brings up another interesting advantage to my extensible scheme: we could also define @blocking in the library to put on I/O calls and then vibe.d does a check for it and complains if you called one.

Though getting this kind of coverage would be hard, a third party lib might not use it and then the custom check would miss the problem. But, in general, I'm sure if we had the capability, uses would come up beyond nogc/noheap.

> I have said on this topic several times - it does not matter what is _possible_ to do with D memory model. It does matter what is _convenient_ to do.

And, of course, "my function didn't compile because phobos uses the gc" is hardly convenient unless phobos offers the functionality (where possible) without allocating. I think output ranges offer a pretty good solution here for a lot of cases, and could be added without breaking the current interfaces - just keep functions that return new strings as alternatives implemented in terms of the base output range function.

On 10/8/13, Adam D. Ruppe <destructionator@gmail.com> wrote:
> That brings up another interesting advantage to my extensible scheme: we could also define @blocking in the library to put on I/O calls and then vibe.d does a check for it and complains if you called one.

Kind of relevant, I've recently filed this: http://d.puremagic.com/issues/show_bug.cgi?id=10979

On Tuesday, 8 October 2013 at 16:29:38 UTC, ponce wrote:
> On Tuesday, 8 October 2013 at 16:22:25 UTC, Dicebot wrote:
>> It is not overblown. It is simply "@nogc" which is lacking but absolutely mandatory. Amount of hidden language allocations makes manually cleaning code of those via runtime asserts completely unreasonable for real project.
>
> Hidden language allocations:
> - concatenation operator   ~
> - homogeneous arguments   void (T[]... args)
> - "real" closures that escapes
> - array literals
> - some phobos calls
>
> What else am I missing?
> I don't see the big problem, and a small fraction of projects will require a complete ban on GC allocation, right?

Johannes Pfau's -vgc pull request[1] had a list of ones he was able to find. It's all allocations, not just hidden allocations:

COV         // Code coverage enabled
NEW         // User called new (and it's not placement new)
ASSERT_USER // A call to assert. This usually throws, but can be overwritten
            // by user
SWITCH_USER // Called on switch error. This usually throws, but can be
            // overwritten by user
HIDDEN_USER // Called on hidden function error. This usually throws, but can
            // be overwritten by user
CONCAT      // a ~ b
ARRAY       // array.length = value, literal, .dup, .idup, .sort
APPEND      // a~= b
AALITERAL   // ["a":1]
CLOSURE

1. https://github.com/D-Programming-Language/dmd/pull/1886

On Tuesday, 8 October 2013 at 17:00:35 UTC, Dicebot wrote:
> On Tuesday, 8 October 2013 at 16:29:38 UTC, ponce wrote:
>> Hidden language allocations:
>> - concatenation operator   ~
>> - homogeneous arguments   void (T[]... args)
>> - "real" closures that escapes
>> - array literals
>> - some phobos calls
>>
>> What else am I missing?
>> I don't see the big problem, and a small fraction of projects will require a complete ban on GC allocation, right?
>
> Should be all I am aware of (though closures sometimes do allocate even without escaping AFAIK). This is more than enough.
>
> Imagine stuff like vibe.d - for proper performance you don't want to make any allocations during request handling. Neither GC, nor malloc. It is still perfectly fine to run GC in background (well, assuming we will get concurrent GC one day) for some persistent tasks but how are you going to verify your request handling is clean? By tracking mentioning of array literals in random places by hand? During every single pull review?
>
> I have said on this topic several times - it does not matter what is _possible_ to do with D memory model. It does matter what is _convenient_ to do. If something is possible but needs more attention than in C++ it will be considered by crowd as impossible and no blog posts will change that.
>
> (loud shouting "@noheap, @noheap, @noheap !")

+1

> Imagine stuff like vibe.d - for proper performance you don't want to make any allocations during request handling. Neither GC, nor malloc.

This is absurd.

O(1) malloc implementations exist, it is a solved problem. (http://www.gii.upv.es/tlsf/)
TLSF executes a maximum of 168 processor instructions in a x86 architecture. Saying that you can't use that during request handling is like saying that you can't afford a cache miss.

On Tuesday, October 08, 2013 18:29:36 ponce wrote:
> On Tuesday, 8 October 2013 at 16:22:25 UTC, Dicebot wrote:
> > It is not overblown. It is simply "@nogc" which is lacking but absolutely mandatory. Amount of hidden language allocations makes manually cleaning code of those via runtime asserts completely unreasonable for real project.
> 
> Hidden language allocations:
> - concatenation operator ~
> - homogeneous arguments void (T[]... args)
> - "real" closures that escapes
> - array literals
> - some phobos calls
> 
> What else am I missing?
> I don't see the big problem, and a small fraction of projects
> will require a complete ban on GC allocation, right?

I think that it's clear that for some projects, it's critical to minimize the GC, and I think that it's clear that we need to do a better job of supporting the folks who want to minimize GC usage, but I also think that for the vast majority of cases, complaints about the GC are way overblown. It becomes an issue when you're doing a lot of heap allocations, but it's frequently easy to design D code so that heap allocations are relatively rare such that they aren't going to be a serious problem outside of code which is performance critical to the point that it would be worrying about the cost of malloc (which most code isn't). Personally, the only time that I've run into issues with the GC is when trying to do use RedBlackTree with a lot of items. That has a tendancy to tank performance.

So, yes, it's problem. Yes, we need to improve the situaton. But for most situations, I think that the concern about the GC is way overblown.

- Jonathan M Davis