On Tuesday, 19 September 2017 at 13:46:20 UTC, Craig Black wrote:

> Thanks, I didn't know you could to that but it still doesn't give me the behavior that I want:
>
> class Foo
> {
> }
>
> struct MyStruct
> {
> @nogc:
> public:
>   Foo foo; // This does not produce an error, but it still requires a GC scan
>   void Bar()
>   {
>     foo = new Foo; // This produces an error
>   }
> }

@nogc applies to functions, not to types or variables. It prevents actions that allocate, like calls to new, appending to or concatenating arrays, and so on, inside the annotated function. By extension, no sort of GC scanning or collections will occur in such functions because that sort of thing can only happen during an allocation.

On Tuesday, 19 September 2017 at 13:11:03 UTC, Craig Black wrote:
> I've recently tried coding in D again after some years.  One of my earlier concerns was the ability to code without the GC, which seemed difficult to pull off.  To be clear, I want my programs to be garbage collected, but I want to use the GC sparingly so that the mark and sweep collections will be fast.  So I want guarantees that certain sections of code and certain structs will not require the GC in any way.
>
> I realize that you can allocate on the non-GC heap using malloc and free and emplace, but I find it troubling that you still need to tell the GC to scan your allocation. What I would like is, for example, to be able to write a @nogc templated struct that guarantees that none of its members require GC scanning.  Thus:
>
> @nogc struct Array(T)
> {
>   ...
> }
>
> class GarbageCollectedClass
> {
> }
>
> void main()
> {
>   Array!int intArray; // fine
>
>
> }

@nogc has nothing to do with whether something needs scanning. It guarantees that code will never allocate with the GC or trigger a GC collection (because the only way to do that is to allocate or to call the functions in core.memory.GC, which are deliberately not marked @nogc).

On Tuesday, 19 September 2017 at 13:59:27 UTC, Jonathan M Davis wrote:
> On Tuesday, September 19, 2017 13:11:03 Craig Black via Digitalmars-d wrote:
>> I've recently tried coding in D again after some years.  One of my earlier concerns was the ability to code without the GC, which seemed difficult to pull off.  To be clear, I want my programs to be garbage collected, but I want to use the GC sparingly so that the mark and sweep collections will be fast.
>>  So I want guarantees that certain sections of code and certain structs will not require the GC in any way.
>>
>> I realize that you can allocate on the non-GC heap using malloc and free and emplace, but I find it troubling that you still need to tell the GC to scan your allocation. What I would like is, for example, to be able to write a @nogc templated struct that guarantees that none of its members require GC scanning.  Thus:
>>
>> @nogc struct Array(T)
>> {
>>    ...
>> }
>>
>> class GarbageCollectedClass
>> {
>> }
>>
>> void main()
>> {
>>    Array!int intArray; // fine
>>
>>
>> }
>
> @nogc is a function attribute. It has no effect on types except on their member functions. All it does is guarantee that a function marked with @nogc cannot call any function which is not @nogc and cannot do any operation which is not considered @nogc. It's to guarantee that a function does not use the GC and has nothing more to do with types than attributes like @safe or nothrow do.
>
> - Jonathan M Davis

Thank you for your response.  The @nogc attribute is good, but in my opinion it is incomplete if all types still require scanning.  The purpose of not employing GC in certain sections of code is performance, and we are sacrificing performance with every allocation unit that is needlessly scanned.

-Craig

On Tuesday, 19 September 2017 at 14:22:21 UTC, Craig Black wrote:

> Thank you for your response.  The @nogc attribute is good, but in my opinion it is incomplete if all types still require scanning.  The purpose of not employing GC in certain sections of code is performance, and we are sacrificing performance with every allocation unit that is needlessly scanned.
>
> -Craig

As I wrote in my previous post, *no* GC activity will happen inside a @nogc function. The only time any scanning or collections can take place is when a allocation is requested. If none are requested, there's no activity.

Aside from that, there are other options. If you don't want your Foo member variable to be scanned, then allocate it outside the GC heap (see Mallocator in std.allocator). You can call GC.disable whenever you want (but be wary of the impact performance when you later call GC.collect). You can allocate outside of your hot loops and use stack allocation where possible.

I suggest you take a look at the ongoing GC series on the D Blog. The next post (coming later this week) covers heap allocations.

https://dlang.org/blog/the-gc-series/

On Tuesday, 19 September 2017 at 14:34:10 UTC, Mike Parker wrote:
> On Tuesday, 19 September 2017 at 14:22:21 UTC, Craig Black wrote:
>
>> Thank you for your response.  The @nogc attribute is good, but in my opinion it is incomplete if all types still require scanning.  The purpose of not employing GC in certain sections of code is performance, and we are sacrificing performance with every allocation unit that is needlessly scanned.
>>
>> -Craig
>
> As I wrote in my previous post, *no* GC activity will happen inside a @nogc function. The only time any scanning or collections can take place is when a allocation is requested. If none are requested, there's no activity.
>
> Aside from that, there are other options. If you don't want your Foo member variable to be scanned, then allocate it outside the GC heap (see Mallocator in std.allocator). You can call GC.disable whenever you want (but be wary of the impact performance when you later call GC.collect). You can allocate outside of your hot loops and use stack allocation where possible.
>
> I suggest you take a look at the ongoing GC series on the D Blog. The next post (coming later this week) covers heap allocations.
>
> https://dlang.org/blog/the-gc-series/

Thank you for the information.  What I would like to do is to create an Array template class that doesn't use GC at all.  Unfortunately I don't think this is possible with D in its current state, at least not safely anyway.  As it is, I can't prevent someone using my Array template to create an array of objects that reference the GC heap.  This means that in order to not break the language and avoid memory leaks, I am forced to tell the GC to scan all of my allocations.  There seems to be no way to avoid this unnecessary overhead.

I realize that the GC will not collect during a @nogc function.  This is a good guarantee, but I want to reduce the amount of time it takes to perform a collection, and there doesn't seem to be a good way to do this.

-Craig

On 9/19/17 10:22 AM, Craig Black wrote:
> On Tuesday, 19 September 2017 at 13:59:27 UTC, Jonathan M Davis wrote:
>> On Tuesday, September 19, 2017 13:11:03 Craig Black via Digitalmars-d wrote:
>>> I've recently tried coding in D again after some years.  One of my earlier concerns was the ability to code without the GC, which seemed difficult to pull off.  To be clear, I want my programs to be garbage collected, but I want to use the GC sparingly so that the mark and sweep collections will be fast.
>>>  So I want guarantees that certain sections of code and certain structs will not require the GC in any way.
>>>
>>> I realize that you can allocate on the non-GC heap using malloc and free and emplace, but I find it troubling that you still need to tell the GC to scan your allocation. What I would like is, for example, to be able to write a @nogc templated struct that guarantees that none of its members require GC scanning.  Thus:
>>>
>>> @nogc struct Array(T)
>>> {
>>>    ...
>>> }
>>>
>>> class GarbageCollectedClass
>>> {
>>> }
>>>
>>> void main()
>>> {
>>>    Array!int intArray; // fine
>>>
>>>
>>> }
>>
>> @nogc is a function attribute. It has no effect on types except on their member functions. All it does is guarantee that a function marked with @nogc cannot call any function which is not @nogc and cannot do any operation which is not considered @nogc. It's to guarantee that a function does not use the GC and has nothing more to do with types than attributes like @safe or nothrow do.
>
> 
> Thank you for your response.  The @nogc attribute is good, but in my opinion it is incomplete if all types still require scanning. The purpose of not employing GC in certain sections of code is performance, and we are sacrificing performance with every allocation unit that is needlessly scanned.
> 

From your posts and responses, it looks like you misunderstand still what the @nogc attribute does.

Note that a type does not bear any relation to whether the memory it lives in is scanned or not -- EXCEPT -- whether the type has indirections (pointers or arrays). A type which contains indirections is scanned, one that does not contain them is not scanned. There is no way to mark a type such that it:

1. Cannot be allocated on the GC*
2. Would not be scanned if it has pointers.

You can manually allocate it elsewhere, and you can manually tell the GC not to scan that block, but those are low-level tools that normally aren't used except by experts.

The @nogc attribute is used to PREVENT any operation that could cause a scan to occur. The idea is to mark areas of your code in such a way that you can predict the execution expense of that code. That is, if you have a tight loop or are in the middle of rendering frames to the screen in a game or something, you want to have the compiler ensure no GC cycles happen. It does not mean "don't ever store this in the GC."

-Steve

* There is a deprecated feature of D that allows specifying how to allocate classes other than heap allocation, but I wouldn't recommend using it. See: https://dlang.org/spec/class.html#allocators

On Tuesday, 19 September 2017 at 15:15:05 UTC, Steven Schveighoffer wrote:
> On 9/19/17 10:22 AM, Craig Black wrote:
>> On Tuesday, 19 September 2017 at 13:59:27 UTC, Jonathan M Davis wrote:
>>> On Tuesday, September 19, 2017 13:11:03 Craig Black via Digitalmars-d wrote:
>>>> I've recently tried coding in D again after some years.  One of my earlier concerns was the ability to code without the GC, which seemed difficult to pull off.  To be clear, I want my programs to be garbage collected, but I want to use the GC sparingly so that the mark and sweep collections will be fast.
>>>>  So I want guarantees that certain sections of code and certain structs will not require the GC in any way.
>>>>
>>>> I realize that you can allocate on the non-GC heap using malloc and free and emplace, but I find it troubling that you still need to tell the GC to scan your allocation. What I would like is, for example, to be able to write a @nogc templated struct that guarantees that none of its members require GC scanning.  Thus:
>>>>
>>>> @nogc struct Array(T)
>>>> {
>>>>    ...
>>>> }
>>>>
>>>> class GarbageCollectedClass
>>>> {
>>>> }
>>>>
>>>> void main()
>>>> {
>>>>    Array!int intArray; // fine
>>>>
>>>>
>>>> }
>>>
>>> @nogc is a function attribute. It has no effect on types except on their member functions. All it does is guarantee that a function marked with @nogc cannot call any function which is not @nogc and cannot do any operation which is not considered @nogc. It's to guarantee that a function does not use the GC and has nothing more to do with types than attributes like @safe or nothrow do.
>>
>> 
>> Thank you for your response.  The @nogc attribute is good, but in my opinion it is incomplete if all types still require scanning. The purpose of not employing GC in certain sections of code is performance, and we are sacrificing performance with every allocation unit that is needlessly scanned.
>> 
>
> From your posts and responses, it looks like you misunderstand still what the @nogc attribute does.
>
> Note that a type does not bear any relation to whether the memory it lives in is scanned or not -- EXCEPT -- whether the type has indirections (pointers or arrays). A type which contains indirections is scanned, one that does not contain them is not scanned. There is no way to mark a type such that it:
>
> 1. Cannot be allocated on the GC*
> 2. Would not be scanned if it has pointers.
>
> You can manually allocate it elsewhere, and you can manually tell the GC not to scan that block, but those are low-level tools that normally aren't used except by experts.
>
> The @nogc attribute is used to PREVENT any operation that could cause a scan to occur. The idea is to mark areas of your code in such a way that you can predict the execution expense of that code. That is, if you have a tight loop or are in the middle of rendering frames to the screen in a game or something, you want to have the compiler ensure no GC cycles happen. It does not mean "don't ever store this in the GC."
>
> -Steve
>
> * There is a deprecated feature of D that allows specifying how to allocate classes other than heap allocation, but I wouldn't recommend using it. See: https://dlang.org/spec/class.html#allocators

Thank you for the clarification. I understand mow that @nogc is only for functions and not for data types.  Thinking out loud, it would seem beneficial if there was a way to mark a pointer or data structure as not pointing to the GC heap. A guarantee to the compiler and run-time to ignore it during GC sweeps.  Now I know that pointer arithmetic breaks every kind of guarantee that would have with pointers, but aside from that it would seem to me that the compiler could help to enforce data marked as non-GC to not be assigned GC heap allocations.  This wouldn't be allowed for classes or class references, since they are always pointing to GC data, but perhaps for pointers and structs.  It seems like it would be a helpful feature, but maybe I'm way off base.

-Craig

On 9/19/17 2:06 PM, Craig Black wrote:
> Thank you for the clarification. I understand mow that @nogc is only for functions and not for data types.  Thinking out loud, it would seem beneficial if there was a way to mark a pointer or data structure as not pointing to the GC heap. A guarantee to the compiler and run-time to ignore it during GC sweeps.  Now I know that pointer arithmetic breaks every kind of guarantee that would have with pointers, but aside from that it would seem to me that the compiler could help to enforce data marked as non-GC to not be assigned GC heap allocations.  This wouldn't be allowed for classes or class references, since they are always pointing to GC data, but perhaps for pointers and structs.  It seems like it would be a helpful feature, but maybe I'm way off base.

In order for that to work, we would first need a precise scanner, as the current scanner scans ALL values in a block to see if any point at any heap memory, regardless of whether it is truly a pointer or not.

Also, if a pointer points to something outside the heap, there is no scanning of that data, since the GC doesn't know about it. The cost of having a pointer that points to a non-GC memory location is small (not entirely trivial, but it's just a few comparisons). The only true benefit would be to avoid scanning the entire block. This means that along with precise scanning, you couldn't have any GC pointers in the block.

As with most things, we need to prove the benefit with profiling before spending any time thinking about how to implement, and certainly before adding any language/library features.

-Steve

On 09/19/2017 08:06 PM, Craig Black wrote:
> This wouldn't be allowed for classes or class references, since they are always pointing to GC data

That's not true. You can put class objects into other places than the GC heap.

On Tuesday, 19 September 2017 at 15:11:31 UTC, Craig Black wrote:
> Thank you for the information.  What I would like to do is to create an Array template class that doesn't use GC at all.

You want to ensure that it can never refer to GC memory. The type system does not contain that information. It doesn't say whether an object was allocated on the GC heap, on the stack, using malloc, using a whole page of memory with mmap, using hardware addresses directly on an embedded system with manually planned memory layout, using a well-known address range like VGA, as part of the binary's static data segment...

So @nogc can't help you. You could, however, replace the GC implementation with one that throws an AssertError whenever the GC runs a collection, allocates memory, etc. Or you could just add @nogc to your main function.

If you want to forbid all reference types, you could write a `containsReferenceTypes` template, something like:

enum containsReferenceTypes(T) =
  is(T == class) ||
  is(T == interface) ||
  isDynamicArray!T ||
  isDelegate!T ||
  (isStruct!T && hasReferenceField!T);

bool hasReferenceField(T)()
{
  static foreach (t; Fields!T)
  {
    if (containsReferenceTypes!t) return true;
  }
  return false;
}

struct Array(T) if (!containsReferenceTypes!T)
{
}

> Unfortunately I don't think this is possible with D in its current state, at least not safely anyway.  As it is, I can't prevent someone using my Array template to create an array of objects that reference the GC heap.  This means that in order to not break the language and avoid memory leaks, I am forced to tell the GC to scan all of my allocations.  There seems to be no way to avoid this unnecessary overhead.

That's use-after-free, not memory leaks. The GC doesn't watch to see when things go out of scope; it periodically scans the world to see what's still in scope.

> I realize that the GC will not collect during a @nogc function.
>  This is a good guarantee, but I want to reduce the amount of time it takes to perform a collection, and there doesn't seem to be a good way to do this.

The GC is smart enough to not scan something for pointers if it doesn't contain pointers. So if you have `new Vec4[100]`, for instance, the GC isn't going to waste time scanning the memory it points to.

So if you have an Array!int variable in a GC'd object, it's basically a pointer and a length. The GC will note that the block of memory it points to has a reference to it and should not be disposed of. It will at some point consider doing a scan on that block of memory, see that it has no pointers in it, and skip over it.