DISCLAIMER: although I've been occasionally using D for a lot of time, I only recently started to follow the forums and use it intensively, so I may miss part of the history / long term vision.

So, the current guidelines to make a function usable in @nogc are:
1) use ranges as much as possible, instead of arrays;
2) don't specify a precise delegate type, but make it templated (so you accept both gc and @nogc delegates);
3) use manual management (e.g. malloc/free) for internal buffers whose lifetime and ownership are easy to track.

Now, there are some cases in which you cannot avoid the managed allocations:
1) throwing exceptions: these should not be abandoned in favor of other solutions; IMHO, they should be easily usable in @nogc code; switching to error codes or user-defined callbacks is not feasible in general;
2) returning arrays: sometimes you just can't avoid this: if your function must return a string, than it has to allocate it (unless it's a slice of some input)

With the new allocators library, we can customize these allocations. There are two possible ways to follow, both with advantages and drawbacks:
1) have all allocating functions take a templated allocator parameter (defaulting to GCAllocator) and use it to allocate returned arrays and thrown exceptions; this allows the compiler to infer @nogc whenever a @nogc allocator is passed, but becomes bloated because you have to carry around another parameter to lots of functions
2) have all allocating functions use theAllocator instead of raw new to perform allocations. This would make the allocator parameter implicit and the code very easy (just set theAllocator on startup), but would not allow the compiler to infer @nogc; IMHO it's not that bad: you can always use the profiler to check that your code is in fact @nogc, even if not stated explicitly; but many will not agree with this.

So my question is: what's the plan? Which road is to be followed? How will Phobos evolve regarding this?

On Sunday, 24 July 2016 at 15:34:55 UTC, Lodovico Giaretta wrote:
> DISCLAIMER: although I've been occasionally using D for a lot of time, I only recently started to follow the forums and use it intensively, so I may miss part of the history / long term vision.
>
> [...]

Note that for the arrays you have @nogc routines in std.experimental.allocators. (expand/shrink can infer @nogc from Mallocator).

On Sunday, 24 July 2016 at 15:44:48 UTC, lqjglkqjsg wrote:
> Note that for the arrays you have @nogc routines in std.experimental.allocators. (expand/shrink can infer @nogc from Mallocator).

I know. In fact I'm proposing two possible solutions based on std.experimental.allocators. Both have advantages and drawbacks, so I'm asking which one shall be followed, which one will be used my Phobos, or if a third solution exists.

On Sunday, 24 July 2016 at 15:34:55 UTC, Lodovico Giaretta wrote:
> DISCLAIMER: although I've been occasionally using D for a lot of time, I only recently started to follow the forums and use it intensively, so I may miss part of the history / long term vision.
>
> So, the current guidelines to make a function usable in @nogc are:
> 1) use ranges as much as possible, instead of arrays;
> 2) don't specify a precise delegate type, but make it templated (so you accept both gc and @nogc delegates);
> 3) use manual management (e.g. malloc/free) for internal buffers whose lifetime and ownership are easy to track.
>
> Now, there are some cases in which you cannot avoid the managed allocations:
> 1) throwing exceptions: these should not be abandoned in favor of other solutions; IMHO, they should be easily usable in @nogc code; switching to error codes or user-defined callbacks is not feasible in general;
> 2) returning arrays: sometimes you just can't avoid this: if your function must return a string, than it has to allocate it (unless it's a slice of some input)
>
> With the new allocators library, we can customize these allocations. There are two possible ways to follow, both with advantages and drawbacks:
> 1) have all allocating functions take a templated allocator parameter (defaulting to GCAllocator) and use it to allocate returned arrays and thrown exceptions; this allows the compiler to infer @nogc whenever a @nogc allocator is passed, but becomes bloated because you have to carry around another parameter to lots of functions

When I had to deal with this problem, I liked this idea too, but

1) Allocations with GCAllocator (e.g. makeArray) are neither @safe, nothrow nor pure.
2) It creates a huge template bloat

For 1) the best solution is WIP here:

https://github.com/dlang/phobos/pull/3891

> 2) have all allocating functions use theAllocator instead of raw new to perform allocations. This would make the allocator parameter implicit and the code very easy (just set theAllocator on startup), but would not allow the compiler to infer @nogc; IMHO it's not that bad: you can always use the profiler to check that your code is in fact @nogc, even if not stated explicitly; but many will not agree with this.

Yep it destroys the point of @nogc, besides different data structures / algorithms profit from different, specialized allocators.

> So my question is: what's the plan? Which road is to be followed? How will Phobos evolve regarding this?

There are also other options [1], e.g.
- using `static if`
- using two separate functions (one with `new`)
- requiring the API user to allocate the data beforehand

[1] https://github.com/dlang/phobos/pull/4190

On Sunday, 24 July 2016 at 17:04:43 UTC, Seb wrote:
> On Sunday, 24 July 2016 at 15:34:55 UTC, Lodovico Giaretta wrote:
>> DISCLAIMER: although I've been occasionally using D for a lot of time, I only recently started to follow the forums and use it intensively, so I may miss part of the history / long term vision.
>>
>> So, the current guidelines to make a function usable in @nogc are:
>> 1) use ranges as much as possible, instead of arrays;
>> 2) don't specify a precise delegate type, but make it templated (so you accept both gc and @nogc delegates);
>> 3) use manual management (e.g. malloc/free) for internal buffers whose lifetime and ownership are easy to track.
>>
>> Now, there are some cases in which you cannot avoid the managed allocations:
>> 1) throwing exceptions: these should not be abandoned in favor of other solutions; IMHO, they should be easily usable in @nogc code; switching to error codes or user-defined callbacks is not feasible in general;
>> 2) returning arrays: sometimes you just can't avoid this: if your function must return a string, than it has to allocate it (unless it's a slice of some input)
>>
>> With the new allocators library, we can customize these allocations. There are two possible ways to follow, both with advantages and drawbacks:
>> 1) have all allocating functions take a templated allocator parameter (defaulting to GCAllocator) and use it to allocate returned arrays and thrown exceptions; this allows the compiler to infer @nogc whenever a @nogc allocator is passed, but becomes bloated because you have to carry around another parameter to lots of functions
>
> When I had to deal with this problem, I liked this idea too, but
>
> 1) Allocations with GCAllocator (e.g. makeArray) are neither @safe, nothrow nor pure.
> 2) It creates a huge template bloat
>
> For 1) the best solution is WIP here:
>
> https://github.com/dlang/phobos/pull/3891
>
>> 2) have all allocating functions use theAllocator instead of raw new to perform allocations. This would make the allocator parameter implicit and the code very easy (just set theAllocator on startup), but would not allow the compiler to infer @nogc; IMHO it's not that bad: you can always use the profiler to check that your code is in fact @nogc, even if not stated explicitly; but many will not agree with this.
>
> Yep it destroys the point of @nogc, besides different data structures / algorithms profit from different, specialized allocators.
>
>> So my question is: what's the plan? Which road is to be followed? How will Phobos evolve regarding this?
>
> There are also other options [1], e.g.
> - using `static if`
> - using two separate functions (one with `new`)
> - requiring the API user to allocate the data beforehand
>
> [1] https://github.com/dlang/phobos/pull/4190

I prefer alias template parameters as they provide maximum flexibility:
https://github.com/dlang/phobos/pull/4288#issuecomment-227609141

@nogc code is quite different from idiomatic "casual" D code and one is expected to make sacrifices to go for it. Here are some approaches I decided for myself:

On Sunday, 24 July 2016 at 15:34:55 UTC, Lodovico Giaretta wrote:
> Now, there are some cases in which you cannot avoid the managed allocations:
> 1) throwing exceptions: these should not be abandoned in favor of other solutions; IMHO, they should be easily usable in @nogc code; switching to error codes or user-defined callbacks is not feasible in general;

Use pre-allocated exception instances. Throwing itself doesn't require GC, only allocating exception does. You can possibly screw up exception chaining this way but this is a very minor loss.

> 2) returning arrays: sometimes you just can't avoid this: if your function must return a string, than it has to allocate it (unless it's a slice of some input)

If it can't be avoided, this is not @nogc code and there is no point in pretending otherwise. Use ranges and sinks instead to move allocations decisions up the call chain. Note that with sink approach you want be able to mark function itself @nogc (because sink delegate allocates) but you can mark unittest block that verifies there are no _other_ allocations:

void foo ( alias sink_dg ) ( )
{
    sink_dg("abc");
}

@nogc unittest
{
    void noop_sink ( const(char)[] chunk ) @nogc { }
    foo!noop_sink();
}

Also in general usage of custom allocator for temporaries is a very bad @nogc approach and is only marginally better than GC using code. Good @nogc must make no decisions about allocations at all (and ideally minimize stack allocations too, especially if fibers are to be used).