I've been trying out an API strategy to use sink functions a lot more to hoist the item allocation/management logic to the user and out from the work routines. This idea has received a lot of attention, and I heard a lot of people saying this was a planned direction to move in phobos and friends.

void parseThings(string s, void delegate(Thing) sink)
{
  //...
  sink(Thing());
  //...
}

We have a serious problem though; a function that receives a sink function
must transfer the attributes from the sink function to itself, and we have
no language to do that...
What are the leading strategies that have been discussed to date? Is there
a general 'plan'?
Restriction attributes like pure/nothrow/@nogc/safe, etc need some
expression to operate in a similar way to; where the attribute-ness of
function matches the attributes of its delegate argument(/s).

Here's the stupidest idea ever: expand inout to take an argument...

void parseThings(string s, void delegate(Thing)  inout(nothrow)
inout(@nogc) sink) inout(nothrow) inout(@nogc)
{
  //...
  sink(Thing());
  //...
}

Surely people have had better ideas? But you get the point, and this is basically essential to make the 'sink' pattern work at all in D.

No, no templates; it's not right to generate multiple copies of identical functions just because something it CALLS would transfer an attribute. The function itself is identical no matter the state of any attribute transference, and so this isn't a template problem; it's a pattern matching problem.

My solution I have proposed plenty, is attribute invalidation.

So the caller sees a more restricted set, depending upon the callback passed in.

However Timon didn't like that, thought it wasn't good enough.

On 12/8/24 06:54, Manu wrote:
> Here's the stupidest idea ever: expand inout to take an argument...
> 
> void parseThings(string s, void delegate(Thing) inout(nothrow) inout(@nogc) sink) inout(nothrow) inout(@nogc)
> {
>    //...
>    sink(Thing());
>    //...
> }
> ...

Issue with this is you will still not know which `inout`s match up. This is already an issue with the existing `inout`, which just arbitrarily selects a convention. In particular, it does not really work with higher-order functions the way you'd need it to here.

> Surely people have had better ideas? But you get the point, and this is basically essential to make the 'sink' pattern work at all in D.
> 
> No, no templates; it's not right to generate multiple copies of identical functions just because something it CALLS would transfer an attribute. The function itself is identical no matter the state of any attribute transference, and so this isn't a template problem; it's a pattern matching problem.

Well, it can be seen as a homogeneous vs heterogeneous compilation problem. The attributes can still act a bit like template parameters, but only one instance must be created that works for all of them. It's sometimes called parametric polymorphism.

On 12/8/24 06:57, Richard (Rikki) Andrew Cattermole wrote:
> My solution I have proposed plenty, is attribute invalidation.
> 
> So the caller sees a more restricted set, depending upon the callback passed in.
> 
> However Timon didn't like that, thought it wasn't good enough.
> 

It is not.

Completeness: This proposal cannot express even one of the simplest forms of composition, function composition. It also will not compose in practice.

Soundness: This proposal changes the meaning of qualified delegates everywhere in the language and invalidates a lot of existing type system invariants. It will be very tricky to maintain both backwards-compatibility with plain attributes and soundness.

In practice, it will be complicated, very incomplete, and unsound, like `inout`.

On Sunday, 8 December 2024 at 05:54:40 UTC, Manu wrote:
>
> Surely people have had better ideas? No, no templates;

I'm pretty sure this only could work with templates and with templates it would just happen.

Or better yet, be inlined by the optimizer using function litteralls

"How do I get rid of this bruise on my head, no without stopping hitting my head with the hammer, surely someone with medical skills could actually help me"

On Sun, 8 Dec 2024 at 16:01, Richard (Rikki) Andrew Cattermole via Digitalmars-d <digitalmars-d@puremagic.com> wrote:

> My solution I have proposed plenty, is attribute invalidation.
>
> So the caller sees a more restricted set, depending upon the callback passed in.
>
> However Timon didn't like that, thought it wasn't good enough.
>

I agree with Timon, that sounds terrible.

On Mon, 9 Dec 2024 at 01:51, Timon Gehr via Digitalmars-d < digitalmars-d@puremagic.com> wrote:

> On 12/8/24 06:54, Manu wrote:
> > Here's the stupidest idea ever: expand inout to take an argument...
> >
> > void parseThings(string s, void delegate(Thing) inout(nothrow)
> > inout(@nogc) sink) inout(nothrow) inout(@nogc)
> > {
> >    //...
> >    sink(Thing());
> >    //...
> > }
> > ...
>
> Issue with this is you will still not know which `inout`s match up. This is already an issue with the existing `inout`, which just arbitrarily selects a convention. In particular, it does not really work with higher-order functions the way you'd need it to here.
>

Yes, this is obviously an issue, and we have it in D comprehensively; it's
the main issue with our 'safe' stuff too; where rust has explicit lifetime
attribution... it's essentially the same problem all round; it needs tags
that specify things which are associated.
In lieu of that though, I would say, if there are multiple things marked
inout(...) in the way I proposed, you would assume the most pessimistic
from the set of possibilities is supplied.

void inheritAttribs(string s, void delegate() inout(nothrow) inout(@nogc)
fun_1, void delegate() inout(nothrow) inout(@nogc) fun_2) inout(nothrow)
inout(@nogc)
{
  //...
  fun_1();
  fun_2();
  //...
}

In this case, `inheritAttribs` would only be nothrow in the event BOTH fun_1 and fun_2 are nothrow, likewise for nogc...


> Surely people have had better ideas? But you get the point, and this is
> > basically essential to make the 'sink' pattern work at all in D.
> >
> > No, no templates; it's not right to generate multiple copies of
> identical functions just because something it CALLS would transfer an attribute. The function itself is identical no matter the state of any attribute transference, and so this isn't a template problem; it's a pattern matching problem.
>
> Well, it can be seen as a homogeneous vs heterogeneous compilation problem. The attributes can still act a bit like template parameters, but only one instance must be created that works for all of them. It's sometimes called parametric polymorphism.
>

Right. Do you have examples of this from other languages and how it's expressed? Lifetime's in Rust are the obvious benchmark, but I don't see any evidence that D has a taste for this sort of thing.

I do think that inout could work for all attributes the same as inout does
for const (with the same limitations). You could see `inout` as shorthand
for `inout(const)` under my suggestion.
Obviously it must enforce the most restrictive implementation inside the
code; my function `inheritAttribs` above must be nothrow and nogc
internally, but externally it would allow a mapping from one of the 'input'
attributes to the 'output' attribute in a non-templated way.

On Mon, 9 Dec 2024 at 03:51, Monkyyy via Digitalmars-d < digitalmars-d@puremagic.com> wrote:

> On Sunday, 8 December 2024 at 05:54:40 UTC, Manu wrote:
> >
> > Surely people have had better ideas? No, no templates;
>
> I'm pretty sure this only could work with templates and with templates it would just happen.
>

This doesn't *require* templates. Obviously there is a template solution,
but I'm very sensitive to code repetition and template bloat.
I'm pretty bored of D's "everything is templates" mentality; it's one of
D's biggest attractions, and also D's single biggest trap.
We need to work towards making more things sane without resorting to the
sledgehammer.

Or better yet, be inlined by the optimizer using function
> litteralls
>

This is not about inlining; these functions are not to be inlined.


> "How do I get rid of this bruise on my head, no without stopping hitting my head with the hammer, surely someone with medical skills could actually help me"
>

🤨

On Mon, 9 Dec 2024 at 05:21, Steven Schveighoffer via Digitalmars-d < digitalmars-d@puremagic.com> wrote:

> On Sunday, 8 December 2024 at 05:54:40 UTC, Manu wrote:
> > I've been trying out an API strategy to use sink functions a lot more to hoist the item allocation/management logic to the user and out from the work routines. This idea has received a lot of attention, and I heard a lot of people saying this was a planned direction to move in phobos and friends.
> >
> > ```d
> > void parseThings(string s, void delegate(Thing) sink)
> > {
> >   //...
> >   sink(Thing());
> >   //...
> > }
> > ```
> >
> > We have a serious problem though; a function that receives a
> > sink function
> > must transfer the attributes from the sink function to itself,
> > and we have
> > no language to do that...
> > What are the leading strategies that have been discussed to
> > date? Is there
> > a general 'plan'?
>
> I've proposed something like the following in the past, but I don't know how much appetite there is for it:
>
> ```d
> // assuming Thing() is @nogc nothrow @safe, but not pure
> void parseThings(string s, @called void delegate(Thing) sink)
> @nogc nothrow @safe
> {
>     sink(Thing());
> }
> ```
>
> Essentially, how this would work is that the compiler would consider all code inside the function except the call of the delegate to be attributed with the provided attributes. However, the delegate call will *always work*. All that will happen is that the call to `parseThings` will have any restrictions removed that aren't on the delegate.
>
> So e.g. a delegate of `@nogc pure` but not `nothrow` or `@safe` will mean `parseThings(dg)` will be treated like it's `@nogc`.
>
> Because the list of attributes `@nogc nothrow @safe` is logic-anded with `@nogc pure`.
>
> No recompile is necessary.
>
> If you want to insist that some attribute must be on the delegate, you can add the attribute to the delegate parameter. So if I wanted to insist that you must pass in at least a `@safe` delegate:
>
> ```d
> void parseThings(string s, @called void delegate(Thing) @safe
> sink) { ... }
> ```
>
> As far as I know, there is no plan to make something like this happen.
>
> -Steve
>

Interesting idea. It seems like a novel solution. Approaching it from this perspective feels hard to reason about though... it feels like it would add serious friction to introspection activities?