Thread overview
Re: Attribute transference from callbacks?
Dec 08
monkyyy
December 08
On Saturday, December 7, 2024 10:54:40 PM MST Manu via Digitalmars-d wrote:
> I've been trying out an API strategy to use sink functions a lot more to hoist the item allocation/management logic to the user and out from the work routines. This idea has received a lot of attention, and I heard a lot of people saying this was a planned direction to move in phobos and friends.
>
> void parseThings(string s, void delegate(Thing) sink)
> {
>   //...
>   sink(Thing());
>   //...
> }
>
> We have a serious problem though; a function that receives a sink function
> must transfer the attributes from the sink function to itself, and we have
> no language to do that...
> What are the leading strategies that have been discussed to date? Is there
> a general 'plan'?
> Restriction attributes like pure/nothrow/@nogc/safe, etc need some
> expression to operate in a similar way to; where the attribute-ness of
> function matches the attributes of its delegate argument(/s).
>
> Here's the stupidest idea ever: expand inout to take an argument...
>
> void parseThings(string s, void delegate(Thing)  inout(nothrow)
> inout(@nogc) sink) inout(nothrow) inout(@nogc)
> {
>   //...
>   sink(Thing());
>   //...
> }
>
> Surely people have had better ideas? But you get the point, and this is basically essential to make the 'sink' pattern work at all in D.
>
> No, no templates; it's not right to generate multiple copies of identical functions just because something it CALLS would transfer an attribute. The function itself is identical no matter the state of any attribute transference, and so this isn't a template problem; it's a pattern matching problem.
>

Unless, I'm missing something (which is quite possible), you have this backwards. The attributes on the delegate have to depend on the attributes of the function, not the other way around - though it's certainly true that if you have to pass a delegate with exactly the same list of attributes that that delegate parameter has, then that's going to cause problems.

Every compiled function has a fixed set of attributes. A template can be used to generate multiple functions, each with its own set of attributes, but the compiled result in each case is a single function with a fixed set of attributes (which even gets compiled into the function's mangling). And those attributes tell the compiler what the function is promising (e.g. nothrow means that it can't throw). And as such, if that function accepts a delegate, that delegate must follow all of the requirements that come with the attributes on the function it's being passed to; otherwise, calling the delegate would violate the function's attributes. For instance, if this code were to compile

```
void foo(void delegate(int) @system d) @safe
{
    d(42);
}
```

then it would result in foo calling @system code without @trusted being involved, so it would violate @safe. Similarly,

```
void foo(void delegate(int) d) nothrow
{
    d(42);
}
```

would result in the function violating nothrow.

If foo were templated, then multiple versions of it could be compiled, and you could templatize foo on the type of the delegate parameter, so each of the compiled functions' attributes would depend on the type of the delegate passed to the function in each case. But as long as you're dealing with a non-templated function, you only get one version, and the delegate can't violate the function's attributes, or the attributes would be meaningless.

So, I don't see how it makes any sense to talk about inferring the attributes on the function based on the attributes on the delegate being passed in unless the function is templated. The attributes were decided when the function was compiled and have to be the same regardless of what's being passed in.

Now, what we need to be able to do is pass more restrictive delegates to less restrictive functions. For instance, if you have

```
void foo(void delegate(int) @system d) @system
{
    d(42);
}
```

then it should be possible to pass an @safe delegate, because an @safe delegate would not violate @system, and since the delegate is essentially just using a pointer, the generated code should be able to work just fine regardless of the attributes on the delegate. As you pointed out, the generated code should be the same. It's just that for the function's attributes to not be violated, the type system has to prevent it from accepting any delegates with looser attributes. So, right now, you can do

```
void foo(void delegate(int) @safe d)
{
    d(42);
}
```

and that compiles just fine, because the function is @system, and calling an @safe delegate is allowed in an @system function. However, if we flip that

```
void foo(void delegate(int) @system d) @safe
{
    d(42);
}
```

then we get a compiler error, because calling an @system delegate in an @safe function is not allowed. And of course, we get the same situation with nothrow, pure, etc.

```
void foo(void delegate(int) nothrow d)
{
    d(42);
}
```

compiles, whereas

```
void foo(void delegate(int) d) nothrow
{
    d(42);
}
```

does not.

So, the compiler already makes sure that the attributes on the delegate parameter don't violate the attributes on the function if that delegate is called (the attributes can be totally different if it's not called, since it's the call that's the problem, but it would be pretty atypical to pass a delegate in and then not call it).

As such, what that leaves is the question of what the compiler does when you pass the function a delegate with attributes which are more restrictive than the parameter's attributes. I'm pretty sure that it used to be the case that the type of the delegate that you passed in had to match the type of the parameter exactly (which would then mean that we'd be screwed with regards to being able to reasonably have non-templated functions take delegates with a variety of attributes). However, from what I can tell, whatever it may have done in the past, the compiler now actually does the right thing. For instance, this code compiles just fine

```
void main()
{
    void delegate(int) @safe a;
    void delegate(int) pure b;
    void delegate(int) @nogc c;
    void delegate(int) nothrow d;

    foo(a);
    foo(b);
    foo(c);
    foo(d);
}

void foo(void delegate(int) d)
{
    d(42);
}
```

and if we change the signature on foo to

```
void foo(void delegate(int) @safe d)
{
    d(42);
}
```

then only the call to foo with the @safe delegate compiles, since the others are @system and would violate the requirement put on the delegate parameter (and the attributes on the delegate parameter have to be at least as restrictive - but no more - than the attributes on the function).

So, from what I can tell, the problem that you're trying solve - that is, have a non-templated function take a delegate where the delegate passed in can have a varying set of attributes - already works. It's just that the type system won't let you pass in a delegate which is less restrictive, since that would violate the attributes. And from the looks of it, it was implemented as a general implict conversion rather than something specific to function calls, e.g.

```
static assert(is(void delegate(int) @safe :
                 void delegate(int) @system));
```

and

```
void delegate(int) @safe a;
void delegate(int) @system b = a;
```

compile just fine - but fail to compile if you flip the attributes.

So, unless I'm missing something here (which is quite possible), this issue
has already been solved.

Now, IIRC, there are issues elsewhere in the language with regards to attributes on delegates - e.g. opApply doesn't handle them very well - but I don't recally the details at the moment, since I basically never use opApply. Timon would be able to explain. So, there may be improvements which still need to be made with regards to delegates, but I don't think that we need anything like what you're suggesting.

- Jonathan M Davis



December 08
On Sunday, December 8, 2024 8:57:46 AM MST Jonathan M Davis via Digitalmars-d wrote:
> So, unless I'm missing something here (which is quite possible), this issue
> has already been solved.

Actually, after thinking about this further, I think that I _might_ get what you're trying to say.

As I said, a function's attributes are fixed and can't be changed, and we can't allow them to be violated. And they're in the name mangling to help with that.

However, in principle, it's possible to write a function that can promise to be @safe or nothrow or whatever as long as its delegate argument is (so all of the checks are done on the basis that as long as the delegate meets the appropriate requirements, the function does). The function can't really be typed as @safe or nothrow or whatever at that point, since if it's saying that it's @safe, then it would violate its attributes to call a delegate that wasn't. However, if we had some form of conditional @safe / conditional nothrow / etc., then a function could theoretically be typed with something like @conditionalsafe to indicate that the as long as the delegate that it's passed is @safe or @trusted, then the function call can be treated as @safe, but if the delegate is @system, then the function can't retain its promise. However, because the @safe was conditional, the type system can then treat the call as @system, because the delegate argument was @system.

The implementation would potentially also have to be different from the normal attributes (e.g. you couldn't remove any exception handling stuff with conditional nothrow), but it could theoretically be implemented from what I can tell at the moment.

I assume that you're suggesting inout on the delegate parameters to tell the compiler that that particular attribute needs to be treated as conditional on the function based on the attribute of the argument to that parameter, which makes some sense.

So, I _think_ that I now get what you're proposing, but I'm not sure what the consequences would be. My gut reaction is that we'd have to be _very_ careful about this to make sure that we weren't opening ourselves up to additional issues. inout already causes us a fair bit of grief, and it's trying something similar, albeit with type qualifiers. I'm also not sure that something like this is needed often enough to be worth the extra complication, but I don't know. It's only needed in cases where you want to be able to make the function be able to be @safe but not require that it be @safe. Normally, we'd just say that the delegate argument had to be compatible with the attributes on the delegate parameter, and if we wanted to make the function's attributes be inferred, we'd make it a template. And in most cases, that's fine - though obviously, it's not fine in cases when you can't use a template.

So, it _might_ be a good idea, but I don't know. I've never personally felt the need, but I'm also perfectly fine with templating most code that would need to take any kind of sink or output range. That's already what happens with output ranges in general. A delegate sink can be an output range, but output ranges are frequently structs of some kind. And particularly for Phobos code, since it's generic, most of that stuff already has to be templated on the element type anyway. So, using a delegate wouldn't avoid templates except in cases where the code was not generic with regards to the element type. But it's certainly true that in less generic code, it can make perfect sense to have an output range or delegate sink which requires a particular element type, and if you use a delegate in such a case, then you can potentially avoid a template.

So, this may very well be an idea worth exploring, but I'm also skeptical that the problem is big enough for it to be worth making attributes even more complicated than they already are in order to solve it. At this point, in most cases, I would suggest to folks that they only use attributes in cases where they're really needed, since experience has shown that once a code base gets large enough, attributes make refactoring a royal pain and start causing more problems than they solve. And if you do need to support attributes on much, then you almost need templates in order to reduce problems with refactoring. And if you're not using attributes all over the place, then this kind of problem with delegates isn't going to come up much.

- Jonathan M Davis



December 09
On 09/12/2024 4:57 AM, Jonathan M Davis wrote:
> Unless, I'm missing something (which is quite possible), you have this
> backwards. The attributes on the delegate have to depend on the attributes
> of the function, not the other way around - though it's certainly true that
> if you have to pass a delegate with exactly the same list of attributes that
> that delegate parameter has, then that's going to cause problems.

I think that you have it backwards here.

The caller is saying, I do not care about these guarantees you offer, I don't want it to infect me.

Rather than the callee saying, I don't care what you want, you will take these guarantees and you will abide by MY restrictions too.

December 08
On Sunday, 8 December 2024 at 18:20:09 UTC, Richard (Rikki) Andrew Cattermole wrote:
>
> The caller is saying, I do not care about these guarantees you offer, I don't want it to infect me.

`void*`