January 20, 2023

On Friday, 20 January 2023 at 05:48:26 UTC, Steven Schveighoffer wrote:

>

On 1/19/23 10:38 PM, Walter Bright wrote:

>

On 1/19/2023 6:18 PM, Adam D Ruppe wrote:

>

Ideally, you'd be able to say process inherits the nogc-ness of userData.

That's exactly what template attribute inference does.

[…]

What we want is the effective attributes of the function to be the most restrictive possible of both the code inside the function, and the argument to the function.

The compiler would need a way to specify this for non-inferred functions, but it could be inferred for templates and auto functions.

I wrote a DIP (Attributes for Higher-Order Functions, AfHOF) for that, but didn’t make it. I tried to be pragmatic and squeeze it into the current syntax via doubling-down on a bug in the type system (i.e. making it a ‘feature’). Timon Gehr made me aware of the bug. Writing the DIP made me aware that higher-order functions are a blocker for making e.g. @safe the default: If you have void f(void delegate() dg), there is no consistent way to add @safe to f and/or dg that won’t break properly annotated @system code.

Mathias Lang has a work-in-progress DIP called Argument dependent attributes (AdA). It introduces syntax to specify that the attributes of higher-order function (i.e. a function with callbacks) depend on the attributes of the callback arguments. It’s a half-way solution that only targets parameters of delegate or function pointer type. A full solution would include parameters of composed types that include delegate or function pointer types somewhere: Slices of delegate or function pointer type could be considered reasonably common. Attribute variables would do that.

  • D has runtime value variables, e.g. your good old x in int x;.
  • D has type variables, e.g. you good old T in template t(T){..}
  • D has symbol variables, e.g. template alias parameters.
  • D has 1 type constructor variable: inout. (D were more consistent if it allowed for two or more independent type constructor variables, but that’s rarely needed practically.)
  • D has no attribute variables.

If D added 1 fixed-name attribute variable per attribute (cf. inout), they could be spelled @safety, purity, @gcness, and throwiness.

void callInSequence(
    void delegate() @safety purity @gcness throwiness[] dgs
) @safety purity @gcness throwiness
{
    foreach (dg; dgs) dg();
}

When called, the argument is a slice of delegates that are @safe or @system@safety is replaced by that; that are pure or not – purity is replaced by that; that are @nogc or not – @gcness is replaced by that; that may throw or not – throwiness by throw or nothrow.

This works the same way as when inout(int)[] f() inout is called on a mutable, const or immutable object, its return type will be int[], const(int)[], or immutable(int)[] after replacing inout with the call-side type constructor applied to the object’s type.

January 20, 2023

On Friday, 20 January 2023 at 13:08:46 UTC, Quirin Schroll wrote:

>

On Friday, 20 January 2023 at 05:48:26 UTC, Steven
... [snip]

If D added 1 fixed-name attribute variable per attribute (cf. inout), they could be spelled @safety, purity, @gcness, and throwiness.

void callInSequence(
    void delegate() @safety purity @gcness throwiness[] dgs
) @safety purity @gcness throwiness
{
    foreach (dg; dgs) dg();
}

When called, the argument is a slice of delegates that are @safe or @system@safety is replaced by that; that are pure or not – purity is replaced by that; that are @nogc or not – @gcness is replaced by that; that may throw or not – throwiness by throw or nothrow.

[snip]

I second this, or a less verbose version of this. One who wishes to vary all of the attributes shouldn't have to explicitly write @safety @gcness purity throwiness every time they want to vary all of the attributes. Perhaps adding a keyword such as attrib to vary all attributes not specifically set is the best course of action. Then we can write:

void callInSequence(void delegate() attrib @safe dgs[])
{
    foreach(dg; dgs)
        dg();
}

This will require that all delegates be @safe, but there are no other restrictions on their attributes.

January 20, 2023
On 1/20/23 8:08 AM, Quirin Schroll wrote:

> 
> If D added 1 fixed-name attribute variable per attribute (cf. `inout`), they could be spelled `@safety`, `purity`, `@gcness`, and `throwiness`.
> 
> ```d
> void callInSequence(
>      void delegate() @safety purity @gcness throwiness[] dgs
> ) @safety purity @gcness throwiness
> {
>      foreach (dg; dgs) dg();
> }
> ```

My proposal was basically to have a `@called` attribute that indicates the function delegate may be called inside the function, meaning the attributes of the call should reflect the attributes of the parameter.

> This works the same way as when `inout(int)[] f() inout` is called on a mutable, `const` or `immutable` object, its return type will be `int[]`, `const(int)[]`, or `immutable(int)[]` after replacing `inout` with the call-side type constructor applied to the object’s type.

Yes, inout is exactly the same type of mechanism

-Steve
January 20, 2023
On Fri, Jan 20, 2023 at 11:49:42AM +0000, Quirin Schroll via Digitalmars-d wrote: [...]
> The correct version is this:
> ```D
> void process(DG : void delegate())(DG callback) { callback(); }
> ```
> 
> This has two drawbacks:
> 1. `process` cannot be virtual.
> 2. the argument bound to `callback` cannot have its parameter types
> inferred.

And 3.: it causes the body of `process` to be duplicated across multiple instantiations, even if the only difference in DG is in attributes, i.e., the generated code is 100% identical.  This leads to template bloat.


[...]
> The drawbacks are lots of template instantiations and that it always generates 16 overloads.

Exactly. The language needs to support this natively, not leave it up to workarounds that bring in lots of template bloat.


T

-- 
Famous last words: I *think* this will work...
January 20, 2023
On 1/19/2023 8:28 PM, H. S. Teoh wrote:
> void process()(void delegate() cb) { cb(); }

void process(T)(T cb) { cb(); }

January 20, 2023
In general, yes, moving towards attribute inference for ordinary functions rather than just templates is where we want to go.

The reason it isn't done currently is because it makes .di declarations incompatible with the .d definitions.
January 20, 2023

On Thursday, 19 January 2023 at 15:22:08 UTC, Steven Schveighoffer wrote:

>

Something like serialization/deserialization (which requires templates) should use attribute inference, and use unittests to prove memory safety/nogc/etc.

Now, this can break down if not explicit since the compiler sometimes gives up on inference, so maybe that's what happened here.

-Steve

That is exactly what happened here. It should just work with inference but it was impossible. Even reducing to a simple case wasn't possible, because it only appears in the large.

Note I wasn't personally involved, just 2nd hand experience.

January 20, 2023
On Friday, 20 January 2023 at 19:15:09 UTC, Walter Bright wrote:
> On 1/19/2023 8:28 PM, H. S. Teoh wrote:
>> void process()(void delegate() cb) { cb(); }
>
> void process(T)(T cb) { cb(); }

Quirin Schroll addresses the drawbacks of that
https://forum.dlang.org/post/fzujvfmtckvcusdelzqj@forum.dlang.org

HS Teoh adds another issue here:
https://forum.dlang.org/post/mailman.8202.1674234814.31357.digitalmars-d@puremagic.com
January 20, 2023
On 1/20/2023 9:13 AM, H. S. Teoh wrote:
> And 3.: it causes the body of `process` to be duplicated across multiple
> instantiations, even if the only difference in DG is in attributes,
> i.e., the generated code is 100% identical.  This leads to template
> bloat.

This is a general issue with templates. Some linkers are able to remove duplicates. The compiler could be enhanced for it, too. It's not a language problem.
January 20, 2023
On 1/20/2023 3:49 AM, Quirin Schroll wrote:
> 1. `process` cannot be virtual.

Virtual functions are meant to be overridden, meaning their attributes are inherited with covariant and contravariant rules. This is incompatible with attribute inference.

> 2. the argument bound to `callback` cannot have its parameter types inferred.

The version of this I posted can.