March 18
On 3/9/24 22:47, Richard (Rikki) Andrew Cattermole wrote:
> On 10/03/2024 10:26 AM, ryuukk_ wrote:
>> If there is a significant build penalty, than i hope it's opt-in, i personally do not use any of the attributes
> 
> It shouldn't be significant.
> 
> It is a minor cost as things go, no reason to start thinking opt-in at this stage.

Really? This proposal requires full semantic analysis of any imported function. It is only minor in certain build setups. (E.g., when the project is compiled in a single invocation.)
March 18
On 18/03/2024 12:12 PM, Timon Gehr wrote:
> On 3/9/24 22:47, Richard (Rikki) Andrew Cattermole wrote:
>> On 10/03/2024 10:26 AM, ryuukk_ wrote:
>>> If there is a significant build penalty, than i hope it's opt-in, i personally do not use any of the attributes
>>
>> It shouldn't be significant.
>>
>> It is a minor cost as things go, no reason to start thinking opt-in at this stage.
> 
> Really? This proposal requires full semantic analysis of any imported function. It is only minor in certain build setups. (E.g., when the project is compiled in a single invocation.)

I am assuming that you generate .di files every time and use that for importing, not the .d files.

These days I'm leaning quite heavily towards projects using .di files for intermediary steps, due to distribution reasons with shared libraries.

It's already impossible to keep bindings to D code up to date manually, I tried pure was a killer on that idea.
March 20
On Monday, 18 March 2024 at 10:30:28 UTC, Richard (Rikki) Andrew Cattermole wrote:
> I am assuming that you generate .di files every time and use that for importing, not the .d files.

Yes, this is one possible mitigation.

The main downside of this is that .di files don't include function bodies, so you lose access to cross-module CTFE. Although I guess we could add a compiler switch to include function bodies in .di files.
April 05

On Thursday, 29 February 2024 at 22:39:43 UTC, Paul Backus wrote:

>

On Thursday, 29 February 2024 at 21:21:12 UTC, Sebastiaan Koppe wrote:

>

What about only doing inference when something fails to compile due to a missing attribute?

This would completely break separate compilation,

Isn't that already broken? See Bugzilla Issue #17541
https://forum.dlang.org/post/gguwkjyiduyxjilyluvf@forum.dlang.org#:~:text=%3A%20Bugzilla%20Issue%20%2317541

It seems inference need to spend more time to find out if some attribute can really be added and not give up early - else there will be always cases where an attribute is expected but was not inferred. Maybe the compiler should even warn, if it is not sure if some attribute is fulfilled or not? Something like: "purity could not be inferred. Please explicitly state if this function is pure or not"
(of course that would require the counterparts of pure, @nogc and @nothrow)

April 05
On Wednesday, 20 March 2024 at 23:59:09 UTC, Paul Backus wrote:
> On Monday, 18 March 2024 at 10:30:28 UTC, Richard (Rikki) Andrew Cattermole wrote:
>> I am assuming that you generate .di files every time and use that for importing, not the .d files.
>
> Yes, this is one possible mitigation.
>
> The main downside of this is that .di files don't include function bodies, so you lose access to cross-module CTFE. Although I guess we could add a compiler switch to include function bodies in .di files.

I would much better like having .di files that don't contain any code (even not for templates) and instead store the template bodies in some intermediary format in another file (of course not an object file, as it need to be possible to create different instances from it, but also not source-code because it should be possible to deliver e.g. a library without sources).
May 01

On Wednesday, 28 February 2024 at 17:18:04 UTC, Paul Backus wrote:

>

The primary goal of universal inference is to solve D's "attribute soup" problem without breaking compatibility with existing code. Compatibility with existing code makes universal inference a better solution to this problem than "@safe by default," "nothrow by default," and other similar proposals.

Overridable functions (that is, non-final virtual functions) are excluded from universal inference because their bodies may be replaced at runtime.

For cases where attribute inference is not desired, an opt-out mechanism will be provided.

Currently, .di files generated by the compiler do not include inferred function attributes. This will have to change.

Related Links

I have a few improvements to the suggestions linked to above. (I'm packing all three of these into one post. But each is probably substantial enough to have its own subthread.)

Suggestion #1: Better syntax for the @default attribute DIP

More specifically, for the alternative mechanism proposed by the DIP, which aimed to provide a generic way of deactivating any currently active attribute, but was rejected in the DIP because the syntax was "too complex and verbose." Here is that syntax:

@nogc nothrow pure @safe:
// ...
void f(T)(T x) @nogc(default) @nothrow(default) pure(default) {}

I've seen some other suggestions for the same feature. But I didn't find any particularly compelling.

But just now I realized we could have: @no(...), where ... contains the list of one or more attributes to deactivate.

So the above syntax transforms into:

@nogc nothrow pure @safe:
// ...

void f(T)(T x) @no(@nogc nothrow pure) {}

void g(T)(T x) @no(@nogc) @no(nothrow pure) {} // alternative grouping

@no(pure): // works for the scope too
...

This feature does not mean that the code it applies to wouldn't pass the checks for pure, @nogc, etc. It simply deactivates the previous label. The compiler can still infer a given attribute. It just wouldn't be explicit.

This syntax requires adding a keyword @no, and the ability to group one or more attributes within parentheses. But that's all. It's very simple, and it makes generically deactivating attributes very easy.

Suggestion #2: Better syntax for the Argument Dependent Attributes DIP

This suggestion requires a little more elaboration to make clear. The DIP in question was linked to in Adam's article. First let's address the question of default attributes for a function which has callable parameter(s). What should the default inference behavior be for func() in the following code? (Just focus on @safe and throw/nothrow)

void func(void delegate(in char[]) sink) @safe {
    sink("Hello World");
}

void g() {
    func((in char[] arg) {
        throw new Exception("Length cannot be 0");
    });
}
void h() {
    func((in char[] arg) {
        return;
    });
}

In my opinion, function func() should be inferred @safe throw when it is called in g(), and @safe nothrow when it is called in h(). In other words, its attributes should be combined at the call site with those of the delegate which is passed to it. These are Argument Dependent Attributes (ADAs).

Moreover, this should be the default behavior. (Note: I'm not 100% certain about this, and would like to be shown otherwise. But I think it's true.)

In other words, any function which takes a delegate or a function as a parameter, should have Argument Dependent Attributes (ADAs) by default.

In the existing situation, however, we have no such attributes at all, let alone by default, and the DIP above suggests the following syntax in order to add them. (Hint: The * means you don't have to specify the specific name of the argument for which the attribute status should propagate to the overall signature.):

// Basic usage, with nothrow and @safe
void func0(void delegate(int) sink) @safe(sink) nothrow(sink);
// Empty argument list, equivalent to @safe
void func1() @safe();
// Equivalent to func0
void func2(void delegate(int)) @safe(*) nothrow(*);
// Equivalent to func0
void func3(void delegate(int) arg) @safe(arg,) nothrow(*);
// Equivalent to func1
void func4(int) @safe(*);
// Equivalent to func0
void func3(void delegate(int) arg) @safe(0) nothrow(0,);

Again, the major problem here is with the chosen syntax. If I were suggesting a solution to the same problem, I would go with the following simple syntax using a new keyword @imply:

void func0(void delegate(int) @imply sink);

@imply simply means: Imply that (all) the attributes of sink() apply to func0() as well, and determine them each time func0() is called. (@imply as a keyword would only have any meaning as part of a callable parameter.) If you want to limit the implication to one or more particular attributes, indicate those in parentheses, using the same syntax as in Suggestion #1. So:

void func0(void delegate(int) @imply(@system throw) sink);

However, as mentioned above, I don't see why adding @imply to every delegate/function passed as an argument shouldn't be the default behavior. After all, generally speaking, why have a callable as a function parameter if you're not going to call it in the body of the function?

Therefore, what we really need is to make @imply the default, and add a way to opt out of it, by indicating that a function call should NOT infer its attributes based on the callable passed. So we are now defaulting to ADAs, and in rare cases adding @noimply() to turn them off:

// @noimply turns the new default off for the specified attribute
void func(void delegate(in char[]) @noimply(throw) sink) @safe {
    try {
        sink("Hello World");
    }
    catch (Exception) {}
}

void g() {
    func((in char[] arg) {
        throw new Exception(".");
    });
}

Since func() above catches the exception, it can be determined and inferred to be nothrow even if sink() throws. @noimply(throw) indicates this.

So, this is a syntax improvement suggestion for the ADAs DIP. But it's also a recognition that if they become the default, the primary need will be for an opt-out syntax rather than an opt-in one.

Suggestion #3: Syntax for explicitly annotating inferred attributes

>

[From the OP:]"Currently, .di files generated by the compiler do not include inferred function attributes. This will have to change."

I assume that the problem is that the mangled names for a function should not include the inferred attributes even if the .di header files do. It may also help with documentation to be able to distinguish officially supported attributes from inferred ones.

For this, I suggest another keyword with the same syntax as my previous proposals. Namely, I suggest putting all inferred attributes into an @inferred() grouping:

void func() @safe @nogc @inferred(pure nothrow);

@inferred() has no effect other than to tell the compiler or the documentation generator that its contents, while accurate, are not part of the official API/ABI of the function. The compiler must naturally keep track of which attributes are explicit as opposed to inferred, in order to be able to generate headings like this.

May 16

On Wednesday, 28 February 2024 at 17:18:04 UTC, Paul Backus wrote:

>

Currently, function attributes and function parameter attributes are only inferred by the D compiler for certain kinds of functions. This DIP idea proposes that such inference be extended to all non-overridable functions with bodies.

Why not just provide a tool or a compiler switch that tells you where something should have been annotated, but isn’t?

Something like -helpAnnotate=safe,system,nogc and the compiler will tell you which functions in your code base could be @safe and/or @nogc, but aren’t annotated @safe and/or @nogc, and which functions are @system, but lack the annotation.

May 16

On Thursday, 16 May 2024 at 17:25:53 UTC, Quirin Schroll wrote:

>

On Wednesday, 28 February 2024 at 17:18:04 UTC, Paul Backus wrote:

>

Currently, function attributes and function parameter attributes are only inferred by the D compiler for certain kinds of functions. This DIP idea proposes that such inference be extended to all non-overridable functions with bodies.

Why not just provide a tool or a compiler switch that tells you where something should have been annotated, but isn’t?

Something like -helpAnnotate=safe,system,nogc and the compiler will tell you which functions in your code base could be @safe and/or @nogc, but aren’t annotated @safe and/or @nogc, and which functions are @system, but lack the annotation.

If programmers are required to take any kind of manual action to add attributes to their functions, no matter how easy or straightforward it is, a large portion of them simply will not do it. The default effect is extremely strong.

The Github comment from Andrei that I linked in the first post tells the same story:

>

[Jonathan] Aldrich has written a number of papers about augmenting Java with annotations for various properties, notably ArchJava and AliasJava. He has conducted thorough experiments with real developers and projects in researching for this work, and has noticed (and mentioned this in his papers, talks, and our group discussions) that as a rule of thumb developers never go back and annotate their code with the most precise signatures. [...] Although Jonathan's early work focused on offering programmers the ability to add annotations, before long he realized he must extend his work to do annotation inference in order to make it useful.

May 23
On Monday, 18 March 2024 at 10:30:28 UTC, Richard (Rikki) Andrew Cattermole wrote:
>
> On 18/03/2024 12:12 PM, Timon Gehr wrote:
>> On 3/9/24 22:47, Richard (Rikki) Andrew Cattermole wrote:
>>> On 10/03/2024 10:26 AM, ryuukk_ wrote:
>>>> If there is a significant build penalty, than i hope it's opt-in, i personally do not use any of the attributes
>>>
>>> It shouldn't be significant.
>>>
>>> It is a minor cost as things go, no reason to start thinking opt-in at this stage.
>> 
>> Really? This proposal requires full semantic analysis of any imported function. It is only minor in certain build setups. (E.g., when the project is compiled in a single invocation.)
>
> I am assuming that you generate .di files every time and use that for importing, not the .d files.
>
> These days I'm leaning quite heavily towards projects using .di files for intermediary steps, due to distribution reasons with shared libraries.
>
> It's already impossible to keep bindings to D code up to date manually, I tried pure was a killer on that idea.

One of my favorite features of D is its compilation model and in particular the ability to import from source files directly. This simplifies fast incremental parallel builds by not requiring a separate step to extract the interface of a module, or worse, requiring that modules are compiled in topological order.

C++20, Fortran, and Haskell modules and Rust crates work the other way; they require interface files to be generated for their compilation units before they can be imported. Consequently they do not integrate well with traditional build systems.

I personally don't mind maintaining the attributes by hand, but I understand other people would gladly trade convenience for build simplicity and performance. Perhaps the compiler could be configured to infer attributes on functions from certain modules and not others, catering to both use cases.
May 24
On Thursday, 23 May 2024 at 01:14:59 UTC, zopsicle wrote:
> One of my favorite features of D is its compilation model and in particular the ability to import from source files directly. This simplifies fast incremental parallel builds by not requiring a separate step to extract the interface of a module, or worse, requiring that modules are compiled in topological order.
>
> C++20, Fortran, and Haskell modules and Rust crates work the other way; they require interface files to be generated for their compilation units before they can be imported. Consequently they do not integrate well with traditional build systems.

I should emphasize that the ability to import directly from .d files is not going anywhere, even if this proposal is accepted. The .di files would function purely as a cache to improve performance, and generating them would never be required.