On Tuesday, 6 January 2015 at 03:29:39 UTC, Zach the Mystic wrote:
> A more likely scenario is that your library starts small enough not to need the @api attribute, then at some point it gets really, really huge. Then in one fell swoop you decide to "@api:" your whole file so that the public interface won't change so often. I'm picking the most extreme case I can think of, in order to argue the point from a different perspective.

Note that if you want auto-inferred attributes during the alpha phase of library development, it's just as trivial to put a general @autoinfer: or @noapi: or whatever you like, and that in turn is a pretty nice signifier to the user "this function's attributes are not guaranteed to be stable".

> Attribute inference provides convenience, not guarantees.

Indeed.  But any publicly-available API is a guarantee of sorts.  From the moment people are using something, you can no longer vicariously break things.

> If a user was relying on the purity of a function which was never marked 'pure', it's only convenience which allows him to do it, both on the part of the user, for adding 'pure', and the library writer, for *not* adding it.

Nevertheless, if a user relies on that inferred purity (which they will do), and you tweak things so the function is no longer pure, you have broken that downstream user's code.  Worse, you'll have done it in a way which flies under the radar until someone actually tries to build against your updated library.  You, as the library developer, won't get any automatic warning that you've broken backwards compatibility with your earlier implementation; the downstream user won't get any automatically-documented warnings of this breaking change.

If instead you have an explicit "please auto-infer attributes for this function" marker, then at least the user has a very clear warning that any attributes possessed by this function cannot be relied on.  (Absence of a guarantee != presence of a statement that there is NO guarantee:-)

> Adding @api (or 'extern (noinfer)') cancels that convenience for the sake of modularity. It's a tradeoff. The problem  itself is solved either by the library writer marking the function 'pure', or the user removing 'pure' from his own function.

As a library writer, I don't think you can responsibly expect users to bear the burden of fixing undocumented breaking change.

> Without @api,  the problem only arises when the library writer
> actually does something impure, which makes perfect sense.
> It's @api (and D's existing default, by the way) which adds
> the artificiality to the process, not my suggested default.

I'm not sure what exactly you mean when you talk about D's existing default, but one aspect that I think is important is: D's default position is that a function has no guarantees, and you _add_ guarantees to it via attributes.

This whole discussion would be quite different if the default was that a function is expected to be @safe, pure, nothrow, etc., and the developer is expected to use attributes to indicate _weakening_ of those guarantees.

>> It's quite analogous in this respect to the argument about final vs. virtual by default for class methods.
>
> I don't think so, because of so-called covariance. Final and virtual each have their own advantages and disadvantages, whereas inferring attributes only goes one way. There is no cost to inferring in the general case.

I think you have missed the point I was making.

If you have final-by-default for classes, and you accidentally forget to tag a public method as 'virtual', then you can fix that without breaking any downstream user's code.  If by contrast you have virtual-by-default and you accidentally forget to tag a public method as 'final', then you can't fix that without the risk of breaking downstream; _someone_ may have relied on that function being virtual.

The situation is very similar here.  If your function has no attributes, and then later you add one (say, 'pure'), then you don't do any downstream user any harm.  If on the other hand your function _does_ have attributes -- whether explicit or inferred -- and then you remove them, you risk breaking downstream code.

If you don't auto-infer, this is not really an issue, because you have to manually add and remove attributes, and so you can never unintentionally or unknowingly remove an attribute.  But if you _do_ auto-infer, then it's very easy indeed to accidentally remove attributes that your downstream may have relied on.

> My suggestion, (I now prefer 'extern(noinfer)'), does absolutely
> nothing except to restore D's existing default, for what I think
> are the rare cases it is needed. I could be wrong about just how
> rare using extern(noinfer) will actually be, but consider that
> phobos, for example, just doesn't need it, because it's too small
> a library to cause trouble if all of a sudden one of its
> non-templated functions becomes impure.

I don't think you can reasonably anticipate how much trouble breaking change can cause for your downstreams.  At a minimum, _accidental_ and undocumented breaking change is completely unacceptable, and this proposal introduces many easy ways to see it happen.

> A quick recompile, a new interface file, and now everyone's using the new thing. Even today, it's not even marked up with attributes completely, thus indicating that you never even *could* have used it for all it's worth.
>
> Have I convinced you?

I understand why you want it, I just think you underestimate the importance of avoiding accidentally breaking things for downstream.

Note that I'd be much more prepared to be convinced if the proposal was that attributes should be auto-inferred _except_ for public functions, although that has its own range of nastinesses in terms of inconsistent behaviour.

> tldr: I like what you're thinking, please can we have this.

+1

Atila

On Tuesday, 6 January 2015 at 12:07:21 UTC, Joseph Rushton Wakeling wrote:
> As a library writer, I don't think you can responsibly expect users to bear the burden of fixing undocumented breaking change.

I agree, maybe just replace "module" with "library". Also make "module" mandatory. It takes no extra keyword if it always is the first token.

Then the compiler can be more lax with "module" files and more strict with "library" files.

On Tuesday, 6 January 2015 at 09:14:00 UTC, John Colvin wrote:
> tldr: I like what you're thinking, please can we have this.

Hey thanks, John.

On Tuesday, 6 January 2015 at 09:12:43 UTC, John Colvin wrote:
> It's worth noting that with full inference then much less code would be explicitly annotated, which goes some way towards ameliorating the stability problem as more client code will be flexible w.r.t. changing attributes on library boundaries.

That's a great point. Explicit attributes will only be for larger projects which really want those guarantees. Everyone else gets all the optimizations for free... pretty cool!

On Tuesday, 6 January 2015 at 12:07:21 UTC, Joseph Rushton Wakeling wrote:
> If you have final-by-default for classes, and you accidentally forget to tag a public method as 'virtual', then you can fix that without breaking any downstream user's code.  If by contrast you have virtual-by-default and you accidentally forget to tag a public method as 'final', then you can't fix that without the risk of breaking downstream; _someone_ may have relied on that function being virtual.

That's a much bigger problem than what you get with inferred attributes. If you have whole class hierarchy built around a function which isn't even supposed to be virtual, you'll have a lot of refactoring to do when it's marked final.

The only way in which downstream code will ever "break" is when the called function is longer expressed by the interface as 'pure'. Under the new default, this will only happen when the function *actually stops being pure*. This is good! The only code which will then break is code which is itself marked 'pure'. The solution is either to remove 'pure' from the user's function, or to stop calling the impure function. It's not the end of the world if a user function stops being pure (@safe, nothrow, etc.). Since the called function was never marked 'pure', and thus never guaranteed to be pure to begin with, it was only luck and convenience which allowed the user to tag his code 'pure' in the first place. And, as John Colvin just pointed out, with the new defaults, the user most likely won't even bother to tag his code 'pure', because he knows he will get all the optimization benefits without the hassle. The only reason to tag 'pure' from now on will be to *guarantee* it. At that point, you *want* your code to break, but only when your function actually stops being pure.

With D's existing default (and with code marked with the suggested new attribute 'extern(noinfer)'), you have another reason why your 'pure' function won't compile. Indeed, it's already broken. I ask you, why should a user have to wait until someone marks a function 'pure' to get the advantages of its purity, when the compiler already has everything it needs in order to grant those advantages? In the rare case where you actually want to revert to the way D is now, then you simply have to pay the price, which is the *same* price that everyone is already paying now.

On Tuesday, 6 January 2015 at 12:07:21 UTC, Joseph Rushton Wakeling wrote:
> I think you have missed the point I was making.
>
> If you have final-by-default for classes, and you accidentally forget to tag a public method as 'virtual', then you can fix that without breaking any downstream user's code.  If by contrast you have virtual-by-default and you accidentally forget to tag a public method as 'final', then you can't fix that without the risk of breaking downstream; _someone_ may have relied on that function being virtual.


For people making libraries which really need to keep ABI compatibility there should be some quick trick to tell the compiler to build those ABI-stable interfaces like:

  class {
  final @api:
    ...

  virtual @api:
    ...
  }

It's imho a small price to pay for not having to always write explicit attributes for most D functions.

Martin Nowak is the one who created the issue for this enhancement request:

https://issues.dlang.org/show_bug.cgi?id=10979

He proposes a different solution though - creating two mangled names for each function. The advantage, as far as I can see it, is that you wouldn't even have to add "@api" or "extern(noinfer)" to link properly. I see two disadvantages: the binary would suffer bloat, and the generated .di signatures would have to default to the conservative explicit signature, since is could not include both signatures without causing a duplicate definition.

On Thursday, 8 January 2015 at 04:43:50 UTC, Zach the Mystic wrote:
> https://issues.dlang.org/show_bug.cgi?id=10979

Whoops! The link is:
https://issues.dlang.org/show_bug.cgi?id=10924

On Monday, 5 January 2015 at 21:15:00 UTC, Zach the Mystic wrote:
> https://github.com/D-Programming-Language/dmd/pull/1877

Deep into the discussion on the above pull, Walter responds to this comment of Kenji's:

"I think this kind of optimization should be hidden after the semantic phase - as like inlining and backend optimizations."

...with this:

"The trouble with doing this is that it is extremely dangerous in a language that supports separate compilation. It means that a function can be treated as pure, but its mangled signature says its impure. Then, the build system swaps in another version of that function, with the same signature, but is impure. Now the code that calls it is silently broken - the worst kind of runtime error.
This is why attribute inference is limited to those functions where the definitions are guaranteed to be available - templates, lambdas, nesteds and autos - then there can be no runtime mismatch."

What bothers me about this is that the compiler could *know* that a function is pure any time it has the function body available to it, regardless of what kind of function it is. The above quote suggests, however, that it's just completely impossible for the compiler to know if the function which will be called is the one that it has just compiled, or if it will be swapped in by a linker with another function with the same name from somewhere else. I honestly don't know enough about how linking works to know if this cancels my proposal. Wouldn't it cause an error if the linker found two of the exact same function?

My naive assumption about it is that when the compiler has the body, it can treat it as pure from within, even if it gets expressed by 'extern(noinfer)' as impure. That is, from within the singly compiled program, it can be treated as if it *is* pure, being one-hundred percent certain that the function it compiles will be the one that gets called at binary time, but for anyone reading the expressed interface, it can only be compiled as if it's *not* pure, since that's all the information the interface tells them.