Thread overview
What is the current state of scope and member functions?
Sep 10, 2023
Jonathan M Davis
Sep 10, 2023
Dennis
Sep 11, 2023
Jonathan M Davis
Sep 12, 2023
Dennis
Sep 12, 2023
Jonathan M Davis
Sep 12, 2023
Dennis
Sep 12, 2023
Jonathan M Davis
September 10, 2023
It is my understanding that with the changes for DIP 1000, scope now applies to member functions, in which case, it marks the implicit this parameter as scope. And I get deprecation messages in some of my code complaining about stuff like popFront on a range type not being scope, which would seem to indicate that scope does indeed now apply to member functions. However, I don't seem to be getting any inference for it. For instance, this very simple example does not compile:

void main()
{
    import std.traits;
    static assert(functionAttributes!(Range!int.popFront) &
                  FunctionAttribute.scope_);
}

struct Range(T)
{
    void popFront()
    {
    }
}

The member function is templated (because the type is templated), and it does literally nothing, so I would expect scope to be inferred, but it clearly isn't. To make matters even weirder, if I explicitly put scope on popFront, it _still_ fails to compile. And printing out the function's type gives

pure nothrow @nogc @safe void()

in both cases, so this doesn't seem to be a problem with std.traits or my usage of it. Rather, it seems to indicate that not only is scope not being inferred for popFront, but it's being outright ignored when it's used explicitly. This would imply that scope is _not_ supposed to be used on member functions.

And whether I use -preview=dip1000 seems to have no effect on this particular example. scope is not inferred if it isn't explicitly there, and it's ignored if it is explicitly there.

So, am I misunderstanding something about how scope is supposed to work, and it's not supposed be used on member functions? And if that's the case, why am I getting deprecation messages about member functions not being scope within an @safe function? Or is this a compiler bug that needs fixing?

- Jonathan M Davis



September 10, 2023

On Sunday, 10 September 2023 at 21:23:06 UTC, Jonathan M Davis wrote:

>

in both cases, so this doesn't seem to be a problem with std.traits or my usage of it. Rather, it seems to indicate that not only is scope not being inferred for popFront, but it's being outright ignored when it's used explicitly.

scope on a variable is implicitly removed when the variable's type has no pointers. This includes the this parameter, and your example struct has 0 members, so 0 pointers.

I removed this behavior a few months ago for regular parameters because checking if the type had pointers caused forward reference errors: https://github.com/dlang/dmd/pull/14561

The this parameter doesn't cause forward reference errors, but perhaps scope should also stay there for consistency and simplicity.

September 10, 2023
On Sunday, September 10, 2023 4:15:48 PM MDT Dennis via Digitalmars-d wrote:
> On Sunday, 10 September 2023 at 21:23:06 UTC, Jonathan M Davis
>
> wrote:
> > in both cases, so this doesn't seem to be a problem with std.traits or my usage of it. Rather, it seems to indicate that not only is scope not being inferred for popFront, but it's being outright ignored when it's used explicitly.
>
> `scope` on a variable is implicitly removed when the variable's type has no pointers. This includes the `this` parameter, and your example struct has 0 members, so 0 pointers.
>
> I removed this behavior a few months ago for regular parameters because checking if the type had pointers caused forward reference errors: https://github.com/dlang/dmd/pull/14561
>
> The `this` parameter doesn't cause forward reference errors, but perhaps `scope` should also stay there for consistency and simplicity.

Well, I don't know what the overall pros and cons of the current behavior are, but I found it to be very confusing. AFAIK, no other attributes magically disappear like that. And having the behavior between the this parameter and explicit parameters differ is likely to add to the confusion for folks as well. DIP 1000 is already arguably overly complicated for what it buys us, and I'd think that removing unnecessary complexity with regards to what's going on with scope would be a good thing.

- Jonathan M Davis



September 12, 2023

On Monday, 11 September 2023 at 00:12:29 UTC, Jonathan M Davis wrote:

>

Well, I don't know what the overall pros and cons of the current behavior are, but I found it to be very confusing.

In a templated struct, you can annotate member functions scope without it cluttering up the function type when the struct is instantiated without pointers.

>

AFAIK, no other attributes magically disappear like that.

__gshared and shared disappear on immutable variables.

>

DIP 1000 is already arguably overly complicated for what it buys us, and I'd think that removing unnecessary complexity with regards to what's going on with scope would be a good thing.

We could try remove it.

September 12, 2023
On Tuesday, September 12, 2023 3:26:02 AM MDT Dennis via Digitalmars-d wrote:
> On Monday, 11 September 2023 at 00:12:29 UTC, Jonathan M Davis
> > AFAIK, no other attributes magically disappear like that.
>
> __gshared and shared disappear on immutable variables.

That's only because they're redundant given that immutable is implicitly shared. It would be like how const disappears on a template parameter when the type itself is already const - e.g. const(T) being const(int) when T is const(int). So, they don't really disappear, and what's happening is very obvious based on the types. This is in stark contrast to scope just outright disappearing even though it's explicitly used just because the compiler decided that there were no indirections. There is logic to it disappearing when there are no indirections, since it doesn't really do anything, but it's not at all intuitively obvious that that's how things would work, whereas with double consts or shared immutable, the redundant attribute basically has to go away, so it's far less surprising or confusing.

> > DIP 1000 is already arguably overly complicated for what it buys us, and I'd think that removing unnecessary complexity with regards to what's going on with scope would be a good thing.
>
> We could try remove it.

Given the incredible complexity that DIP 1000 has, I really think that inconsistencies like this should be removed. Honestly, I have a hard time believing that DIP 1000 is going to go over well in the long run if it stays anywhere near as complex as it is. IMHO, it's currently well beyond what the average programmer is going to reasonably be able to understand, and my gut reaction is that it would be better to just slap @trusted in a bunch of places to shut the compiler up about scope than it would be to try to actually make it work - though unfortunately, that's not going to work very well with templated code (particularly in libraries), which is precisely where I'm seeing the compiler complain about scope even though I'm not doing stuff like taking the address of local variables.

So, anything that can be done to reduce the complexity (particularly unnecessary complexity) and reduce the number of special cases is likely to be a good thing. And having parameters and this parameters behave differently with regards to scope seems unnecessarily confusing.

- Jonathan M Davis



September 12, 2023

On Tuesday, 12 September 2023 at 09:52:58 UTC, Jonathan M Davis wrote:

>

and my gut reaction is that it would be better to just slap @trusted in a bunch of places to shut the compiler up about scope than it would be to try to actually make it work - though unfortunately, that's not going to work very well with templated code (particularly in libraries), which is precisely where I'm seeing the compiler complain about scope even though I'm not doing stuff like taking the address of local variables.

Please file a bug when you get lifetime errors while not taking the address of a local. Note that slicing a local static array is taking the address, and it being allowed in @safe code without dip1000 is a long standing accepts-invalid bug. dip1000 is strictly allowing more code to compile, and all the breakage comes from the accepts-invalid bug. Robert's DConf '23 proposal to simplify by disallowing @safe code slicing local static arrays altogether strictly breaks more code.

As for removing the scope-stripping of member functions, I remember I tried that, but failed because it would break code in an unexpected way (forward references again):

https://github.com/dlang/dmd/pull/14232#issuecomment-1162906573

September 12, 2023
On Tuesday, September 12, 2023 4:48:13 AM MDT Dennis via Digitalmars-d wrote:
> On Tuesday, 12 September 2023 at 09:52:58 UTC, Jonathan M Davis
>
> wrote:
> > and my gut reaction is that it would be better to just slap @trusted in a bunch of places to shut the compiler up about scope than it would be to try to actually make it work - though unfortunately, that's not going to work very well with templated code (particularly in libraries), which is precisely where I'm seeing the compiler complain about scope even though I'm not doing stuff like taking the address of local variables.
>
> Please file a bug when you get lifetime errors while not taking the address of a local. Note that slicing a local static array is taking the address, and it being allowed in @safe code without dip1000 is a long standing accepts-invalid bug. dip1000 is strictly allowing more code to compile, and all the breakage comes from the accepts-invalid bug. Robert's DConf '23 proposal to simplify by disallowing @safe code slicing local static arrays altogether  strictly breaks more code.

IMHO, the big mistake was making static arrays implicitly convert to dynamic arrays. Being able to slice them to get dynamic arrays is fine, but automatically slicing them is a lot like automatically taking the address of a local variable to get an implicit conversion to a pointer. The need for scope would go down considerably if such conversions weren't happening in an essentially invisible manner, and the programmer was required to explicitly slice a static array to get a dynamic array just like they're required to explicitly take the address of a local variable to get a pointer. Unfortunately, Walter didn't agree with that idea (probably in large part because it would have required deprecating existing behavior), so instead of fixing it and reducing the need for something like scope, we got DIP 1000 instead.

In general, I'm not in favor of being as extreme as Robert was proposing with regards to removing stuff from @safe, but DIP 1000 seems like a pretty extreme set of changes to solve a fairly small problem that most D programmers who aren't constantly interacting with C code should only rarely need to worry about. Simply getting rid of the implicit conversion from static arrays to dynamic arrays would go a _long_ way IMHO.

As for reporting stuff, I will if I can, but even figuring out what's supposed to be happening - particularly in highly templated code - can be difficult if you're not really well-versed in exactly how scope is supposed to work right now. scope seems to have become very complicated in the search to make more and more code work with it, thus reducing the need for @trusted, instead of just requiring that the programmer use @trusted when taking the address of a local (or slicing a static array) and then letting them make sure that they're doing it right. I can appreciate the sentiment and goal behind scope, but the more I deal with it, the more it seems like it's a nuke trying to take out an ant. At the moment, I'm inclined to believe that it adds _way_ too much complication to the language for minimal benefit.

Quite possibly the biggest design smell with regards to scope is return scope and how the order and adjacency of the attributes changes the semantics. Almost no one is going to remember that kind of thing. But scope has become ridiculously complicated in general. And honestly, as things stand, if DIP 1000 were enabled by default, I would probably stop using @safe entirely just to save myself the headaches that scope introduces - or at least, I'd stop using it whenever the compiler complained about scope.

> As for removing the scope-stripping of member functions, I remember I tried that, but failed because it would break code in an unexpected way (forward references again):
>
> https://github.com/dlang/dmd/pull/14232#issuecomment-1162906573

Well, I certainly don't know what is going to be possible on the implementation side of things, but every inconsistency that's added to scope's design is going to make it harder to understand. And if it's too hard to understand, most people will just avoid it rather than benefiting from the feature. So, if inconsistencise can reasonably be removed, they should be.

- Jonathan M Davis