Thread overview
in; scope; scope ref; DIP1000; documentation
Aug 27, 2020
James Blachly
Aug 27, 2020
ag0aep6g
Aug 28, 2020
Mathias LANG
August 27, 2020
Peeling off from Mathias Lang's thread in General about making 'in' useful, for some novice questions:


1. The thread involves 'in' qualifier. Documentation (https://dlang.org/spec/function.html#param-storage) indicates that `in` is defined as `scope const` and should not be used as it is not implemented. **Is this [fact and recommendation] still true?**


2. Regarding scope:

On 8/5/20 3:27 AM, Fynn Schröder wrote:
> On Friday, 31 July 2020 at 21:49:25 UTC, Mathias LANG wrote:
>> I hope this will generate interest with people hitting the same problem.
> 
> I've literally yesterday written some new code with `const scope ref` in almost every function to pass large, complex structs. Occasionally, I had to store rvalues temporarily to pass as lvalues (non-templated code). I would rather simply put `in` on those parameters :-) It's a lot easier to grasp function signatures only using `in` and `out` on parameters (and their effect/purpose being immediately obvious to new D programmers!)

Is "scope ref" documented somewhere specifically? I found https://dlang.org/spec/function.html#scope-parameters which discusses the use of `scope` with ref type parameters, but the example given is pointer-based. Is it correct that `scope ref T` behaves the same as `scope T*` ?

Regarding `scope` more generally, DIP1000 shows as "superseded" -- **can I still rely on this document for guidance?** We have a `-dip1000` flag but a superseded DIP. The discordance is extremely confusing.


I am glad D is iterating quickly and improving on safety, but I have found that documentation may not well recent changes in this area. Consequently I am reluctant to use (newer) features related to memory safety. If this is all comprehensively documented somewhere please let me know!

August 27, 2020
On 27.08.20 20:49, James Blachly wrote:
> 1. The thread involves 'in' qualifier. Documentation (https://dlang.org/spec/function.html#param-storage) indicates that `in` is defined as `scope const` and should not be used as it is not implemented. **Is this [fact and recommendation] still true?**

There is this:

    https://dlang.org/changelog/2.092.0.html#preview-in

Which says that `in` is planned to become `const scope` in the future and there is a preview switch for it: `-preview=in`.

But these are both false (with and without `-preview=in`):

    is(void function (in int* x) == void function (const int* x))
    is(void function (in int* x) == void function (scope const int* x))

So `in` is not strictly equivalent to anything else. I'd say the recommendation is good: Avoid `in` until that stuff is sorted out and `in` actually gets lowered to `scope const` (or whatever it ends up meaning).

[...]
> Is "scope ref" documented somewhere specifically? I found https://dlang.org/spec/function.html#scope-parameters which discusses the use of `scope` with ref type parameters, but the example given is pointer-based. Is it correct that `scope ref T` behaves the same as `scope T*` ?

I don't think there's documentation for `scope ref`. That may be because `scope` doesn't affect `ref`. This is a corner of DIP 1000 that keeps confusing me.

Consider this function:

    int* fp(scope int** p) { return *p; }

This compiles with `-preview=dip1000`. That's because the `scope` only applies to the outer pointer. It doesn't apply to the inner one. So returning the inner pointer is fine. At least, that's how I understand DIP 1000.

Now consider this one:

    int* fr(scope ref int* r) { return r; }

One might expect the same result for fr. A `ref` is pretty much the same thing as a pointer, isn't it? But you actually get an error: "scope variable `r` may not be returned". I think that's because the `scope` doesn't apply to the `ref` part of the parameter; it applies to the pointer.

Similarly, this compiles:

    int* fp(int* p) { return p; }

But this doesn't:

    int* fr(ref int r) { return &r; }

Apparently, a `ref` is not "pretty much the same thing as a pointer". It's more restricted. It acts like a `scope` pointer without needing the `scope` annotation. Unfortunately, this isn't documented, as far as I can tell.

> Regarding `scope` more generally, DIP1000 shows as "superseded" -- **can I still rely on this document for guidance?** We have a `-dip1000` flag but a superseded DIP. The discordance is extremely confusing.
> 
> 
> I am glad D is iterating quickly and improving on safety, but I have found that documentation may not well recent changes in this area.

Agreed. The documentation is in a bad state. The information is spread over documents, forum posts, and of course the implementation in DMD. And all of those are wrong/outdated in parts.

Regarding the different sources of information:

1) Start with the spec (<https://dlang.org/spec/function.html#scope-parameters>). Unlike DIP documents, the spec is being updated. It might be incomplete or have errors, but it at least has a chance to get fixed.

2) Ignore the DIP document (<https://github.com/dlang/DIPs/blob/master/DIPs/other/DIP1000.md>). Only refer to it as a last resort. Any other source is more likely to be correct.

3) Do check what the compiler does, but don't trust it blindly. The implementation still has some serious bugs. A personal favorite: <https://issues.dlang.org/show_bug.cgi?id=20150>

4) Bugzilla (<https://issues.dlang.org>): If a bug report demonstrates a safety violation with only @safe code, it's valid. If it doesn't, there's a significant chance that the reporter missed some detail about `scope` and the issue ends up being invalid.

> Consequently I am reluctant to use (newer) features related to memory safety. If this is all comprehensively documented somewhere please let me know!

For the time being, I think it's perfectly fine to ignore `-preview=dip1000`. The feature is clearly not finished.
August 28, 2020
On Thursday, 27 August 2020 at 18:49:19 UTC, James Blachly wrote:
> Peeling off from Mathias Lang's thread in General about making 'in' useful, for some novice questions:
>
>
> 1. The thread involves 'in' qualifier. Documentation (https://dlang.org/spec/function.html#param-storage) indicates that `in` is defined as `scope const` and should not be used as it is not implemented. **Is this [fact and recommendation] still true?**

I have a PR to update the documentation: https://github.com/dlang/dlang.org/pull/2845
I got feedback on it, just didn't have time to complete / finish it, will come back to it soon.

> Is "scope ref" documented somewhere specifically? I found https://dlang.org/spec/function.html#scope-parameters which discusses the use of `scope` with ref type parameters, but the example given is pointer-based. Is it correct that `scope ref T` behaves the same as `scope T*` ?
>
> Regarding `scope` more generally, DIP1000 shows as "superseded" -- **can I still rely on this document for guidance?** We have a `-dip1000` flag but a superseded DIP. The discordance is extremely confusing.

It is, for users and compilers developers alike. We're still waiting for Walter to write down the docs on that. If you look in the forums / PRs, you'll see that it's the most common feedback.

> I am glad D is iterating quickly and improving on safety, but I have found that documentation may not well recent changes in this area. Consequently I am reluctant to use (newer) features related to memory safety. If this is all comprehensively documented somewhere please let me know!

We have a policy that every language change should come with a spec PR. It has improved documentation drastically, but there's still a way to go.

I hope the aforementioned PR will clarify the status of `in`. I'm currently focusing on making sure it works properly everywhere before the next release (in 1 or 2 weeks), hence why fixing that PR has taken the back-seat (the website gets auto-updated, so even if the PR doesn't make the deadline, it'll be deployed as soon as it gets merged).