Thread overview
`in` on function parameters: const, scope const, ref scope const ?
Apr 03, 2020
Mathias LANG
Apr 05, 2020
tsbockman
Apr 05, 2020
tsbockman
Apr 05, 2020
tsbockman
April 03, 2020
There's been a discussion recently about `-preview` switches (https://forum.dlang.org/thread/r627m8$2p6c$1@digitalmars.com).

It turns out one of the preview switch that is to be introduced next release is `-preview=in`, meant to make `in` on parameters (not to be confused with opIn) means `scope const` instead of just `const`.
Now I wasn't particularly fond of this PR, since it doesn't seem logical to me to have 2 switches for the same thing (`in` was changed to mean `const` when DIP1000 was a PR, aiming to reduce the breakage, but before it was behind a switch, which completely nullified the need for changing it in the first place).

But since leadership decided to go with the switch anyway, why not go all the way and make it means what it was actually supposed to mean in the first place ? My experience with D is that I had to stick quite a few `const ref scope` (in varying order) on my parameters to mean "this is an input parameter. I'm going to read it, not modify it. And I will not keep a reference to it, because I just need it for the processing of this function". However I couldn't use `in` because it would pass things by value, meaning my `struct`s would get blitted over and over.

I have submitted a PR for it: https://github.com/dlang/dmd/pull/11000 and would like to know how other users of `in` expect it to behave, or would want it to behave.
Bear in mind this is a `-preview` switch: it won't break your code.
One of the downside I'm aware of is passing rvalue. But luckily, we got another preview for this (`-preview=rvaluerefparam`).
April 05, 2020
On Friday, 3 April 2020 at 06:19:38 UTC, Mathias LANG wrote:
> I have submitted a PR for it: https://github.com/dlang/dmd/pull/11000 and would like to know how other users of `in` expect it to behave, or would want it to behave.

Given the possibility of aliasing, adding `ref` to parameters that are merely `const`, but not `immutable`, could break existing code in very subtle ways. (I know there has been talk of disallowing aliasing of `ref` arguments, but I don't think D's tracking of such things is reliable enough to depend on to prevent code breakage yet. Also, I'm hoping that proposal isn't actually implemented, anyway, because I find intentionally aliasing `ref` parameters useful.) Less importantly, adding `ref` where it wasn't intended may cause minor performance problems.

However, if I ignore the code breakage issue, then `in` == `ref scope const` is the most aesthetically pleasing option for the symmetry with `out`. I'll also guess that by-value `scope const` is needed a lot less often than `ref scope const`, so this is probably the better long-term option to reduce the considerable verbosity of the language - IF we can avoid breaking too much existing code due the altered run-time semantics from adding `ref`.
April 05, 2020
On Sunday, 5 April 2020 at 02:01:05 UTC, tsbockman wrote:
> On Friday, 3 April 2020 at 06:19:38 UTC, Mathias LANG wrote:
>> I have submitted a PR for it: https://github.com/dlang/dmd/pull/11000 and would like to know how other users of `in` expect it to behave, or would want it to behave.
>
> Given the possibility of aliasing, adding `ref` to parameters that are merely `const`, but not `immutable`, could break existing code in very subtle ways.

Interesting, I wasn't aware of this. Can you elaborate more?

> (I know there has been talk of disallowing aliasing of `ref`
> arguments, but I don't think D's tracking of such things is
> reliable enough to depend on to prevent code breakage yet. Also,
> I'm hoping that proposal isn't actually implemented, anyway,
> because I find intentionally aliasing `ref` parameters useful.)

Can you provide an example?

> However, if I ignore the code breakage issue, then `in` == `ref scope const` is the most aesthetically pleasing option for the symmetry with `out`.

I completely agree.

> I'll also guess that by-value `scope const` is needed a lot less
> often than `ref scope const`, so this is probably the better
> long-term option to reduce the considerable verbosity of the
> language - IF we can avoid breaking too much existing code due the
> altered run-time semantics from adding `ref`.

I think the best option would be what @kinke proposed in that pull request:

> I think the optimal solution would be either `const scope` or `const scope ref`, depending on the type and target ABI, for the most efficient way to pass an input parameter. I.e., no ref (extra indirection) for class references and small PODs (incl. delegates and slices for most ABIs), but a ref for non-PODs and large PODs. To be coupled with `-preview=rvaluerefparam` as you've mentioned in the forum thread.

https://github.com/dlang/dmd/pull/11000#issuecomment-608382442
April 05, 2020
On Friday, 3 April 2020 at 06:19:38 UTC, Mathias LANG wrote:
> [..]

One more thought:

`in` needs be its own parameter storage class, like `out` and `ref` on the name-mangling / ABI level. This is the only way to change its meaning while preserving it in compiler-generated *.di headers and preventing subtle errors while linking different libraries compiled with and without the preview switch.

April 05, 2020
On Sunday, 5 April 2020 at 12:54:01 UTC, Petar Kirov [ZombineDev] wrote:
> On Sunday, 5 April 2020 at 02:01:05 UTC, tsbockman wrote:
>> Given the possibility of aliasing, adding `ref` to parameters that are merely `const`, but not `immutable`, could break existing code in very subtle ways.
>
> Interesting, I wasn't aware of this. Can you elaborate more?

`const` marks a read-only view of some data, for which there *may* currently be mutable aliases, as well. `immutable` means that all current aliases for the data are read-only, and so it is guaranteed not to change as long as the type system is not subverted through unsafe casting.

import std.stdio;

bool isPositive(in int x, scope ref int callCount) pure @safe {
    callCount += 1;
    return (x > 0);
}

void main() @safe {
    int a = 0;
    writeln(isPositive(a, a));
}

The above should print "true" if `in` means `ref scope const`, and "false" if `in` means just `scope const`. This minimal example is completely contrived, of course, but the issue does appear in reasonable real world code, too.

Aliasing can occur via `ref`, but also via class references, pointers, delegates, array slices, and user-defined data structures containing or referencing any of those things at any level of indirection. And, those are just things that need to be tracked to detect aliasing in a program that is 100% @safe code; with @trusted, there is the possibility that additional aliases may be encoded, and correctly managed, in arbitrary data types that are not understood as pointer types by the semantic analyzer:

import std.stdio;

struct CustomRef {
    private size_t address;
    void opAssign(int* target) pure @trusted nothrow @nogc {
        address = cast(size_t) target;
    }
    ref inout(int) access() inout pure @trusted nothrow @nogc {
        return *cast(inout(int)*) address;
    }
}

bool isPositive(ref scope const int x, scope ref int callCount) pure @safe {
    callCount += 1;
    return (x > 0);
}

int a = 0;
CustomRef q;

void main() @safe {
    q = &a;
    writeln(isPositive(a, q.access));
}

Again, this is a contrived example, and simple enough that the semantic analyzer could no doubt be extended to catch it. But, there are valid practical applications for code like this, some of which may be far more difficult, or even impossible, to analyze automatically in the general case.

So, changing `in` to be pass-by-ref is, at least in theory, a very serious breaking change - *far* more so than adding `scope`, which was always planned anyway. If we're going to do this I think a long deprecation cycle is needed for the existing behavior, with aggressive warnings about the upcoming change.

I don't think it will break very much code in the real world, it's just that some of the code it does break will probably fail in such a subtle way, for certain inputs only, that it will be a real pain to debug.
April 05, 2020
On Sunday, 5 April 2020 at 12:59:23 UTC, Petar Kirov [ZombineDev] wrote:
> `in` needs be its own parameter storage class, like `out` and `ref` on the name-mangling / ABI level.

If we go with @kinke 's suggestion, then yes `in` definitely needs to be its own storage class, and we'll need good meta-programming tools for working with it.