Is there a way to prevent this?

import std.stdio;
struct S(size_t elems)
{
    int[elems] data;
}

void foo(T)(in T constdata, ref T normaldata)
{
    normaldata.data[0] = 1;
    writeln(constdata.data[0]);
}
void main()
{
    S!1 smallval;
    foo(smallval, smallval);
    S!100 largeval;
    foo(largeval, largeval);
}


Compile without -preview=in, it prints:

0
0

Compile with -preview=in, it prints:

0
1

-Steve

On 10/2/20 10:08 AM, Steven Schveighoffer wrote:
> Is there a way to prevent this?

Or at least warn about it?

-Steve

On Friday, 2 October 2020 at 14:08:29 UTC, Steven Schveighoffer wrote:
> Is there a way to prevent this?
>
> import std.stdio;
> struct S(size_t elems)
> {
>     int[elems] data;
> }
>
> void foo(T)(in T constdata, ref T normaldata)
> {
>     normaldata.data[0] = 1;
>     writeln(constdata.data[0]);
> }
> void main()
> {
>     S!1 smallval;
>     foo(smallval, smallval);
>     S!100 largeval;
>     foo(largeval, largeval);
> }
>
>
> Compile without -preview=in, it prints:
>
> 0
> 0
>
> Compile with -preview=in, it prints:
>
> 0
> 1
>
> -Steve

Hmm, that doesn't look good 🤔

On 10/2/20 10:08 AM, Steven Schveighoffer wrote:
> Is there a way to prevent this?
> 
> import std.stdio;
> struct S(size_t elems)
> {
>      int[elems] data;
> }
> 
> void foo(T)(in T constdata, ref T normaldata)
> {
>      normaldata.data[0] = 1;
>      writeln(constdata.data[0]);
> }
> void main()
> {
>      S!1 smallval;
>      foo(smallval, smallval);
>      S!100 largeval;
>      foo(largeval, largeval);
> }
> 
> 
> Compile without -preview=in, it prints:
> 
> 0
> 0
> 
> Compile with -preview=in, it prints:
> 
> 0
> 1

Finally, my "told you so" moment has come! :o)

https://forum.dlang.org/post/rhmst4$1vmc$1@digitalmars.com

On 10/2/20 10:32 AM, Andrei Alexandrescu wrote:
> On 10/2/20 10:08 AM, Steven Schveighoffer wrote:
>> Is there a way to prevent this?
>>
>> import std.stdio;
>> struct S(size_t elems)
>> {
>>      int[elems] data;
>> }
>>
>> void foo(T)(in T constdata, ref T normaldata)
>> {
>>      normaldata.data[0] = 1;
>>      writeln(constdata.data[0]);
>> }
>> void main()
>> {
>>      S!1 smallval;
>>      foo(smallval, smallval);
>>      S!100 largeval;
>>      foo(largeval, largeval);
>> }
>>
>>
>> Compile without -preview=in, it prints:
>>
>> 0
>> 0
>>
>> Compile with -preview=in, it prints:
>>
>> 0
>> 1
> 
> Finally, my "told you so" moment has come! :o)
> 
> https://forum.dlang.org/post/rhmst4$1vmc$1@digitalmars.com

My problem with it isn't necessarily that it uses references in some cases vs. copies in others, it's that the decision is arbitrary and implementation defined.

Legitimately, you can have code that compiles fine on one compiler and breaks subtly on others.

But good to see this was at least discussed. However, I'm not sure the result is what should have happened...

Just noticed too, if you want to *force* ref by using in ref, you get this message:

Error: attribute ref is redundant with previously-applied in

That is... not good.

I think in should always mean ref. If you want to pass not by ref, use const.

-Steve

> Is there a way to prevent this?

In the general case, no. You can have two distinct pointers with the same value, and there's nothing the frontend can do to detect it.

This scenario has been brought up during the review. I doubt it will, in practice, be an issue though. This is not a common pattern, nor does it seems useful. It rather looks like a code smell.
Bear in mind that in D, `const` data can change under your feet. The following case is similar to your example:
```
char[16] buffer;
foo(buffer, buffer);
void foo (const(char)[] data, char[] buff);
```
And yet, no one complains that it breaks `const`. The fact that `in` sometimes means by value, and sometimes by `ref`, seems to be the problem. I think, in our explanation of `in`, we ought to phrase things this way: `in` passes by `ref` *unless* it is more efficient to pass by value. With the added note that mutating the parameter through an indirection should not be relied on.


On Friday, 2 October 2020 at 14:48:18 UTC, Steven Schveighoffer wrote:
>
> My problem with it isn't necessarily that it uses references in some cases vs. copies in others, it's that the decision is arbitrary and implementation defined.

Not *quite* arbitrary. It's only passed by value if it's relatively efficient to do so. That means that anything that'd triggers a copy constructor, dtor, postblit, etc... is guaranteed to be passed by `ref`.

> Just noticed too, if you want to *force* ref by using in ref, you get this message:
>
> Error: attribute ref is redundant with previously-applied in
>
> That is... not good.
>
> I think in should always mean ref. If you want to pass not by ref, use const.

This limitation was, I think, the main pain point for people during the review.
The main reason for disallowing it is that allowing it would open the door for overloading based on `in`, which is *definitely* not something we want, since it would mean people relying on by-value passing of `in`.
While that was the main reason why I went this way (remember I experimented with several designs), there were two additional benefits that cemented the conviction that it was the way to go.
First, it creates a nice separation between the three parameters storage classes: `in`, `ref`, `out`, hopefully simplifying the language and making it easier to explain.
Second, it allowed me to bake in a little trick: When `ref` is inferred for `in`, the parameter *actually* mangles as `in ref`. The point of doing that was to prevent code compiled with different compilers, or version of the same compilers, to link if they had a mismatch in their inference rules.

Regarding the "in should always be `ref`", I following Kinke's advice here, which seems sensible: it should be up to the ABI to decide what is the most efficient way to pass a parameter. But there is also a reason why you don't want everything to be `ref`. Consider the following two types:
```
alias A = void delegate(in char[]);
alias B = void delegate(const(char)[]);
```
I wanted `A` to be implicitly convertible to `B`, because not only does it make sense, but it avoids a lot of the code breakage I was seeing in druntime.

On Friday, 2 October 2020 at 14:32:50 UTC, Andrei Alexandrescu wrote:
> On 10/2/20 10:08 AM, Steven Schveighoffer wrote:
>> Is there a way to prevent this?
>> 
>> import std.stdio;
>> struct S(size_t elems)
>> {
>>      int[elems] data;
>> }
>> 
>> void foo(T)(in T constdata, ref T normaldata)
>> {
>>      normaldata.data[0] = 1;
>>      writeln(constdata.data[0]);
>> }
>> void main()
>> {
>>      S!1 smallval;
>>      foo(smallval, smallval);
>>      S!100 largeval;
>>      foo(largeval, largeval);
>> }
>> 
>> 
>> Compile without -preview=in, it prints:
>> 
>> 0
>> 0
>> 
>> Compile with -preview=in, it prints:
>> 
>> 0
>> 1
>
> Finally, my "told you so" moment has come! :o)
>
> https://forum.dlang.org/post/rhmst4$1vmc$1@digitalmars.com

Could we be ambitious and aim to have ownership taken to the max and catch this statically? This particular case is relatively low hanging fruit but having the in parameter work this way would be nice if it was safe.

On 10/2/20 1:01 PM, Mathias LANG wrote:
>> Is there a way to prevent this?
> 
> In the general case, no. You can have two distinct pointers with the same value, and there's nothing the frontend can do to detect it.
> 
> This scenario has been brought up during the review. I doubt it will, in practice, be an issue though. This is not a common pattern, nor does it seems useful. It rather looks like a code smell.

Of course, the exact sample that I wrote is not what happens. What happens is something more convoluted. But it will happen.

> Bear in mind that in D, `const` data can change under your feet. The following case is similar to your example:
> ```
> char[16] buffer;
> foo(buffer, buffer);
> void foo (const(char)[] data, char[] buff);
> ```
> And yet, no one complains that it breaks `const`. The fact that `in` sometimes means by value, and sometimes by `ref`, seems to be the problem. I think, in our explanation of `in`, we ought to phrase things this way: `in` passes by `ref` *unless* it is more efficient to pass by value. With the added note that mutating the parameter through an indirection should not be relied on.

Yes, the problem is the "sometimes ref". Because ref changes the semantics.

I read it as, in means by ref, unless the compiler can prove it's the same to pass by value, and that is more efficient. But if it's for *optimization*, it shouldn't change the effective semantics. The optimizer should be invisible.

In practice, I don't think the compiler can prove that.

> 
> 
> On Friday, 2 October 2020 at 14:48:18 UTC, Steven Schveighoffer wrote:
>>
>> My problem with it isn't necessarily that it uses references in some cases vs. copies in others, it's that the decision is arbitrary and implementation defined.
> 
> Not *quite* arbitrary. It's only passed by value if it's relatively efficient to do so. That means that anything that'd triggers a copy constructor, dtor, postblit, etc... is guaranteed to be passed by `ref`.

One point of decision is arbitrary -- is it big enough to be worth it to pass by ref. That "big enough" decision is clearly specified to depend on compiler/ABI. Quoting from the changelog:

"Otherwise, if the type's size requires it, it will be passed by reference. Currently, types which are over twice the machine word size will be passed by reference, however this is controlled by the backend and can be changed based on the platform's ABI."

>> Just noticed too, if you want to *force* ref by using in ref, you get this message:
>>
>> Error: attribute ref is redundant with previously-applied in
>>
>> That is... not good.
>>
>> I think in should always mean ref. If you want to pass not by ref, use const.
> 
> This limitation was, I think, the main pain point for people during the review.
> The main reason for disallowing it is that allowing it would open the door for overloading based on `in`, which is *definitely* not something we want, since it would mean people relying on by-value passing of `in`.

Why would it be any different? What I mean is pass by ref always, but still allow binding to lvalues and rvalues.

> While that was the main reason why I went this way (remember I experimented with several designs), there were two additional benefits that cemented the conviction that it was the way to go.
> First, it creates a nice separation between the three parameters storage classes: `in`, `ref`, `out`, hopefully simplifying the language and making it easier to explain.

All three of them are consistent, if `in` always means `by reference`.

> Second, it allowed me to bake in a little trick: When `ref` is inferred for `in`, the parameter *actually* mangles as `in ref`. The point of doing that was to prevent code compiled with different compilers, or version of the same compilers, to link if they had a mismatch in their inference rules.

So you like the situation that 2 compilers will not be compatible? Instead of they just work because everyone does the same thing?

> 
> Regarding the "in should always be `ref`", I following Kinke's advice here, which seems sensible: it should be up to the ABI to decide what is the most efficient way to pass a parameter.

This is NOT about how to pass a parameter. The ABI does not make a decision on by value vs. by ref. The semantics are different.

The only way it can make this decision and still be sane is if passing by ref does not change the semantics of the resulting code.

> But there is also a reason why you don't want everything to be `ref`. Consider the following two types:
> ```
> alias A = void delegate(in char[]);
> alias B = void delegate(const(char)[]);
> ```
> I wanted `A` to be implicitly convertible to `B`, because not only does it make sense, but it avoids a lot of the code breakage I was seeing in druntime.

And this might not be true on a different compiler.

-Steve

On Friday, 2 October 2020 at 17:16:46 UTC, Max Haughton wrote:
> Could we be ambitious and aim to have ownership taken to the max and catch this statically? This particular case is relatively low hanging fruit but having the in parameter work this way would be nice if it was safe.

For which parameter types should such ownership checking be performed?

All kinds of references types including ref params, classes and pointers or a subset of them?

On Friday, 2 October 2020 at 17:31:06 UTC, Steven Schveighoffer wrote:
> Yes, the problem is the "sometimes ref". Because ref changes the semantics.
>
> I read it as, in means by ref, unless the compiler can prove it's the same to pass by value, and that is more efficient. But if it's for *optimization*, it shouldn't change the effective semantics. The optimizer should be invisible.
>
> In practice, I don't think the compiler can prove that.

A good backend most certainly can? This ought to be a pure backend issue and not affect the fontend at all if the separation front/back is as it should be.

Calling conventions do not belong in a language spec...