On Monday, 8 December 2014 at 16:04:42 UTC, Steven Schveighoffer wrote:
> On 12/8/14 10:45 AM, "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm@gmx.net>" wrote:
>> On Monday, 8 December 2014 at 15:12:51 UTC, Steven Schveighoffer wrote:
>>> I think you should eliminate scope returns then. They are not useful.
>>> I can't think of a single reason why a newly allocated via GC or
>>> global reference return should have to be restricted to exist only
>>> within the statement. Both have infinite lifetimes.
>>
>> It's for references to objects that are owned by the function (or object
>> of which the function is a method). These don't have infinite lifetimes.
>
> Why not? An object is allocated on the heap, and has infinite lifetime.

"object" as in "instance of struct" ;-)

And I was referring to objects owned by something with finite lifetime, e.g. a container.

    struct Array(T) {
        scope ref T opIndex(size_t);
    }

    Array!int container;
    ref x = container[42];  // no
    container[42]++;        // yes
    writeln(container[42]); // yes

On Monday, 8 December 2014 at 16:25:22 UTC, Dicebot wrote:
> On Sunday, 7 December 2014 at 21:29:50 UTC, Walter Bright wrote:
>> On 12/7/2014 6:12 AM, Dicebot wrote:
>>> But from existing cases it doesn't seem working good enough. For example, not
>>> being able to represent idiom of `scope ref int foo(scope ref int x) { return x;
>>> }` seems very limiting.
>>
>>  scope ref int foo(ref int x);
>>
>> will do it.
>
> This isn't the same as it does not propagate scope but just restricts return value. Difference is that it cannot be chained. Let's consider practical example based on Phobos:
>
> there was an issue with byLine range that it has reused same buffer internally which sometimes caught users off guard when trying to save slice. It is a natural fit for `scope` - make it return `scope string` instead to ensure that no slices get stored.
>
> Two issues immediately pop up:
>
> 1) scope is not transitive thus it doesn't work at all - you still can store slice of `scope string` as only actual ptr+length struct is protected.
>
> 2) even if it worked, existing definition of scope return value makes it impossible to use in typical idiomatic pipeline: `file.byLine.algo1.algo2`. Either algoX is defined to take `scope ref` and thus can't return it or it is defined to take `ref` and can't take another `scope ref` as an argument.

That's why I asked the question in http://forum.dlang.org/post/xdjsmwocbtxovjnathek@forum.dlang.org . It seems Walter wants to allow passing `scope ref` to `ref`, but then automatically treat a normal `ref` of the second function as if it were `scope ref`. But I can't quite see through it :-(

>
> At least this is what I get from reading existing examples in DIP69
>
>>> I also don't consider `ref` design as a storage class any kind of success at all
>>> and generally agree with Manu on this topic. At the same time alternative
>>> proposals that make it a qualifier (like Marc did) do impact existing language
>>> much more and this no small concern.
>>
>> My experience with C++ ref as type qualifier is very, very bad. It's a special case EVERYWHERE. Doing type deduction with it is an exercise in a completely baffling set of rules and a different rule for every occasion - Scott Meyers has a great piece on this.
>>
>> There are probably only a handful of people on the planet who actually understand C++ ref. I wished very hard to avoid that with D ref.
>
> While there is no argument that C++ ref is screwed, it is rather hard to say if this is inherent consequence of ref being a type qualifier or just C++ being C++. I mean how many C++ type system features in general are understood my more than a handful of people on the planet? For me `ref` is essentially just a different flavor of `*` - and if the latter can be part of type, I see no reasons why former can't

I think most problems would come from type deduction. All the other qualifiers and function attributes, like const and pure, just either work, or they don't, so they can be tested easily, and you want to apply these qualifiers wherever possible. This also applies to scope. But for `ref`, it's presence influences behaviour, therefore I think it would be best if it were never inferred implicitly. Actually that too is just what we do with pointers. We don't change non-pointer types to pointers automatically.

On 05.12.14 21:55, Walter Bright wrote:
> 
> It means that this code will be safe:
> 
>     void foo(scope int* p);
> 
>     p = malloc(n);
>     foo(p);
>     free(p);
> 
> The rest is all the nuts and bolts of making that work.
> 

If it is main goal, opposite looks more natural.

void foo(int* p);
void foo1(escape int* p);

p = malloc(n);
p1 = malloc(n);
foo(p);
free(p);  // OK
foo1(p1);
free(p1);  // no

On 12/8/14 11:27 AM, "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm@gmx.net>" wrote:
> On Monday, 8 December 2014 at 16:04:42 UTC, Steven Schveighoffer wrote:
>> On 12/8/14 10:45 AM, "Marc =?UTF-8?B?U2Now7x0eiI=?=
>> <schuetzm@gmx.net>" wrote:
>>> On Monday, 8 December 2014 at 15:12:51 UTC, Steven Schveighoffer wrote:
>>>> I think you should eliminate scope returns then. They are not useful.
>>>> I can't think of a single reason why a newly allocated via GC or
>>>> global reference return should have to be restricted to exist only
>>>> within the statement. Both have infinite lifetimes.
>>>
>>> It's for references to objects that are owned by the function (or object
>>> of which the function is a method). These don't have infinite lifetimes.
>>
>> Why not? An object is allocated on the heap, and has infinite lifetime.
>
> "object" as in "instance of struct" ;-)
>
> And I was referring to objects owned by something with finite lifetime,
> e.g. a container.
>
>      struct Array(T) {
>          scope ref T opIndex(size_t);
>      }
>
>      Array!int container;
>      ref x = container[42];  // no

Why not? x has the same lifetime as container!

This is an incorrect arbitrary decision:

module foo;

Array!int arr;

int *x;

void bar()
{
   x = &arr[0]; // should be ok.
}

A struct's member function has no idea of its lifetime, except what it's told.

Now, it would make more sense for this:

ref T opIndex(size_t);
scope ref T opIndex(size_t) scope;

But it's sounding to me (from Walter's other comments) like the latter function can't exist, because you can't return scope from scope? And furthermore, there's no way to overload based on scope? I'm really not liking this whole proposal, it either needs to be explained better, or rewritten.

-Steve

On Monday, 8 December 2014 at 16:57:44 UTC, Marc Schütz wrote:
>> Two issues immediately pop up:
>>
>> 1) scope is not transitive thus it doesn't work at all - you still can store slice of `scope string` as only actual ptr+length struct is protected.
>>
>> 2) even if it worked, existing definition of scope return value makes it impossible to use in typical idiomatic pipeline: `file.byLine.algo1.algo2`. Either algoX is defined to take `scope ref` and thus can't return it or it is defined to take `ref` and can't take another `scope ref` as an argument.
>
> That's why I asked the question in http://forum.dlang.org/post/xdjsmwocbtxovjnathek@forum.dlang.org . It seems Walter wants to allow passing `scope ref` to `ref`, but then automatically treat a normal `ref` of the second function as if it were `scope ref`. But I can't quite see through it :-(

Yes I have seen that thread and have no idea what Walter has meant either. His answer only makes sense if return value scope is supposed to be inferred from body - what when bodies are guaranteed to be there everything can be inferred anyway.

On 12/8/2014 8:25 AM, Dicebot wrote:
> 2) even if it worked, existing definition of scope return value makes it
> impossible to use in typical idiomatic pipeline: `file.byLine.algo1.algo2`.
> Either algoX is defined to take `scope ref` and thus can't return it or it is
> defined to take `ref` and can't take another `scope ref` as an argument.
>
> At least this is what I get from reading existing examples in DIP69

The difference between 'scope ref' and 'ref' parameters is that the former cannot be returned by reference.

The difference between 'scope ref' and 'ref' function returns is that the former cannot be saved by the caller.

You can still safely pass the address of a stack variable by 'ref' - it will not escape. Under the current proposal, as now, you cannot store a ref by ref, and cannot take the address of a ref variable.


>> There are probably only a handful of people on the planet who actually
>> understand C++ ref. I wished very hard to avoid that with D ref.
> While there is no argument that C++ ref is screwed, it is rather hard to say if
> this is inherent consequence of ref being a type qualifier or just C++ being
> C++.

It's not because C++ designers are idiots. It inevitably follows from what ref is. ref grew, brick by brick, into a monster, each brick being inevitable.


> I mean how many C++ type system features in general are understood my more
> than a handful of people on the planet? For me `ref` is essentially just a
> different flavor of `*` - and if the latter can be part of type, I see no
> reasons why former can't

I agree it's a seductively simple idea. The trouble starts happening when you start when making ref idempotent, when ref can only be at the 'head' of a data structure, when trying to do type deduction of a ref type (do you get the ref, or do you look 'through' the ref?), what happens with overloading, etc., and on and on.

On Monday, 8 December 2014 at 19:44:48 UTC, Walter Bright wrote:
> The difference between 'scope ref' and 'ref' parameters is that the former cannot be returned by reference.
>
> The difference between 'scope ref' and 'ref' function returns is that the former cannot be saved by the caller.
>
> You can still safely pass the address of a stack variable by 'ref' - it will not escape. Under the current proposal, as now, you cannot store a ref by ref, and cannot take the address of a ref variable.

Easier to go straight with pseudo-code:

struct ByLine
{
    scope string front();
    // ...
}

auto byLine(File file)
{
    return ByLine(file);
}

scope /* ref */ string foo(scope /* ref */ string input)
{
    return input[1..$];
}

void main()
{
    auto r = file.byLine.map!foo;
    string s = r.front; // this should not compile
    string s = r.front.dup; // this should compile

    // how foo signature should look like for this to work?
}


>> I mean how many C++ type system features in general are understood my more
>> than a handful of people on the planet? For me `ref` is essentially just a
>> different flavor of `*` - and if the latter can be part of type, I see no
>> reasons why former can't
>
> I agree it's a seductively simple idea. The trouble starts happening when you start when making ref idempotent, when ref can only be at the 'head' of a data structure, when trying to do type deduction of a ref type (do you get the ref, or do you look 'through' the ref?), what happens with overloading, etc., and on and on.

But was there any reason why those traits (alien to type qualifiers) were pursued? What is the problem with `ref` simply meaning `non-null pointer` and allowing non-idempotent ref(ref(int))?

On 12/8/2014 12:54 PM, Dicebot wrote:
> On Monday, 8 December 2014 at 19:44:48 UTC, Walter Bright wrote:
>> I agree it's a seductively simple idea. The trouble starts happening when you
>> start when making ref idempotent, when ref can only be at the 'head' of a data
>> structure, when trying to do type deduction of a ref type (do you get the ref,
>> or do you look 'through' the ref?), what happens with overloading, etc., and
>> on and on.
>
> But was there any reason why those traits (alien to type qualifiers) were
> pursued? What is the problem with `ref` simply meaning `non-null pointer` and
> allowing non-idempotent ref(ref(int))?

Because it isn't just a non-null pointer (and ref's can still be null in C++!), it's an auto-dereferencing pointer.

On 12/8/2014 12:54 PM, Dicebot wrote:
> struct ByLine
> {
>      scope string front();
>      // ...
> }
>
> auto byLine(File file)
> {
>      return ByLine(file);
> }
>
> scope /* ref */ string foo(scope /* ref */ string input)
> {
>      return input[1..$];
> }
>
> void main()
> {
>      auto r = file.byLine.map!foo;
>      string s = r.front; // this should not compile
>      string s = r.front.dup; // this should compile
>
>      // how foo signature should look like for this to work?
> }

front() should return a 'scope ref string'.

On 12/7/2014 2:46 AM, Sebastiaan Koppe wrote:
> On Friday, 5 December 2014 at 23:58:41 UTC, Walter Bright wrote:
>> On 12/5/2014 8:48 AM, "Marc Schütz" <schuetzm@gmx.net>" wrote:
>>>     scope ref int foo();
>>>     scope ref int bar1(ref int a) {
>>>         return a;
>>>     }
>>>     scope ref int bar2(scope ref int a) {
>>>         return a;
>>>     }
>>>     ref int bar3(ref int a) {
>>>         return a;
>>>     }
>>>     ref int bar4(scope ref int a) {
>>>         return a;
>>>     }
>>>     void baz(scope ref int a);
>>>
>>> Which of the following calls would work?
>>>
>>>     foo().bar1().baz();
>>
>> yes
>>
>>>     foo().bar2().baz();
>>
>> no - cannot return scope ref parameter
>>
>>>     foo().bar3().baz();
>>
>> yes
>>
>>>     foo().bar4().baz();
>>
>> no, cannot return scope ref parameter
>
> I understand that scope will not allow the contents of the variable to escape
> the lifetime of a declaration. But can you explain why bar1() works, but bar2()
> doesn't?

A 'scope ref' parameter may not be returned as a 'ref' or a 'scope ref'.

> Isn't the body of bar2() in the line `foo().bar2();` part of the
> declaration?
>
> Besides, what does it mean to return a `scope ref int`? Does it mean that the
> content of the variable that is returned is not allowed to escape the scope of
> the calling site? Huh?

It means the reference itself (the pointer) does not escape.