On 5/14/2014 12:29 AM, Idan Arye wrote:
> On Tuesday, 13 May 2014 at 17:44:02 UTC, Walter Bright wrote:
>> On 5/13/2014 6:50 AM, Dicebot wrote:
>>> Walter's initial post implies that he wanted to re-used `ref` for borrowed
>>> pointer (which would mean same semantics as `scope` parameter qualifier)
>>
>> 'ref' already is to much extent, for example:
>>
>>   @safe int foo(ref int x) {
>>     auto a = &x;
>>     return 3;
>>   }
>>
>> dmd foo -c
>> foo.d(4): Error: cannot take address of parameter x in @safe function foo
>
> This has nothing to do with `ref`. If you remove the `ref` you get the exact
> same error.

Right, but the thing is, the only address of a local/parameter you can take is by ref, not &.

On Tuesday, 13 May 2014 at 19:17:16 UTC, Jacob Carlborg wrote:
> What is "scope ref" supposed to do in this example, compared to just "scope"?

To be a reference ;) But yeah, it is not important in this example, plain scope should behave the same if transitive.

On Tuesday, 13 May 2014 at 20:46:19 UTC, Walter Bright wrote:
> On 5/13/2014 12:06 PM, Dicebot wrote:
>> No, it still can be necessary. `scope` can greatly help not only with resource
>> releasing, it is also missing tool to safely cast from shared. Locking shared
>> variable can return same variable casted to scope qualifier which will
>> guarantee  that no reference has been stored to shared object by the time lock
>> is released.
>
> I believe that is the role of `unique`. DIP69 addresses making unique pointers in D, and there have been several PR's implementing aspects of it.

I don't really understand how you see this working. Sure, unique concept from that DIP can somewhat replace `scope` storage class. But you can't prove that some function does not leak unique variable internals if this is not annotated as such.

>> And "if those are marked as refcounted" as assumption is no better than "if
>> those are owned by GC" ;)
>
> I think that an object that wants to completely own its resources must properly encapsulate and restrict unsafe access to them itself.

This statement is not much different from "any programmer who cares about memory should manage it manually" or "instead of const qualifier you can simply use convention". Doing this without compiler help is tedious. Mistakes result in bugs that are insanely hard to debug (storing non-shared reference to shared object by an accident).

>> Also A can only control escaping of any internal references only by completely
>> prohibiting access to it which is not good. You have no means to say "feel free
>> to use this reference as long as you don't keep it outside of current scope".
>> And you effectively say "make all your array members private to keep borrowing
>> guarantees".
>
> You can by only returning ref's.

Also slices and pointers (or structs with pointers inside).

On 2014-05-14 15:00, Dicebot wrote:

> To be a reference ;) But yeah, it is not important in this example,
> plain scope should behave the same if transitive.

I though that "A" was a class in the previous example, but now I see that it was a struct.

-- 
/Jacob Carlborg

On 5/14/2014 6:09 AM, Dicebot wrote:
> On Tuesday, 13 May 2014 at 20:46:19 UTC, Walter Bright wrote:
>> On 5/13/2014 12:06 PM, Dicebot wrote:
>>> No, it still can be necessary. `scope` can greatly help not only with resource
>>> releasing, it is also missing tool to safely cast from shared. Locking shared
>>> variable can return same variable casted to scope qualifier which will
>>> guarantee  that no reference has been stored to shared object by the time lock
>>> is released.
>>
>> I believe that is the role of `unique`. DIP69 addresses making unique pointers
>> in D, and there have been several PR's implementing aspects of it.
>
> I don't really understand how you see this working. Sure, unique concept from
> that DIP can somewhat replace `scope` storage class. But you can't prove that
> some function does not leak unique variable internals if this is not annotated
> as such.

I'm not suggesting unique can replace scope, I am suggesting they are separate concepts.


>>> And "if those are marked as refcounted" as assumption is no better than "if
>>> those are owned by GC" ;)
>>
>> I think that an object that wants to completely own its resources must
>> properly encapsulate and restrict unsafe access to them itself.
>
> This statement is not much different from "any programmer who cares about memory
> should manage it manually" or "instead of const qualifier you can simply use
> convention". Doing this without compiler help is tedious. Mistakes result in
> bugs that are insanely hard to debug (storing non-shared reference to shared
> object by an accident).

My comment applies when the destructor behaves as if it owns resources other than the object instance itself, in which case the class designer is already manually managing them.


>>> Also A can only control escaping of any internal references only by completely
>>> prohibiting access to it which is not good. You have no means to say "feel free
>>> to use this reference as long as you don't keep it outside of current scope".
>>> And you effectively say "make all your array members private to keep borrowing
>>> guarantees".
>>
>> You can by only returning ref's.
>
> Also slices and pointers (or structs with pointers inside).

The idea is that 'ref' are borrowed pointers, so if you're returning pointers, the borrowed semantics do not apply.

On Wednesday, 14 May 2014 at 19:03:20 UTC, Walter Bright wrote:
>>>> Also A can only control escaping of any internal references only by completely
>>>> prohibiting access to it which is not good. You have no means to say "feel free
>>>> to use this reference as long as you don't keep it outside of current scope".
>>>> And you effectively say "make all your array members private to keep borrowing
>>>> guarantees".
>>>
>>> You can by only returning ref's.
>>
>> Also slices and pointers (or structs with pointers inside).
>
> The idea is that 'ref' are borrowed pointers, so if you're returning pointers, the borrowed semantics do not apply.

Somewhat more extended example:

struct Buffer
{
    private byte[] data;

    this() { this.data = (cast(byte*)malloc(42))[0..42]; }
    ~this() { free(this.data.ptr); }

    byte[] get(size_t a, size_t b) { return this.data[a..b]; }
}

void foo(ref Buffer buff)
{
    // here be trouble
    static int[] slice = buff.get(10, 20);
}

void foo2()
{
    Buffer buff;
    foo(buff);
    // destructor gets called, foo now has pointer to freed memory
}

Transitivity of borrowing ensures that you can use any object as an argument for function that takes a borrowed pointer and no reference to its internals will persist. Whatever memory management model of object type is.

With such borrowing implementation this example code is also totally @safe in spirit (assignment to static var will result in compile-time error).

On 5/15/2014 5:18 AM, Dicebot wrote:
> On Wednesday, 14 May 2014 at 19:03:20 UTC, Walter Bright wrote:
>> The idea is that 'ref' are borrowed pointers, so if you're returning pointers,
>> the borrowed semantics do not apply.
>
> Somewhat more extended example:
>
> struct Buffer
> {
>      private byte[] data;
>
>      this() { this.data = (cast(byte*)malloc(42))[0..42]; }
>      ~this() { free(this.data.ptr); }
>
>      byte[] get(size_t a, size_t b) { return this.data[a..b]; }
> }
>
> void foo(ref Buffer buff)
> {
>      // here be trouble
>      static int[] slice = buff.get(10, 20);
> }
>
> void foo2()
> {
>      Buffer buff;
>      foo(buff);
>      // destructor gets called, foo now has pointer to freed memory
> }
>
> Transitivity of borrowing ensures that you can use any object as an argument for
> function that takes a borrowed pointer and no reference to its internals will
> persist. Whatever memory management model of object type is.
>
> With such borrowing implementation this example code is also totally @safe in
> spirit (assignment to static var will result in compile-time error).

get() is returning a pointer to its internally managed data (in the form of []). You're right that transitivity of borrowing would support this safely, but I am not proposing that for ref. To make slicing Buffer safe, one would have to overload opSlice and then manage access to the slice.

On Thursday, 15 May 2014 at 18:08:06 UTC, Walter Bright wrote:
> get() is returning a pointer to its internally managed data (in the form of []). You're right that transitivity of borrowing would support this safely, but I am not proposing that for ref. To make slicing Buffer safe, one would have to overload opSlice and then manage access to the slice.

Sure, I simply question its practical applicability in that case. It is not like I want borrowed pointer semantics only because it looks cool :) Transitive borrowing solves certain class of issues that currently rely on convention, enabling whole new type of verified safe code (both memory safe and concurrency safe). Head-only? Doesn't look so.

On 5/16/2014 9:43 AM, Dicebot wrote:
> Transitive
> borrowing solves certain class of issues that currently rely on convention,
> enabling whole new type of verified safe code (both memory safe and concurrency
> safe). Head-only? Doesn't look so.

I'm concerned that transitive borrowing will *preclude* a number of useful cases.

On Friday, 16 May 2014 at 17:22:21 UTC, Walter Bright wrote:
> On 5/16/2014 9:43 AM, Dicebot wrote:
>> Transitive
>> borrowing solves certain class of issues that currently rely on convention,
>> enabling whole new type of verified safe code (both memory safe and concurrency
>> safe). Head-only? Doesn't look so.
>
> I'm concerned that transitive borrowing will *preclude* a number of useful cases.

Which is why `ref` itself can't be used for that and usage of `scope` as qualifier is necessary to enable transitive behavior :)