View mode: basic / threaded / horizontal-split · Log in · Help
November 11, 2012
Re: Const ref and rvalues again...
11/10/2012 11:07 PM, Jonathan M Davis пишет:
> And actually, to make matters worse, I'm not sure that scope on delegates is
> working correctly. I thought that it was, but this code compiles:
>
> import std.stdio;
>
> void delegate() global;
>
> void foo(scope void delegate() del)
> {
>      global = del;
> }
>
> void main()
> {
>      {
>          char[5] bar = "hello";
>          foo((){writeln(bar);});
>      }
>      char[7] baz = "goodbye";
>
>      global();
> }
>
> It also prints out "hello", and if a closure had not been allocated,
I would
> have at least half-expected it to print out "goodb", because I'd have thought
> that baz would have been taking up the same memory that bar had been.

Nope. It's just that the stack is intact and contains: hello and goodbye 
one after another. Without optimizations { } scope doesn't mean reuse 
stack space.

Now if play with stack a bit, for me the next one prints:
­-²↑

import std.stdio;

void delegate() global;

void foo(scope void delegate() del)
{
    global = del;
}


void f()
{
    {
        char[5] bar = "hello";
        foo((){writeln(bar);});
    }
}

void main()
{
    char[7] baz = "goodbye";
    f();

    global();
}

-- 
Dmitry Olshansky
November 11, 2012
Re: Const ref and rvalues again...
On Sunday, November 11, 2012 13:36:05 Dmitry Olshansky wrote:
> Nope. It's just that the stack is intact and contains: hello and goodbye
> one after another. Without optimizations { } scope doesn't mean reuse
> stack space.
> 
> Now if play with stack a bit, for me the next one prints:
> ­-²↑
> 
> import std.stdio;
> 
> void delegate() global;
> 
> void foo(scope void delegate() del)
> {
>      global = del;
> }
> 
> 
> void f()
> {
>      {
>          char[5] bar = "hello";
>          foo((){writeln(bar);});
>      }
> }
> 
> void main()
> {
>      char[7] baz = "goodbye";
>      f();
> 
>      global();
> }

It still prints "hello", even with full optimations turned on. So, it must be 
allocating a closure in spite of scope. So, it looks to me like scope is just 
completely ignored and does absolutely nothing at this point, unless I'm just 
completely missing something here.

- Jonathan M Davis
November 11, 2012
Re: Const ref and rvalues again...
On Sunday, 11 November 2012 at 10:09:17 UTC, Jonathan M Davis 
wrote:
> It still prints "hello", even with full optimations turned on. 
> So, it must be
> allocating a closure in spite of scope. So, it looks to me like 
> scope is just
> completely ignored and does absolutely nothing at this point, 
> unless I'm just
> completely missing something here.

Try this:

---
import std.stdio;

void delegate() global;

void foo(scope void delegate() del)
{
     global = del;
}


void f()
{
     {
         char[5] bar = "hello";
         foo((){writeln(bar);});
     }
}

void smashStack() {
    uint[1000] dummy = 0xbadcab1e;
    asm { nop; }
}

void main()
{
     char[7] baz = "goodbye";
     f();
     smashStack();
     global();
}
---

David
November 11, 2012
Re: Const ref and rvalues again...
11/11/2012 2:08 PM, Jonathan M Davis пишет:
> On Sunday, November 11, 2012 13:36:05 Dmitry Olshansky wrote:
>> Nope. It's just that the stack is intact and contains: hello and goodbye
>> one after another. Without optimizations { } scope doesn't mean reuse
>> stack space.
>>
>> Now if play with stack a bit, for me the next one prints:
>> ­-²↑
>>
>> import std.stdio;
>>
>> void delegate() global;
>>
>> void foo(scope void delegate() del)
>> {
>>       global = del;
>> }
>>
>>
>> void f()
>> {
>>       {
>>           char[5] bar = "hello";
>>           foo((){writeln(bar);});
>>       }
>> }
>>
>> void main()
>> {
>>       char[7] baz = "goodbye";
>>       f();
>>
>>       global();
>> }
>
> It still prints "hello", even with full optimations turned on.

I tried with and without optimizations. I get garbage as expected.

So, it must be
> allocating a closure in spite of scope. So, it looks to me like scope is just
> completely ignored and does absolutely nothing at this point, unless I'm just
> completely missing something here.

Something must be screwed up. I dunno what, I use near-latest DMD from 
github and Win32 binaries.
For good measure try making stack variables larger if you are on 64bit. 
Drop in a  couple of calls to writeln before and after calling 'f' it 
should scramble the stack.

-- 
Dmitry Olshansky
November 11, 2012
Re: Const ref and rvalues again...
On 10 November 2012 21:07, Jonathan M Davis <jmdavisProg@gmx.com> wrote:

> On Saturday, November 10, 2012 13:21:42 Manu wrote:
> > I'm still not buying this. Here's a common struct I will pass by ref
> > (perhaps the most common struct in my industry):
> >
> > struct Vector { float, x,y,z,w; }
> > struct Matrix { Vector xRow, yRow, zRow, wRow; }
> >
> > Vector mul( scope const ref Matrix m, scope const Vector v)
> > {
> >   Vector v;
> >   // perform a matrix multiply against the vector...
> >   // this work uses every single field of the inputs given, but the
> result
> > it produces has to references to the sources.
> >   // everything is operated on and copied to the output struct, which is
> > returned.
> >   return result;
> > }
> >
> > Why should this be a problem?
> > The majority of my work-horse structs apply to this pattern. This is
> what I
> > imagine 'scope' to be for...
> > The main advantage I expect is that I can have confidence that passing
> > rvalues (temporaries) is safe, and that external code won't take
> references
> > to memory that I may not own/control. Is that not the point?
> >
> > Surely the problem that scope should be protecting against is a pointer
> to
> > any part of the argument escaping. *Copies* of values contained in the
> > argument/s are fine.
>
> Hmmmm. scope on value types is pointless, because there are no references
> to
> escape, but if you pass by ref, then it does become possible for a pointer
> to
> the argument to escape,


Precisely, to me, this seems like the ENTIRE POINT of 'in'?  (I originally
presumed 'in' implied ref, but I was wrong, 'in ref' is supported however)


but I don't know that that's actually actually covered
> by scope. The description for scope in docs is that "ref­er­ences in the
> pa­
> ra­me­ter can­not be es­caped (e.g. as­signed to a global vari­able)." And
> taking the address of a local variable (which is the only way that any
> sort of
> reference to the data could escape) is never @safe anyway.


When did '@safe'ty enter into it? Are you saying that a ref variable is
somehow a local variable? It's a local pointer to a foreign variable... and
a function can usually operate on that data however it likes.
Scope would promise that nothing other than the function I give it to will
get its grubby little hands on it.
Let's say that function wanted to call through to some other function and
pass the variable along (by ref). Obviously, the second function would also
have to have it's inputs marked scope, to promise that it never escapes
from there.
I imagine scope similarly to const, once it goes scope, the whole callstack
must maintain the scope property, otherwise the outermost function can't
trust it anymore.

This makes perfect sense for any function that is likely to receive
immediate or local variables by reference (which is extremely common).
It also seems absolutely relevant to the rvalues -> ref thing.

If you passed in a
> pointer, and scope were fully working, then you'd be protected against the
> pointer escaping, but passing by ref isn't really the same thing. I'd have
> thought that taking the address of a variable passed by ref would fall into
> pretty much the same camp as taking the address of any other local
> variable,
> which is completely unsafe to escape to the point that I'm not sure that
> there's any point in protecting against it. It's just completely stupid to
> do
> anyway and is definitely @system. Outside of taking the address of a ref
> parameter, taking the address of a local variable and escpaing it is
> _always_
> going to result in garbage, and ref parameters aren't really references in
> the
> normal sense, so I don't know.
>

What do you mean 'aren't really references in the normal sense'?


You bring up a good point, but I don't know if it's applicable. Certainly,
> without the ref there (like is the case with the Vector that you're passing
> in), scope would never do anything, because it doesn't even theoretically
> have
> anything to do. It's purely a value type that's not even being passed by
> ref.
>

Correct, scope on a purely value type passed by-value means absolutely
nothing.
scope on a pointer parameter means something; I would expect the pointer
its self, nor a pointer INTO anything under the pointer could escape.
scope on a by-value parameter that contains pointers (like
delegates/slices) has meaning, I presume scope would be transitive like
const, apply the pointer rule above.
ref is sugar for a pointer, and the above applies verbatim. If you take the
address of the ref argument, you have the pointer, and it mustn't escape,
likewise, no pointers or pointer to anything beneath it.

If my imagination of this concept completely wrong?
This sounds useful to me, I can't imagine another scenario where the
keyword a) makes sense, and b) is useful...

In general though, putting scope on struct parameters would cause a lot of
> problems, because of arrays that they might hold and whatnot. Slices
> wouldn't
> be able to escape (and so copies of the struct wouldn't be able escape
> without
> deep copying, let alone the array itself).


That would be the point though. If you don't want that, then you don't want
scope.

That said, I would suggest that it may be considered, for practicality,
that immutable members WERE allowed to escape from scope controlled
structs...
I think this makes sense, and is certainly practical. Strings are the most
likely thing to fall under the scenario you illustrated above. Deep-copying
strings (or any similarly immutable data) would be a bit silly.

So, while scope may be very useful
> in some such cases (assuming that it worked), it's not necessarily
> something
> that you'd want as a matter of course.


Of course not.

Part of it probably depends on your
> programming style though. If you have a lot of functions that take
> arguments
> and don't return anything that was in them ever, then scope is less of a
> big
> deal, but that's the sort of thing that happens a _lot_ in my experience,
> so
> scope would very quickly become extremely annoying.
>
> And actually, to make matters worse, I'm not sure that scope on delegates
> is
> working correctly. I thought that it was, but this code compiles:
>
> import std.stdio;
>
> void delegate() global;
>
> void foo(scope void delegate() del)
> {
>     global = del;
> }
>
> void main()
> {
>     {
>         char[5] bar = "hello";
>         foo((){writeln(bar);});
>     }
>     char[7] baz = "goodbye";
>
>     global();
> }
>
> It also prints out "hello", and if a closure had not been allocated, I
> would
> have at least half-expected it to print out "goodb", because I'd have
> thought
> that baz would have been taking up the same memory that bar had been. So,
> it
> looks like scope may be completely and utterly broken at this point. I
> don't
> know.
>
> - Jonathan M Davis
>
November 11, 2012
Re: Const ref and rvalues again...
On Sunday, November 11, 2012 12:10:59 David Nadlinger wrote:
> On Sunday, 11 November 2012 at 10:09:17 UTC, Jonathan M Davis
> 
> wrote:
> > It still prints "hello", even with full optimations turned on.
> > So, it must be
> > allocating a closure in spite of scope. So, it looks to me like
> > scope is just
> > completely ignored and does absolutely nothing at this point,
> > unless I'm just
> > completely missing something here.
> 
> Try this:
> 
> ---
> import std.stdio;
> 
> void delegate() global;
> 
> void foo(scope void delegate() del)
> {
>       global = del;
> }
> 
> 
> void f()
> {
>       {
>           char[5] bar = "hello";
>           foo((){writeln(bar);});
>       }
> }
> 
> void smashStack() {
>      uint[1000] dummy = 0xbadcab1e;
>      asm { nop; }
> }
> 
> void main()
> {
>       char[7] baz = "goodbye";
>       f();
>       smashStack();
>       global();
> }
> ---

That did it. And if scope is removed, it works again. I clearly don't get the 
low level stuff though, since it's just plain bizarre to me that the previous 
example didn't do it.

- Jonathan M Davis
November 11, 2012
Re: Const ref and rvalues again...
On Sunday, November 11, 2012 15:17:03 Dmitry Olshansky wrote:
> Something must be screwed up. I dunno what, I use near-latest DMD from
> github and Win32 binaries.

I'm on 64-bit Linux, so that may change things a bit.

- Jonathan M Davis
November 11, 2012
Re: Const ref and rvalues again...
On Sunday, November 11, 2012 13:30:12 Manu wrote:
> What do you mean 'aren't really references in the normal sense'?

ref is not really part of a variable's type. It's just a storage class and is 
really only applicable in very specific circumstances. A function parameter is 
a local variable, but ref makes it so that it just so happens to affect a 
variable outside the function as well. The semantics of the function itself or 
how the parameter is used aren't any different either way. So, if you were to 
remove ref from the parameter, the function itself would be unaffected 
semantically-speaking (it would affect the code generation some though). It's 
the caller whose semantics change.

This is in direct contrast with a pointer or reference type where the fact 
that it refers to something else outside the function is fully part of the 
type.

Taking the address of a local variable is already something that is incredibly 
unsafe and doesn't work. So, anyone doing it is being an idiot anyway. And if 
it's _always_ stupid, then protecting it with scope isn't really necessary. 
pure mostly stops anything like that from happening in the general case, but 
doesn't prevent people from doing stupid stuff like

int* foo() pure
{
   int i;
   return bar(&i);
}

int* bar(int* p) pure
{
   return p;
}

Regardless, my point was that the refness of the parameter isn't really part 
of the type of the parameter, whereas all other issues with reference escaping 
that scope would affect _are_ part of the type. So, you're talking about scope 
protecting against something quite different from what it would protect from in 
all other circumstances. Protecting against escaping a ref parameter and a 
reference type escaping are two different (albeit not completely unrelated) 
things.

Honestly, before you brought up the possibility of scope protecting against 
pointers to ref parameters escaping, it had never occurred to me. I've never 
seen anyone bring it up before. And as you're talking about protecting against 
escaping a pointer to a local variable, which is an incredibly stupid thing to 
do anyway, I'm not sure that the protection is really needed.

But I don't know what Walter's or Andrei's intentions were with regards to 
whether scope would extend to the fact that the parameter is ref. I don't 
think that it's necessarily the case that it wouldn't, but you're then 
protecting against escaping a pointer to a local variable (albeit one which 
affects a variable outside of the function) rather than against escaping a 
reference type. And from everything I've seen, it's protecting against 
escaping reference types which was its purpose.

Unfortunately, TDPL doesn't seem to discuss scope parameters at all (I thought 
that it did, but I can't find it anywhere now if it does), meaning that the 
online docs are the only official documentation on scope, and all other 
information on it comes from newsgroup discussions on the matter. It's quite 
possible that many of us have misunderstood aspects of what scope parameters 
were intended to do.

So, assuming that if/when scope actually starts affecting more than delegates 
as quite a few us think that it's supposed to do, then what you're looking for 
may very well be on the table, much as I wouldn't have expected it to be. I 
don't know.

Feel free to create an enhancement request for it. Even it wasn't part of the 
original intention of scope parameters, Walter may think that it's worth 
making it so that it is.

> That would be the point though. If you don't want that, then you don't want
> scope.

The problem is that in most cases, I really don't think that that is what the 
average programmer wants, and there _are_ people who use in as a matter 
course. It would be one thing if you had to explicitly use scope, then I 
suspect that it wouldn't be used as a matter of course by much of anyone. For 
the most part, it would then just be used when the programmer knew that that's 
what they wanted. But with in, so many people seem to really like the concept 
of it being the opposite of out that they use it as a matter of course without 
understanding the consequences.

- Jonathan M Davis
November 12, 2012
Re: Const ref and rvalues again...
What about making this a default behavior and introducing a new keyword 
if the function wants to modify the argument but it is not ref (pass by 
value) ? The reason I think that this should be a default behavior 
because not many functions actually modify their arguments and so it 
leaves a lot of space for optimization.

For example:

void f (int x, val int y, ref int z) {
  x = 1; // x is not copied
         // compiler throws an error, x is not passed by value
         // and therefor could not / should not be changed
  y = 2; // ok, y is copied
  z = 3; // ok, z is a reference
}

On 18.10.2012 5:07, Malte Skarupke wrote:
> Hello,
>
> I realize that this has been discussed before, but so far there is no
> solution and this really needs to be a high priority:
>
> We need a way for a function to declare that it doesn't want it's
> argument to be copied, but it also doesn't care whether the argument is
> an rvalue or an lvalue.
>
> The C++ way of doing this would be to declare the argument as a const &.
> Apparently it is not desired that we do the same thing for const ref.
>
> Currently, if you want that behavior, you have to write 2^n permutations
> of your function, with n being the number of arguments that the function
> takes.
>
> Here's my attempt at passing a struct to a function that takes three
> arguments without the struct being copied:
>
> int copyCounter = 0;
> struct CopyCounter
> {
> this(this) { ++copyCounter; }
> }
> void takeThree(ref in CopyCounter a, ref in CopyCounter b, ref in
> CopyCounter c)
> {
> writeln("took three");
> }
> void takeThree(in CopyCounter a, ref in CopyCounter b, ref in
> CopyCounter c)
> {
> takeThree(a, b, c);
> }
> void takeThree(ref in CopyCounter a, in CopyCounter b, ref in
> CopyCounter c)
> {
> takeThree(a, b, c);
> }
> void takeThree(ref in CopyCounter a, ref in CopyCounter b, in
> CopyCounter c)
> {
> takeThree(a, b, c);
> }
> void takeThree(in CopyCounter a, in CopyCounter b, ref in CopyCounter c)
> {
> takeThree(a, b, c);
> }
> void takeThree(in CopyCounter a, ref in CopyCounter b, in CopyCounter c)
> {
> takeThree(a, b, c);
> }
> void takeThree(ref in CopyCounter a, in CopyCounter b, in CopyCounter c)
> {
> takeThree(a, b, c);
> }
> void takeThree(in CopyCounter a, in CopyCounter b, in CopyCounter c)
> {
> takeThree(a, b, c);
> }
> static CopyCounter createCopyCounter()
> {
> return CopyCounter();
> }
> void main()
> {
> CopyCounter first;
> CopyCounter second;
> CopyCounter third;
> takeThree(first, second, third);
> takeThree(createCopyCounter(), second, createCopCounter());
> assert(copyCounter == 0); // yay, works
> }
>
>
> My propsed solution is this:
> - Make functions that take "ref in" arguments also accept rvalues.
> - The user can still provide an overload that accepts an rvalue, using
> the "in" keyword, and that one will be preferred over the "ref in" version.
>
>
> What do you think?
>
> Malte
November 13, 2012
Re: Const ref and rvalues again...
On Monday, 12 November 2012 at 23:38:43 UTC, luka8088 wrote:
> What about making this a default behavior and introducing a new 
> keyword if the function wants to modify the argument but it is 
> not ref (pass by value) ? The reason I think that this should 
> be a default behavior because not many functions actually 
> modify their arguments and so it leaves a lot of space for 
> optimization.
>
> For example:
>
> void f (int x, val int y, ref int z) {
>   x = 1; // x is not copied
>          // compiler throws an error, x is not passed by value
>          // and therefor could not / should not be changed
>   y = 2; // ok, y is copied
>   z = 3; // ok, z is a reference
> }

Your proposal isn't really related to this thread's topic, but I 
understand what you mean (although your code comments distract me 
a bit):

void f(const int x, int y, ref int z); =>
void f(int x, val/mutable int y, ref int z);

I use const/in ;) parameters a lot in my code too to prevent 
accidental modifications, so my function signatures may be more 
compact by treating normal pass-by-value parameters as const if 
not denoted with a special keyword. I guess it wouldn't be very 
important for optimization though because I'd expect the 
optimizer to detect unchanged parameters. Anyway, your proposal 
would completely break existing code.
6 7 8 9 10 11
Top | Discussion index | About this forum | D home