The following code did not produce results I expected:

int delegate() dg;

alias int delegate() dg_t ;

dg_t fn(int a )
{
  int inner() { int i = a; return i + 5; }
  dg_t result = &inner;
  return result;
}


void main()
{
  dg_t    delfunct, delfunct2;

  delfunct = fn( 100 );
  delfunct2 = fn( 55 );

  printf( "%i\n", delfunct() );
  printf( "%i\n", delfunct2() );
  printf( "%i\n", delfunct() );
  printf( "%i\n", delfunct2() );
}

After reading the spec I see this is normal behavior:

"The stack variables, however, are not valid once the function declaring them has exited, in the same manner that pointers to stack variables are not valid upon exit from a function.."

My question is why this limitation exists?
The only difference for a delegate and a normal function pointer is one uses the stack for its local variables and a delegate would use a local block of memory(which can change).  Maybe I am not seeing some other underlying technical issue.

Sorry if this topic has come up before( I am a little new to D ).
Thanks,
David

David Medlock wrote:
>
> "The stack variables, however, are not valid once the function declaring them has exited, in the same manner that pointers to stack variables are not valid upon exit from a function.."
> 
> My question is why this limitation exists?

This has been discussed a bit, but it's been a few months.  I think the primary reason was performance.  If 95% of the delgates used are not returned from functions in this manner then it's unnecessarily costly to always copy the stack frame.  Some more advanced lexing might be possible to determine when this is necessary but I think the response was "maybe but not right now."  One slightly lame fix that might work in a single-threaded environment is:

dg_t fn(int a )
{
  static int cpy = a;
  int inner() { int i = cpy; return i + 5; }
  dg_t result = &inner;
  return result;
}

I suppose if this works you could use thread local storage to do the same thing in a multithreaded program.


Sean

Sean Kelly wrote:
> David Medlock wrote:
> 
>>
>> "The stack variables, however, are not valid once the function declaring them has exited, in the same manner that pointers to stack variables are not valid upon exit from a function.."
>>
>> My question is why this limitation exists?
> 
> 
> This has been discussed a bit, but it's been a few months.  I think the primary reason was performance.  If 95% of the delgates used are not returned from functions in this manner then it's unnecessarily costly to always copy the stack frame.  Some more advanced lexing might be possible to determine when this is necessary but I think the response was "maybe but not right now."  One slightly lame fix that might work in a single-threaded environment is:
> 
> dg_t fn(int a )
> {
>   static int cpy = a;
>   int inner() { int i = cpy; return i + 5; }
>   dg_t result = &inner;
>   return result;
> }
> 
> I suppose if this works you could use thread local storage to do the same thing in a multithreaded program.
> 
> 
> Sean

Hmmm.  The only performance hit would be at creation and not during use, so I would consider that a minor issue at best.  And since the delegate 'carries' its local variables, its pretty thread safe isn't it?

Adding a 'const int i = cpy' would perhaps cause the compiler to inline the whole thing, but as you say more advanced checking could optimize even in the absence of const.

It just borders so close to true closures, just seems a shame they aren't there, especially since delegate is D-only and doesn't rely on C linkability(or does it?).  Adding this in makes D much more functional and may attract people from functional language communities, imo.

In article <cdm2ml$2qfb$1@digitaldaemon.com>, David Medlock says...
>
>Sean Kelly wrote:
>> David Medlock wrote:
>> 
>>>
>>> "The stack variables, however, are not valid once the function declaring them has exited, in the same manner that pointers to stack variables are not valid upon exit from a function.."
>>>
>>> My question is why this limitation exists?
>> 
>> 
>> This has been discussed a bit, but it's been a few months.  I think the primary reason was performance.  If 95% of the delgates used are not returned from functions in this manner then it's unnecessarily costly to always copy the stack frame.  Some more advanced lexing might be possible to determine when this is necessary but I think the response was "maybe but not right now."  One slightly lame fix that might work in a single-threaded environment is:
>> 
>> dg_t fn(int a )
>> {
>>   static int cpy = a;
>>   int inner() { int i = cpy; return i + 5; }
>>   dg_t result = &inner;
>>   return result;
>> }
>> 
>> I suppose if this works you could use thread local storage to do the same thing in a multithreaded program.
>> 
>> 
>> Sean
>
>Hmmm.  The only performance hit would be at creation and not during use, so I would consider that a minor issue at best.  And since the delegate 'carries' its local variables, its pretty thread safe isn't it?

The thread-safety issue was only with my workaround, not with a language-based stack copy.  Assume this bit of code:

void func() {
char buf1[65536*64];
void deleg() { ... }
for( ... ) { deleg(); }
}

The delegate is never used outside the function but the stack would have to be copied every time func() is called (assuming no advanced lexing) to make sure that deleg() always has correct data in its copy of buf.  And because buf is pretty large there's a risk of out of memory conditions plus the cost of allocating the memory and copying the data.  Things get a bit more complicated when you consider class method functions, as a delegate should have access to not only its own stack but to class member data as well, though I think the already existing hidden this pointer may suffice in that case.

>Adding a 'const int i = cpy' would perhaps cause the compiler to inline the whole thing, but as you say more advanced checking could optimize even in the absence of const.

True enough.  Though I'm not sure a const would work--do const values have static linkage?

>It just borders so close to true closures, just seems a shame they aren't there, especially since delegate is D-only and doesn't rely on C linkability(or does it?).  Adding this in makes D much more functional and may attract people from functional language communities, imo.

I agree.  I think if this were to be done the best thing would be to add a new keyword that signals to the compiler that the stack must be copied.  Perhaps use "auto" in the function declaration to signal that it is a truly local functions, while the new default behavior is to copy the stack?


Sean

I am a big proponent of having some functionality that enables us to copy stack delegates.  We don't want copying to be the default, however.  Consider this code:


int foo() {
  int counter = 0;

  bar(void delegate(int val) { counter += arg; });

  return counter;
}
void bar(void delegate() dg) {
  foreach(whatever)
    dg(something);
}


You see, in this code, you want to give the delegate the capability to modify the real values on the stack.  You don't want it to be using a duplicate.

So what I've advocated is that we have the option to copy or not.

Sean Kelly wrote:
> David Medlock wrote:
> 
>>
>> "The stack variables, however, are not valid once the function declaring them has exited, in the same manner that pointers to stack variables are not valid upon exit from a function.."
>>
>> My question is why this limitation exists?
> 
> 
> This has been discussed a bit, but it's been a few months.  I think the primary reason was performance.  If 95% of the delgates used are not returned from functions in this manner then it's unnecessarily costly to always copy the stack frame.  Some more advanced lexing might be possible to determine when this is necessary but I think the response was "maybe but not right now."  One slightly lame fix that might work in a single-threaded environment is:
> 
> dg_t fn(int a )
> {
>   static int cpy = a;
>   int inner() { int i = cpy; return i + 5; }
>   dg_t result = &inner;
>   return result;
> }
> 
> I suppose if this works you could use thread local storage to do the same thing in a multithreaded program.
> 
> 
> Sean

Russ Lewis wrote:

> I am a big proponent of having some functionality that enables us to copy stack delegates.  We don't want copying to be the default, however.  Consider this code:
> 
> 
> int foo() {
>   int counter = 0;
> 
>   bar(void delegate(int val) { counter += arg; });
> 
>   return counter;
> }
> void bar(void delegate() dg) {
>   foreach(whatever)
>     dg(something);
> }
> 
> 
> You see, in this code, you want to give the delegate the capability to modify the real values on the stack.  You don't want it to be using a duplicate.
> 
> So what I've advocated is that we have the option to copy or not.
> 
> Sean Kelly wrote:

I assume you meant to write 'counter += val'.

Actually that seems an error in my definition of a closure. In the
above situation the behavior I would expect from the compiler is all
accessible variables not passed as params are *in*, so counter would
have to be redeclared within the inner function.

Pseudo example of what I am thinking of(internals):
struct delegate { int size; void * data; void *funct; }

Assigning a function to a delegate would mean ensuring the size/data parameters are valid.

The only problem that remains would be giving a delegate function access to the *data* within the struct.

The compiler could:
Have code inserted before and after a delegate is called to handle copying the data to and from the stack(as mentioned).  Performance is extremely minimal because in most cases its just a few Push/Pops.

-or-

For each non-static function, have another version internally(AST) which is used when an assignment to a delegate happens.  The delegate version accesses local variables in its storage space instead of the stack(using the data*). Having a different version probably wouldnt be that bad, templates do this already.

I realize this would be a PITA, but this would allow many functional idioms to be used.

Sorry for the rambling, I hope this makes some sense.

David Medlock wrote:
> Russ Lewis wrote:
> 
>> I am a big proponent of having some functionality that enables us to copy stack delegates.  We don't want copying to be the default, however.  Consider this code:
>>
>>
>> int foo() {
>>   int counter = 0;
>>
>>   bar(void delegate(int val) { counter += arg; });
>>
>>   return counter;
>> }
>> void bar(void delegate() dg) {
>>   foreach(whatever)
>>     dg(something);
>> }
>>
> I assume you meant to write 'counter += val'.

Yes.  and I also noticed that the argument to bar() has the wrong argument type.  My bad. :(

> Actually that seems an error in my definition of a closure. In the
> above situation the behavior I would expect from the compiler is all
> accessible variables not passed as params are *in*, so counter would
> have to be redeclared within the inner function.

I see what you're thinking of, and it makes sense.  However, what I'm pointing out is that what you're suggesting is only appropriate in some designs.  In your design, the creator of the delegate cannot get any results out of the delegate (unless, of course, you pass pointers into the delegate).

What I was pointing out is that there are two different paradigms here.  One paradigm is supported by D currently; you are saying that it should be something else.

I would like the compiler to support both paradigms.

> Pseudo example of what I am thinking of(internals):
> struct delegate { int size; void * data; void *funct; }

Essentially, a stack delegate is syntax sugar for declaring a new struct type.  Here's old C++ code:

struct func_workingData { int counter; }
int func() {
  func_workingData data;
  data.counter = 0;
  bar(&func_inner_delegate, &data);
  return data.counter;
}
void func_inner_delegate(void *arg, int val) {
  func_workingData *this = (func_workingData*)arg;
  this.counter += val;
}
void bar(void (*callback)(void*,int), void *arg) {
  for(<whatever>)
    callback(arg, <something>);
}

So you see, when you create a stack delegate you are essentially passing a pointer to an anonymous struct on the stack.  When you duplicate, you are doing something else:

void func2() {
  func_workingData data;
  data.counter = 0;
  func_workingData *copy = malloc(sizeof(data));
  memcpy(copy, &data, sizeof(data));
  bar(&func_inner_delegate, copy);
}

Note that there's no need to push/pop the duplicated stack variables. You can simply access them directly from the heap.  The delegate code doesn't need to know whether the pointer points to the heap or the stack.

In article <cdn6a7$76i$1@digitaldaemon.com>, Russ Lewis says...
>
>I am a big proponent of having some functionality that enables us to copy stack delegates.  We don't want copying to be the default, however.

I agree.  The only reason I suggested that was because it would allow us to use the existing "auto" keyword in a consistent manner.


Sean

Sean Kelly wrote:
> In article <cdn6a7$76i$1@digitaldaemon.com>, Russ Lewis says...
> 
>>I am a big proponent of having some functionality that enables us to copy stack delegates.  We don't want copying to be the default, however. 
> 
> 
> I agree.  The only reason I suggested that was because it would allow us to use
> the existing "auto" keyword in a consistent manner.

I apologize, but I'm clueless about how 'auto' comes in here.  Can you give an example?  Sounds interesting.

In article <cdp115$vo6$1@digitaldaemon.com>, Russ Lewis says...
>
>Sean Kelly wrote:
>> In article <cdn6a7$76i$1@digitaldaemon.com>, Russ Lewis says...
>> 
>>>I am a big proponent of having some functionality that enables us to copy stack delegates.  We don't want copying to be the default, however.
>> 
>> 
>> I agree.  The only reason I suggested that was because it would allow us to use the existing "auto" keyword in a consistent manner.
>
>I apologize, but I'm clueless about how 'auto' comes in here.  Can you give an example?  Sounds interesting.

It was in an earlier post in this thread.  Here's an example:

# void func()
# {
#    void d1() { ... }
#    auto void d2() { ... }
# }

In the above code, d1 will get a copy of the stack frame that it can carry outside of func, while d2 will not since it is declared as "auto."  This seems consistent with the existing auto semantics for variables.  Only catch is that, as you said, I think the default behavior should remain how it is.


Sean