View mode: basic / threaded / horizontal-split · Log in · Help
July 21, 2004
Delegates
The following code did not produce results I expected:

int delegate() dg;

alias int delegate() dg_t ;

dg_t fn(int a )
{
  int inner() { int i = a; return i + 5; }
  dg_t result = &inner;
  return result;
}


void main()
{
  dg_t    delfunct, delfunct2;

  delfunct = fn( 100 );
  delfunct2 = fn( 55 );

  printf( "%i\n", delfunct() );
  printf( "%i\n", delfunct2() );
  printf( "%i\n", delfunct() );
  printf( "%i\n", delfunct2() );
}

After reading the spec I see this is normal behavior:

"The stack variables, however, are not valid once the function declaring 
them has exited, in the same manner that pointers to stack variables are 
not valid upon exit from a function.."

My question is why this limitation exists?
The only difference for a delegate and a normal function pointer is one 
uses the stack for its local variables and a delegate would use a local 
block of memory(which can change).  Maybe I am not seeing some other 
underlying technical issue.

Sorry if this topic has come up before( I am a little new to D ).
Thanks,
David
July 21, 2004
Re: Delegates
David Medlock wrote:
>
> "The stack variables, however, are not valid once the function declaring 
> them has exited, in the same manner that pointers to stack variables are 
> not valid upon exit from a function.."
> 
> My question is why this limitation exists?

This has been discussed a bit, but it's been a few months.  I think the 
primary reason was performance.  If 95% of the delgates used are not 
returned from functions in this manner then it's unnecessarily costly to 
always copy the stack frame.  Some more advanced lexing might be 
possible to determine when this is necessary but I think the response 
was "maybe but not right now."  One slightly lame fix that might work in 
a single-threaded environment is:

dg_t fn(int a )
{
  static int cpy = a;
  int inner() { int i = cpy; return i + 5; }
  dg_t result = &inner;
  return result;
}

I suppose if this works you could use thread local storage to do the 
same thing in a multithreaded program.


Sean
July 21, 2004
Re: Delegates
Sean Kelly wrote:
> David Medlock wrote:
> 
>>
>> "The stack variables, however, are not valid once the function 
>> declaring them has exited, in the same manner that pointers to stack 
>> variables are not valid upon exit from a function.."
>>
>> My question is why this limitation exists?
> 
> 
> This has been discussed a bit, but it's been a few months.  I think the 
> primary reason was performance.  If 95% of the delgates used are not 
> returned from functions in this manner then it's unnecessarily costly to 
> always copy the stack frame.  Some more advanced lexing might be 
> possible to determine when this is necessary but I think the response 
> was "maybe but not right now."  One slightly lame fix that might work in 
> a single-threaded environment is:
> 
> dg_t fn(int a )
> {
>   static int cpy = a;
>   int inner() { int i = cpy; return i + 5; }
>   dg_t result = &inner;
>   return result;
> }
> 
> I suppose if this works you could use thread local storage to do the 
> same thing in a multithreaded program.
> 
> 
> Sean

Hmmm.  The only performance hit would be at creation and not during use, 
so I would consider that a minor issue at best.  And since the delegate 
'carries' its local variables, its pretty thread safe isn't it?

Adding a 'const int i = cpy' would perhaps cause the compiler to inline 
the whole thing, but as you say more advanced checking could optimize 
even in the absence of const.

It just borders so close to true closures, just seems a shame they 
aren't there, especially since delegate is D-only and doesn't rely on C 
linkability(or does it?).  Adding this in makes D much more functional 
and may attract people from functional language communities, imo.
July 21, 2004
Re: Delegates
In article <cdm2ml$2qfb$1@digitaldaemon.com>, David Medlock says...
>
>Sean Kelly wrote:
>> David Medlock wrote:
>> 
>>>
>>> "The stack variables, however, are not valid once the function 
>>> declaring them has exited, in the same manner that pointers to stack 
>>> variables are not valid upon exit from a function.."
>>>
>>> My question is why this limitation exists?
>> 
>> 
>> This has been discussed a bit, but it's been a few months.  I think the 
>> primary reason was performance.  If 95% of the delgates used are not 
>> returned from functions in this manner then it's unnecessarily costly to 
>> always copy the stack frame.  Some more advanced lexing might be 
>> possible to determine when this is necessary but I think the response 
>> was "maybe but not right now."  One slightly lame fix that might work in 
>> a single-threaded environment is:
>> 
>> dg_t fn(int a )
>> {
>>   static int cpy = a;
>>   int inner() { int i = cpy; return i + 5; }
>>   dg_t result = &inner;
>>   return result;
>> }
>> 
>> I suppose if this works you could use thread local storage to do the 
>> same thing in a multithreaded program.
>> 
>> 
>> Sean
>
>Hmmm.  The only performance hit would be at creation and not during use, 
>so I would consider that a minor issue at best.  And since the delegate 
>'carries' its local variables, its pretty thread safe isn't it?

The thread-safety issue was only with my workaround, not with a language-based
stack copy.  Assume this bit of code:

void func() {
char buf1[65536*64];
void deleg() { ... }
for( ... ) { deleg(); }
}

The delegate is never used outside the function but the stack would have to be
copied every time func() is called (assuming no advanced lexing) to make sure
that deleg() always has correct data in its copy of buf.  And because buf is
pretty large there's a risk of out of memory conditions plus the cost of
allocating the memory and copying the data.  Things get a bit more complicated
when you consider class method functions, as a delegate should have access to
not only its own stack but to class member data as well, though I think the
already existing hidden this pointer may suffice in that case. 

>Adding a 'const int i = cpy' would perhaps cause the compiler to inline 
>the whole thing, but as you say more advanced checking could optimize 
>even in the absence of const.

True enough.  Though I'm not sure a const would work--do const values have
static linkage?

>It just borders so close to true closures, just seems a shame they 
>aren't there, especially since delegate is D-only and doesn't rely on C 
>linkability(or does it?).  Adding this in makes D much more functional 
>and may attract people from functional language communities, imo.

I agree.  I think if this were to be done the best thing would be to add a new
keyword that signals to the compiler that the stack must be copied.  Perhaps use
"auto" in the function declaration to signal that it is a truly local functions,
while the new default behavior is to copy the stack?


Sean
July 22, 2004
Re: Delegates
I am a big proponent of having some functionality that enables us to 
copy stack delegates.  We don't want copying to be the default, however. 
 Consider this code:


int foo() {
  int counter = 0;

  bar(void delegate(int val) { counter += arg; });

  return counter;
}
void bar(void delegate() dg) {
  foreach(whatever)
    dg(something);
}


You see, in this code, you want to give the delegate the capability to 
modify the real values on the stack.  You don't want it to be using a 
duplicate.

So what I've advocated is that we have the option to copy or not.

Sean Kelly wrote:
> David Medlock wrote:
> 
>>
>> "The stack variables, however, are not valid once the function 
>> declaring them has exited, in the same manner that pointers to stack 
>> variables are not valid upon exit from a function.."
>>
>> My question is why this limitation exists?
> 
> 
> This has been discussed a bit, but it's been a few months.  I think the 
> primary reason was performance.  If 95% of the delgates used are not 
> returned from functions in this manner then it's unnecessarily costly to 
> always copy the stack frame.  Some more advanced lexing might be 
> possible to determine when this is necessary but I think the response 
> was "maybe but not right now."  One slightly lame fix that might work in 
> a single-threaded environment is:
> 
> dg_t fn(int a )
> {
>   static int cpy = a;
>   int inner() { int i = cpy; return i + 5; }
>   dg_t result = &inner;
>   return result;
> }
> 
> I suppose if this works you could use thread local storage to do the 
> same thing in a multithreaded program.
> 
> 
> Sean
July 22, 2004
Re: Delegates
Russ Lewis wrote:

> I am a big proponent of having some functionality that enables us to 
> copy stack delegates.  We don't want copying to be the default, however. 
>  Consider this code:
> 
> 
> int foo() {
>   int counter = 0;
> 
>   bar(void delegate(int val) { counter += arg; });
> 
>   return counter;
> }
> void bar(void delegate() dg) {
>   foreach(whatever)
>     dg(something);
> }
> 
> 
> You see, in this code, you want to give the delegate the capability to 
> modify the real values on the stack.  You don't want it to be using a 
> duplicate.
> 
> So what I've advocated is that we have the option to copy or not.
> 
> Sean Kelly wrote:

I assume you meant to write 'counter += val'.

Actually that seems an error in my definition of a closure. In the
above situation the behavior I would expect from the compiler is all
accessible variables not passed as params are *in*, so counter would
have to be redeclared within the inner function.

Pseudo example of what I am thinking of(internals):
struct delegate { int size; void * data; void *funct; }

Assigning a function to a delegate would mean ensuring the size/data 
parameters are valid.

The only problem that remains would be giving a delegate function access 
to the *data* within the struct.

The compiler could:
Have code inserted before and after a delegate is called to handle 
copying the data to and from the stack(as mentioned).  Performance is 
extremely minimal because in most cases its just a few Push/Pops.

-or-

For each non-static function, have another version internally(AST) which 
is used when an assignment to a delegate happens.  The delegate version 
accesses local variables in its storage space instead of the stack(using 
the data*). Having a different version probably wouldnt be that bad, 
templates do this already.

I realize this would be a PITA, but this would allow many functional 
idioms to be used.

Sorry for the rambling, I hope this makes some sense.
July 22, 2004
Re: Delegates
David Medlock wrote:
> Russ Lewis wrote:
> 
>> I am a big proponent of having some functionality that enables us to 
>> copy stack delegates.  We don't want copying to be the default, 
>> however.  Consider this code:
>>
>>
>> int foo() {
>>   int counter = 0;
>>
>>   bar(void delegate(int val) { counter += arg; });
>>
>>   return counter;
>> }
>> void bar(void delegate() dg) {
>>   foreach(whatever)
>>     dg(something);
>> }
>>
> I assume you meant to write 'counter += val'.

Yes.  and I also noticed that the argument to bar() has the wrong 
argument type.  My bad. :(

> Actually that seems an error in my definition of a closure. In the
> above situation the behavior I would expect from the compiler is all
> accessible variables not passed as params are *in*, so counter would
> have to be redeclared within the inner function.

I see what you're thinking of, and it makes sense.  However, what I'm 
pointing out is that what you're suggesting is only appropriate in some 
designs.  In your design, the creator of the delegate cannot get any 
results out of the delegate (unless, of course, you pass pointers into 
the delegate).

What I was pointing out is that there are two different paradigms here. 
 One paradigm is supported by D currently; you are saying that it 
should be something else.

I would like the compiler to support both paradigms.

> Pseudo example of what I am thinking of(internals):
> struct delegate { int size; void * data; void *funct; }

Essentially, a stack delegate is syntax sugar for declaring a new struct 
type.  Here's old C++ code:

struct func_workingData { int counter; }
int func() {
  func_workingData data;
  data.counter = 0;
  bar(&func_inner_delegate, &data);
  return data.counter;
}
void func_inner_delegate(void *arg, int val) {
  func_workingData *this = (func_workingData*)arg;
  this.counter += val;
}
void bar(void (*callback)(void*,int), void *arg) {
  for(<whatever>)
    callback(arg, <something>);
}

So you see, when you create a stack delegate you are essentially passing 
a pointer to an anonymous struct on the stack.  When you duplicate, you 
are doing something else:

void func2() {
  func_workingData data;
  data.counter = 0;
  func_workingData *copy = malloc(sizeof(data));
  memcpy(copy, &data, sizeof(data));
  bar(&func_inner_delegate, copy);
}

Note that there's no need to push/pop the duplicated stack variables. 
You can simply access them directly from the heap.  The delegate code 
doesn't need to know whether the pointer points to the heap or the stack.
July 22, 2004
Re: Delegates
In article <cdn6a7$76i$1@digitaldaemon.com>, Russ Lewis says...
>
>I am a big proponent of having some functionality that enables us to 
>copy stack delegates.  We don't want copying to be the default, however. 

I agree.  The only reason I suggested that was because it would allow us to use
the existing "auto" keyword in a consistent manner.


Sean
July 22, 2004
Re: Delegates
Sean Kelly wrote:
> In article <cdn6a7$76i$1@digitaldaemon.com>, Russ Lewis says...
> 
>>I am a big proponent of having some functionality that enables us to 
>>copy stack delegates.  We don't want copying to be the default, however. 
> 
> 
> I agree.  The only reason I suggested that was because it would allow us to use
> the existing "auto" keyword in a consistent manner.

I apologize, but I'm clueless about how 'auto' comes in here.  Can you 
give an example?  Sounds interesting.
July 22, 2004
Re: Delegates
In article <cdp115$vo6$1@digitaldaemon.com>, Russ Lewis says...
>
>Sean Kelly wrote:
>> In article <cdn6a7$76i$1@digitaldaemon.com>, Russ Lewis says...
>> 
>>>I am a big proponent of having some functionality that enables us to 
>>>copy stack delegates.  We don't want copying to be the default, however. 
>> 
>> 
>> I agree.  The only reason I suggested that was because it would allow us to use
>> the existing "auto" keyword in a consistent manner.
>
>I apologize, but I'm clueless about how 'auto' comes in here.  Can you 
>give an example?  Sounds interesting.

It was in an earlier post in this thread.  Here's an example:

# void func()
# {
#    void d1() { ... }
#    auto void d2() { ... }
# }

In the above code, d1 will get a copy of the stack frame that it can carry
outside of func, while d2 will not since it is declared as "auto."  This seems
consistent with the existing auto semantics for variables.  Only catch is that,
as you said, I think the default behavior should remain how it is.


Sean
« First   ‹ Prev
1 2 3 4
Top | Discussion index | About this forum | D home