View mode: basic / threaded / horizontal-split · Log in · Help
October 25, 2008
Re: Dynamic closure vs static closure
Jarrett Billingsley wrote:
> On Sat, Oct 25, 2008 at 9:30 AM, Yigal Chripun <yigal100@gmail.com> wrote:
>> anyway, I think you got my intention by now.
>> I really don't want to have 3 kinds of function types. I think the
>> already existing separation between function pointers and delegates
>> could be handled better by adding an implicit cast.
> 
> It would start to become untenable, but I would imagine closures would
> be implicitly convertible to scope delegates, since in effect they are
> a subtype, and anywhere a scope delegate could be used, a closure
> could be used as well.  Add to that implicit conversion from functions
> to delegates using thunks and bam, you could have a function take one
> type and it could accept all three.

OK, But my question is why do you need that separation in the first
place? to me it seems an unnecessary distinction between scope delegates
and "regular" delegates.
regarding functions and delegates - I must be missing something but why
do you need a thunk? just allocate a delegate, assign its function
pointer to the function and leave the "context" pointer null. isn't that
enough?
October 25, 2008
Re: Dynamic closure vs static closure
On Sat, Oct 25, 2008 at 5:08 PM, Yigal Chripun <yigal100@gmail.com> wrote:
> Jarrett Billingsley wrote:
>> On Sat, Oct 25, 2008 at 9:30 AM, Yigal Chripun <yigal100@gmail.com> wrote:
>>> anyway, I think you got my intention by now.
>>> I really don't want to have 3 kinds of function types. I think the
>>> already existing separation between function pointers and delegates
>>> could be handled better by adding an implicit cast.
>>
>> It would start to become untenable, but I would imagine closures would
>> be implicitly convertible to scope delegates, since in effect they are
>> a subtype, and anywhere a scope delegate could be used, a closure
>> could be used as well.  Add to that implicit conversion from functions
>> to delegates using thunks and bam, you could have a function take one
>> type and it could accept all three.
>
> OK, But my question is why do you need that separation in the first
> place? to me it seems an unnecessary distinction between scope delegates
> and "regular" delegates.

Did you see the thread on performance?

D1's nested functions are great because they're very efficient.  D2's
"allocate everything on the heap just in case" is, in most cases,
completely unnecessary, and performancewise it's terrible too.  Yes,
with a perfect compiler, it would be able to statically ensure that a
delegate doesn't need to be allocated on the heap; but since we have
separate compilation and use ancient object formats and linkers that
don't understand such analysis, it can't be done.

> regarding functions and delegates - I must be missing something but why
> do you need a thunk? just allocate a delegate, assign its function
> pointer to the function and leave the "context" pointer null. isn't that
> enough?

No.  Functions and delegates have similar, but slightly different
calling conventions.  Translation from one to the other requires a bit
of register/stack shuffling.  Implicit conversion from function to
delegate only requires a single statically allocated thunk for each
set of parameter types; the delegate would have the thunk as the
funcptr and the original function as the ptr.  The thunk would then be
able to shift the params around and call the real function.
October 26, 2008
Re: Dynamic closure vs static closure
Frank Benoit wrote:
> It is great to hear that this issue is getting solved.
> How will be the now syntax?
> 
> I wonder if the distinction between dynamic/static closure shall be done
> on the calling site, or the called site.
> 
> void foo( void delegate() dg ){
> }
> // -or-
> void foo2( void delegate() dg ){
> }
> 
> void bar(){
>   int i;
>   foo({
>     i++;
>   });
>   // -or-
>   foo( scope {
>     i++;
>   });
> }
> 
> Because I think, the foo method/function signature has to define if the
> delegate is escaping or not. The caller might not know it.
> 
> If the signature defines this, the compiler can check that and give more
> safety.

What if heap delegates were implicitly castable to stack delegates? That 
way, things that needed heap delegates could demand them, and things 
that didn't care would be OK either way?
October 26, 2008
Re: Dynamic closure vs static closure
"Frank Benoit" wrote
> It is great to hear that this issue is getting solved.
> How will be the now syntax?
>
> I wonder if the distinction between dynamic/static closure shall be done
> on the calling site, or the called site.
>
> void foo( void delegate() dg ){
> }
> // -or-
> void foo2( void delegate() dg ){
> }
>
> void bar(){
>  int i;
>  foo({
>    i++;
>  });
>  // -or-
>  foo( scope {
>    i++;
>  });
> }
>
> Because I think, the foo method/function signature has to define if the
> delegate is escaping or not. The caller might not know it.
>
> If the signature defines this, the compiler can check that and give more
> safety.

I think possibly, there cannot be enforcement of this.

Here is a case, where the constructor needs to be able to take both types 
(scope and heap), and the compiler will not be able to which to use without 
semantic analysis.

class DelegateCaller
{
  private delegate int _foo();
  this(int delegate() foo) { _foo = foo; }
  int callit() { return _foo();}
}

int f1()
{
   int x() { return 5; }
   auto dc = new DelegateCaller(&x); // should allocate on stack
   return dc.callit() * dc.callit();
}

DelegateCaller f2()
{
  int x() { return 5;}
  auto dc =  new DelegateCaller(&x); // allocate on heap
  return dc;
}

So the author of DelegateCaller cannot require a heap delegate, even though 
it's possible the act of creating a DelegateCaller can cause an escape.

Note that the first two lines of each function are identical, so the 
compiler needs to do a semantic analysis of whether the use of a delegate 
requires a heap allocation.  It would be even more sticky with multiple 
function calls, or functions that call eachother.

The only thing I can think of is to leave it up to the caller to decide 
whether he thinks the delegate should be heap-allocated, and the author of 
the function should document how it will use the delegate.

Fortunately, this is not a huge deal, as most of the time a developer 
expects a stack-allocated delegate (scope).  So make scope the default and 
allow some syntax to let the author define that a function call warrants 
allocating the frame on the heap.

-Steve
October 27, 2008
Re: Dynamic closure vs static closure
Jarrett Billingsley wrote:
> On Sat, Oct 25, 2008 at 5:08 PM, Yigal Chripun <yigal100@gmail.com> wrote:
>> Jarrett Billingsley wrote:
>>> On Sat, Oct 25, 2008 at 9:30 AM, Yigal Chripun <yigal100@gmail.com> wrote:
>>>> anyway, I think you got my intention by now.
>>>> I really don't want to have 3 kinds of function types. I think the
>>>> already existing separation between function pointers and delegates
>>>> could be handled better by adding an implicit cast.
>>> It would start to become untenable, but I would imagine closures would
>>> be implicitly convertible to scope delegates, since in effect they are
>>> a subtype, and anywhere a scope delegate could be used, a closure
>>> could be used as well.  Add to that implicit conversion from functions
>>> to delegates using thunks and bam, you could have a function take one
>>> type and it could accept all three.
>> OK, But my question is why do you need that separation in the first
>> place? to me it seems an unnecessary distinction between scope delegates
>> and "regular" delegates.
> 
> Did you see the thread on performance?
> 

I've read that thread and understand the benefits of D1 style delegates,
performance wise. What I meant to say was Why do we need two separate
*types* for that?
both styles can be used with one type and with a scope modifier as I
replied to Kenny.

> D1's nested functions are great because they're very efficient.  D2's
> "allocate everything on the heap just in case" is, in most cases,
> completely unnecessary, and performancewise it's terrible too.  Yes,
> with a perfect compiler, it would be able to statically ensure that a
> delegate doesn't need to be allocated on the heap; but since we have
> separate compilation and use ancient object formats and linkers that
> don't understand such analysis, it can't be done.
> 
>> regarding functions and delegates - I must be missing something but why
>> do you need a thunk? just allocate a delegate, assign its function
>> pointer to the function and leave the "context" pointer null. isn't that
>> enough?
> 
> No.  Functions and delegates have similar, but slightly different
> calling conventions.  Translation from one to the other requires a bit
> of register/stack shuffling.  Implicit conversion from function to
> delegate only requires a single statically allocated thunk for each
> set of parameter types; the delegate would have the thunk as the
> funcptr and the original function as the ptr.  The thunk would then be
> able to shift the params around and call the real function.

I see. Thanks for explaining this.
October 27, 2008
Re: Dynamic closure vs static closure
Yigal Chripun wrote:
> Jarrett Billingsley wrote:
>> On Sat, Oct 25, 2008 at 5:08 PM, Yigal Chripun <yigal100@gmail.com> wrote:
>>> Jarrett Billingsley wrote:
>>>> On Sat, Oct 25, 2008 at 9:30 AM, Yigal Chripun <yigal100@gmail.com> wrote:
>>>>> anyway, I think you got my intention by now.
>>>>> I really don't want to have 3 kinds of function types. I think the
>>>>> already existing separation between function pointers and delegates
>>>>> could be handled better by adding an implicit cast.
>>>> It would start to become untenable, but I would imagine closures would
>>>> be implicitly convertible to scope delegates, since in effect they are
>>>> a subtype, and anywhere a scope delegate could be used, a closure
>>>> could be used as well.  Add to that implicit conversion from functions
>>>> to delegates using thunks and bam, you could have a function take one
>>>> type and it could accept all three.
>>> OK, But my question is why do you need that separation in the first
>>> place? to me it seems an unnecessary distinction between scope delegates
>>> and "regular" delegates.
>> Did you see the thread on performance?
>>
> 
> I've read that thread and understand the benefits of D1 style delegates,
> performance wise. What I meant to say was Why do we need two separate
> *types* for that?
> both styles can be used with one type and with a scope modifier as I
> replied to Kenny.
> 
>> -- snip --

The scope solution is very good since whether the frame is allocated on 
heap or on stack is determined only at the point of declaring the DG. 
After the declaration a closure is no different from a delegate -- both 
has the same structure {ptr, funcptr} and even the calling convention is 
the same. The two are indistinguishable after declaration.

But you'll need two types if you want to distinguish them. Such as to 
enforce a non-scope constraint:

  // f1's ptr can be on stack or on heap.
  void T1 (scope delegate f1) { ... }

  // f2's ptr must be on heap.
  void T2 (delegate f2) { ... }

  ...
  {
    ...
    scope f4 = delegate { ... }; // frame allocated on stack.
    ...
    {
       ...
       T2(f4);
       //  T2: Hi, f4, is your .ptr on stack or on heap?
       //      For safety, only (.ptr)s on heap are allowed.
       //  f4: I... don't know. Just let me in? I'll behave.
       ...

    }
  }
  ...

  // later on an access violation/stack overflow appears mysteriously.

Solutions I can think of :-
 (a) Use some algorithm to check if the .ptr is on heap or on stack 
(indeed the address for them are significantly different);
 (b) Add a field to indicate if f4 is allocated on stack or on heap;
 (c) The “scope” keyword for delegates do nothing, let the programmer 
take the risk (current behavior);
 (d) Automatically .dup the frame when a delegate is passed into an 
argument without “scope”.

Problems for each solution :-
 (a) is probably not reliable (is it? can the GC help?), and DG & CL 
now behaves like 2 different types.
 (b) is effectively adding a new type.
 (c) is probably unsafe.
 (d) is slow.

The move I'd take for now would be (c) if no new types are wanted, 
because it's the current behavior and if you want significant 
performance out of nowhere there should be some risk trade off. The most 
urgent issue it seems right now is to restore the performance of 
delegates to D1 level.

If (a) can be done reliably and quickly than I vote for (a).

There could be other solutions as well I don't know yet.
Next ›   Last »
1 2
Top | Discussion index | About this forum | D home