October 25, 2008
Jarrett Billingsley wrote:
> On Sat, Oct 25, 2008 at 9:30 AM, Yigal Chripun <yigal100@gmail.com> wrote:
>> anyway, I think you got my intention by now.
>> I really don't want to have 3 kinds of function types. I think the
>> already existing separation between function pointers and delegates
>> could be handled better by adding an implicit cast.
> 
> It would start to become untenable, but I would imagine closures would be implicitly convertible to scope delegates, since in effect they are a subtype, and anywhere a scope delegate could be used, a closure could be used as well.  Add to that implicit conversion from functions to delegates using thunks and bam, you could have a function take one type and it could accept all three.

OK, But my question is why do you need that separation in the first
place? to me it seems an unnecessary distinction between scope delegates
and "regular" delegates.
regarding functions and delegates - I must be missing something but why
do you need a thunk? just allocate a delegate, assign its function
pointer to the function and leave the "context" pointer null. isn't that
enough?
October 25, 2008
On Sat, Oct 25, 2008 at 5:08 PM, Yigal Chripun <yigal100@gmail.com> wrote:
> Jarrett Billingsley wrote:
>> On Sat, Oct 25, 2008 at 9:30 AM, Yigal Chripun <yigal100@gmail.com> wrote:
>>> anyway, I think you got my intention by now.
>>> I really don't want to have 3 kinds of function types. I think the
>>> already existing separation between function pointers and delegates
>>> could be handled better by adding an implicit cast.
>>
>> It would start to become untenable, but I would imagine closures would be implicitly convertible to scope delegates, since in effect they are a subtype, and anywhere a scope delegate could be used, a closure could be used as well.  Add to that implicit conversion from functions to delegates using thunks and bam, you could have a function take one type and it could accept all three.
>
> OK, But my question is why do you need that separation in the first place? to me it seems an unnecessary distinction between scope delegates and "regular" delegates.

Did you see the thread on performance?

D1's nested functions are great because they're very efficient.  D2's "allocate everything on the heap just in case" is, in most cases, completely unnecessary, and performancewise it's terrible too.  Yes, with a perfect compiler, it would be able to statically ensure that a delegate doesn't need to be allocated on the heap; but since we have separate compilation and use ancient object formats and linkers that don't understand such analysis, it can't be done.

> regarding functions and delegates - I must be missing something but why do you need a thunk? just allocate a delegate, assign its function pointer to the function and leave the "context" pointer null. isn't that enough?

No.  Functions and delegates have similar, but slightly different calling conventions.  Translation from one to the other requires a bit of register/stack shuffling.  Implicit conversion from function to delegate only requires a single statically allocated thunk for each set of parameter types; the delegate would have the thunk as the funcptr and the original function as the ptr.  The thunk would then be able to shift the params around and call the real function.
October 26, 2008
Frank Benoit wrote:
> It is great to hear that this issue is getting solved.
> How will be the now syntax?
> 
> I wonder if the distinction between dynamic/static closure shall be done
> on the calling site, or the called site.
> 
> void foo( void delegate() dg ){
> }
> // -or-
> void foo2( void delegate() dg ){
> }
> 
> void bar(){
>   int i;
>   foo({
>     i++;
>   });
>   // -or-
>   foo( scope {
>     i++;
>   });
> }
> 
> Because I think, the foo method/function signature has to define if the
> delegate is escaping or not. The caller might not know it.
> 
> If the signature defines this, the compiler can check that and give more
> safety.

What if heap delegates were implicitly castable to stack delegates? That way, things that needed heap delegates could demand them, and things that didn't care would be OK either way?
October 26, 2008
"Frank Benoit" wrote
> It is great to hear that this issue is getting solved.
> How will be the now syntax?
>
> I wonder if the distinction between dynamic/static closure shall be done on the calling site, or the called site.
>
> void foo( void delegate() dg ){
> }
> // -or-
> void foo2( void delegate() dg ){
> }
>
> void bar(){
>  int i;
>  foo({
>    i++;
>  });
>  // -or-
>  foo( scope {
>    i++;
>  });
> }
>
> Because I think, the foo method/function signature has to define if the delegate is escaping or not. The caller might not know it.
>
> If the signature defines this, the compiler can check that and give more safety.

I think possibly, there cannot be enforcement of this.

Here is a case, where the constructor needs to be able to take both types (scope and heap), and the compiler will not be able to which to use without semantic analysis.

class DelegateCaller
{
   private delegate int _foo();
   this(int delegate() foo) { _foo = foo; }
   int callit() { return _foo();}
}

int f1()
{
    int x() { return 5; }
    auto dc = new DelegateCaller(&x); // should allocate on stack
    return dc.callit() * dc.callit();
}

DelegateCaller f2()
{
   int x() { return 5;}
   auto dc =  new DelegateCaller(&x); // allocate on heap
   return dc;
}

So the author of DelegateCaller cannot require a heap delegate, even though it's possible the act of creating a DelegateCaller can cause an escape.

Note that the first two lines of each function are identical, so the compiler needs to do a semantic analysis of whether the use of a delegate requires a heap allocation.  It would be even more sticky with multiple function calls, or functions that call eachother.

The only thing I can think of is to leave it up to the caller to decide whether he thinks the delegate should be heap-allocated, and the author of the function should document how it will use the delegate.

Fortunately, this is not a huge deal, as most of the time a developer expects a stack-allocated delegate (scope).  So make scope the default and allow some syntax to let the author define that a function call warrants allocating the frame on the heap.

-Steve


October 27, 2008
Jarrett Billingsley wrote:
> On Sat, Oct 25, 2008 at 5:08 PM, Yigal Chripun <yigal100@gmail.com> wrote:
>> Jarrett Billingsley wrote:
>>> On Sat, Oct 25, 2008 at 9:30 AM, Yigal Chripun <yigal100@gmail.com> wrote:
>>>> anyway, I think you got my intention by now.
>>>> I really don't want to have 3 kinds of function types. I think the
>>>> already existing separation between function pointers and delegates
>>>> could be handled better by adding an implicit cast.
>>> It would start to become untenable, but I would imagine closures would be implicitly convertible to scope delegates, since in effect they are a subtype, and anywhere a scope delegate could be used, a closure could be used as well.  Add to that implicit conversion from functions to delegates using thunks and bam, you could have a function take one type and it could accept all three.
>> OK, But my question is why do you need that separation in the first place? to me it seems an unnecessary distinction between scope delegates and "regular" delegates.
> 
> Did you see the thread on performance?
> 

I've read that thread and understand the benefits of D1 style delegates,
performance wise. What I meant to say was Why do we need two separate
*types* for that?
both styles can be used with one type and with a scope modifier as I
replied to Kenny.

> D1's nested functions are great because they're very efficient.  D2's "allocate everything on the heap just in case" is, in most cases, completely unnecessary, and performancewise it's terrible too.  Yes, with a perfect compiler, it would be able to statically ensure that a delegate doesn't need to be allocated on the heap; but since we have separate compilation and use ancient object formats and linkers that don't understand such analysis, it can't be done.
> 
>> regarding functions and delegates - I must be missing something but why do you need a thunk? just allocate a delegate, assign its function pointer to the function and leave the "context" pointer null. isn't that enough?
> 
> No.  Functions and delegates have similar, but slightly different calling conventions.  Translation from one to the other requires a bit of register/stack shuffling.  Implicit conversion from function to delegate only requires a single statically allocated thunk for each set of parameter types; the delegate would have the thunk as the funcptr and the original function as the ptr.  The thunk would then be able to shift the params around and call the real function.

I see. Thanks for explaining this.

October 27, 2008
Yigal Chripun wrote:
> Jarrett Billingsley wrote:
>> On Sat, Oct 25, 2008 at 5:08 PM, Yigal Chripun <yigal100@gmail.com> wrote:
>>> Jarrett Billingsley wrote:
>>>> On Sat, Oct 25, 2008 at 9:30 AM, Yigal Chripun <yigal100@gmail.com> wrote:
>>>>> anyway, I think you got my intention by now.
>>>>> I really don't want to have 3 kinds of function types. I think the
>>>>> already existing separation between function pointers and delegates
>>>>> could be handled better by adding an implicit cast.
>>>> It would start to become untenable, but I would imagine closures would
>>>> be implicitly convertible to scope delegates, since in effect they are
>>>> a subtype, and anywhere a scope delegate could be used, a closure
>>>> could be used as well.  Add to that implicit conversion from functions
>>>> to delegates using thunks and bam, you could have a function take one
>>>> type and it could accept all three.
>>> OK, But my question is why do you need that separation in the first
>>> place? to me it seems an unnecessary distinction between scope delegates
>>> and "regular" delegates.
>> Did you see the thread on performance?
>>
> 
> I've read that thread and understand the benefits of D1 style delegates,
> performance wise. What I meant to say was Why do we need two separate
> *types* for that?
> both styles can be used with one type and with a scope modifier as I
> replied to Kenny.
> 
>> -- snip --

The scope solution is very good since whether the frame is allocated on heap or on stack is determined only at the point of declaring the DG. After the declaration a closure is no different from a delegate -- both has the same structure {ptr, funcptr} and even the calling convention is the same. The two are indistinguishable after declaration.

But you'll need two types if you want to distinguish them. Such as to enforce a non-scope constraint:

  // f1's ptr can be on stack or on heap.
  void T1 (scope delegate f1) { ... }

  // f2's ptr must be on heap.
  void T2 (delegate f2) { ... }

  ...
  {
    ...
    scope f4 = delegate { ... }; // frame allocated on stack.
    ...
    {
       ...
       T2(f4);
       //  T2: Hi, f4, is your .ptr on stack or on heap?
       //      For safety, only (.ptr)s on heap are allowed.
       //  f4: I... don't know. Just let me in? I'll behave.
       ...

    }
  }
  ...

  // later on an access violation/stack overflow appears mysteriously.

Solutions I can think of :-
 (a) Use some algorithm to check if the .ptr is on heap or on stack (indeed the address for them are significantly different);
 (b) Add a field to indicate if f4 is allocated on stack or on heap;
 (c) The “scope” keyword for delegates do nothing, let the programmer take the risk (current behavior);
 (d) Automatically .dup the frame when a delegate is passed into an argument without “scope”.

Problems for each solution :-
 (a) is probably not reliable (is it? can the GC help?), and DG & CL now behaves like 2 different types.
 (b) is effectively adding a new type.
 (c) is probably unsafe.
 (d) is slow.

The move I'd take for now would be (c) if no new types are wanted, because it's the current behavior and if you want significant performance out of nowhere there should be some risk trade off. The most urgent issue it seems right now is to restore the performance of delegates to D1 level.

If (a) can be done reliably and quickly than I vote for (a).

There could be other solutions as well I don't know yet.
1 2
Next ›   Last »