Thread overview
how to correctly populate an array of dynamic closures?
Mar 29, 2018
Ivan Kazmenko
Mar 29, 2018
ag0aep6g
Mar 29, 2018
Ivan Kazmenko
Mar 29, 2018
Dennis
Mar 29, 2018
kdevel
Mar 29, 2018
ag0aep6g
Mar 29, 2018
kdevel
Mar 29, 2018
ag0aep6g
March 29, 2018
Here's a simplified example of what I want to achieve.

I first create funs, an array of two delegates.
I want funs[0] to always return 0 and funs[1] to always return 1.

By assigning the constants directly (see the code below), I achieve exactly that.
Now, I want to use a loop to assign the values, and this is where things stop working.

My first try is guns, populated with `foreach (i; 0..2) guns ~= () => i;`.
But both guns[0] and guns[1] return 1.

I tried to circumvent that by populating another array, huns, with functions returning immutable copies of the loop variable, but the effect was the same.

import std.stdio;
void main () {
	int delegate () [] funs;
	funs ~= () => 0;
	funs ~= () => 1;
	foreach (i; 0..2) writeln (funs[i] ());  // 0 and 1 as expected

	int delegate () [] guns;
	foreach (i; 0..2) guns ~= () => i;
	foreach (i; 0..2) writeln (guns[i] ());  // 1 and 1, why?

	int delegate () [] huns;
	foreach (i; 0..2) {
		immutable int j = i;
		huns ~= () => j;
	}
	foreach (i; 0..2) writeln (huns[i] ());  // 1 and 1, why?
}

In my real use case, the delegates actually get stored in different structs or classes instead of a single array, and instead of returning 0 and 1, they call another function with argument 0 and 1, respectively.
Also, the number of delegates to create is known only at runtime.
However, I believe that won't be a problem once I grasp how to do this basic example.

So, why do delegates of guns[] and huns[] all return 1, and how to correctly reproduce the behavior of funs[] while populating it in a loop?

Ivan Kazmenko.

March 29, 2018
On 03/29/2018 05:16 PM, Ivan Kazmenko wrote:
>      int delegate () [] guns;
>      foreach (i; 0..2) guns ~= () => i;
>      foreach (i; 0..2) writeln (guns[i] ());  // 1 and 1, why?

Because there's only variable `i`. All delegates refer to that same one. With `i` being mutable, this could maybe be argued to be acceptable.

>      int delegate () [] huns;
>      foreach (i; 0..2) {
>          immutable int j = i;
>          huns ~= () => j;
>      }
>      foreach (i; 0..2) writeln (huns[i] ());  // 1 and 1, why?

Same here. There's only one `j`. With immutable, this is certainly a problem. https://issues.dlang.org/show_bug.cgi?id=2043

Two possible workarounds:

    int delegate () [] iuns;
    foreach (i; 0..2) iuns ~= (j) { return () => j; } (i);
    foreach (i; 0..2) writeln (iuns[i] ());  /* 0 and 1 */

    static struct S
    {
        int i;
        int m() { return i; }
    }
    int delegate () [] juns;
    foreach (i; 0..2) juns ~= &(new S(i)).m;
    foreach (i; 0..2) writeln (juns[i] ());  /* 0 and 1 */
March 29, 2018
On Thursday, 29 March 2018 at 15:16:07 UTC, Ivan Kazmenko wrote:
> So, why do delegates of guns[] and huns[] all return 1, and how to correctly reproduce the behavior of funs[] while populating it in a loop?

A delegate is a function with a pointer to the stack frame where it was created. It doesn't copy or insert the value of 'i', it still refers to the very same location in memory as the i from the for-loop. After the for-loop, that value is 1, so all delegates refering to that i return 1. The solution is to generate a new local variable for each closure with a helper function:

```
import std.stdio: writeln;

void main () {
    int delegate () [] funs;
    foreach(i; 0..2) {
        funs ~= constantDelegate(i);
    }

    writeln(funs[0]()); //prints 0
    writeln(funs[1]()); //prints 1
}

auto constantDelegate(int num) {
    return () => num;
}
```

Note that since the delegate leaves the scope of constantDelegate, the stack frame with the value for 'num' will be allocated to the heap because local variables normally don't persist after returning from a function.

March 29, 2018
On Thursday, 29 March 2018 at 15:38:14 UTC, ag0aep6g wrote:
> <...> With immutable, this is certainly a problem. https://issues.dlang.org/show_bug.cgi?id=2043

Wow, such history for the bug!

> Two possible workarounds:
>
>     int delegate () [] iuns;
>     foreach (i; 0..2) iuns ~= (j) { return () => j; } (i);
>     foreach (i; 0..2) writeln (iuns[i] ());  /* 0 and 1 */
>
>     static struct S
>     {
>         int i;
>         int m() { return i; }
>     }
>     int delegate () [] juns;
>     foreach (i; 0..2) juns ~= &(new S(i)).m;
>     foreach (i; 0..2) writeln (juns[i] ());  /* 0 and 1 */

Thank you ag0aep6g and Dennis for showing the possible workarounds!

On Thursday, 29 March 2018 at 15:47:33 UTC, Dennis wrote:
> A delegate is a function with a pointer to the stack frame where it was created. It doesn't copy or insert the value of 'i', it still refers to the very same location in memory as the i from the for-loop. After the for-loop, that value is 1, so all delegates refering to that i return 1. The solution is to generate a new local variable for each closure with a helper function:

So, basically, one has to create and call another function, or explicitly copy onto the heap, in order to ensure the copy of the loop variable is stored for the closure.
My (mis)understanding was that there's some additional magic happening for closures that get stored on the heap, as opposed to delegates used before their context goes out of scope.
As long as the whole story is that simple, fine.

Ivan Kazmenko.

March 29, 2018
On Thursday, 29 March 2018 at 15:16:07 UTC, Ivan Kazmenko wrote:
> import std.stdio;
> void main () {
> 	int delegate () [] funs;
> 	funs ~= () => 0;
> 	funs ~= () => 1;
> 	foreach (i; 0..2) writeln (funs[i] ());  // 0 and 1 as expected
>
> 	int delegate () [] guns;
> 	foreach (i; 0..2) guns ~= () => i;
> 	foreach (i; 0..2) writeln (guns[i] ());  // 1 and 1, why?

Isn't this undefined behavior? The first loop variable named "i" already went out of scope when the delegate is invoked.

March 29, 2018
On Thursday, 29 March 2018 at 19:02:51 UTC, kdevel wrote:
> On Thursday, 29 March 2018 at 15:16:07 UTC, Ivan Kazmenko wrote:
[...]
>> 	int delegate () [] guns;
>> 	foreach (i; 0..2) guns ~= () => i;
>> 	foreach (i; 0..2) writeln (guns[i] ());  // 1 and 1, why?
>
> Isn't this undefined behavior? The first loop variable named "i" already went out of scope when the delegate is invoked.

Not undefined behavior. At least, not for that reason.

The compiler sees that the delegate references `i`. So it puts `i` on the heap where it survives beyond the `foreach` scope. That's a closure.

https://dlang.org/spec/function.html#closures
March 29, 2018
On Thursday, 29 March 2018 at 20:05:35 UTC, ag0aep6g wrote:
> On Thursday, 29 March 2018 at 19:02:51 UTC, kdevel wrote:
>> On Thursday, 29 March 2018 at 15:16:07 UTC, Ivan Kazmenko wrote:
> [...]
>>> 	int delegate () [] guns;
>>> 	foreach (i; 0..2) guns ~= () => i;
>>> 	foreach (i; 0..2) writeln (guns[i] ());  // 1 and 1, why?
>>
>> Isn't this undefined behavior? The first loop variable named "i" already went out of scope when the delegate is invoked.
>
> Not undefined behavior.

What is the lifetime of the first loop's variable i? What about this example:

``` bug2.d
import std.stdio;

void main ()
{
   int delegate () [] dg;
   foreach (i; 0..2) {
      int *j;
      if (i == 0) {
         auto k = i;
         j = &k;
      }
      else {
         auto l = i;
         j = &l;
      }
      dg ~= () => *j;
   }
   foreach (p; dg)
      p ().writeln;
}
```

March 29, 2018
On Thursday, 29 March 2018 at 20:26:59 UTC, kdevel wrote:
> What is the lifetime of the first loop's variable i?

It lives as long as the delegate.

> What about this example:
>
> ``` bug2.d
> import std.stdio;
>
> void main ()
> {
>    int delegate () [] dg;
>    foreach (i; 0..2) {
>       int *j;
>       if (i == 0) {
>          auto k = i;
>          j = &k;
>       }
>       else {
>          auto l = i;
>          j = &l;
>       }
>       dg ~= () => *j;
>    }
>    foreach (p; dg)
>       p ().writeln;
> }
> ```

As far as I can understand it, you get a closure for j, but not for i, k, or l. So `*j` will be garbage.

The compiler doesn't consider where j points. It just goes by the variables that are used in the delegate.