void func1(scope lazy string msg) @nogc {
}

void func2(scope lazy string msg) @nogc {
    func1(msg);
}

What? Why is msg GC allocating, especially since I scoped the lazy? Why is msg even evaluated?

Something seems off here.

Thanks,
Shachar

On Sunday, December 10, 2017 12:54:00 Shachar Shemesh via Digitalmars-d wrote:
> void func1(scope lazy string msg) @nogc {
> }
>
> void func2(scope lazy string msg) @nogc {
>      func1(msg);
> }
>
> What? Why is msg GC allocating, especially since I scoped the lazy? Why is msg even evaluated?
>
> Something seems off here.

Well, I'm not exactly sure how lazy is implemented underneath the hood (other than the fact that it generates a delegate and potentially a closure), but based on the error message, it sounds like the delegate that's being generated isn't @nogc, so it can't be called within the function, which would be a completely different issue from allocating a closure.

Curiously, pure doesn't seem to have the same problem, and I would have guessed that it would given that @nogc does.

But if it matters that the delegate be pure or @nogc because the function is marked that way, then that's a bit of a problem, because presumably, there can only be one delegate (since the function isn't templated), and the same function could be called with an expression that allocated and with an expression that didn't allocate. So, if the delegate is restricted like that, then that would restrict the caller, which doesn't follow how non-lazy arguments work, and an argument could be made that whether calling the delegate allocates memory or not or is pure or not doesn't matter even if the function being called is @nogc or pure on the basis that it's conceptually just a delayed evaluation of the argument in the caller's scope. But for that to work, the generated delegate can't be checked for @nogc or pure or nothrow or any of that inside the function with the lazy parameter. Given that pure works but @nogc doesn't, it makes it seem like the compiler was made smart enough to ignore purity for lazy parameters but not smart enough to ignore the lack of @nogc.

As for scope, I don't know if it really applies to lazy parameters or not at this point. Without -dip1000, all it applies to is delegates, but lazy parameters are turned into delegates, so you would _think_ that scope would apply, but I don't know. scope has always been underimplemented, though Walter's work on -dip1000 is fixing that. Regardless, the error message makes it sound like scope and closures have nothing to do with the problem (though it could potentially be a problem once the @nogc problem with the delegate is fixed).

In any case, I think that it's pretty clear that this merits being reported in bugzilla. The fact that pure works while @nogc doesn't strongly indicates that something is off with @nogc - especially if scope is involved. Worst case, a fix will have to be lumped in with -dip1000, depending on what scope is supposed to be doing now and exactly how lazy is working underneath the hood, but I definitely think that your code should work.

- Jonathan M Davis

On 10/12/17 13:44, Jonathan M Davis wrote:
> it sounds like the delegate that's
> being generated isn't @nogc, so it can't be called within the function,
> which would be a completely different issue from allocating a closure.

Here's the thing, though. There is no reason for the delegate to be called at all!

Since func1's msg is also lazy, I expected the lowering to look like this:

void func1(/*scope*/ string delegate() msg) @nogc {
}

void func2(/*scope*/ string delegate() msg) @nogc {
  func1(msg);
}

Even with scope commented out, the above compiles just fine. The fact that msg is not @nogc doesn't matter at all, simply because no one is evaluating the delegate.

On 10/12/17 14:00, Shachar Shemesh wrote:
> On 10/12/17 13:44, Jonathan M Davis wrote:
>> it sounds like the delegate that's
>> being generated isn't @nogc, so it can't be called within the function,
>> which would be a completely different issue from allocating a closure.
> 
> Here's the thing, though. There is no reason for the delegate to be called at all!
> 
> Since func1's msg is also lazy, I expected the lowering to look like this:
> 
> void func1(/*scope*/ string delegate() msg) @nogc {
> }
> 
> void func2(/*scope*/ string delegate() msg) @nogc {
>    func1(msg);
> }
> 
> Even with scope commented out, the above compiles just fine. The fact that msg is not @nogc doesn't matter at all, simply because no one is evaluating the delegate.

I think I got it. I think the lowering is this:

void func1(/*scope*/ string delegate() msg) @nogc {
}

void func2(/*scope*/ string delegate() msg) @nogc {
  func1((){ return msg(); });
}

With "scope" commented out, this generates the following error:
test.d(15): Error: function test.func2 is @nogc yet allocates closures with the GC
test.d(16):        test.func2.__lambda2 closes over variable msg at test.d(15)

So there are two problems here. The first is that a "lazy" generated delegate does not get forwarded to a function, and instead gets wrapped by a completely useless temporary delegate. The second is that scope on lazy is not honored.

I'll file a bug report.

Shachar

On Sunday, 10 December 2017 at 11:44:20 UTC, Jonathan M Davis wrote:
> On Sunday, December 10, 2017 12:54:00 Shachar Shemesh via Digitalmars-d wrote:
>> void func1(scope lazy string msg) @nogc {
>> }
>>
>> void func2(scope lazy string msg) @nogc {
>>      func1(msg);
>> }
>>
>> What? Why is msg GC allocating, especially since I scoped the lazy? Why is msg even evaluated?
>>
>> Something seems off here.
>
> Well, I'm not exactly sure how lazy is implemented underneath the hood (other than the fact that it generates a delegate and potentially a closure), but based on the error message, it sounds like the delegate that's being generated isn't @nogc, so it can't be called within the function, which would be a completely different issue from allocating a closure.

> Curiously, pure doesn't seem to have the same problem, and I would have guessed that it would given that @nogc does.
>
> But if it matters that the delegate be pure or @nogc because the function is marked that way, then that's a bit of a problem, because presumably, there can only be one delegate (since the function isn't templated), and the same function could be called with an expression that allocated and with an expression that didn't allocate. So, if the delegate is restricted like that, then that would restrict the caller, which doesn't follow how non-lazy arguments work, and an argument could be made that whether calling the delegate allocates memory or not or is pure or not doesn't matter even if the function being called is @nogc or pure on the basis that it's conceptually just a delayed evaluation of the argument in the caller's scope. But for that to work, the generated delegate can't be checked for @nogc or pure or nothrow or any of that inside the function with the lazy parameter. Given that pure works but @nogc doesn't, it makes it seem like the compiler was made smart enough to ignore purity for lazy parameters but not smart enough to ignore the lack of @nogc.
>
> As for scope, I don't know if it really applies to lazy parameters or not at this point. Without -dip1000, all it applies to is delegates, but lazy parameters are turned into delegates, so you would _think_ that scope would apply, but I don't know. scope has always been underimplemented, though Walter's work on -dip1000 is fixing that. Regardless, the error message makes it sound like scope and closures have nothing to do with the problem (though it could potentially be a problem once the @nogc problem with the delegate is fixed).
>
> In any case, I think that it's pretty clear that this merits being reported in bugzilla. The fact that pure works while @nogc doesn't strongly indicates that something is off with @nogc - especially if scope is involved. Worst case, a fix will have to be lumped in with -dip1000, depending on what scope is supposed to be doing now and exactly how lazy is working underneath the hood, but I definitely think that your code should work.
>
> - Jonathan M Davis

What does D throw so many errors on code people don't write? I realize that "lowering" has some uses and I'm not advocating *not* doing it, but why not after a semantic pass? Why is it so consistently throwing semantic errors on code it's users did not write...

It would cut down on the cryptic errors...

Submitted
https://issues.dlang.org/show_bug.cgi?id=18058

On 10/12/17 14:07, Shachar Shemesh wrote:
> On 10/12/17 14:00, Shachar Shemesh wrote:
>> On 10/12/17 13:44, Jonathan M Davis wrote:
>>> it sounds like the delegate that's
>>> being generated isn't @nogc, so it can't be called within the function,
>>> which would be a completely different issue from allocating a closure.
>>
>> Here's the thing, though. There is no reason for the delegate to be called at all!
>>
>> Since func1's msg is also lazy, I expected the lowering to look like this:
>>
>> void func1(/*scope*/ string delegate() msg) @nogc {
>> }
>>
>> void func2(/*scope*/ string delegate() msg) @nogc {
>>    func1(msg);
>> }
>>
>> Even with scope commented out, the above compiles just fine. The fact that msg is not @nogc doesn't matter at all, simply because no one is evaluating the delegate.
> 
> I think I got it. I think the lowering is this:
> 
> void func1(/*scope*/ string delegate() msg) @nogc {
> }
> 
> void func2(/*scope*/ string delegate() msg) @nogc {
>    func1((){ return msg(); });
> }
> 
> With "scope" commented out, this generates the following error:
> test.d(15): Error: function test.func2 is @nogc yet allocates closures with the GC
> test.d(16):        test.func2.__lambda2 closes over variable msg at test.d(15)
> 
> So there are two problems here. The first is that a "lazy" generated delegate does not get forwarded to a function, and instead gets wrapped by a completely useless temporary delegate. The second is that scope on lazy is not honored.
> 
> I'll file a bug report.
> 
> Shachar