We have this cool feature in D where if you try take the address of a function, and the compiler can't prove that it doesn't escape where the actual stack frame is, it allocates the stack frame on the GC, and then it becomes a "closure".

Why can't we just do this in @safe code whenever it has the same problem? Consider this code snippet:

void bar(int[]) @safe;
void foo() @safe
{
   int[5] arr;
   bar(arr[]);
}

Currently, without DIP1000 enabled, this compiles, and if bar squirrels away the array, you have a memory issue.

With DIP1000, this becomes an error, the compiler yells at you to put scope on the bar parameter (and then you can't squirrel it away).

But what if instead, with DIP1000 seeing that, it just says now we have a closure situation, and allocates foo's frame on the heap.

A sufficiently smart optimizer might be able to detect that actually bar doesn't squirrel it away, and will still allocate on the stack.

If you want to ensure this doesn't happen, just like with other closures, you annotate with @nogc. And then you can have suggestions about putting scope on bar's parameter.

This gives us safe code that may not perform as expected, but at least it is safe. And it doesn't spew endless errors to the user. Consider that there are already so many cases where closures are allocated, with std.algorithm and lambdas, and mostly nobody bats an eye.

Just another possible idea for debate.

-Steve

On Friday, 27 May 2022 at 22:16:30 UTC, Steven Schveighoffer wrote:

The fundamental problem here, is that proving whether it squirrels away or not is actually very difficult, and therefore, you really don't want to rely on this, especially if the end result is as drastic as making most functions closures.

On Friday, 27 May 2022 at 22:16:30 UTC, Steven Schveighoffer wrote:

Object oriented languages has often used only closures conceptually (no assumptions about a contiguous stack), it allows for high degree of concurrency etc. It is a common strategy for high level languages, yes.

On Friday, 27 May 2022 at 22:16:30 UTC, Steven Schveighoffer wrote:

void bar(int[]) @safe;
void foo() @safe
{
   int[5] arr;
   bar(arr[]);
}

Good brainstorming. But I don't think it can work:

void foo(ref int[5] arr)
{ // It's not up to this function to decide where
  // arr resides in memory. Thus we can't make a
  // closure out of this one without breaking epected
  // behaviour.
  bar(arr[]);
}

On Saturday, 28 May 2022 at 08:12:53 UTC, Dukc wrote:

The root cause is that Walter does not want @safe to be a high level feature, as a result it will be easier to write @system code than @safe no matter what features he comes up with!!

If one instead frame @safe as easy high level programming and add some reasonable constraints and owning pointers/objects then things will fall into place. Then optimize.

Without that it will be a neverending tail chasing dance. Or you will end up with something that constantly gets in your way, like Rust. And that makes @system or a switch to Rust more attractive, so nothing to gain there...

On Saturday, 28 May 2022 at 10:24:40 UTC, Ola Fosheim Grøstad wrote:

@safe code is almost always higher level than @system code on average. In both theory and practice.

You are right in the sense that you can do everything in @system that you can do in @safe and more, so it's more expressive and higher level in the same sense C90 is "higher-level" than C99. And just like it may not be worth it to add proper function prototypes to already well-tested C90 codebase, it's not always worthwhile to add @safe to D a stable D codebase that is not going to see any major refactoring.

But woe to one who chooses @system just to "be higher level". While it does enable slightly shorter code in some cases, much more time will be lost debugging the error-prone practices it leads to. It's just as "good" choice as not using function prototypes in a new C codebase because it saves some LOC.

On Saturday, 28 May 2022 at 10:59:58 UTC, Dukc wrote:

What I mean is that DSP code will most likely be @system, but the UI could essentially be written in C#, so it would be acceptable to limit @safe to high level constructs and put the burden on @system code calling @safe code (eg provide owning pointers and other useful constructs). That has value.

I am not sure if there is all that much value in making low level code @safe, if you look at status quo for research languages... that is a tall mountain to climb and not a pretty view.

On 5/28/22 4:12 AM, Dukc wrote:

void bar(int[]) @safe;
void foo() @safe
{
   int[5] arr;
   bar(arr[]);
}

void foo(ref int[5] arr)
{ // It's not up to this function to decide where
   // arr resides in memory. Thus we can't make a
   // closure out of this one without breaking epected
   // behaviour.
   bar(arr[]);
}

Well, we have to decide if taking a value by ref means it should be allocated, or if it should be scope (like DIP1000). If we go with the latter, then you start getting error messages, and we are kind of back to square one. If we go with the former, then any simple use of struct methods is going to allocate a closure.

So yeah, that pretty much destroys this idea.

-Steve

On Friday, 27 May 2022 at 22:16:30 UTC, Steven Schveighoffer wrote:

Nobody bats an eye when it's DMD doing something as silly as allocating closures on heap, because low performance is pretty much expected and normal for it. But if this problem also affects GDC, then I'm very much concerned about it: https://forum.dlang.org/thread/myiqlzkghnnyykbyksga@forum.dlang.org

On Saturday, 28 May 2022 at 14:50:39 UTC, Steven Schveighoffer wrote:

void foo(ref int[5] arr)
{ // It's not up to this function to decide where
   // arr resides in memory. Thus we can't make a
   // closure out of this one without breaking epected
   // behaviour.
   bar(arr[]);
}

I'm necrobumping an old thread, because I made a discovery that I think brings this idea back to the table.

The closures we already have suffer from the same problem I wrote about here! Behold:

int delegate(int) @safe escape1(scope int* x) @safe
{   return (int y) => *x += y;
}

int delegate(int) @safe escape2(ref int x) @safe
{   return (int y) => x += y;
}

These compile, but they shouldn't. They should require annotating their parameters with return. Now in itself this is just a DIP1000 bug, but consider the situation once it's fixed. If you create a pointer to a local variable and try to return it, you get an error, but if you create a delegate using it and return it, you get a closure. This is a language inconsistency for no good reason.

So we won't get rid of DIP1000 with this observation, but for sake of consistency maybe we still should reconsider the idea this thread is about.