September 04

On Wednesday, 4 September 2024 at 03:02:10 UTC, Richard (Rikki) Andrew Cattermole wrote:

>

It would be an easy enough swap to change @safe to @tsafe. But that isn't a decision we need to make here. We can make that prior to launch.

@safe @trusted and @system are already misunderstood as they are, we really don't want to throw a fourth attribute into the mix.

>

But I do want to make a point here, owner escape analysis only kicks in and forces effectively const on the owner if:

That's not consistent with this example from the DIP, where there's no scope or &field:

struct Top {
	int* field;
}

void func(ref Top owner) @safe {
	int* field = owner.field;
	// owner is now effectively const, it cannot be mutated
	
	owner = Top.init; // Error: The variable `owner` has a borrow and cannot be mutated
>

This is the same meaning it has today with DIP1000.

Today, scope doesn't imply 'strong relationship with input variable', only return ref does.

>

It needs to mean something, so got an alternative?

No, because I don't like this escape set definition syntax in the first place.

>

Giving scope a default escape set is to allow it to match existing understanding, which does help with communicability.

That's sending mixed messages. On the one hand, this DIP completely redefines lifetime semantics and syntax, trying to forget DIP1000 ever existed. On the other hand, it adds a special meaning to scope feigning some sort of backward compatibility, but adding a new double meaning to the keyword, which is the very thing the new syntax is supposed to fix!

>

Yes you are correct.

If you were allowed to take a pointer to a by-ref variable and then store it some place you are most likely escaping a pointer.

The address of the variable and the pointer value it holds are two different things. So the following becomes impossible to express with this DIP:

int* global;

int** f(return ref int* v) @safe
{
    global = v;
    return &v;
}
>

I'm going to need an example of what you think is not addressed here.

To clarify, the headings in my post are common DIP1000 woes that alternative DIPs should have an answer to. Timon has brought up the composability problem before:

import std.typecons;
int* y;
int* foo(){
    int x;
    auto t=tuple(&x,y); // type has to be Tuple!(scope(int*),int*)
    return t[1];
}

https://forum.dlang.org/post/qqgjop$kan$1@digitalmars.com

The example could compile, but it doesn't because the entire tuple shares one lifetime.
Another example is item 1 of my post: https://forum.dlang.org/post/icoavlbaxqpcnkhijcpy@forum.dlang.org

>

From my perspective the field gets conflated with its containing instance variable and that covers composability.

So this DIP's answer is: tough luck, we're still conflating.

>

scope is not transitive, at least as far as the language knows transitive to mean.

Same here, I meant to say that "lack of transitive scope" is a DIP1000 woe that the DIP should address. The DIP doesn't have a single example where a pointer gets dereferenced and then escaped. What happens to the following examples?

// Assuming -preview=dip1000

int* deref(scope int** x) @safe => *x; // currently allowed
// because x gets dereferenced and scope only applies to first indirection

void main() @safe
{
    int x, y;
    scope int[] arr = [&x, &y]; // currently not allowed
    // because it requires scope to apply to two levels of pointer indirection
}
>

I focus upon multiple outputs, because to make flattening to a function signature to work, you have to do this. If you don't you are not going to model enough code, and will be going against the literature on this subject making it harder to use.

Walter has stated that he's not looking for a complete lifetime tracking solution for all possible situations, just something simple and pragmatic to cover common cases. In the DIP1000 woes thread, the only multiple output-related issue is with swap. This DIP's syntax is overkill to solve just that problem. It would help if there were examples of actual code that really needs to use @escape(parametername).

September 04
On 04/09/2024 10:24 PM, Dennis wrote:
> On Wednesday, 4 September 2024 at 03:02:10 UTC, Richard (Rikki) Andrew Cattermole wrote:
>> It would be an easy enough swap to change ``@safe`` to ``@tsafe``. But that isn't a decision we need to make here. We can make that prior to launch.
> 
> `@safe` `@trusted` and `@system` are already misunderstood as they are, we really don't want to throw a fourth attribute into the mix.

Indeed, there are some interesting trade offs here.

But it is an option to give people a way to buy into it (by not being forced to use it).

>> But I do want to make a point here, owner escape analysis only kicks in and forces effectively const on the owner if:
> 
> That's not consistent with this example from the DIP, where there's no `scope` or `&field`:
> 
> ```D
> struct Top {
>      int* field;
> }
> 
> void func(ref Top owner) @safe {
>      int* field = owner.field;
>      // owner is now effectively const, it cannot be mutated
> 
>      owner = Top.init; // Error: The variable `owner` has a borrow and cannot be mutated
> ```

Okay that example is wrong, it was copied from an earlier iteration and I didn't think it through.

Will fix.

```d
struct Top {
	int* field;
}

void func(ref Top owner) @safe {
	int** field = &owner.field;
	// owner is now effectively const, it cannot be mutated
	
	owner = Top.init; // Error: The variable `owner` has a borrow and cannot be mutated
	owner.field = null; // Error: The variable `owner` has a borrow and cannot be mutated

	if (field !is null) {
		writeln(**field);
		**field = 2; // ok, fully mutable
	}
}
```

>> Giving ``scope`` a default escape set is to allow it to match existing understanding, which does help with communicability.
> 
> That's sending mixed messages. On the one hand, this DIP completely redefines lifetime semantics and syntax, trying to forget DIP1000 ever existed. On the other hand, it adds a special meaning to `scope` feigning some sort of backward compatibility, but adding a new double meaning to the keyword, which is the very thing the new syntax is supposed to fix!

Okay, I can entirely ditch the default escape set for ``scope``. Its not required, it only exists as a QoL thing.

Done.

Now ``scope`` by itself won't reflect existing behaviors and require additional annotation to make it completely consistent within the proposal.

>> Yes you are correct.
>>
>> If you were allowed to take a pointer to a by-ref variable and then store it some place you are most likely escaping a pointer.
> 
> The address of the variable and the pointer value it holds are two different things. So the following becomes impossible to express with this DIP:
> 
> ```D
> int* global;
> 
> int** f(return ref int* v) @safe
> {
>      global = v;
>      return &v;
> }
> ```

Yes, that is intentional.

>> I'm going to need an example of what you think is not addressed here.
> 
> To clarify, the headings in my post are common DIP1000 woes that alternative DIPs should have an answer to. Timon has brought up the composability problem before:
> 
> ```D
> import std.typecons;
> int* y;
> int* foo(){
>      int x;
>      auto t=tuple(&x,y); // type has to be Tuple!(scope(int*),int*)
>      return t[1];
> }
> ```
> 
> https://forum.dlang.org/post/qqgjop$kan$1@digitalmars.com
> 
> The example could compile, but it doesn't because the entire tuple shares one lifetime.
> Another example is item 1 of my post: https://forum.dlang.org/post/icoavlbaxqpcnkhijcpy@forum.dlang.org

Yes I'm aware of this one.

It is a complicating factor in the analysis and should be developed later on.

We did talk about it on Discord.

If we were to do it right now, we can do POD structs and static arrays. But more adjustment would be needed later on for language tuples.

>> From my perspective the field gets conflated with its containing instance variable and that covers composability.
> 
> So this DIP's answer is: tough luck, we're still conflating.

Yes.

>> ``scope`` is not transitive, at least as far as the language knows transitive to mean.
> 
> Same here, I meant to say that "lack of transitive scope" is a DIP1000 woe that the DIP should address. The DIP doesn't have a single example where a pointer gets dereferenced and then escaped. What happens to the following examples?
> 
> ```D
> // Assuming -preview=dip1000
> 
> int* deref(scope int** x) @safe => *x; // currently allowed
> // because x gets dereferenced and scope only applies to first indirection

Allowed too, but the return value will have a strong relationship.

``int* deref(@escape(return) int** x) @safe => *x;``

Annotating ``scope`` is optional, as it'll be upgraded by caller if needed.

This function is effectively the ``identity`` functions that I use throughout the document. So this is covered.

> void main() @safe
> {
>      int x, y;
>      scope int[] arr = [&x, &y]; // currently not allowed
>      // because it requires scope to apply to two levels of pointer indirection

That is safe due to conflation and reverse order of cleanup.

> }
> ```

Okay this would be a good addition.

```d
int* transformation(int* input) {
	int value;

	int*[3] array;
	array[0] = input; // `array` has a weak relationship to `input`
	array[1] = new int; // GC allocation has no relationships without a constructor call or initializer to form one
	array[2] = &value;
	return array[0]; // Error: Variable `array` is owned by the stack due to the variable `value` and cannot be returned
}
```
September 04

On Tuesday, 3 September 2024 at 03:00:20 UTC, Richard (Rikki) Andrew Cattermole wrote:

>

An example of this is with a global, in the case of a variable thread local storage, it is possible in fully @safe code with DIP1000 turned on to cause a segfault.

import std;

int* tlsGlobal;

@safe:

void main() {
    tlsGlobal = new int(2);
    assert(*tlsGlobal == 2);

    toCall();
    assert(*tlsGlobal == 2); // Segfault
}

void toCall() {
    tlsGlobal = null;
}

But aren’t segfault always meant to be @safe anyway?

int* x;
void main() @safe{
  auto y = *x;
}
September 05
On 05/09/2024 2:28 AM, IchorDev wrote:
> On Tuesday, 3 September 2024 at 03:00:20 UTC, Richard (Rikki) Andrew Cattermole wrote:
>> An example of this is with a global, in the case of a variable thread local storage, it is possible in fully ``@safe`` code with DIP1000 turned on to cause a segfault.
>>
>> ```d
>> import std;
>>
>> int* tlsGlobal;
>>
>> @safe:
>>
>> void main() {
>>     tlsGlobal = new int(2);
>>     assert(*tlsGlobal == 2);
>>
>>     toCall();
>>     assert(*tlsGlobal == 2); // Segfault
>> }
>>
>> void toCall() {
>>     tlsGlobal = null;
>> }
>> ```
> 
> But aren’t segfault always meant to be @safe anyway?
> ```d
> int* x;
> void main() @safe{
>    auto y = *x;
> }
> ```

In theory yes it's perfectly safe. However this example isn't meant to show that a solution to nullability is needed, but instead to show that you cannot make assumptions based upon what code is locally analyzed for things outside of it.

To assume that a non-function local variable will have a value that is known to the analysis over the course of a function body isn't correct and that pokes a massive hole in the analysis capabilities.

September 04

On Wednesday, 4 September 2024 at 10:24:51 UTC, Dennis wrote:

>

[snip]
Walter has stated that he's not looking for a complete lifetime tracking solution for all possible situations, just something simple and pragmatic to cover common cases. In the DIP1000 woes thread, the only multiple output-related issue is with swap. This DIP's syntax is overkill to solve just that problem. It would help if there were examples of actual code that really needs to use @escape(parametername).

Walter has stated that in the past, but it shouldn't necessarily mean we should put ourselves in a straitjacket if another solution is better (not saying this one is). I think the interpolation changes are apropos. The difference is that more people can understand positives and negatives with competing interpolation designs vs. competing lifetime analysis designs.

September 05

On Wednesday, 4 September 2024 at 03:02:10 UTC, Richard (Rikki) Andrew Cattermole wrote:

>

owner escape analysis only kicks in and forces effectively const on the owner if:

  1. You take a pointer to stack memory
  2. You receive memory that has a strong relationship (perhaps done explicitly for reference counting!)
  3. You take a pointer to a field of struct/class/union

[…]

In a way its a hole in the design, but an intentional one as it makes for a very good user experience and doesn't really have a lot of down sides.

I was going to fill in that hole, but @system variables covers it enough that I kinda just went meh.

Wait, so how would one force owner escape analysis to be enabled for manually heap-allocated memory? This DIP is meant to replace @live, after all.

> >
  • @escape is the opposite of @escape(), which could be confusing

Originally I was going to make this to mean 'inferred', but it's better if everything gets inferred by default.

It needs to mean something, so got an alternative?

Maybe add a special case for something like @escape(false)?

September 05
On 05/09/2024 9:53 PM, IchorDev wrote:
> On Wednesday, 4 September 2024 at 03:02:10 UTC, Richard (Rikki) Andrew Cattermole wrote:
>> owner escape analysis only kicks in and forces effectively const on the owner if:
>>
>> 1. You take a pointer to stack memory
>> 2. You receive memory that has a strong relationship (perhaps done explicitly for reference counting!)
>> 3. You take a pointer to a field of struct/class/union
>>
>> […]
>>
>> In a way its a hole in the design, but an intentional one as it makes for a very good user experience and doesn't really have a lot of down sides.
>>
>> I was going to fill in that hole, but ``@system`` variables covers it enough that I kinda just went meh.
> 
> Wait, so how would one force owner escape analysis to be enabled for manually heap-allocated memory? This DIP is meant to replace @live, after all.

You need to establish a strong relationship either to a variable, or from it.

For a method call add ``scope`` on the this pointer:

```d
struct RC {
	int* borrow() scope;
}
```

For variable declaration:

```d
scope int* owner = new int;
```

Take a pointer:

```d
struct Top {
	int field;
}

Top top;
int* borrow = &top.field;
```

I have not added meaning for ``scope`` on a field, although I can see that this might be nice to add. I'm not sure if that is needed. Is this a hole for you?

Are you expecting a type qualifier? It is not needed for this.

```d
struct Thing {
	int* ptr;
}

void caller() {
	scope Thing owner;
	called(owner); // Error: owner would escape to an unknown location
}

int* global;

void called(@escape(__unknown) /*weak*/ Thing thing) {
	global = thing.ptr;
}
```

This is a rather clever aspect of weak vs strong relationships, a weak relationship tells the analysis about how memory is moving around. You do not need to understand the full graph, as long you understand your own function body and those that you call function signatures.

In general I strongly suggest wrapping raw memory in an RC owner, this allows you move it around safely, and then borrow from it (kicking off owner escape analysis).

```d
struct Wrapper {
	private @system {
		int* ptr;
	}

	int* borrow() @escape(return) scope @trusted {
		return ptr;
	}
}

Wrapper acquire() {
	Wrapper wrapper = ...;

	{
		int* borrowed = wrapper.borrow();
		...
	}

	return wrapper;
}
```

I have a feeling that this won't be answering your question, is there something I'm for whatever reason not understanding about it?

>>> - `@escape` is the opposite of `@escape()`, which could be confusing
>>
>> Originally I was going to make this to mean 'inferred', but it's better if everything gets inferred by default.
>>
>> It needs to mean something, so got an alternative?
> 
> Maybe add a special case for something like `@escape(false)`?

The question is for what ``@escape()`` would do. Which Dennis has not counter proposal for.
September 22

On Thursday, 5 September 2024 at 10:28:45 UTC, Richard (Rikki) Andrew Cattermole wrote:

>

On 05/09/2024 9:53 PM, IchorDev wrote:

>

Wait, so how would one force owner escape analysis to be enabled for manually heap-allocated memory? This DIP is meant to replace @live, after all.

You need to establish a strong relationship either to a variable, or from it.

For a method call add scope on the this pointer:

struct RC {
	int* borrow() scope;
}

For variable declaration:

scope int* owner = new int;

Take a pointer:

struct Top {
	int field;
}

Top top;
int* borrow = &top.field;

I see, thank you. So scope x = malloc(10);.

>

I have not added meaning for scope on a field, although I can see that this might be nice to add. I'm not sure if that is needed. Is this a hole for you?

I can certainly think of some times where forcing certain escape analysis pattens when heap memory is taken from a struct would be useful. That said, I’d usually be wrapping that memory in a @property method anyway.

>

I have a feeling that this won't be answering your question, is there something I'm for whatever reason not understanding about it?

I think you understood just fine.

> > > >
  • @escape is the opposite of @escape(), which could be confusing

Originally I was going to make this to mean 'inferred', but it's better if everything gets inferred by default.

It needs to mean something, so got an alternative?

Maybe add a special case for something like @escape(false)?

The question is for what @escape() would do. Which Dennis has not counter proposal for.

I see. @escape is meant to tell you how the variable escapes, but on its own it implicitly uses a default set of escapes. If the parentheses are empty, surely that should just be the same as @escape?

September 23
On 22/09/2024 11:03 PM, IchorDev wrote:
>                   * |@escape| is the opposite of |@escape()|, which
>                     could be confusing
> 
>             Originally I was going to make this to mean 'inferred', but
>             it's better if everything gets inferred by default.
> 
>             It needs to mean something, so got an alternative?
> 
>         Maybe add a special case for something like |@escape(false)|?
> 
>     The question is for what |@escape()| would do. Which Dennis has not
>     counter proposal for.
> 
> I see. |@escape| is meant to tell you how the variable escapes, but on its own it implicitly uses a default set of escapes. If the parentheses are empty, surely that should just be the same as |@escape|?

No, ``@escape()`` specifies the empty set. As in, it does not escape anywhere.

With ``@escape`` no set has been given.

I'm wondering if it shouldn't be specified but instead let buildkite determine what ``@escape`` does based upon statistics.
1 2
Next ›   Last »