Jump to page: 1 24  
Page
Thread overview
August 25

DIP1000 keeps rearing its head in discussions touching on safety in our monthly meetings. So we recently held a planning session focused exclusively on DIP1000. Two things came out of it.

First, it's pretty clear that inference by default is necessary to make DIP1000 easier to use. It's not clear at the moment how this can be achieved, but it's something that we need to work out.

Second, we'd like to get a number of examples of problems people have had with using DIP1000 that haven't shown up in Bugzilla. Walter wants to be as certain as he can whether such issues are fixable or if the design is fundamentally flawed.

If you've had problems using DIP1000, please post here with a description of what you encountered and any details about it you think we may find helpful. Please provide example code where possible.

As an example of the sort of thing we're looking for, someone in the meeting brought up the problems encountered with trying to add DIP1000 support to vibe.d.

Please do not reply to posts in this thread unless you have additional relevant information regarding the problem described. We want to collect examples, and any side discussions will just be noise.

Thanks!

August 26
I'll include the items I said at the meeting just so we can hopefully minimize extras.

From meeting:

- Reference counted types (not in language) get affected by scope, like any other pointer contained struct. Which is a problem since it has runtime lifetime, not a CT analyzed one.
- Only one output is described, (returned value or this pointer), this means ref and out parameters aren't outputs and therefore cannot be @safe.
- The abomination that is scope return + return scope (this is easier to resolve in terms of new UDA's, but does not solve communicability of behavior).

Other ones:

- It does not model owners and borrows, this isn't in its scope, it also means in practice it has to try and act as a guarantee both up the call stack and down it, which is not what such escape analysis is designed for. Which makes it a lot harder to use.
- Does not attempt at modelling globals including TLS, so with @safe code you can absolutely escape memory from the thread completely unintentionally.

As someone who has over 120k LOC fully annotated DIP1000 code, my only conclusion is that DIP1000 has some serious scope of proposal level issues and cannot be rectified. This has been my conclusion *after* trying to fix it for a few years and having no proposals that could.

Depending upon how DConf goes, I might be looking at dumping it, because as-is it has too many scope-level holes that will not be fixed. But in saying that, Dennis I sincerely hope you can do some serious magic here because it'll be impressive!

August 25

On Sunday, 25 August 2024 at 13:10:22 UTC, Mike Parker wrote:

>

If you've had problems using DIP1000, please post here with a description of what you encountered and any details about it you think we may find helpful. Please provide example code where possible.

Here, I just made a PR to mysql-native, with dip1000 enabled. The last time I went down this rabbit hole, it was clear this is unworkable.

You are welcome to submit PRs to my PR to show me how to fix it, please explain how everything works.

I don't have time to play around with dip1000 normally -- it generally just results in bizarre and unfixable errors.

https://github.com/mysql-d/mysql-native/pull/283

-Steve

August 25
On 8/25/24 19:02, Steven Schveighoffer wrote:
> 
> I don't have time to play around with dip1000 normally -- it generally just results in bizarre and unfixable errors.
> 
> https://github.com/mysql-d/mysql-native/pull/283



diff --git a/source/mysql/protocol/comms.d b/source/mysql/protocol/comms.d
index ec2a3ea..0238926 100644
--- a/source/mysql/protocol/comms.d
+++ b/source/mysql/protocol/comms.d
@@ -689,7 +689,7 @@ package(mysql) ubyte[] getPacket(Connection conn)
 	return packet;
 }

-package(mysql) void send(MySQLSocket _socket, const(ubyte)[] packet)
+package(mysql) void send(MySQLSocket _socket, scope const(ubyte)[] packet)
 in
 {
 	assert(packet.length > 4); // at least 1 byte more than header
diff --git a/source/mysql/protocol/sockets.d b/source/mysql/protocol/sockets.d
index a582fc9..7da43d7 100644
--- a/source/mysql/protocol/sockets.d
+++ b/source/mysql/protocol/sockets.d
@@ -39,7 +39,7 @@ interface MySQLSocket
 @safe:
 	void close();
 	@property bool connected() const;
-	void read(ubyte[] dst);
+	void read(scope ubyte[] dst);
 	void write(const scope ubyte[] bytes);

 	void acquire();
@@ -78,7 +78,7 @@ class MySQLSocketPhobos : MySQLSocket
 		return socket.isAlive;
 	}

-	void read(ubyte[] dst)
+	void read(scope ubyte[] dst)
 	{
 		// Note: I'm a little uncomfortable with this line as it doesn't
 		// (and can't) update Connection._open. Not sure what can be done,

August 26

On Sunday, 25 August 2024 at 13:10:22 UTC, Mike Parker wrote:

>

As an example of the sort of thing we're looking for, someone in the meeting brought up the problems encountered with trying to add DIP1000 support to vibe.d.

My first attempt at adding DIP1000-compatibility to oceandrift/http was a disaster. An hour or two of attribute tuning in a whack-a-mole fashion during which I was asking myself why I had to do this manually as I was mostly just making the changes suggested by the compiler. It all had an end when I ran into an presumably return ref return scope case; something that isn’t doable in D.

The second attempt was a bit more successful. I’ve committed a scarily big number of patches to a feature branch, pushed it to GitHub and let it be: https://github.com/oceandrift/http/commit/996b743f1a7b5f4799dcde802971bfeb5a675cf1

My third attempt was more pragmatic and meant “not doing anything that would make me bother with DIP1000’s scope gulasch myself”. I was quite happy with the end result.

August 26
On Sunday, 25 August 2024 at 13:10:22 UTC, Mike Parker wrote:
> If you've had problems using DIP1000, please post here with a description of what you encountered and any details about it you think we may find helpful.

One of the most common issues I run into when talking about DIP1000 is that DIP1000 is actually two things:

1. Plugging holes in `@safe`.
2. Providing a mechanism the return/ref/scope mechanism re-enables most of the sort of code that is flagged `@system` after implementing the former.

Furthermore, I have yet to meet anyone who tells me that adding 2-word attributes (`scope return` vs `return scope`) to the soup was a great choice. I see a major comprehensibility problem with these.

It’s also a bit funny that all of those things are still referred to as “DIP1000”. If one takes just a slight glimpse at the document of the same name, it clearly says “Superseded”.

I do understand why the article “Coralling Wild Pointers With ref return scope” explains most of the concepts on raw pointers. But on the other hand I’m not sure how useful those are to the unsuspecting users who run into DIP1000 violations with static arrays or refs.

Or they even just wanted to use `std.getopt` in `@safe` code and ended up dealing with DIP1000. That’s what made me turn on the compiler switch for first time.
August 26
I found the following inconsistency:

```
@safe ref int abc(int i) { return ghi(i); }
@trusted ref int ghi(ref int r) { return r; }

@safe int* foo(int i) { return bar(&i); } // Error: reference to local variable `i` assigned to non-scope parameter `p` calling `bar`
@trusted int* bar(int* p) { return p; }
```

Compiling with -dip1000 gives the error indicated. Note that it did not give an error for the semantically equivalent call to ghi(i).

Possible resolutions:

1. give an error for both

2. give no error for both, as @trusted implementations are the user's problem

3. disallow taking the address of a local variable in @safe code


August 26
On 26/08/2024 8:17 PM, Walter Bright wrote:
> I found the following inconsistency:
> 
> ```
> @safe ref int abc(int i) { return ghi(i); }
> @trusted ref int ghi(ref int r) { return r; }
> 
> @safe int* foo(int i) { return bar(&i); } // Error: reference to local variable `i` assigned to non-scope parameter `p` calling `bar`
> @trusted int* bar(int* p) { return p; }
> ```
> 
> Compiling with -dip1000 gives the error indicated. Note that it did not give an error for the semantically equivalent call to ghi(i).
> 
> Possible resolutions:
> 
> 1. give an error for both
> 
> 2. give no error for both, as @trusted implementations are the user's problem
> 
> 3. disallow taking the address of a local variable in @safe code


There is a fourth option:

Infer for ``@trusted`` functions, but don't validate.

Support describing the empty escape set separately from wanting it inferred.

If set, trust it, otherwise infer.

Will infer:

```d
@trusted ref int ghi(ref int r) { return r; }
```

Will use annotation:

```d
@trusted ref int ghi(@escapevia(return) ref int r) { return r; }
@trusted ref int ghi(@escapevia() ref int r) { return r; }
```

First and second will error, but third won't.

This appears to be an interaction between DIP1000 and the fact that ``@trusted`` functions have an interface that is required to be ``@safe`` but not its body validated.
August 26

On Sunday, 25 August 2024 at 13:10:22 UTC, Mike Parker wrote:

>

Please do not reply to posts in this thread unless you have additional relevant information regarding the problem described. We want to collect examples, and any side discussions will just be noise.

Thanks!

I am probably adding to the noise, but I think this is relevant.
I can not be the only one, that can not provide a bug report or a code snipped because I just lost interest in pursuing DIP1000 compatibility years ago.

In the end the whole thing just feels off, and is one of my major concerns about going @safe by default.

In all seriousness I am not willing to look at a function signature full of return keywords and figuring out what it means. More so, not gonna spend my energy writing such thing.

>

First, it's pretty clear that inference by default is necessary to make DIP1000 easier to use. It's not clear at the moment how this can be achieved, but it's something that we need to work out.

In my opinion, without this, there will just not be enough users.

(I written a bigger rant, but lets keep it simple)

Juraj

August 26
On 8/26/24 10:17, Walter Bright wrote:
> I found the following inconsistency:
> 
> ```
> @safe ref int abc(int i) { return ghi(i); }
> @trusted ref int ghi(ref int r) { return r; }
> 
> @safe int* foo(int i) { return bar(&i); } // Error: reference to local variable `i` assigned to non-scope parameter `p` calling `bar`
> @trusted int* bar(int* p) { return p; }
> ```
> ...

This is exactly how it should work.

What's inconsistent here is your code. The first `@trusted` function is invalid. The second `@trusted` function is fine because the parameter is not `scope`. The second function can even be `@safe`. The first memory safety bug is therefore not caught (because the bug is in the `@trusted` function), while the second one is caught (because the bug is in the `@safe` function).

> Compiling with -dip1000 gives the error indicated. Note that it did not give an error for the semantically equivalent call to ghi(i).
> 

If you want to attribute blame for a bug, not only the semantics matter, also the contracts at function interfaces matter. This is exactly why DIP1000 comes with annotations to make the intent clear and checkable modularly.

There is no analogy between raw pointers and `ref`, there is an analogy between `scope` pointers and `ref`.

(If you want to continue this discussion, we should probably take it out of the thread.)
« First   ‹ Prev
1 2 3 4