On 17.06.21 04:02, Walter Bright wrote:
> The ()@trusted{ ... }() is equally bad. Note that there are no parameters
> to it, hence NO interface. This was discovered as a way to evade the
> intent that @trusted was to be applied to a function interface. It
> had never occurred to me to add semantics to disallow it. Even worse,
> I myself have been seduced into using this trick.

It still has an interface, of course. The surrounding context acts as one large `ref` parameter. Strictly speaking, the programmer must ensure that the @trusted nested function doesn't create unsafe values in the outer function.

Everyone conveniently forgets that when writing @trusted nested functions. Just like everyone conveniently forgets that all(!) @trusted functions must have safe interfaces. But a review process that is serious about @trusted could catch those errors and force programmers to use it as intended.

However, even the standard library has more than enough instances of strictly-speaking-incorrect @trusted. So maybe it's time to give the users more features that make help with writing correct @trusted code. @system variables is one [1]. @system blocks in @trusted functions with special semantics as proposed earlier in the discussion is another.



[1] https://github.com/dlang/DIPs/blob/master/DIPs/DIP1035.md

On Thursday, 17 June 2021 at 11:16:47 UTC, Ola Fosheim Grøstad wrote:
> On Thursday, 17 June 2021 at 10:57:01 UTC, ag0aep6g wrote:
>...
>> True system level programming is going to be @system in D. I don't think that's much of a surprise.
>
> That makes Rust a much better option for people who cares about safety. That is a problem.

Actually that is the same road taken by Rust, all interop with C libraries is considered unsafe.

You can enjoy endless amount of unsafe on Microsoft code samples for Windows coding with Rust and Win32 APIs.

https://github.com/microsoft/windows-rs/tree/master/examples

On 17.06.21 13:16, Ola Fosheim Grøstad wrote:
> That basically means that all interesting system level code is @system, including all the code that calls it. That also means that you prevent system level programmers from benefiting from language safety checks!?

Yes.

> Here is the problem with that viewpoint, there is no way for the function to prove that the memory it receives has not been freed. So there is in fact no way for the function to ensure that it is @trusted. That applies to @safe functions too. There has to be a contract between caller and callee, those are the invariants that the unsafe code (and safe code) depends on.

It's not a viewpoint. It's how @system/@trusted/@safe are defined.

Part of that definition is that pointer arguments to @safe and @trusted functions must be valid (not freed). If a freed pointer ends up as the argument to an @safe/@trusted function, you have an error in your @system/@trusted code. @safe code can't produce such a pointer, because it can't call `free`.

> So I strongly disagree with the viewpoint that @trusted cannot assume invariants to hold for the data it receives. That is mandatory for all correct code of some complexity.

It can assume the invariants that are guaranteed by the language. The language guarantees (and demands) that pointer arguments are valid.

> For instance, in order to make the dmd lexer @trusted you would then require the lexer to do the allocation itself. If it accepts a filebuffer allocated outside the lexer then there is no way for the lexer to ensure that the sentinels (zeros at the end) are not overwritten by other code.

I don't know DMD's source very well, so I can't make statements about that piece of code. But it wouldn't surprise me if it can't be validly @trusted. If you provide a concrete example (that is digestible in size), I can give my take on it.

> That is an unreasonable restriction that makes @trusted and @safe useless. The lexer should be allowed to assume that the invariants of the filebuffer holds when it takes ownership of it. It is difficult to prove without language level unique ownership, but it is unreasonable to make the lexer and everything that calls it @system, just because it accepts a filebuffer object.

I don't mind you thinking that @trusted is useless. It is what it is. If you want something different, you'll have to push for change (i.e. nag Walter, write a DIP, make DMD pull requests).

Please don't mistake my insistence on the definition of @trusted as a defense of it. If @trusted falls short, then we need something better. But you can't just assert that @trusted really means something else beyond what's in the spec, something that isn't backed by Walter or DMD. That just adds to the confusion about @trusted which is already high.

>> That's also not a valid @trusted function. "It's safe as long as [some condition that's not guaranteed by the language]" describes an @system function, not an @trusted one.
> 
> What are the invariants that are guaranteed by the language in a multi-threaded program that calls C code?
> 
> What can you depend on?

@trusted functions can assume that they're only called with "safe values" and "safe aliasing" in the parameters.

For details, see the spec:

https://dlang.org/spec/function.html#safe-interfaces
https://dlang.org/spec/function.html#safe-values
https://dlang.org/spec/function.html#safe-aliasing

Note that that part of the spec is largely my attempt at pinning down what Walter means by "safe interface". There are certainly still some things missing. But the gist is there, and it has Walter's blessing.

> Is it at all possible to write a performant 3D game that isn't @system?

I don't know. I would expect that you need some @system code in there. But maybe the higher-level abstractions can be @trusted.

[...]
> But that is the signature of a very high level language, not of a system level language. In system level programming you cannot have dynamic checks all over the place, except in debug builds.

D is both high and low level. At least, it tries to be. High level: garbage collection enables code to be @safe. Low level: You can avoid garbage collection in @system code. DIP 1000 tries to make some lower-level code @safe, but it's clearly not a cure-all.

[...]
> That makes Rust a much better option for people who cares about safety. That is a problem.

I don't have an opinion on this.

On Thursday, 17 June 2021 at 10:18:57 UTC, Walter Bright wrote:

>
> I actually agree with you.

That's a miracle!

> The lambdas should be replaced with a static nested functions, so the arguments come in through the front door.

I had skipped over the other part of your post where you admitted that trusted lambdas had been an oversight. I apologize for that.

On 17.06.21 14:52, Ola Fosheim Grøstad wrote:
> When you ask for the next lexeme the lexer advances a pointer, if it hits a zero character it stops advancing.
> 
> For this to be @trusted, by the safe-requirements, the lexer cannot accept a filebuffer it has not allocated itself, as that makes it possible for external code to overwrite the zeros.

That's right. An @trusted function cannot ever advance a pointer it received from the outside. It can only assume that the pointer can be dereferenced.

[...]
> The lexer should only require that the filebuffer invariant of unique ownership and no borrowed pointers to hold. Then the lexer can add the zero-character at the end of the buffer and it will be @safe.

You're losing me. You wrote that the lexer advances a pointer to a "character". I figure that means it has a `char*` parameter. What's a filebuffer? If it's a struct around a `char*`, why is the lexer manipulating the pointer directly instead of calling some method of the filebuffer?

An example in code (instead of descriptions) would go a long way.

[...]
> I don't think this is enough to prevent @safe code from tripping up @trusted code as it would prevent many interesting ADTs from being implemented efficently. Meaning, you would have to restrict yourself to @safe practices (like bounds checks).

I'm sure many interesting types are not compatible with @safe/@trusted.

DIP 1000, Walter's @live experiments, and @system variables (DIP 1035 [1]) might enable some more. The ideas in this thread might go somewhere, too.

[...]
> Ok, but it is not _realistic_ to think that D users will not write code that they think is _good enough_ for their purpose. Since there is no way to verify that they  adhere to idealistic principles, it won't happen.

I think there's almost a consensus that @trusted isn't quite good enough, exactly because no one can be bothered to use it as intended. That's why the incremental improvements mentioned above are being worked on. I don't think anyone is working on redesigning (or reinterpreting) the whole thing from the ground up. And I would expect strong push-back from Walter if someone tried.

> So, you can get Phobos to adhere to it, but basically no other libraries will. And applications will most certainly make choices on a case-by-case evaluation.

Phobos isn't even close to adhering to it. Yes, that's a problem.


[1] https://github.com/dlang/DIPs/blob/master/DIPs/DIP1035.md