On Friday, 6 February 2015 at 17:13:09 UTC, Zach the Mystic wrote:
> It's been suggested that '@system' be used to mark the blocks in question. The point would be to emphasize the dangerousness of the operation. The function is still @trusted, but inside, the @system code is marked as such.

This is precisely equivalent to just making the function @safe and using @trusted blocks inside (which is incidentally my preferred approach, and similar to what parts of Phobos try to emulate).

As I argued almost three years ago in my "@trusted considered harmful" post, @trusted doesn't differ in meaning from @safe for API clients. Both mean that you can call the function from @safe code, nothing more, nothing less. I hope we agree on that.

You might argue that having @trusted in the function declaration serves as an indicator that the implementation requires extra careful review. However, there will *still* be a @trusted keyword sitting in the source code and staring at your face even with @trusted blocks. And if you don't have the source code for a function you call to begin with, all bets are off anyway (i.e. you need to trust its author) as @safe functions may always internally call @trusted ones.

If the leadership feels that your proposal is to be preferred because it keeps @trusted in the function signature (as he correctly remarked, the safety of @trusted blocks might depend on surrounding @safe code), then so be it. However, it makes an implementation detail part of the API (see "@trusted considered harmful"), and furthermore is not backwards-compatible.

David

On 2/6/15 12:36 PM, Atila Neves wrote:
>> I'm trying to promote suggesting '@system' blocks instead of
>> '@trusted'. '@trusted' functions, but '@system' blocks - which can
>> only go in @trusted functions (@system block in @system functions are
>> redundant). It's the same semantics, but it might win the day because
>> the intent is to isolate the @system code, while still presenting a
>> @trusted interface, as seems so important to the leadership.
>
> That might be better than using @safe inside @trusted:
>
> @trusted void myfunc() {
> //implicitly safe
> ....
> @system { //wouldn't compile otherwise.
>     auto ptr = cast(ubyte*)4;
> }
>
> //implicitly safe again
> }

BTW, this is H.S. Teoh's suggestion too, and I like it better than mine after much thought.

-Steve

On Fri, Feb 06, 2015 at 10:52:45AM -0800, Andrei Alexandrescu via Digitalmars-d wrote:
> On 2/6/15 10:42 AM, David Nadlinger wrote:
> >On Friday, 6 February 2015 at 18:39:28 UTC, Andrei Alexandrescu wrote:
> >>It's clear. I just don't think it's a good point. -- Andrei
> >
> >I'm not making a point; I'm posing a problem. What is your solution?
> 
> I think the problem is overstated. -- Andrei

This is precisely why I have lost all interest in @safe. It's clear that the present problematic situation will continue to hold, and the decision makers are not interested to address it. I am not going to waste any more time and energy on this topic.


T

-- 
Question authority. Don't ask why, just do it.

On 2/6/15 11:11 AM, H. S. Teoh via Digitalmars-d wrote:
> On Fri, Feb 06, 2015 at 10:52:45AM -0800, Andrei Alexandrescu via Digitalmars-d wrote:
>> On 2/6/15 10:42 AM, David Nadlinger wrote:
>>> On Friday, 6 February 2015 at 18:39:28 UTC, Andrei Alexandrescu wrote:
>>>> It's clear. I just don't think it's a good point. -- Andrei
>>>
>>> I'm not making a point; I'm posing a problem. What is your solution?
>>
>> I think the problem is overstated. -- Andrei
>
> This is precisely why I have lost all interest in @safe. It's clear that
> the present problematic situation will continue to hold, and the
> decision makers are not interested to address it. I am not going to
> waste any more time and energy on this topic.

I've asked repeatedly for evidence of the "problematic situation", and all I got was doomsday predictions "maintenance nightmare!". If you have such, please show it. If not, thanks for a good course of action. -- Andrei

On Friday, 6 February 2015 at 18:58:27 UTC, David Nadlinger wrote:
> On Friday, 6 February 2015 at 17:13:09 UTC, Zach the Mystic wrote:
>> It's been suggested that '@system' be used to mark the blocks in question. The point would be to emphasize the dangerousness of the operation. The function is still @trusted, but inside, the @system code is marked as such.
>
> This is precisely equivalent to just making the function @safe and using @trusted blocks inside (which is incidentally my preferred approach, and similar to what parts of Phobos try to emulate).

No it's not. Forcing @trusted functions, with @system blocks inside builds a redundancy into the system. Now someone from outside the can read the signature and know there might be a problem, and someone inside can find everywhere that problem could possibly have arisen from. The difference is you now have two signals instead of one - one in the function header, and the other(s) in the code itself.

> As I argued almost three years ago in my "@trusted considered harmful" post, @trusted doesn't differ in meaning from @safe for API clients. Both mean that you can call the function from @safe code, nothing more, nothing less. I hope we agree on that.

I know. This was recently pointed out again by Steven Schveighoffer, and it's a great insight.

> You might argue that having @trusted in the function declaration serves as an indicator that the implementation requires extra careful review. However, there will *still* be a @trusted keyword sitting in the source code and staring at your face even with @trusted blocks. And if you don't have the source code for a function you call to begin with, all bets are off anyway (i.e. you need to trust its author) as @safe functions may always internally call @trusted ones.

It is a redundancy, yes, but one people might actually *appreciate*. People around here take memory safety very seriously, and probably with good reason!

> If the leadership feels that your proposal is to be preferred because it keeps @trusted in the function signature (as he correctly remarked, the safety of @trusted blocks might depend on surrounding @safe code), then so be it. However, it makes an implementation detail part of the API (see "@trusted considered harmful"), and furthermore is not backwards-compatible.

It's not backward-compatible, but it's not hard to fix at all. You get an error, and you enclose your unsafe code in a @system block. And the quality of your code improves at the same time.

The only reason the implementation detail, @trusted, was made part of the API was precisely because memory safety is taken extremely seriously by the language designers. There's no other reason I can think of, but for some reason, I can appreciate the thought. Also, I want to find a real solution and I doubt anyone is suggesting getting rid of @trusted at this point.

On Friday, 6 February 2015 at 19:16:13 UTC, Andrei Alexandrescu wrote:
>> This is precisely why I have lost all interest in @safe. It's clear that
>> the present problematic situation will continue to hold, and the
>> decision makers are not interested to address it. I am not going to
>> waste any more time and energy on this topic.
>
> I've asked repeatedly for evidence of the "problematic situation", and all I got was doomsday predictions "maintenance nightmare!". If you have such, please show it. If not, thanks for a good course of action. -- Andrei

The best evidence I have it that everyone who actually worked on the phobos code you disapprove of says it's a problem. Argumentum ad populem, I know, but it calls for close scrutiny, to say the least.

My attitude is not based on evidence. It's based on just thinking about the problem:

http://forum.dlang.org/post/eeglnychgudcffpjcdvy@forum.dlang.org

I really can't answer this question.

On 2/6/15 2:16 PM, Andrei Alexandrescu wrote:
> On 2/6/15 11:11 AM, H. S. Teoh via Digitalmars-d wrote:
>> On Fri, Feb 06, 2015 at 10:52:45AM -0800, Andrei Alexandrescu via
>> Digitalmars-d wrote:
>>> On 2/6/15 10:42 AM, David Nadlinger wrote:
>>>> On Friday, 6 February 2015 at 18:39:28 UTC, Andrei Alexandrescu wrote:
>>>>> It's clear. I just don't think it's a good point. -- Andrei
>>>>
>>>> I'm not making a point; I'm posing a problem. What is your solution?
>>>
>>> I think the problem is overstated. -- Andrei
>>
>> This is precisely why I have lost all interest in @safe. It's clear that
>> the present problematic situation will continue to hold, and the
>> decision makers are not interested to address it. I am not going to
>> waste any more time and energy on this topic.
>
> I've asked repeatedly for evidence of the "problematic situation", and
> all I got was doomsday predictions "maintenance nightmare!". If you have
> such, please show it. If not, thanks for a good course of action. -- Andrei

I think your strawman is overstated. The "doomsday" is the current situation to which you and Walter have objected. If you think having "better discipline" in reviews is going to fix it, I guess we will have to wait and see what the evidence eventually does show. There isn't evidence that either solution has worked, because neither has been employed yet.

Logically, it makes sense to me that we should adjust how @trusted operates to prevent preventable problems that you have identified. But we can just keep the status quo and rely on manual process improvements instead.

It's not terribly important to fix it right now, we can try your way first, I don't see how adjusting the meaning of @trusted in the future would be any more disruptive than it would be now.

If this is how it is to be, can we get some guidelines as to what should and should not pass review for @trusted?

-Steve

On 2/6/15 11:29 AM, Zach the Mystic wrote:
> The best evidence I have it that everyone who actually worked on the
> phobos code you disapprove of says it's a problem.

I see more like two. -- Andrei

On 2/6/15 11:53 AM, Steven Schveighoffer wrote:
>
> I think your strawman is overstated. The "doomsday" is the current
> situation to which you and Walter have objected.

I see that just as: code in poor style made its way in Phobos. It doesn't improve anything (e.g. didn't find bugs in std.file.read) and is just a net negative resulting from the corrupted use of a feature.

There's no completely automated protection against poor style.

> If you think having
> "better discipline" in reviews is going to fix it, I guess we will have
> to wait and see what the evidence eventually does show. There isn't
> evidence that either solution has worked, because neither has been
> employed yet.
>
> Logically, it makes sense to me that we should adjust how @trusted
> operates to prevent preventable problems that you have identified. But
> we can just keep the status quo and rely on manual process improvements
> instead.
>
> It's not terribly important to fix it right now, we can try your way
> first, I don't see how adjusting the meaning of @trusted in the future
> would be any more disruptive than it would be now.
>
> If this is how it is to be, can we get some guidelines as to what should
> and should not pass review for @trusted?

Wise words. Walter has a PR on docs, you may want to review it: https://github.com/D-Programming-Language/dlang.org/pull/890


Andrei

On Friday, 6 February 2015 at 18:51:34 UTC, Steven Schveighoffer wrote:
> My point was that if you have @trusted escapes inside a function, whether it's marked @safe or not, you still have to review the whole function. If the compiler disallowed this outright, then you don't have that issue.

Ok, I also prefer that @trusted only apply to whole functions. I don't agree with the argument, but let's leave it at that ;-).

> Separating the trusted code from the safe code via an API barrier has merits when it comes to code review.

Yes. And I actually don't think @trusted functions in Phobos should contain version() or other forms for conditional compilation since that makes it much harder to reason about.

> I see the point now that making sure @safe functions don't have escapes has the advantage of not requiring *as much* review as a @system or @trusted function. I am leaning so much towards H.S. Teoh's solution of making @trusted safe by default, and allowing escapes into @system code. That seems like the right abstraction.

Just to make sure that I got this right:

I don't really understand why you need to escape to @system from @trusted. Isn't @trusted the same as @system but with a seal that says that it has been manually verified to be memory safe?  @system simply allows the same internal semantics as @trusted but with no such declared guarantee to the caller?

Except that in @system you could potentially switch the stacks and do other unsafe operations that are not brought back to normal before leaving the system context... In @trusted you are required to restore the context to normal before returning.

So in the type system you only have @safe and @system, @trusted is just @safe with flexible manual verification rather than the limited automated verification DMD is capable of. Thus you only need to export @safe vs @system for separate compilation...?

Isn't that how it is supposed to work already?

> I see what you mean, but there are also really dumb things that people miss that a compiler won't. Having a mechanical set of eyes in addition to human eyes is still more eyes ;)

Well, if you can come up with something sound. The way I see it, @trusted functions are allowed to create a new context on entry as long as it restores the previous context before exiting.

This essentially means that what is @safe before entering @trusted no longer can be guaranteed to be safe. The floating point unit might work differently and result in memory unsafe operations etc etc.

So calling @safe from @trusted means that you are calling @safe as if it was @system. And therefore @safe code called from @trusted has to be proven correct just like @system code called from @trusted.

> The way reviews are done isn't anything the language can require. Certainly we can provide guidelines, and we can require such review processes for phobos and druntime.

Yes, in Phobos you need to impose additional stricter guarantees in order to support the formal review process and ensure that the formal review cannot be broken without a new review taking place. And yes, that is a process requirement, not a language requirement. A different process might impose other requirements...

I think you will have to define such a process though, because I don't think there is a solution for fully automated verification for D without going for a much more complex type system and mechanics (which I actually think is out of reach unless everything is designed around it).

Fortunately with 3 reviewers you can do quite well if the proof is available and they work independently without sharing results before everyone is done. If each reviewer has a 10% chance of failure then you end up with only 0.1% chance of all of them failing... So it is possible to get decent results with a formal process in place.

Complicated @trusted code (hard to prove safe) should be rejected and unnecessary @trusted code should be fixed in the compiler optimizer or by adding language features (like SIMD).