On 2/5/15 2:17 PM, H. S. Teoh via Digitalmars-d wrote:

> I mostly like this idea, except that foo() should not be marked @safe.
> It should be marked @trusted because it still needs review, but the
> meaning of @trusted should be changed so that it still enforces
> @safe-ty, except that now @trusted variables are permitted. Or rather, I
> would call them @system variables -- they *cannot* be trusted, and must
> be manually verified.

I still am not sure I get this idea. If @safe code can call @trusted code, how is it any more mechanically verified than @trusted code?

API-wise, there is no difference. The whole idea of @trusted is that you don't need to go read the implementation, you have to trust the person writing it. As a user of @trusted function, I really don't care whether it's marked @safe or @trusted. And I shouldn't be able to break my @safe function by calling it.

> @safe code should not allow any @system variables or any cast(@safe)
> operations, period. Otherwise, anybody can wrap unsafe operations inside
> their @safe function and still clothe it with the sheep's clothing of
> @safe, and @safe becomes a meaningless annotation.

The only way to fix this is to ban @trusted altogether.

Which makes @safe quite useless indeed.

-Steve

On Thursday, 5 February 2015 at 19:19:51 UTC, H. S. Teoh wrote:
> - @safe should continue being @safe -- no (potentially) unsafe
>   operations are allowed, period.
> 
>   Rationale: allowing @system variables in @safe code makes the function
>   non-verifiable mechanically. This completely breaks the whole point of
>   @safe.
>
> - Change the meaning of @trusted (as applied to a function) to require
>   @safe inside the function body, but in addition permit @system
>   variables and cast(@safe).
> 
>   Rationale: the function cannot be verified mechanically to be safe,
>   therefore it cannot be marked @safe. It must be marked @trusted to
>   draw attention to the fact that manual review is required. However,
>   this does not constitute license to perform arbitrary @system
>   operations. Instead, any @system code/variable inside the @trusted
>   function must be explicitly marked as such, to indicate that these
>   items require special attention during review. Everything else must
>   still conform to @safe requirements.
>
> - Introduce @system variables for holding tainted values that the
>   compiler cannot guarantee the safety of, as well as cast(@safe), as
>   described in Steven's post. These constructs are only permitted inside
>   @trusted functions. They are prohibited in @safe code, and are no-ops
>   in @system code.
>
>   Rationale: to reduce the maintainability problem, @trusted functions
>   should not allow @system code by default. Rather, the scope of @system
>   code/data inside a @trusted function should be restricted by requiring
>   explicit marking. The compiler then helps the verification process by
>   ensuring that anything not explicitly marked is still @safe.
>
>
> T

This is another interesting addition to the original proposal. My initial response is that if I were building the system from the ground up, I might not understand why a function needs to be redundantly marked @trusted, since the unsafe data is already casted(@safe) inside the function. Yet two factors start to sway me in favor of what you're saying, despite my initial doubt:

1. Memory safety is serious business. Forcing people to mark potentially unsafe functions @trusted instead of @safe, while (as Steven Schveighoffer pointed out) actually meaning nothing to the caller, will allow maintainers to more easily detect possible problems. In other words, it's a redundancy, but it's a GOOD redundancy because of how serious safety issues are. I don't know how strong this argument is, but it seems sound and I'm willing to go with it, barring good reasons not to.

2. It's already there in the language. This heavily reduces the "switching costs". While not a great argument in itself, to my mind anyway (#pleasebreakourcode ;-), it does seem more likely to be accepted by people who fear the change.

On Thu, Feb 05, 2015 at 11:49:40AM -0800, Andrei Alexandrescu via Digitalmars-d wrote:
> On 2/5/15 11:17 AM, H. S. Teoh via Digitalmars-d wrote:
> >In short, my proposal is:
> 
> Tainted variables are an interesting topic, but quite distinct from the notion of separating safe code from unsafe code.
> 
> As much as I was shocked about the use of @trusted/@safe/@system in std.file, std.array and sadly possibly in other places, I found no evidence that the feature is misdesigned. I continue to consider it a simple, sound, and very effective method of building and interfacing robust code. An excellent engineering solution that offers a lot of power at a modest cost.

Frankly, I don't understand how you could fail to see the maintenance nightmare @trusted, as you envisioned it, is causing. The wholesale license to freely perform @system operations inside a @trusted function greatly undermines the trustworthiness of SafeD as a whole. The problem is that human verification only happens once, and thereafter @trusted continues to be applied to that function, even if later on, further downstream in the call graph, a remote helper function has changed in a way that compromises that trustworthiness. This happens with no warnings whatsoever from the compiler, and so we continue to build our house upon sinking sand.

To truly be sure that our @trusted functions are actually trusted, we have to continually review them not just for changes within the function bodies, but also all downstream functions used within. The review requirements for maintaining such a standard is impractically onerous. And, judging from the trends in our industry, what will likely happen is that this review is never actually carried out until a nasty security exploit is discovered in @safe code. By then, it will be far too late. Maybe to you this situation is acceptable, but to me, this is an utter maintenance nightmare.


> I do not support this proposal to change the semantics of @trusted/@safe/@system. A separate tainted data proposal might be of interest for loosely related topics.
[...]

That is your own prerogative, but this discussion has convinced me that @safe is a joke, and your continual refusal to admit the existence of any problems despite many active Phobos contributors describing it to you in detail has made me completely lose interest in this subject. From now on I will not bother with @safe in my projects anymore, since it has become clear that it does not live up to its promise and probably never will (this specific issue is only the tip of the iceberg; there are many other problems with @safe that you may find in bugzilla), and I have no further interest in contributing to @safe-related issues in Phobos.


T

-- 
Mediocrity has been pushed to extremes.

On 2/5/2015 11:49 AM, Andrei Alexandrescu wrote:
> As much as I was shocked about the use of @trusted/@safe/@system in std.file,
> std.array and sadly possibly in other places, I found no evidence that the
> feature is misdesigned. I continue to consider it a simple, sound, and very
> effective method of building and interfacing robust code. An excellent
> engineering solution that offers a lot of power at a modest cost.
>
> I do not support this proposal to change the semantics of
> @trusted/@safe/@system.

I agree.

So the question is, what does @trusted actually buy you, since the compiler can't check it?

It serves as notice that "This function merits special attention during code review to check that it has a safe interface and that its implementation is correct."

On 2/5/15 2:43 PM, H. S. Teoh via Digitalmars-d wrote:

> The idea is that while we would like the compiler to mechanically verify
> *everything*, in practice there are some things that the compiler simply
> cannot verify. Since those remaining things require human effort to
> verify and humans are prone to errors, we would like to limit the scope
> of those things by confining them inside @trusted functions, which,
> ideally, would be few in number and limited in scope. Everything else
> should be relegated to @safe functions, where we *require* completely
> automated verification by the compiler.

What's the difference between an internal scope and a separate function scope? That is, a static internal function can simply be a private module function and have the same effect.

I don't see how your proposal is more safe than mine, or that somehow I can expect a @safe function never to have manually verified code that it uses.

-Steve

On 2/5/15 2:49 PM, Andrei Alexandrescu wrote:
> On 2/5/15 11:17 AM, H. S. Teoh via Digitalmars-d wrote:
>> In short, my proposal is:
>
> Tainted variables are an interesting topic, but quite distinct from the
> notion of separating safe code from unsafe code.
>
> As much as I was shocked about the use of @trusted/@safe/@system in
> std.file, std.array and sadly possibly in other places, I found no
> evidence that the feature is misdesigned. I continue to consider it a
> simple, sound, and very effective method of building and interfacing
> robust code. An excellent engineering solution that offers a lot of
> power at a modest cost.
>
> I do not support this proposal to change the semantics of
> @trusted/@safe/@system. A separate tainted data proposal might be of
> interest for loosely related topics.

The proposal (the original one I stated, not H.S.'s) is to do 2 things:

1. Clean up the syntax for @trusted escapes inside @safe code that we have settled on.
2. Add a mechanism to make those escapes safer and more reviewable.

I don't think the idea behind @trusted is incorrect, just that the idea it's a function attribute is mis-designed.

Note that in my proposal, you can essentially create a @trusted function just by marking the whole thing @trusted:

-Steve

On Thursday, 5 February 2015 at 19:49:41 UTC, Andrei Alexandrescu wrote:
> On 2/5/15 11:17 AM, H. S. Teoh via Digitalmars-d wrote:
>> In short, my proposal is:
>
> Tainted variables are an interesting topic, but quite distinct from the notion of separating safe code from unsafe code.
>
> As much as I was shocked about the use of @trusted/@safe/@system in std.file, std.array and sadly possibly in other places, I found no evidence that the feature is misdesigned. I continue to consider it a simple, sound, and very effective method of building and interfacing robust code. An excellent engineering solution that offers a lot of power at a modest cost.
>
> I do not support this proposal to change the semantics of @trusted/@safe/@system. A separate tainted data proposal might be of interest for loosely related topics.
>
>
> Andrei

At minimum, there needs to be official documented guidance on how to use @trusted. If phobos developers got this far without knowing how to use it (assuming their complaints on its design are indeed meritless), how can anyone else be expected to?

On 2/5/15 3:13 PM, Walter Bright wrote:
> On 2/5/2015 11:49 AM, Andrei Alexandrescu wrote:
>> As much as I was shocked about the use of @trusted/@safe/@system in
>> std.file,
>> std.array and sadly possibly in other places, I found no evidence that
>> the
>> feature is misdesigned. I continue to consider it a simple, sound, and
>> very
>> effective method of building and interfacing robust code. An excellent
>> engineering solution that offers a lot of power at a modest cost.
>>
>> I do not support this proposal to change the semantics of
>> @trusted/@safe/@system.
>
> I agree.
>
> So the question is, what does @trusted actually buy you, since the
> compiler can't check it?
>
> It serves as notice that "This function merits special attention during
> code review to check that it has a safe interface and that its
> implementation is correct."
>

That also applies to @safe functions since they can call @trusted functions. In essense, @trusted buys you headaches. I think we should try to lessen them.

-Steve

On Thu, Feb 05, 2015 at 03:14:18PM -0500, Steven Schveighoffer via Digitalmars-d wrote:
> On 2/5/15 2:43 PM, H. S. Teoh via Digitalmars-d wrote:
> 
> >The idea is that while we would like the compiler to mechanically verify *everything*, in practice there are some things that the compiler simply cannot verify. Since those remaining things require human effort to verify and humans are prone to errors, we would like to limit the scope of those things by confining them inside @trusted functions, which, ideally, would be few in number and limited in scope. Everything else should be relegated to @safe functions, where we *require* completely automated verification by the compiler.
> 
> What's the difference between an internal scope and a separate function scope? That is, a static internal function can simply be a private module function and have the same effect.
> 
> I don't see how your proposal is more safe than mine, or that somehow I can expect a @safe function never to have manually verified code that it uses.
[...]

It's as Walter just said: @safe means the compiler has mechanically verified it, @trusted means the compiler has *not* verified it but that a human did (or so we hope). If you like, think of it as @safe-compiler-verified vs. @safe-human-verified. By segregating the two, you limit the scope of code that needs to be reviewed.  Of course, this is only of interest to the maintainer of the code, really, to the user both sport a @safe API and there is no distinction.

In any case, it doesn't look like anything is going to change after all, so this discussion has is just another of those what-could-have-beens rather than what could be.


T

-- 
Klein bottle for rent ... inquire within. -- Stephen Mulraney

On 2/5/15 3:23 PM, H. S. Teoh via Digitalmars-d wrote:
> On Thu, Feb 05, 2015 at 03:14:18PM -0500, Steven Schveighoffer via Digitalmars-d wrote:
>> On 2/5/15 2:43 PM, H. S. Teoh via Digitalmars-d wrote:
>>
>>> The idea is that while we would like the compiler to mechanically
>>> verify *everything*, in practice there are some things that the
>>> compiler simply cannot verify. Since those remaining things require
>>> human effort to verify and humans are prone to errors, we would like
>>> to limit the scope of those things by confining them inside @trusted
>>> functions, which, ideally, would be few in number and limited in
>>> scope. Everything else should be relegated to @safe functions, where
>>> we *require* completely automated verification by the compiler.
>>
>> What's the difference between an internal scope and a separate
>> function scope? That is, a static internal function can simply be a
>> private module function and have the same effect.
>>
>> I don't see how your proposal is more safe than mine, or that somehow
>> I can expect a @safe function never to have manually verified code
>> that it uses.
> [...]
>
> It's as Walter just said: @safe means the compiler has mechanically
> verified it, @trusted means the compiler has *not* verified it but that
> a human did (or so we hope). If you like, think of it as
> @safe-compiler-verified vs. @safe-human-verified. By segregating the
> two, you limit the scope of code that needs to be reviewed.  Of course,
> this is only of interest to the maintainer of the code, really, to the
> user both sport a @safe API and there is no distinction.

I'll put out a strawman similar to my example response to Zach:

@trusted int[] tmalloc(size_t x) { ... }
@trusted void tfree(int[] x) { ... }

Now, let's say these are in some module you use, and your code is:

void foo() @safe
{
   auto x = tmalloc(100);
   tfree(x);

   ...

   x[0] = 1;
}

foo is "mechanically verified", but it's not really, because tmalloc and tfree are not. Now, you may just trust that tfree is fine, you may go and verify what tfree does. But in either case, you still have the problem that tfree(x) and the usage of x may be far away from each other, and may even be written by different people at different times. The compiler will still fail you in this regard, because it will not complain.

Understand that I don't disagree with your proposal, I just think it can be reduced to mine, and is unnecessarily complicated.

I think the *fundamental* problem with @trusted (currently) is that it assumes all the code it covers was written simultaneously and is not allowed to morph. This isn't the way code is written, it's massaged and tweaked over long periods of time by different people.

> In any case, it doesn't look like anything is going to change after all,
> so this discussion has is just another of those what-could-have-beens
> rather than what could be.

Don't give up so easily ;)

-Steve