July 28, 2012
On 28-Jul-12 04:08, David Nadlinger wrote:
> Let me make something clear first: I am _not_ intending to remove
> @trusted from the language. As a bridge between the @safe and @system
> worlds, it is an integral part of SafeD. What I'm proposing is:
>
>   1) Remove the distinction between @safe and @trusted at the interface
> (ABI, API) level. This implies changing the name mangling of @trusted to
> Nf, and consequently removing the distinction in DMD altogether (at
> least in user-facing parts like .stringof and error messages). In
> theory, this is a breaking change, but as any code that doesn't treat
> them the same is buggy anyway, it shouldn't be in practice. As for
> std.traits.FunctionAttribute, we could either make trusted an alias for
> safe, or just remove documentation for the former and keep it around for
> some time (there is no way to deprecate an enum member).
>

No question here, @trusted should be usable in place of @safe _transparently_.

>   2) The first step is necessary, but mainly of cosmetic nature (think
> `pure`, `pure2`). We still need to address for the granularity and
> attribute inference problem. The obvious solution is to add a "@trusted"
> declaration/block, which would allow unsafe code in a certain region.
> Putting @trusted in the function header would still be allowed for
> backwards compatibility (but discouraged), and would have the same
> effect as marking the function @safe and wrapping its whole body in a
> @trusted block. It could e.g. look something like this (the @ prefix
> definitely looks weird, but I didn't want to introduce a new keyword):
>
> ---
>   void foo(T)(T t) {
>     t.doSomething();
>     @trusted {
>       // Do something dirty.
>     }
>     t.doSomethingElse();
>     @trusted phobosFunctionWhichHasNotBeenMarkedSafeYet();
>   }
> ---
>
> This is similar to other constructs we have today, for example debug {},
> which allows impure code. It can be debated whether a block »argument«
> should introduce a new scope or not (like static if). The latter
> currently seems much more attractive to me, but I suppose it could be
> confusing for some.
>

Finally proper granularity for trusted!
I'd say that SafeD is still unusable mostly because @trusted is too blunt (your example from std.uuid).
The day writeln works for all safe types (with safe/trusted toString or whatever) I'd call SafeD barely usable.

I believe it need not introduce another scope.



-- 
Dmitry Olshansky
July 28, 2012
On 07/28/12 02:08, David Nadlinger wrote:
> @trusted in its current form needs to go. Its design is badly broken, as it leaks implementation details and encourages writing unsafe code.

The problem with @trusted is that it is transitive.

@trusted should allow unsafe operations in the covered scope (right now - the function), but disallow calling unsafe (@system) code. IOW a @safe function should only be able to call a @safe or @trusted one, and the same restriction should apply to @trusted functions. This way you can actually audit the code - something which isn't typically possible right now, when you have no control over what the "trusted" code might call.

There are two issues w/ such a change
1. Backward compatibility
2. Being able to override the restrictions (eg for debugging)

Both can be addressed by having a trust-me-harder mode, which can be implemented as "@trusted @safe", ie functions marked with both attributes behave just as @trusted code does right now.

Yes, @safe and @trusted do not need different name mangling.

The '@attribute {}' syntax is something that is needed, but should probably wait for /proper/ attribute handling, which should also allow for a saner 'synchronized' etc.

'{}' not introducing a scope is not the most intuitive approach (yes, this includes the currently existing cases too).

artur
July 28, 2012
On 7/28/12 7:05 AM, Artur Skawina wrote:
> On 07/28/12 02:08, David Nadlinger wrote:
>> @trusted in its current form needs to go. Its design is badly broken, as it leaks implementation details and encourages writing unsafe code.
>
> The problem with @trusted is that it is transitive.
>
> @trusted should allow unsafe operations in the covered scope (right now -
> the function), but disallow calling unsafe (@system) code.

No. Trusted means "hand-checked, good to go". It can do anything.

Andrei
July 28, 2012
On 7/27/12 8:08 PM, David Nadlinger wrote:
> First, there is no point in having @trusted in the function signature.
> Why? From the perspective of the caller of the function in question,
> @safe and @trusted mean exactly the same thing. If you are not convinced
> about that, just consider that you can wrap any @trusted function into a
> @safe function to make it @safe, and vice versa.

If @trusted is not part of the signature, we can't enable e.g. analyzers that verify an entire program or package to be safe. This is not something that's currently used, but I'd hate to look back and say, "heck, I hate that we conflated @trusted with @safe!"

> One issue is that the distinction unnecessarily restricts the
> implementation in terms of interface stability. Yes, @safe and @trusted
> are equivalent from the caller's perspective, but they are mangled
> differently. This means that changing a function signature from one to
> the other is a breaking change to the ABI, and as the mangled name is
> available in the program (which is e.g. what
> std.traits.FunctionAttributes), also to the API.

I salute this quest for stability, but I don't find it the argument all that compelling. If one makes changes to a library, well, some changes will require clients to relink. Some don't. I don't see why we'd make a thing of the particular change @safe <-> @trusted. Is this often, fundamental, big,...?

> Thus, you can't just change @trusted to @safe or vice versa on the
> implementation side if you make changes code which require @trusted,
> resp. cause it to be no longer needed.

Can't parse this sentence following "if you make changes code..."

> But the much bigger problem is that @trusted doesn't play well with
> template attribute inference and makes it much too easy to accidentally
> mark a function as safe to call if it really isn't. Both things are a
> consequence of the fact that it can be applied at the function level
> only; there is no way to apply it selectively to only a part of the
> function.

This could be a more serious problem. Could you please write a brief example that shows attribute deduction messing things up? I don't understand how marking a template as @trusted is bad.

> The obvious solution is to add a "@trusted"
> declaration/block, which would allow unsafe code in a certain region.

This is sensible, but I fail to figure how it adds value over marking functions as @trusted. Sure, it's finer-grained, but it's also less structured.


Andrei
July 28, 2012
On Saturday, 28 July 2012 at 14:02:44 UTC, Andrei Alexandrescu wrote:
> If @trusted is not part of the signature, we can't enable e.g. analyzers that verify an entire program or package to be safe. This is not something that's currently used, but I'd hate to look back and say, "heck, I hate that we conflated @trusted with @safe!"

Could you elaborate on that? A @safe function is _identical_, from a client point of view, to a @trusted one. It can always call a @trusted function under the hood without the caller noticing, there is no way around that.

Thus, to be able to check that a program consists only of @safe code [1], you would need its complete source, i.e. including all the functions it can possibly invoke, to be able to check if @trusted code is called in any place. But with all the source available, you can just check the implementation for @trusted blocks [2], there is no advantage over having it in the signature.

Destroyed? :P

David


[1] Which is highly unlikely, by the way, as many parts of druntime just can't be safe.
[2] Or @trusted attributes in the function header – as described in the original post, they won't go away for backwards compatibility.
July 28, 2012
On 07/28/12 15:47, Andrei Alexandrescu wrote:
> On 7/28/12 7:05 AM, Artur Skawina wrote:
>> On 07/28/12 02:08, David Nadlinger wrote:
>>> @trusted in its current form needs to go. Its design is badly broken, as it leaks implementation details and encourages writing unsafe code.
>>
>> The problem with @trusted is that it is transitive.
>>
>> @trusted should allow unsafe operations in the covered scope (right now -
>> the function), but disallow calling unsafe (@system) code.
> 
> No. Trusted means "hand-checked, good to go". It can do anything.

Exactly, but the only way for it to mean anything is if it really /can/
be hand-checked. A "trusted" function that calls arbitrary, potentially
unsafe code cannot be trusted. You can't audit code that isn't available.
So the result is bugs (like the ones mentioned in this thread), where @safe
is bypassed, because the @trusted functions aren't expecting to be used
with "unsafe" ones. @trusted bypasses *all* safety checks, not just those
in the hand-checked code. This is something that you will want sometimes,
but in most cases is neither necessary nor desirable. When dealing with
safety one has to be conservative. The proposals to limit the scope of
@trusted only address the symptoms, not the cause. If the design is fixed,
many of the reasons for introducing the finer-grained @trusted disappear,
and it is the truly unsafe (@trusted that calls @system) code that needs
the extra annotations -- which is a good thing. Papering over design bugs
never is.

artur
July 28, 2012
On Saturday, 28 July 2012 at 14:02:44 UTC, Andrei Alexandrescu wrote:
>> But the much bigger problem is that @trusted doesn't play well with
>> template attribute inference and makes it much too easy to accidentally
>> mark a function as safe to call if it really isn't. Both things are a
>> consequence of the fact that it can be applied at the function level
>> only; there is no way to apply it selectively to only a part of the
>> function.
>
> This could be a more serious problem. Could you please write a brief example that shows attribute deduction messing things up? I don't understand how marking a template as @trusted is bad.

See the std.uuid discussion I linked in the original post for a real-world example of this bug.

The gist is: You can't ever mark a function which can end up execute code coming from a template parameter, for example a function accepting a range, as @trusted, because then you would vouch for all the passed in code as well, which might be @system. [1]

Templates parameters which just supply data are obviously not a problem.

David


[1] Unless you explicitly check whether the passed code is @safe, that is. If you go down this route, though, you need to duplicate the function declaration, which isn't pretty. See std.range.RefRange.save for an example of this.
July 28, 2012
On Saturday, 28 July 2012 at 11:05:34 UTC, Artur Skawina wrote:
> On 07/28/12 02:08, David Nadlinger wrote:
>> @trusted in its current form needs to go. Its design is badly broken, as it leaks implementation details and encourages writing unsafe code.
>
> The problem with @trusted is that it is transitive.
>
> @trusted should allow unsafe operations in the covered scope (right now -
> the function), but disallow calling unsafe (@system) code. IOW a @safe
> function should only be able to call a @safe or @trusted one, and the same
> restriction should apply to @trusted functions. This way you can actually
> audit the code - something which isn't typically possible right now, when
> you have no control over what the "trusted" code might call.

Sorry, while I think I know which problem you are referring to, I don't see what your suggestion does to address it. There is no such thing as allowing unsafe code »only in the current scope«. As soon as you are allowed to do unsafe things, you can e.g. just cast a @system function to @safe and call it. Or jump to it using inline assembly, etc.

My suggestion might not be perfect, but it addresses the problem by allowing you to trust only the specific piece of your code which actually need to perform unsafe things. Reviewers can concentrate on the smaller sections to make sure that what they do is indeed @safe (e.g. that they don't call unsafe external code), and if something unsafe is accidentally done outside, the compiler will catch it.

I also think implementing your suggestion, i.e. distinguishing between »local« and »external« code, would add much value for another reason: In my experience, the most frequent use case for @trusted is precisely to interface with external code that is @system, either for legacy reasons of because it just can't be safe in the general case (e.g. grep the Phobos sources for @trusted). It seems like the actual low-level pointer/inline asm magic is often encapsulated into separate (@system) functions anyway.

> The '@attribute {}' syntax is something that is needed, but should probabl
> wait for /proper/ attribute handling, which should also allow for a saner
> 'synchronized' etc.

What is »proper attribute handling« supposed to be? How would it influence the implementation of @trusted{}? How is synchronized related to @trusted?

David
July 28, 2012
On 7/28/12 10:34 AM, Artur Skawina wrote:
> On 07/28/12 15:47, Andrei Alexandrescu wrote:
>> On 7/28/12 7:05 AM, Artur Skawina wrote:
>>> On 07/28/12 02:08, David Nadlinger wrote:
>>>> @trusted in its current form needs to go. Its design is badly broken, as it leaks implementation details and encourages writing unsafe code.
>>>
>>> The problem with @trusted is that it is transitive.
>>>
>>> @trusted should allow unsafe operations in the covered scope (right now -
>>> the function), but disallow calling unsafe (@system) code.
>>
>> No. Trusted means "hand-checked, good to go". It can do anything.
>
> Exactly, but the only way for it to mean anything is if it really /can/
> be hand-checked.

It means someone stares at it until the goat dies.

> A "trusted" function that calls arbitrary, potentially
> unsafe code cannot be trusted.

I think you have it all wrong. Trusted means it's verified by a human, not by a formal method. The compiler allows it to do anything.

> You can't audit code that isn't available.

Correct. If you make e.g. syscalls into a closed-source OS you trust that function to not have bugs. It's a decision made by the human who annotates @trusted.

> So the result is bugs (like the ones mentioned in this thread), where @safe
> is bypassed, because the @trusted functions aren't expecting to be used
> with "unsafe" ones. @trusted bypasses *all* safety checks, not just those
> in the hand-checked code. This is something that you will want sometimes,
> but in most cases is neither necessary nor desirable. When dealing with
> safety one has to be conservative. The proposals to limit the scope of
> @trusted only address the symptoms, not the cause. If the design is fixed,
> many of the reasons for introducing the finer-grained @trusted disappear,
> and it is the truly unsafe (@trusted that calls @system) code that needs
> the extra annotations -- which is a good thing. Papering over design bugs
> never is.

I don't understand what you suggest here. Is it a sort of a refinement of @trusted?


Andrei

July 28, 2012
On 07/28/12 17:05, David Nadlinger wrote:
> On Saturday, 28 July 2012 at 11:05:34 UTC, Artur Skawina wrote:
>> On 07/28/12 02:08, David Nadlinger wrote:
>>> @trusted in its current form needs to go. Its design is badly broken, as it leaks implementation details and encourages writing unsafe code.
>>
>> The problem with @trusted is that it is transitive.
>>
>> @trusted should allow unsafe operations in the covered scope (right now - the function), but disallow calling unsafe (@system) code. IOW a @safe function should only be able to call a @safe or @trusted one, and the same restriction should apply to @trusted functions. This way you can actually audit the code - something which isn't typically possible right now, when you have no control over what the "trusted" code might call.
> 
> Sorry, while I think I know which problem you are referring to, I don't see what your suggestion does to address it. There is no such thing as allowing unsafe code »only in the current scope«. As soon as you are allowed to do unsafe things, you can e.g. just cast a @system function to @safe and call it. Or jump to it using inline assembly, etc.

Of course inside a @trusted scope you will always be able to bypass every restriction. But that does not mean that all restrictions should be disabled by default, so that you are not able to take advantage of them and have to reimplement them by hand, like checking if every called function is 'safe'.

> My suggestion might not be perfect, but it addresses the problem by allowing you to trust only the specific piece of your code which actually need to perform unsafe things. Reviewers can concentrate on the smaller sections to make sure that what they do is indeed @safe (e.g. that they don't call unsafe external code), and if something unsafe is accidentally done outside, the compiler will catch it.

Well, what you're proposing is already possible (other than the scope-lessness), your suggestion would result in a nicer syntax. Which would be an improvement, I agree. but it's not really enough.  Say somebody writes a function like this one:

   struct S { size_t m() @safe { return 42; } }

   T* t(T)(T* p, size_t idx) {
      auto s = p+idx;
      auto i = s.m();
      return p+i;
   }

then later realizes it can't be called from @safe code. The easiest solution will be to mark the 't' function is trusted, which will work.

   T* t(T)(T* p, size_t idx) @trusted {
      auto s = p+idx;
      auto i = s.m();
      return p+i;
   }

Until 't' (much) later gets called with a different struct

   struct U { size_t m()  { return 666; } }

Which will succeed and @system code will silently run in a @safe context; w/o even a warning.

What should have happened is that the function, instead of being marked as @trusted, should have been rewritten as:

   T* t(T)(T* p, size_t idx) @safe {
      @trusted gets() { return p+idx; }
      auto s = gets();
      auto i = s.m();
      @trusted getn(size_t i) { return p+i; }
      return  getn(i);
   }

But, realistically, that is not what's going to happen most of the time...

Your suggestion is an improvement, in that it makes the above look like:

   T* t(T)(T* p, size_t idx) @safe {
      @trusted { auto s = p+idx; }
      auto i = s.m();
      @trusted { return p+i; }
   }

which is much better. But I'm afraid marking the whole function as trusted
will still be the 'easier' choice - so the problem won't go away, just become
less frequent.
Accidentally 'breaking' @safeness is a serious problem, at least if @safe is to
be taken seriously.

> I also think implementing your suggestion, i.e. distinguishing between »local« and »external« code, would add much value for another reason: In my experience, the most frequent use case for @trusted is precisely to interface with external code that is @system, either for legacy reasons of because it just can't be safe in the general case (e.g. grep the Phobos sources for @trusted). It seems like the actual low-level pointer/inline asm magic is often encapsulated into separate (@system) functions anyway.
> 
>> The '@attribute {}' syntax is something that is needed, but should probabl wait for /proper/ attribute handling, which should also allow for a saner 'synchronized' etc.
> 
> What is »proper attribute handling« supposed to be? How would it influence the implementation of @trusted{}? How is synchronized related to @trusted?

There have been some discussions about attributes in the past on these lists. I have some thoughts on »proper attribute handling«, but that's a different topic. What I mean here is that the /syntax/ could also be used for marking scopes/blocks with other attributes besides @trusted; synchronized is an example that would also fit in the '@attr {...}' scheme and is interesting because that also needs an optional declaration block.

artur