View mode: basic / threaded / horizontal-split · Log in · Help
July 28, 2012
Re: @trusted considered harmful
On 07/28/12 18:16, Andrei Alexandrescu wrote:
> On 7/28/12 10:34 AM, Artur Skawina wrote:
>> So the result is bugs (like the ones mentioned in this thread), where @safe
>> is bypassed, because the @trusted functions aren't expecting to be used
>> with "unsafe" ones. @trusted bypasses *all* safety checks, not just those
>> in the hand-checked code. This is something that you will want sometimes,
>> but in most cases is neither necessary nor desirable. When dealing with
>> safety one has to be conservative. The proposals to limit the scope of
>> @trusted only address the symptoms, not the cause. If the design is fixed,
>> many of the reasons for introducing the finer-grained @trusted disappear,
>> and it is the truly unsafe (@trusted that calls @system) code that needs
>> the extra annotations -- which is a good thing. Papering over design bugs
>> never is.
> 
> I don't understand what you suggest here. Is it a sort of a refinement of @trusted?

Maybe my other response to David makes things more clear. 

Yes, there would be a cost to this change (some lib and user code would need
additional annotations), but I think it'd be worth it, as right now it is much
too easy to 'break' the @safe guarantees, often w/o even realizing it.

artur
July 28, 2012
Re: @trusted considered harmful
On Saturday, 28 July 2012 at 02:33:54 UTC, Jonathan M Davis wrote:
> On Saturday, July 28, 2012 04:05:14 Jesse Phillips wrote:
>> You aren't supposed to do dirty things in @trusted code. You're
>> supposed to  safely wrap a system function to be usable by a 
>> safe
>> function. The system function is supposed to be short and 
>> getting
>> its hands dirty. Remember this is about memory safety and not
>> lack of bugs safety.
>
> It's perfectly acceptable to put "dirty code" in an @trusted
> function. The difference between @trusted and @system is that 
> with @trusted,
> the programmer is guaranteeing that it's actually @safe 
> regardless of what
> arguments it's given, whereas @system is not only doing unsafe 
> things, but
> whether they're ultimately @safe or not depends on arguments 
> and/or other
> code, so the programmer _can't_ guarantee that it's @safe.

I realize you can, David Piepgrass was able to read into an idea 
which I didn't want to specifically state as it isn't as 
concerning. But why not limit @trusted to only do @safe and call 
@trusted. But I could see that as an extra layer of annoying, but 
it would keep the unsafe arithmetic and parameter passing in the 
@system functions. But again, not much thought on it and probably 
just be a layer of annoying.

For your example of the proposal, when does the compiler mark 
save() as @system over @trusted?

@property auto save()
{
   import std.conv;
   alias typeof((*_range).save) S;
   static assert(isForwardRange!S, S.stringof ~ " is not a 
forward range.");

   @trusted
   {
       auto mem = new void[S.sizeof];`
       emplace!S(mem, cast(S)(*_range).save);`
       return RefRange!S(cast(S*)mem.ptr);
   }
}

You make a call to _range.save in the @trusted block. So why 
isn't the function marked @trusted?

Having taken some more thought on the guarantee provided by @safe 
even when calling @trusted code, I do see the issue of templates 
always being trusted. As it could then be the logic in of the 
@safe which is doing something unsafe. This could be true with 
@trusted but at least the author had the opportunity to make it 
safe.

But at this point I'm not seeing how this proposal is helping. 
You do want save() to be @system when the underlining save() is 
@system, and @trusted otherwise? So I'm not seeing that here.
July 28, 2012
Re: @trusted considered harmful
On Saturday, 28 July 2012 at 02:33:54 UTC, Jonathan M Davis wrote:
> @property auto save()
> {
>     import std.conv;
>     alias typeof((*_range).save) S;
>     static assert(isForwardRange!S, S.stringof ~ " is not a 
> forward range.");
>
>     @trusted
>     {
>         auto mem = new void[S.sizeof];`
>         emplace!S(mem, cast(S)(*_range).save);`
>         return RefRange!S(cast(S*)mem.ptr);
>     }
> }
>
> That is _way_ cleaner.

But unfortunately wrong – you call S.save in the @trusted 
block… ;)

David
July 28, 2012
Re: @trusted considered harmful
On Saturday, July 28, 2012 22:08:42 David Nadlinger wrote:
> On Saturday, 28 July 2012 at 02:33:54 UTC, Jonathan M Davis wrote:
> > @property auto save()
> > {
> > 
> >     import std.conv;
> >     alias typeof((*_range).save) S;
> >     static assert(isForwardRange!S, S.stringof ~ " is not a
> > 
> > forward range.");
> > 
> >     @trusted
> >     {
> >     
> >         auto mem = new void[S.sizeof];`
> >         emplace!S(mem, cast(S)(*_range).save);`
> >         return RefRange!S(cast(S*)mem.ptr);
> >     
> >     }
> > 
> > }
> > 
> > That is _way_ cleaner.
> 
> But unfortunately wrong – you call S.save in the @trusted
> block… ;)

Yeah. I screwed that up. I was obviously in too much of a hurry when I wrote 
it. And actually, in this particular case, since the part that can't be 
@trusted is in the middle of an expression doing @system stuff, simply using an 
@trusted block wouldn't do the trick. The basic principle is still good 
though. Being able to mark specific statements or sections of code @trusted 
would greatly simplify templated functions which are making function calls 
which _can't_ necessarily be @trusted.

- Jonathan M Davis
July 28, 2012
Re: @trusted considered harmful
On Saturday, July 28, 2012 21:53:25 Jesse Phillips wrote:
> For your example of the proposal, when does the compiler mark
> save() as @system over @trusted?
> 
> @property auto save()
> {
>     import std.conv;
>     alias typeof((*_range).save) S;
>     static assert(isForwardRange!S, S.stringof ~ " is not a
> forward range.");
> 
>     @trusted
>     {
>         auto mem = new void[S.sizeof];`
>         emplace!S(mem, cast(S)(*_range).save);`
>         return RefRange!S(cast(S*)mem.ptr);
>     }
> }
> 
> You make a call to _range.save in the @trusted block. So why
> isn't the function marked @trusted?

I screwed up my changes there. I made them in a bit of a hurry. And actually, 
in this particular case, using an @trusted block is still a bit entertaining, 
since the save call is in middle of an expression which is doing @system stuff. 
But the principle is the same regardless. If you need to do @system stuff in a 
function where you know that the @system stuff that you're doing is @safe but 
the function is also calling templated stuff which may or may not be @safe, you 
can't mark the whole function as @trusted, or you'll be marking code which is 
potentially truly unsafe as @trusted. With an @trusted block, you can mark 
sections of the code @trusted and let the rest be properly inferred based on 
what the the template was instantiated with.

In this particular case, it could still drastically reduce save, because only 
the line with @save would need to be duplicated. So, it could become something  
more like

@property auto save()
{
   import std.conv;
   alias typeof((*_range).save) S;
   static assert(isForwardRange!S, S.stringof ~ " is not a forward range.");

   @trusted {auto mem = new void[S.sizeof];}

   static if(isSafelyCallable!((){(*_range).save;}))
   {
       @trusted { emplace!S(mem, cast(S)(*_range).save); }
   }
   else
   {
       emplace!S(mem, cast(S)(*_range).save);
   }

   @trusted {return RefRange!S(cast(S*)mem.ptr);}
}

It's still a bit ugly, but it's far better than what we can currently do. As 
it stands, you can't even put the emplace call in a separate function and mark 
it @trusted or @system depending on save, because the only way to mark the 
rest of the function as @trusted is to mark the whole thing as @trusted. You'd 
have to specifically put _everything else_ in its own function _and_ emplace in 
its own function (one @trusted, one @system) and have the outer function call 
both. It's a mess. Being able to mark statements as @trusted rather than just 
entire functons would be _huge_.

- Jonathan M Davis
July 28, 2012
Re: @trusted considered harmful
On Saturday, July 28, 2012 10:02:43 Andrei Alexandrescu wrote:
> On 7/27/12 8:08 PM, David Nadlinger wrote:
> > First, there is no point in having @trusted in the function signature.
> > Why? From the perspective of the caller of the function in question,
> > @safe and @trusted mean exactly the same thing. If you are not convinced
> > about that, just consider that you can wrap any @trusted function into a
> > @safe function to make it @safe, and vice versa.
> 
> If @trusted is not part of the signature, we can't enable e.g. analyzers
> that verify an entire program or package to be safe. This is not
> something that's currently used, but I'd hate to look back and say,
> "heck, I hate that we conflated @trusted with @safe!"

But from the caller's perspective, @safe and @trusted are identical. And if 
you want a function to be @safe rather than @trusted, all you have to do is 
create an @trusted helper function which has the implementation and have the 
actual function call it, and the the function can be @safe. I don't see how it 
makes any difference at all whether a function is marked @safe or @trusted 
except for the fact that with @trusted, the implementation can _directly_ use 
@system stuff, whereas with @safe, there has to be another layer in between. 
There's really no difference from the outside and no difference in terms of the 
actual level of safety.

As far as I can tell, having @trusted being treated differently in the function 
signature buys us nothing. All of the difference is in whether the compiler 
should error out on @system stuff being use inside the functions. So, if we had 
@trusted blocks, and @trusted on a function meant that it was @safe with the 
entire body being inside of an @trusted block, as far as the caller and 
signature go, the behavior would be identical to now except that the mangling 
would be different.

> > But the much bigger problem is that @trusted doesn't play well with
> > template attribute inference and makes it much too easy to accidentally
> > mark a function as safe to call if it really isn't. Both things are a
> > consequence of the fact that it can be applied at the function level
> > only; there is no way to apply it selectively to only a part of the
> > function.
> 
> This could be a more serious problem. Could you please write a brief
> example that shows attribute deduction messing things up? I don't
> understand how marking a template as @trusted is bad.

Marking pretty much _any_ templated function - especially range-based 
functions -  as @trusted is bad. If I have

auto func(R)(R range) if(isForwardRange!R)
{
   while(!r.empty && !condition)
   {
       //do @system stuff
   }

   return range.front;
}

and func does @system stuff which I _know_ is valid based on how it's used 
(emplace, scoped, pointer arithmetic, etc.), then ideally I'd be able to mark 
those operations as @trusted, but what about front, popFront, empty, save, 
etc.? I don't know what they do internally. I don't know whether they're @safe 
or @system. So, I can't mark func as @trusted, or I could be marking @system 
code as @trusted when it really isn't safe.

Take the new std.range.RefRange for example. The body of Its save function is 
supposed to look like this:

import std.conv;
alias typeof((*_range).save) S;
static assert(isForwardRange!S, S.stringof ~ " is not a forward range.");
auto mem = new void[S.sizeof];
emplace!S(mem, cast(S)(*_range).save);
return RefRange!S(cast(S*)mem.ptr);

All of the stuff related to emplace can be marked as @trusted, but save can't 
be marked as @trusted because of that call to _range's save, which may or may 
not be @safe. So, RefRange uses static ifs and mixins to create multiple 
versions of the function which are marked as @system or @trusted based on 
whether _range's save is @system or @safe. It's made even worse by the fact 
that constness isn't inferred (issue# 8407), but the problem with @safe 
remains even if const or inout starts being inferred.

In some cases, you can use helper functions which are marked as @trusted to 
segregate the @system code that you know is safe, but in others, that just 
doesn't work (or is really messy if it does), and you need to use static ifs 
to provide multiple versions of the function. It's exactly the kind of 
situation for which we introduced attribute inferrence in the first place, 
except in this case, you _can't_ use inferrence, because you obviously can't 
infer @trusted.

By being able to mark blocks of code as @trusted instead of whole functions, 
the required code duplication is significantly reduced, and if the functions of 
unknown safety being called are separate enough from the code marked as 
@trusted (i.e. not intertwined like they are with the line using save in 
RefRange's save), then _no_ code duplication is required. But even if the 
potentially unsafe code is intertwined with the @trusted code, at least the 
code duplication can be restricted to just a few lines. So, with const/inout 
inferred and @trusted blocks, RefRange's save should be able to become 
something like this;

@property auto save()
{
   import std.conv;
   alias typeof((*_range).save) S;
   static assert(isForwardRange!S, S.stringof ~ " is not a forward range.");

   auto mem = new void[S.sizeof];

   static if(isSafelyCallable!((){(*_range).save;}))
       @trusted { emplace!S(mem, cast(S)(*_range).save); }
   else
       emplace!S(mem, cast(S)(*_range).save);

   @trusted { return RefRange!S(cast(S*)mem.ptr); }
}

which is _far_ shorter than what it is now (around 50 lines of code), since 
you only need _one_ function declaration instead of four.

- Jonathan M Davis
July 28, 2012
Re: @trusted considered harmful
On Saturday, 28 July 2012 at 20:22:27 UTC, Jonathan M Davis wrote:
> @property auto save()
> {
>     import std.conv;
>     alias typeof((*_range).save) S;
>     static assert(isForwardRange!S, S.stringof ~ " is not a 
> forward range.");
>
>     @trusted {auto mem = new void[S.sizeof];}
>
>     static if(isSafelyCallable!((){(*_range).save;}))
>     {
>         @trusted { emplace!S(mem, cast(S)(*_range).save); }
>     }
>     else
>     {
>         emplace!S(mem, cast(S)(*_range).save);
>     }
>
>     @trusted {return RefRange!S(cast(S*)mem.ptr);}
> }
>
> It's still a bit ugly, but it's far better than what we can 
> currently do. As
> it stands, you can't even put the emplace call in a separate 
> function and mark
> it @trusted or @system depending on save, because the only way 
> to mark the
> rest of the function as @trusted is to mark the whole thing as 
> @trusted. You'd
> have to specifically put _everything else_ in its own function 
> _and_ emplace in
> its own function (one @trusted, one @system) and have the outer 
> function call
> both. It's a mess. Being able to mark statements as @trusted 
> rather than just
> entire functons would be _huge_.

For the specific case of calling unsafe functions from otherwise 
safe code, which occurs quite frequently due to parts of Phobos 
not being properly annotated yet, I've been experimenting with an 
alternative solution in my code. It allows you to »apply 
trusted« at a function call level, and is easily implemented in 
the library with today's D – using it RefRange.save would look 
like this using it:

---
@property auto save()
{
    import std.conv;
    alias typeof((*_range).save) S;
    static assert(isForwardRange!S, S.stringof ~ " is not a 
forward range.");

    auto mem = new void[S.sizeof];
    TRUSTED!(emplace!S)(mem, cast(S)(*_range).save);
    @trusted return RefRange!S(cast(S*)mem.ptr);
}
---

TRUSTED is just a @trusted template function which accepts a 
callable via an alias parameter and invokes it with the given 
parameters, adding @trusted to the function type via a cast (an 
application of std.traits.SetFunctionAttributes, by the way).

It seems to work fine, but there is an obvious catch: Just like 
@trusted, TRUSTED circumvents safety, and all of its uses must be 
carefully examined. But in contrast to the former, it is not part 
of the language, so anyone working on the code base, especially 
reviewers, must be aware of implications. The all-caps name is 
meant to help drawing attention to .

Maybe it would be a good idea to also allow 
`@trusted(emplace!S)(mem, cast(S)(*_range).save)`, with semantics 
similar to TRUSTED? Or even applying @trusted to arbitrary 
expressions, similar to `checked` in C#?

David
July 28, 2012
Re: @trusted considered harmful
On Saturday, 28 July 2012 at 20:52:33 UTC, David Nadlinger wrote:
> The all-caps name is meant to help drawing attention to .

Oh, I should proof-read my posts before submitting, but I guess 
it was clear what I meant – because TRUSTED can only rely on 
convention, I chose a name which I hoped would immediately catch 
one's eye when skimming through the code.

David
July 28, 2012
Re: @trusted considered harmful
On Saturday, July 28, 2012 22:52:32 David Nadlinger wrote:
> Maybe it would be a good idea to also allow
> `@trusted(emplace!S)(mem, cast(S)(*_range).save)`, with semantics
> similar to TRUSTED? Or even applying @trusted to arbitrary
> expressions, similar to `checked` in C#?

Even the cast is unsafe. Basically, that entire line is @system and needs to 
be @trusted except for (*_range).save. Attribute inferrence needs to be used 
on that portion. So, aside from inventing yet more syntax like

@trusted { emplace!S(mem, cast(S)@infer{(*_range).save}); }

I don't know how you'd avoid the code duplication here. But at least with 
@trusted blocks, it can be reduced to a single line rather than having to 
duplicate the whole function.

- Jonathan M Davis
July 28, 2012
Re: @trusted considered harmful
On Saturday, 28 July 2012 at 21:03:20 UTC, Jonathan M Davis wrote:
> On Saturday, July 28, 2012 22:52:32 David Nadlinger wrote:
>> Maybe it would be a good idea to also allow
>> `@trusted(emplace!S)(mem, cast(S)(*_range).save)`, with 
>> semantics
>> similar to TRUSTED? Or even applying @trusted to arbitrary
>> expressions, similar to `checked` in C#?
>
> Even the cast is unsafe. Basically, that entire line is @system 
> and needs to
> be @trusted except for (*_range).save.

Aww, snap, missed that. Saving (*_range).save to a temporary 
would introduce an unnecessary copy, right?

In any case, I've found TRUSTED to be convenient when dealing 
with incorrectly marked Phobos/C library functions, but I'm still 
unsure it is worth the added maintenance liabilities incurred by 
it being non-standard. It certainly isn't a replacement for 
@trusted blocks.

Maybe allowing to apply @trusted at both expression and »block« 
level would really be an interesting direction: I've had a look 
at C#'s checked/unchecked keywords [1], which enable/disable 
integer overflow checking, and they work exactly like that.

I must admit that I never actually used them in the little amount 
of C# code I wrote so far, but C# is generally regarded to be a 
well-designed language and similar enough to D that this makes me 
quite confident that implementing @trusted like that could be 
pulled off without feeling overly alien.

This still wouldn't solve your save() problem, though, as it acts 
»the wrong way« round, so I'm not sure if it would be worth the 
added complexity over allowing it just at the statement level…

David


[1] http://msdn.microsoft.com/en-us/library/74b4xzyw(v=vs.80).aspx
1 2 3 4
Top | Discussion index | About this forum | D home