On 07/28/12 18:16, Andrei Alexandrescu wrote:
> On 7/28/12 10:34 AM, Artur Skawina wrote:
>> So the result is bugs (like the ones mentioned in this thread), where @safe is bypassed, because the @trusted functions aren't expecting to be used with "unsafe" ones. @trusted bypasses *all* safety checks, not just those in the hand-checked code. This is something that you will want sometimes, but in most cases is neither necessary nor desirable. When dealing with safety one has to be conservative. The proposals to limit the scope of @trusted only address the symptoms, not the cause. If the design is fixed, many of the reasons for introducing the finer-grained @trusted disappear, and it is the truly unsafe (@trusted that calls @system) code that needs the extra annotations -- which is a good thing. Papering over design bugs never is.
> 
> I don't understand what you suggest here. Is it a sort of a refinement of @trusted?

Maybe my other response to David makes things more clear.

Yes, there would be a cost to this change (some lib and user code would need additional annotations), but I think it'd be worth it, as right now it is much too easy to 'break' the @safe guarantees, often w/o even realizing it.

artur

On Saturday, 28 July 2012 at 02:33:54 UTC, Jonathan M Davis wrote:
> On Saturday, July 28, 2012 04:05:14 Jesse Phillips wrote:
>> You aren't supposed to do dirty things in @trusted code. You're
>> supposed to  safely wrap a system function to be usable by a safe
>> function. The system function is supposed to be short and getting
>> its hands dirty. Remember this is about memory safety and not
>> lack of bugs safety.
>
> It's perfectly acceptable to put "dirty code" in an @trusted
> function. The difference between @trusted and @system is that with @trusted,
> the programmer is guaranteeing that it's actually @safe regardless of what
> arguments it's given, whereas @system is not only doing unsafe things, but
> whether they're ultimately @safe or not depends on arguments and/or other
> code, so the programmer _can't_ guarantee that it's @safe.

I realize you can, David Piepgrass was able to read into an idea which I didn't want to specifically state as it isn't as concerning. But why not limit @trusted to only do @safe and call @trusted. But I could see that as an extra layer of annoying, but it would keep the unsafe arithmetic and parameter passing in the @system functions. But again, not much thought on it and probably just be a layer of annoying.

For your example of the proposal, when does the compiler mark save() as @system over @trusted?

@property auto save()
{
   import std.conv;
   alias typeof((*_range).save) S;
   static assert(isForwardRange!S, S.stringof ~ " is not a forward range.");

   @trusted
   {
       auto mem = new void[S.sizeof];`
       emplace!S(mem, cast(S)(*_range).save);`
       return RefRange!S(cast(S*)mem.ptr);
   }
}

You make a call to _range.save in the @trusted block. So why isn't the function marked @trusted?

Having taken some more thought on the guarantee provided by @safe even when calling @trusted code, I do see the issue of templates always being trusted. As it could then be the logic in of the @safe which is doing something unsafe. This could be true with @trusted but at least the author had the opportunity to make it safe.

But at this point I'm not seeing how this proposal is helping. You do want save() to be @system when the underlining save() is @system, and @trusted otherwise? So I'm not seeing that here.

On Saturday, 28 July 2012 at 02:33:54 UTC, Jonathan M Davis wrote:
> @property auto save()
> {
>     import std.conv;
>     alias typeof((*_range).save) S;
>     static assert(isForwardRange!S, S.stringof ~ " is not a forward range.");
>
>     @trusted
>     {
>         auto mem = new void[S.sizeof];`
>         emplace!S(mem, cast(S)(*_range).save);`
>         return RefRange!S(cast(S*)mem.ptr);
>     }
> }
>
> That is _way_ cleaner.

But unfortunately wrong – you call S.save in the @trusted block… ;)

David

On Saturday, July 28, 2012 22:08:42 David Nadlinger wrote:
> On Saturday, 28 July 2012 at 02:33:54 UTC, Jonathan M Davis wrote:
> > @property auto save()
> > {
> > 
> >     import std.conv;
> >     alias typeof((*_range).save) S;
> >     static assert(isForwardRange!S, S.stringof ~ " is not a
> > 
> > forward range.");
> > 
> >     @trusted
> >     {
> > 
> >         auto mem = new void[S.sizeof];`
> >         emplace!S(mem, cast(S)(*_range).save);`
> >         return RefRange!S(cast(S*)mem.ptr);
> > 
> >     }
> > 
> > }
> > 
> > That is _way_ cleaner.
> 
> But unfortunately wrong – you call S.save in the @trusted block… ;)

Yeah. I screwed that up. I was obviously in too much of a hurry when I wrote it. And actually, in this particular case, since the part that can't be @trusted is in the middle of an expression doing @system stuff, simply using an @trusted block wouldn't do the trick. The basic principle is still good though. Being able to mark specific statements or sections of code @trusted would greatly simplify templated functions which are making function calls which _can't_ necessarily be @trusted.

- Jonathan M Davis

On Saturday, July 28, 2012 21:53:25 Jesse Phillips wrote:
> For your example of the proposal, when does the compiler mark
> save() as @system over @trusted?
> 
> @property auto save()
> {
>     import std.conv;
>     alias typeof((*_range).save) S;
>     static assert(isForwardRange!S, S.stringof ~ " is not a
> forward range.");
> 
>     @trusted
>     {
>         auto mem = new void[S.sizeof];`
>         emplace!S(mem, cast(S)(*_range).save);`
>         return RefRange!S(cast(S*)mem.ptr);
>     }
> }
> 
> You make a call to _range.save in the @trusted block. So why isn't the function marked @trusted?

I screwed up my changes there. I made them in a bit of a hurry. And actually, in this particular case, using an @trusted block is still a bit entertaining, since the save call is in middle of an expression which is doing @system stuff. But the principle is the same regardless. If you need to do @system stuff in a function where you know that the @system stuff that you're doing is @safe but the function is also calling templated stuff which may or may not be @safe, you can't mark the whole function as @trusted, or you'll be marking code which is potentially truly unsafe as @trusted. With an @trusted block, you can mark sections of the code @trusted and let the rest be properly inferred based on what the the template was instantiated with.

In this particular case, it could still drastically reduce save, because only the line with @save would need to be duplicated. So, it could become something more like

@property auto save()
{
    import std.conv;
    alias typeof((*_range).save) S;
    static assert(isForwardRange!S, S.stringof ~ " is not a forward range.");

    @trusted {auto mem = new void[S.sizeof];}

    static if(isSafelyCallable!((){(*_range).save;}))
    {
        @trusted { emplace!S(mem, cast(S)(*_range).save); }
    }
    else
    {
        emplace!S(mem, cast(S)(*_range).save);
    }

    @trusted {return RefRange!S(cast(S*)mem.ptr);}
}

It's still a bit ugly, but it's far better than what we can currently do. As it stands, you can't even put the emplace call in a separate function and mark it @trusted or @system depending on save, because the only way to mark the rest of the function as @trusted is to mark the whole thing as @trusted. You'd have to specifically put _everything else_ in its own function _and_ emplace in its own function (one @trusted, one @system) and have the outer function call both. It's a mess. Being able to mark statements as @trusted rather than just entire functons would be _huge_.

- Jonathan M Davis

On Saturday, July 28, 2012 10:02:43 Andrei Alexandrescu wrote:
> On 7/27/12 8:08 PM, David Nadlinger wrote:
> > First, there is no point in having @trusted in the function signature. Why? From the perspective of the caller of the function in question, @safe and @trusted mean exactly the same thing. If you are not convinced about that, just consider that you can wrap any @trusted function into a @safe function to make it @safe, and vice versa.
> 
> If @trusted is not part of the signature, we can't enable e.g. analyzers that verify an entire program or package to be safe. This is not something that's currently used, but I'd hate to look back and say, "heck, I hate that we conflated @trusted with @safe!"

But from the caller's perspective, @safe and @trusted are identical. And if you want a function to be @safe rather than @trusted, all you have to do is create an @trusted helper function which has the implementation and have the actual function call it, and the the function can be @safe. I don't see how it makes any difference at all whether a function is marked @safe or @trusted except for the fact that with @trusted, the implementation can _directly_ use @system stuff, whereas with @safe, there has to be another layer in between. There's really no difference from the outside and no difference in terms of the actual level of safety.

As far as I can tell, having @trusted being treated differently in the function signature buys us nothing. All of the difference is in whether the compiler should error out on @system stuff being use inside the functions. So, if we had @trusted blocks, and @trusted on a function meant that it was @safe with the entire body being inside of an @trusted block, as far as the caller and signature go, the behavior would be identical to now except that the mangling would be different.

> > But the much bigger problem is that @trusted doesn't play well with template attribute inference and makes it much too easy to accidentally mark a function as safe to call if it really isn't. Both things are a consequence of the fact that it can be applied at the function level only; there is no way to apply it selectively to only a part of the function.
> 
> This could be a more serious problem. Could you please write a brief example that shows attribute deduction messing things up? I don't understand how marking a template as @trusted is bad.

Marking pretty much _any_ templated function - especially range-based functions -  as @trusted is bad. If I have

auto func(R)(R range) if(isForwardRange!R)
{
    while(!r.empty && !condition)
    {
        //do @system stuff
    }

    return range.front;
}

and func does @system stuff which I _know_ is valid based on how it's used (emplace, scoped, pointer arithmetic, etc.), then ideally I'd be able to mark those operations as @trusted, but what about front, popFront, empty, save, etc.? I don't know what they do internally. I don't know whether they're @safe or @system. So, I can't mark func as @trusted, or I could be marking @system code as @trusted when it really isn't safe.

Take the new std.range.RefRange for example. The body of Its save function is supposed to look like this:

import std.conv;
alias typeof((*_range).save) S;
static assert(isForwardRange!S, S.stringof ~ " is not a forward range.");
auto mem = new void[S.sizeof];
emplace!S(mem, cast(S)(*_range).save);
return RefRange!S(cast(S*)mem.ptr);

All of the stuff related to emplace can be marked as @trusted, but save can't be marked as @trusted because of that call to _range's save, which may or may not be @safe. So, RefRange uses static ifs and mixins to create multiple versions of the function which are marked as @system or @trusted based on whether _range's save is @system or @safe. It's made even worse by the fact that constness isn't inferred (issue# 8407), but the problem with @safe remains even if const or inout starts being inferred.

In some cases, you can use helper functions which are marked as @trusted to segregate the @system code that you know is safe, but in others, that just doesn't work (or is really messy if it does), and you need to use static ifs to provide multiple versions of the function. It's exactly the kind of situation for which we introduced attribute inferrence in the first place, except in this case, you _can't_ use inferrence, because you obviously can't infer @trusted.

By being able to mark blocks of code as @trusted instead of whole functions, the required code duplication is significantly reduced, and if the functions of unknown safety being called are separate enough from the code marked as @trusted (i.e. not intertwined like they are with the line using save in RefRange's save), then _no_ code duplication is required. But even if the potentially unsafe code is intertwined with the @trusted code, at least the code duplication can be restricted to just a few lines. So, with const/inout inferred and @trusted blocks, RefRange's save should be able to become something like this;

@property auto save()
{
    import std.conv;
    alias typeof((*_range).save) S;
    static assert(isForwardRange!S, S.stringof ~ " is not a forward range.");

    auto mem = new void[S.sizeof];

    static if(isSafelyCallable!((){(*_range).save;}))
        @trusted { emplace!S(mem, cast(S)(*_range).save); }
    else
        emplace!S(mem, cast(S)(*_range).save);

    @trusted { return RefRange!S(cast(S*)mem.ptr); }
}

which is _far_ shorter than what it is now (around 50 lines of code), since you only need _one_ function declaration instead of four.

- Jonathan M Davis

On Saturday, 28 July 2012 at 20:22:27 UTC, Jonathan M Davis wrote:
> @property auto save()
> {
>     import std.conv;
>     alias typeof((*_range).save) S;
>     static assert(isForwardRange!S, S.stringof ~ " is not a forward range.");
>
>     @trusted {auto mem = new void[S.sizeof];}
>
>     static if(isSafelyCallable!((){(*_range).save;}))
>     {
>         @trusted { emplace!S(mem, cast(S)(*_range).save); }
>     }
>     else
>     {
>         emplace!S(mem, cast(S)(*_range).save);
>     }
>
>     @trusted {return RefRange!S(cast(S*)mem.ptr);}
> }
>
> It's still a bit ugly, but it's far better than what we can currently do. As
> it stands, you can't even put the emplace call in a separate function and mark
> it @trusted or @system depending on save, because the only way to mark the
> rest of the function as @trusted is to mark the whole thing as @trusted. You'd
> have to specifically put _everything else_ in its own function _and_ emplace in
> its own function (one @trusted, one @system) and have the outer function call
> both. It's a mess. Being able to mark statements as @trusted rather than just
> entire functons would be _huge_.

For the specific case of calling unsafe functions from otherwise safe code, which occurs quite frequently due to parts of Phobos not being properly annotated yet, I've been experimenting with an alternative solution in my code. It allows you to »apply trusted« at a function call level, and is easily implemented in the library with today's D – using it RefRange.save would look like this using it:

---
@property auto save()
{
    import std.conv;
    alias typeof((*_range).save) S;
    static assert(isForwardRange!S, S.stringof ~ " is not a forward range.");

    auto mem = new void[S.sizeof];
    TRUSTED!(emplace!S)(mem, cast(S)(*_range).save);
    @trusted return RefRange!S(cast(S*)mem.ptr);
}
---

TRUSTED is just a @trusted template function which accepts a callable via an alias parameter and invokes it with the given parameters, adding @trusted to the function type via a cast (an application of std.traits.SetFunctionAttributes, by the way).

It seems to work fine, but there is an obvious catch: Just like @trusted, TRUSTED circumvents safety, and all of its uses must be carefully examined. But in contrast to the former, it is not part of the language, so anyone working on the code base, especially reviewers, must be aware of implications. The all-caps name is meant to help drawing attention to .

Maybe it would be a good idea to also allow `@trusted(emplace!S)(mem, cast(S)(*_range).save)`, with semantics similar to TRUSTED? Or even applying @trusted to arbitrary expressions, similar to `checked` in C#?

David

On Saturday, 28 July 2012 at 20:52:33 UTC, David Nadlinger wrote:
> The all-caps name is meant to help drawing attention to .

Oh, I should proof-read my posts before submitting, but I guess it was clear what I meant – because TRUSTED can only rely on convention, I chose a name which I hoped would immediately catch one's eye when skimming through the code.

David

On Saturday, July 28, 2012 22:52:32 David Nadlinger wrote:
> Maybe it would be a good idea to also allow
> `@trusted(emplace!S)(mem, cast(S)(*_range).save)`, with semantics
> similar to TRUSTED? Or even applying @trusted to arbitrary
> expressions, similar to `checked` in C#?

Even the cast is unsafe. Basically, that entire line is @system and needs to be @trusted except for (*_range).save. Attribute inferrence needs to be used on that portion. So, aside from inventing yet more syntax like

@trusted { emplace!S(mem, cast(S)@infer{(*_range).save}); }

I don't know how you'd avoid the code duplication here. But at least with @trusted blocks, it can be reduced to a single line rather than having to duplicate the whole function.

- Jonathan M Davis

On Saturday, 28 July 2012 at 21:03:20 UTC, Jonathan M Davis wrote:
> On Saturday, July 28, 2012 22:52:32 David Nadlinger wrote:
>> Maybe it would be a good idea to also allow
>> `@trusted(emplace!S)(mem, cast(S)(*_range).save)`, with semantics
>> similar to TRUSTED? Or even applying @trusted to arbitrary
>> expressions, similar to `checked` in C#?
>
> Even the cast is unsafe. Basically, that entire line is @system and needs to
> be @trusted except for (*_range).save.

Aww, snap, missed that. Saving (*_range).save to a temporary would introduce an unnecessary copy, right?

In any case, I've found TRUSTED to be convenient when dealing with incorrectly marked Phobos/C library functions, but I'm still unsure it is worth the added maintenance liabilities incurred by it being non-standard. It certainly isn't a replacement for @trusted blocks.

Maybe allowing to apply @trusted at both expression and »block« level would really be an interesting direction: I've had a look at C#'s checked/unchecked keywords [1], which enable/disable integer overflow checking, and they work exactly like that.

I must admit that I never actually used them in the little amount of C# code I wrote so far, but C# is generally regarded to be a well-designed language and similar enough to D that this makes me quite confident that implementing @trusted like that could be pulled off without feeling overly alien.

This still wouldn't solve your save() problem, though, as it acts »the wrong way« round, so I'm not sure if it would be worth the added complexity over allowing it just at the statement level…

David


[1] http://msdn.microsoft.com/en-us/library/74b4xzyw(v=vs.80).aspx