On 6/1/13 3:59 PM, monarch_dodra wrote:
> Yeah, overall, I'm confused as to what "@safe" means from an interface
> point of view :(

If you call the function from a program with memory integrity and it returns, it hasn't compromised the memory integrity of that program.

Homework: define memory integrity :o).


Andrei

On Saturday, 1 June 2013 at 21:41:40 UTC, Jonathan M Davis wrote:
> They're guaranteed to not introduce any such behavior. They can't possibly
> make any guarantees if the caller did @system operations and passed a bad
> pointer to the @safe function. But if all of the functions in the call stack
> are @safe, and you call an @safe function, then you can't get any memory
> corruption unless it (or a function that it calls) calls an @trusted function
> which was incorrectly verified by the programmer who marked it as @trusted.
>
> - Jonathan M Davis

Updated example from above to show how @safe can introduce UB.

import std.stdio;

class A
{
   int[] data;
   ~this()
   {
      writeln(data);
   }
}

void foo(int[] a) @safe
{
   A x = new A;
   x.data = a;
}

void main() @safe
{
   int[4] y;
   foo(y);
}

On Saturday, 1 June 2013 at 21:46:01 UTC, Maxim Fomin wrote:
> Updated example from above to show how @safe can introduce UB.
>
> void main() @safe
> {
>    int[4] y;
>    foo(y);
> }

I believe that's a compiler bug.

@safe requires:
- No taking the address of a local variable or function parameter.

A slice of a local static array should count as this, but it currently doesn't.

On Saturday, June 01, 2013 23:45:59 Maxim Fomin wrote:
> On Saturday, 1 June 2013 at 21:41:40 UTC, Jonathan M Davis wrote:
> > They're guaranteed to not introduce any such behavior. They
> > can't possibly
> > make any guarantees if the caller did @system operations and
> > passed a bad
> > pointer to the @safe function. But if all of the functions in
> > the call stack
> > are @safe, and you call an @safe function, then you can't get
> > any memory
> > corruption unless it (or a function that it calls) calls an
> > @trusted function
> > which was incorrectly verified by the programmer who marked it
> > as @trusted.
> > 
> > - Jonathan M Davis
> 
> Updated example from above to show how @safe can introduce UB.
> 
> import std.stdio;
> 
> class A
> {
>     int[] data;
>     ~this()
>     {
>        writeln(data);
>     }
> }
> 
> void foo(int[] a) @safe
> {
>     A x = new A;
>     x.data = a;
> }
> 
> void main() @safe
> {
>     int[4] y;
>     foo(y);
> }

That's a known bug in @safe. Slicing a static array should be considered @system just like taking the address of a local variable is considered @system:

http://d.puremagic.com/issues/show_bug.cgi?id=8838

The guarantees of @safe hold only so long as there are no holes in it, but any and all holes we find get fixed. Making ref be truly @safe has been a large part of the recent ref discussions, as you can currently get away with doing something like

ref int id(ref int i) { return i; }

ref int foo()
{
    int j;
    return id(j);
}

What it looks like we're going to do in this case is detect when this situation might happen and insert a runtime check which throws an Error if a reference to a local variable tries to escape, but regardless of the solution, it's an example of something that is currently considered @safe by the compiler when it really isn't. All such holes need to be plugged, or @safe isn't doing its job.

So, if you find any more holes in @safe, please report them in bugzilla:

http://d.puremagic.com/issues

- Jonathan M Davis

On Saturday, 1 June 2013 at 21:45:18 UTC, Andrei Alexandrescu wrote:
> On 6/1/13 3:59 PM, monarch_dodra wrote:
>> Yeah, overall, I'm confused as to what "@safe" means from an interface
>> point of view :(
>
> If you call the function from a program with memory integrity and it returns, it hasn't compromised the memory integrity of that program.
>
> Homework: define memory integrity :o).
>
>
> Andrei

OK. In a word, I guess that makes sense. I'll stick to that standard.

But there is still the "emplace" question: When I call "emplace" on a pointer to a built object, is does the program still have memory integrity? At the end of the call, was it emplace that compromised it? Was it the exact instance the "call was initialized"?

On Sunday, June 02, 2013 00:04:18 monarch_dodra wrote:
> On Saturday, 1 June 2013 at 21:45:18 UTC, Andrei Alexandrescu
> 
> wrote:
> > On 6/1/13 3:59 PM, monarch_dodra wrote:
> >> Yeah, overall, I'm confused as to what "@safe" means from an
> >> interface
> >> point of view :(
> > 
> > If you call the function from a program with memory integrity and it returns, it hasn't compromised the memory integrity of that program.
> > 
> > Homework: define memory integrity :o).
> > 
> > 
> > Andrei
> 
> OK. In a word, I guess that makes sense. I'll stick to that standard.
> 
> But there is still the "emplace" question: When I call "emplace" on a pointer to a built object, is does the program still have memory integrity? At the end of the call, was it emplace that compromised it? Was it the exact instance the "call was initialized"?

Because the safety of the function depends on the caller, I'd say that it should be @system.

- Jonathan M Davis

On Saturday, June 01, 2013 23:41:32 monarch_dodra wrote:
> On Saturday, 1 June 2013 at 21:02:44 UTC, Jonathan M Davis wrote:
> > On Saturday, June 01, 2013 21:59:18 monarch_dodra wrote:
> >> The way I understood it, @safe defines a list of things that
> >> are
> >> or aren't legal inside the implementation of a function. It
> >> also
> >> changes the scheme of bounds checking, in release code.
> >> 
> >> What bothers me though, is that from an interface point of
> >> view,
> >> it doesn't really mean anything (or at least, I haven't really
> >> understood anything). AFAIK: if I call something "@safe",
> >> chances
> >> of a core dump are relatively "lower", but they can still
> >> happen:
> >> * A function that accepts a pointer as an argument can be
> >> marked
> >> safe, so all bets are off there, no, since the pointer can be
> >> dereferenced?
> >> * Member functions for structs that have pointers, too, can be
> >> marked safe...
> >> 
> >> Or does it only mean "if you give me valid pointers, I can't
> >> core
> >> dump*"?
> >> (*ignoring current flaws, such as escaping slices from static
> >> arrays)
> >> 
> >> The main reason about this question is that now I'm confused
> >> about @trusted: what are the conditions a developer needs to
> >> take
> >> into account before marking a function "@trusted" ?
> >> 
> >> Ditto for member functions, when they operate on pointer
> >> members.
> >> Can those be @safe?
> >> 
> >> Yeah, overall, I'm confused as to what "@safe" means from an interface point of view :(
> > 
> > @safe is for memory safety, meaning that @safe code cannot
> > corrupt memory. You
> > can get segfaults due to null pointers and the like, but you
> > can't have code
> > which writes passed the end of a buffer, or which uses a freed
> > memory, or does
> > anything else which involves writing or reading from memory
> > which variables
> > aren't supposed to have access to.
> > 
> > Assuming that there are no bugs in @safe, the one thing that
> > can invalidate it
> > is @trusted. With @trusted code, it is the _programmer_ who is
> > then
> > guaranteeing that the code is actually @safe. The code is doing
> > something
> > which is potentially not safe (and therefore is considered
> > @system by the
> > compiler) but which _could_ be safe if the code is correct, and
> > if the
> > programmer is marking the code as @trusted, they are then
> > telling the compiler
> > that they've verified that the code isn't doing anything which
> > could corrupt
> > memory. As long as the programmer doesn't screw that up, then
> > any @safe code
> > calling that @trusted function is indeed @safe, but if the
> > programmer screwed
> > it up, then you could still get memory corruption. However,
> > here's really no
> > way to get around that problem with a systems language, since
> > most code needs
> > to eventually call something that's @system (e.g. all I/O needs
> > @system stuff
> > internally). But by limiting how much code is @system or
> > @trusted, most code
> > is @safe with a minimal amount of code having to have been
> > verified by an
> > appropriately competent programmer as being @trusted.
> > 
> > - Jonathan M Davis
> 
> OK. But by that standard, can't (mostly) anything be trusted? What about something that writes garbage, to a memory location it was *asked* to write to? Or if wrong usage of the function can lead to an inconsistence memory state, but without "out of bounds accesses"?

When a programmer marks a function as @trusted, they are saying that they guarantee that the function will not do anything to corrupt memory. So, yes, a programmer could mark absolutely anything as @trusted - including stuff that is blatantly unsafe and will do all kinds of nasty stuff - but that's the programmer's fault. The compiler told them that it was @system and therefore could not be verified to be memory safe, and the programmer insisted that it was memory safe.

Any code which cannot be guaranteed to be actually safe should not be marked with @trusted. It's up to the programmer figure out whether what they're doing is valid or not.

> For instance: "emplace!T(T* p)": This function takes the address of a T, and writes T.init over it. It does a memcopy, so it can't be @safe, but I can 100% guarantee I'm not doing anything wrong, so I'm marking it as @trusted. This should be fine, right? Or is raw memory copying alway unsafe?

Raw memory copying should be fine as long as the memory being copied from and the memory being copied to is valid.

> Now, technically, emplace can't be called in @safe code, since it requires a pointer to begin with.

Pointers can be in @safe code. They're perfectly safe in and of themselves. It's certain operations on pointers which are unsafe (such as pointer arithmetic).

> But still, if I were to give emplace an "already constructed object", it will happily clobber that object for me, leaking the destructor, and possibly putting the program in an invalid memory state.
> 
> Now, it was *my* fault for calling emplace with an already built object, but it was the (@trusted) emplace that clobbered-it.

Well, given that the safety of the operation relies on what's being passed in, the operation itself can't reasonably be marked as @safe, because you can't guarantee that the operation isn't going to corrupt memory.

- Jonathan M Davis

W dniu 01.06.2013 23:55, Jonathan M Davis pisze:
> The guarantees of @safe hold only so long as there are no holes in it, but any
> and all holes we find get fixed. Making ref be truly @safe has been a large part
> of the recent ref discussions, as you can currently get away with doing
> something like
>
> ref int id(ref int i) { return i; }
>
> ref int foo()
> {
>      int j;
>      return id(j);
> }

I know that introducing another type qualifier may complicate things
but this would be a compile time solution.

I mean _scope_ type qualifier, so your example could be rewritten as:

ref int id(ref int i) { return i; }

ref int foo()
{
    int j;
    return id(j); // error: could not pass scope(int) as ref int parameter
}

Taking an address of local variable would always yield a scope
qualified type, scope(int) in this example.

Obviously, scope qualified type could be passed to functions taking
scope parameters:

void bar(scope ref int i) { i = 10; }

void foo()
{
    int j = 0;
    bar(j); // ok
    assert(i == 10);
}

I think this could fill the @safety holes.

On Sunday, June 02, 2013 01:12:53 Piotr Szturmaj wrote:
> W dniu 01.06.2013 23:55, Jonathan M Davis pisze:
> > The guarantees of @safe hold only so long as there are no holes in it, but any and all holes we find get fixed. Making ref be truly @safe has been a large part of the recent ref discussions, as you can currently get away with doing something like
> > 
> > ref int id(ref int i) { return i; }
> > 
> > ref int foo()
> > {
> > 
> >      int j;
> >      return id(j);
> > 
> > }
> 
> I know that introducing another type qualifier may complicate things but this would be a compile time solution.
> 
> I mean _scope_ type qualifier, so your example could be rewritten as:
> 
> ref int id(ref int i) { return i; }
> 
> ref int foo()
> {
>      int j;
>      return id(j); // error: could not pass scope(int) as ref int parameter
> }
> 
> Taking an address of local variable would always yield a scope qualified type, scope(int) in this example.
> 
> Obviously, scope qualified type could be passed to functions taking scope parameters:
> 
> void bar(scope ref int i) { i = 10; }
> 
> void foo()
> {
>      int j = 0;
>      bar(j); // ok
>      assert(i == 10);
> }
> 
> I think this could fill the @safety holes.

Except that that makes it so that you can't return a ref argument, which is completely unacceptable. You need to be able to pass local variables to functions which accept ref in @safe code, and you need functions which ref arguments to be able to return those arguments by ref. The only thing that we want to prevent is a local variable from escaping its original scope. It's perfectly valid that the id function accept a local variable by ref and returns it by ref. What's invalid is that the function that the local variable was declared in then returns it by ref.

Manu suggested something similar to what you're suggesting with the addition of having making it so that you can then return variables as scope ref, in which case, the caller would see that the function was accepting by scope ref and returning by scope ref and that none of the variables that it accepted were scope ref in the caller. But this requires having having yet another annotation - scope ref - which Andrei was completely against (and Walter too IIRC), and it actually would end up making something like this illegal as well, went it shouldn't:

scope ref int foo(scope ref int i, scope ref int j)
{
    return j;
}

scope ref bar(scope ref int q)
{
    int i;
    return foo(i, q);
}

The compiler can't know whether it's i or q that's being returned from foo, so it would have to given a compilation error, which is more restrictive than the runtime solution that has been proposed. So, you can do less, and you have to have mark up your functions with even more attributes, and it's yet another attribute for those learning the language to have to learn.

Contrast this with simply inserting a very cheap runtime check in the rare cases where the compiler detects that a local variable might escape. No additional attributes are needed. So, the code is simpler, and there's less for people to learn. There's almost no performance hit (and if you want it to be zero, then use -noboundscheck). And we lose _zero_ functionality. None of that is the case with the scope ref proposal.

Walter and Andrei do not like the idea of introducing even more attributes to solve this problem and were very excited to have this solution proposed (unfortunately, I'm not sure who proposed it though, since I missed that part of the conversation). And I'm inclined to agree with them. It's very simple and cheap. The only real downside is that it's caught at runtime rather than compile time, but it's quickly and easily caught at runtime, and the simplicity of it makes it seem like a great solution.

- Jonathan M Davis

On Saturday, 1 June 2013 at 22:15:00 UTC, Jonathan M Davis wrote:
> On Saturday, June 01, 2013 23:41:32 monarch_dodra wrote:
>> OK. But by that standard, can't (mostly) anything be trusted?
>> What about something that writes garbage, to a memory location it
>> was *asked* to write to? Or if wrong usage of the function can
>> lead to an inconsistence memory state, but without "out of bounds
>> accesses"?
>
> When a programmer marks a function as @trusted, they are saying that they
> guarantee that the function will not do anything to corrupt memory. So, yes, a
> programmer could mark absolutely anything as @trusted - including stuff that is
> blatantly unsafe and will do all kinds of nasty stuff - but that's the
> programmer's fault.

Well, by "mostly", I did mean stuff that's not blatantly wrong. I don't usually write stuff with the express objective of clobbering memory.

But given the previous answers, I think I see why anything that should work can't be marked @safe.

>> But still, if I were to give emplace an "already constructed
>> object", it will happily clobber that object for me, leaking the
>> destructor, and possibly putting the program in an invalid memory
>> state.
>> 
>> Now, it was *my* fault for calling emplace with an already built
>> object, but it was the (@trusted) emplace that clobbered-it.
>
> Well, given that the safety of the operation relies on what's being passed in,
> the operation itself can't reasonably be marked as @safe, because you can't
> guarantee that the operation isn't going to corrupt memory.

But isn't that exactly the same as my "void foo(int* p) @safe{*p = 0}" example ? That relies on what is being passed in to guarantee safety :/

@confused