On Wednesday, 25 February 2015 at 22:16:14 UTC, Ola Fosheim Grøstad wrote:
> My point was that there is no conceptual difference between having a named function trusted_malloc!int() and trusted_free() and wrapping them up individually unnamed.

An ad-hoc declared @trusted malloc is just as unsafe as a public one, of course. But there's a difference in exposure. People working on RCArray are supposed to know about the rule-breaking that's going on there. A public trusted_malloc would invite the un-initiated to shoot their feet.

>> RCArray as a whole is the actually trusted region, yes, since it must be manually verified that RCArray.array isn't leaked. But you can't mark it @trusted, because E may be unsafe.
>
> But the semantic analysis should verify that code isn't injected unless it is also @trusted?

You mean the compiler should enforce E to be @safe/@trusted? That wouldn't happen with an @trusted RCArray, because @trusted code may call @system code.

It would be done with an @safe RCArray. But I guess that's deemed too limiting. RCArray is supposed to work with unsafe E types, too.

>>> And that assumes strong typing, which D currently does not provide. Without strong typing it will be very difficult for the compiler to infer anything across compilation units.
>>
>> I don't follow.
>
> C is not strongly typed, and neither is D. That means there are holes in the type system.

I'm a bit lost. What I meant was that the compiler infers @safe for methods of templated structs when they don't call any @system code.

Here: RCArray's this/~this are inferred @safe iff E's __postblit/__dtor are @safe/@trusted.

This is how it works right now, regardless of any holes in the type system.

On Wednesday, 25 February 2015 at 22:59:01 UTC, anonymous wrote:
> rule-breaking that's going on there. A public trusted_malloc would invite the un-initiated to shoot their feet.

That's entirely social...

> You mean the compiler should enforce E to be @safe/@trusted? That wouldn't happen with an @trusted RCArray, because @trusted code may call @system code.

If you can inject new code that is not marked @trusted explicitly into already written code that is marked @trusted, then the "trust type system" is broken.

> I'm a bit lost. What I meant was that the compiler infers @safe for methods of templated structs when they don't call any @system code.

It infers "@safe", but it does not provide validation.

On Thursday, 26 February 2015 at 10:15:07 UTC, Ola Fosheim Grøstad wrote:
> On Wednesday, 25 February 2015 at 22:59:01 UTC, anonymous wrote:
>> rule-breaking that's going on there. A public trusted_malloc would invite the un-initiated to shoot their feet.
>
> That's entirely social...

Sure.

A @trusted function that's not actually memory-safe is against the definition of @trusted. Breaking that rule on a public function is not acceptable. Breaking it locally is apparently ok as long as the surrounding entity (RCArray) is actually memory-safe in the end.

I'm not trying to argue that this is good. Maybe the current system allows for a better way to solve such issues. If it doesn't, maybe the @safe/@trusted system needs an upgrade.

[...]
> If you can inject new code that is not marked @trusted explicitly into already written code that is marked @trusted, then the "trust type system" is broken.

The whole point of @trusted is to be able to call @system code. It doesn't matter if that code is "injected" or not. @safe prevents calling @system code.

[...]
> It infers "@safe", but it does not provide validation.

Yup. RCArray is de-facto trusted, i.e. verified memory-safe by the programmer. It's not compiler verified safe, even though it ends up being labeled @safe.

On 2/24/15 5:37 PM, "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= <ola.fosheim.grostad+dlang@gmail.com>" wrote:
> Since neither Andrei or Walter are able to say something sensible on
> these issues when asked, I apparently need to learn something about the
> "consistency" of C memory safety.
>
> I'm happy to listen to anyone who can explain this to me:
>
> 1. My understanding is that @trusted is supposed to give memory safety
> escapes by providing a context which reestablish a memory safety context
> on return.
>
> Yet in this thread
> http://forum.dlang.org/thread/mcik3j$153g$1@digitalmars.com it is stated
> that this paradigm is an example of «careful use of @trusted»:
>
>      count = (() @trusted => cast(uint*) malloc(uint.sizeof))();
>      …arbitrary code…
>      (() @trusted => free(count))();

This isn't exactly what is happening.

First, malloc should be safe, in the same way new is safe. The fact that @trusted is needed is somewhat incorrect in my opinion.

Where @trusted SHOULD be needed is for `free`.

Now, I disagree that using a @trusted delegate for the free is the right thing to do. As soon as you free count, it is no longer safe, because it's a dangling pointer.

I would say THIS is somewhat correct:

(() @trusted {free(count); count=null;})();

This takes something that is validly safe, and keeps the safety by ensuring the pointer is no longer dangling.

However, we have an issue here. At any point inside the code, you could do:

oldcount = count;

And now, there is still potentially a dangling pointer somewhere. This means every place count is used must be checked. In this case, all uses of count have to be re-checked when the file is edited.

Because count leaks the memory details of the trusted call to free (In other words, the fact that count, or any copy of it, can become a dangling pointer), I believe every call in that type that deals with count should be marked trusted.

This is a great example of why @trusted is mostly a convention thing, and not an enforceable thing. And it's very difficult to get right. There is no way to say "compiler, count is tainted. Please error whenever anyone uses count without marking that code as @trusted."

> They way I see it, this is equivalent to typing a reinterpret_casting
> malloc and free as memorysafe operations in isolation, basically
> providing a malloc!int() and free() as memory safe functions. But why is
> malloc and free not considered safe by default then? These @trusted
> functions clearly cannot prevent leaks within their own context. You
> would need a @trusted-only storage class on the receiving pointer to do
> that and a @trusted move type.
>
> If this is careful use of @trusted, then I don't see the point of having
> @trusted at all. What is the purpose? What is it meant to cover? In
> order for @trusted to make sense in this code segment (
> http://dpaste.dzfl.pl/f3d854feede9 ) I would say that the whole class
> will have to be marked @trusted. Is that possible?

I agree. I think it's possible, like this:

struct RCArray(E) {
@trusted:

But it doesn't help with mechanical checking -- @trusted is still basically more dangerous @system code.

-Steve

On Thursday, 26 February 2015 at 16:25:59 UTC, Steven Schveighoffer wrote:
> First, malloc should be safe, in the same way new is safe.

If it is typed and do the sizeof...

> I would say THIS is somewhat correct:
>
> (() @trusted {free(count); count=null;})();
>
> This takes something that is validly safe, and keeps the safety by ensuring the pointer is no longer dangling.
>
> However, we have an issue here. At any point inside the code, you could do:
>
> oldcount = count;
>
> And now, there is still potentially a dangling pointer somewhere. This means every place count is used must be checked. In this case, all uses of count have to be re-checked when the file is edited.

Yes, so count should only be accessible directly from @trusted. So you need to mark it "@trusted-only".

> This is a great example of why @trusted is mostly a convention thing, and not an enforceable thing.

Well, it would be enforceable if you had a proper type system built around it.

> I agree. I think it's possible, like this:
>
> struct RCArray(E) {
> @trusted:
>
> But it doesn't help with mechanical checking -- @trusted is still basically more dangerous @system code.

What you need is proper encapsulation. So that "dangerous state" and actions on that "dangerous state" is tied together.

And a strong type system.

On Thursday, 26 February 2015 at 12:50:04 UTC, anonymous wrote:
> The whole point of @trusted is to be able to call @system code. It doesn't matter if that code is "injected" or not. @safe prevents calling @system code.

But it should matter, because when you mark a unit @trusted you basically are signing off a "certificate" that says it acts like @safe in @safe code. How can you verify anything if you allow injections? You can only verify what is known when the verification took place. After that it should be frozen so that injections of non-verified code only goes into @safe "slots".

> [...]
>> It infers "@safe", but it does not provide validation.
>
> Yup. RCArray is de-facto trusted, i.e. verified memory-safe by the programmer. It's not compiler verified safe, even though it ends up being labeled @safe.

Well, but @safe code is not verified either... It is inferred @safe based on a fixed set of criterions, but not verified. To verify you need more, and you have to start with strong typing.

On 2/26/15 3:49 PM, "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= <ola.fosheim.grostad+dlang@gmail.com>" wrote:
> On Thursday, 26 February 2015 at 16:25:59 UTC, Steven Schveighoffer wrote:
>> First, malloc should be safe, in the same way new is safe.
>
> If it is typed and do the sizeof...

Right, one can easily make a @safe wrapper around malloc as long as free is never called :)

>
>> I would say THIS is somewhat correct:
>>
>> (() @trusted {free(count); count=null;})();
>>
>> This takes something that is validly safe, and keeps the safety by
>> ensuring the pointer is no longer dangling.
>>
>> However, we have an issue here. At any point inside the code, you
>> could do:
>>
>> oldcount = count;
>>
>> And now, there is still potentially a dangling pointer somewhere. This
>> means every place count is used must be checked. In this case, all
>> uses of count have to be re-checked when the file is edited.
>
> Yes, so count should only be accessible directly from @trusted. So you
> need to mark it "@trusted-only".

Right, the lure of @trusted delegates is that it does not take into account the leaking of memory details. You essentially make any @safe code that uses the same data as @trusted code now @trusted, but without a marking. It was the reason I was suggesting at one point that @trusted should internally mark variables as @trusted accessed so they could only be used in @trusted code.

Even with the suggestion I had above, the danger is still there for maintenance to this code to foil any guarantees made on the previous version.

>
>> This is a great example of why @trusted is mostly a convention thing,
>> and not an enforceable thing.
>
> Well, it would be enforceable if you had a proper type system built
> around it.

I'm not sure it's worth it. For something like RCSlice, it might just be good enough to mark the whole thing as @trusted, so the danger signs are put into the correct place. This is going to be a low-level primitive, that hopefully is fully encapsulated.

But of course, this is still by convention. I agree that any mechanical checking of @trusted is going to be impossible without a type notation for the data it touches.

I think one thing is for certain that I have learned through the discussions about @trusted -- we should avoid @trusted as much as possible in all code.

-Steve

On Thursday, 26 February 2015 at 20:56:52 UTC, Ola Fosheim Grøstad wrote:
> But it should matter, because when you mark a unit @trusted you basically are signing off a "certificate" that says it acts like @safe in @safe code. How can you verify anything if you allow injections?

If you allow @system injections, you can't verify safety, obviously. Such code must not be @trusted. That's why RCArray is not @trusted.

> You can only verify what is known when the verification took place. After that it should be frozen so that injections of non-verified code only goes into @safe "slots".

I don't know if it "should" work like that. I don't have a good enough understanding of the matter to argue for or against any one design.

[...]
> Well, but @safe code is not verified either... It is inferred @safe based on a fixed set of criterions, but not verified. To verify you need more, and you have to start with strong typing.

The criterion being: doesn't call @system code. If that's not enough to ensure memory-safety (given proper usage of @trusted) then I guess the current system has failed, or the implementation has bugs.

I don't think that the ability to break the type system (as done in RCArray) means that @safe has failed, though. We can forcefully break the type system in many ways. That doesn't mean it's not sound.

I'm probably not the guy to discuss "weak" and "strong" typing with. As I don't have a good understanding of type system theory.

On Thursday, 26 February 2015 at 16:25:59 UTC, Steven Schveighoffer wrote:
> However, we have an issue here. At any point inside the code, you could do:
>
> oldcount = count;
>
> And now, there is still potentially a dangling pointer somewhere. This means every place count is used must be checked. In this case, all uses of count have to be re-checked when the file is edited.
>
> Because count leaks the memory details of the trusted call to free (In other words, the fact that count, or any copy of it, can become a dangling pointer), I believe every call in that type that deals with count should be marked trusted.

The counter is freed in the destructor, nothing can happen after that.

On Thursday, 26 February 2015 at 20:56:52 UTC, Ola Fosheim Grøstad wrote:
> Well, but @safe code is not verified either... It is inferred @safe based on a fixed set of criterions, but not verified. To verify you need more, and you have to start with strong typing.

@safe is supposed to provide safety, if you can give an example when it doesn't, you can report a bug. There are indeed bugs in implementation of safety, like escaping of local variables, but they are supposed to be fixed eventually.