Since neither Andrei or Walter are able to say something sensible on these issues when asked, I apparently need to learn something about the "consistency" of C memory safety.

I'm happy to listen to anyone who can explain this to me:

1. My understanding is that @trusted is supposed to give memory safety escapes by providing a context which reestablish a memory safety context on return.

Yet in this thread http://forum.dlang.org/thread/mcik3j$153g$1@digitalmars.com it is stated that this paradigm is an example of «careful use of @trusted»:

    count = (() @trusted => cast(uint*) malloc(uint.sizeof))();
    …arbitrary code…
    (() @trusted => free(count))();

They way I see it, this is equivalent to typing a reinterpret_casting malloc and free as memorysafe operations in isolation, basically providing a malloc!int() and free() as memory safe functions. But why is malloc and free not considered safe by default then? These @trusted functions clearly cannot prevent leaks within their own context. You would need a @trusted-only storage class on the receiving pointer to do that and a @trusted move type.

If this is careful use of @trusted, then I don't see the point of having @trusted at all. What is the purpose? What is it meant to cover? In order for @trusted to make sense in this code segment ( http://dpaste.dzfl.pl/f3d854feede9 ) I would say that the whole class will have to be marked @trusted. Is that possible?


2. In a recent change D now has implemented "return ref", but when I ask if this has implications for coroutines that take ref as parameters, I get no answer. So what is the deal here? Are coroutines banned from taking ref arguments? Will it be possible to retain unsafe references by yielding? Since the feature is implemented I presume there is a clear answer to this.

What is the scope of memory safety in D? Is it a lint-like feature, or is it something to be relied upon?

On Tuesday, 24 February 2015 at 22:37:58 UTC, Ola Fosheim Grøstad wrote:
> If this is careful use of @trusted, then I don't see the point of having @trusted at all. What is the purpose? What is it meant to cover? In order for @trusted to make sense in this code segment ( http://dpaste.dzfl.pl/f3d854feede9 ) I would say that the whole class will have to be marked @trusted. Is that possible?

In general, @trusted means "I have proven myself that this code is actually safe, eeven though it uses unsafe features." The compiler has to be pessimistic and assume that everything which can be used unsafely will be used unsafely. @trusted, as it is used here, is used to say, "I assure you I have used this in a safe manner."

I would like to see @trusted blocks, although I can see Andrei's argument of not making it easy to do intentionally.

On Tuesday, 24 February 2015 at 22:37:58 UTC, Ola Fosheim Grøstad wrote:
> 1. My understanding is that @trusted is supposed to give memory safety escapes by providing a context which reestablish a memory safety context on return.

Yep, that's how I got it, too. A @trusted function is supposed to be memory-safe.

> Yet in this thread http://forum.dlang.org/thread/mcik3j$153g$1@digitalmars.com it is stated that this paradigm is an example of «careful use of @trusted»:
>
>     count = (() @trusted => cast(uint*) malloc(uint.sizeof))();
>     …arbitrary code…
>     (() @trusted => free(count))();
>
> They way I see it, this is equivalent to typing a reinterpret_casting malloc and free as memorysafe operations in isolation, basically providing a malloc!int() and free() as memory safe functions.

Yep. "Careful use": You have be careful when you (ab)use @trusted like this.

The idea is that the compiler enforces safety for the rest of the code. You have to be cautious about the effects of the @trusted malloc/free, but the compiler checks the other stuff. If the whole function was @trusted, the compiler wouldn't catch other safety violations that are not related to malloc/free.

The downside is that @safe on that function then doesn't mean "compiler verified memory-safe" anymore. Instead it means "compiler assisted @trusted".

There's also the other way around: Mark the function as @trusted and throw ()@safe{...}() covers over the non-problematic parts. This doesn't work when a template parameter affects the safety, though.

> But why is malloc and free not considered safe by default then?

Well, because they aren't.

> These @trusted functions clearly cannot prevent leaks within their own context. You would need a @trusted-only storage class on the receiving pointer to do that and a @trusted move type.
>
> If this is careful use of @trusted, then I don't see the point of having @trusted at all. What is the purpose? What is it meant to cover? In order for @trusted to make sense in this code segment ( http://dpaste.dzfl.pl/f3d854feede9 ) I would say that the whole class will have to be marked @trusted. Is that possible?

The goal is to have human verified, compiler recognized memory-safety, when E allows for it.

You can't:
* mark nothing @safe/@trusted, because malloc/free are not safe;
* mark the methods @trusted, because E may be unsafe.

@trusted malloc/free is a hack, but it allows the compiler to infer @safe iff E is safe.

On Tuesday, 24 February 2015 at 22:49:17 UTC, w0rp wrote:
> In general, @trusted means "I have proven myself that this code is actually safe, eeven though it uses unsafe features." The compiler has to be pessimistic and assume that everything which can be used unsafely will be used unsafely. @trusted, as it is used here, is used to say, "I assure you I have used this in a safe manner."

From http://dlang.org/function.html#trusted-functions :

«Trusted functions are guaranteed by the programmer to not exhibit any undefined behavior if called by a safe function.»

I take this to mean that anything that is wrapped up in @trusted should not violate memory safety when in injected into any arbitrary context marked as @safe.

> I would like to see @trusted blocks, although I can see Andrei's argument of not making it easy to do intentionally.

If @trusted is meant to be used the way he is doing it in this example then there is no reason to not provide @trusted-blocks. An inlined lambda is a block...

But it provides poor encapsulation and makes for weak typing.

On Wednesday, 25 February 2015 at 00:12:41 UTC, anonymous wrote:
> If the whole function was @trusted, the compiler wouldn't catch other safety violations that are not related to malloc/free.

You don't need to. In C++ you use a separate "@trusted" data-structure for capturing ownership, aka unique_ptr (which provides a type related to linear typing).

> The downside is that @safe on that function then doesn't mean "compiler verified memory-safe" anymore. Instead it means "compiler assisted @trusted".

Not only on that function, on the whole data structure? You no longer have a scope for what code is considered dangerous.


> There's also the other way around: Mark the function as @trusted and throw ()@safe{...}() covers over the non-problematic parts. This doesn't work when a template parameter affects the safety, though.

That sounds more attractive than the provided example, but the right thing to do is to establish proper encapsulation. That means you need a protection level that is stronger than "private" that restricts "unsafe state" to a @trusted vetted construct. Like unique_ptr informally does in C++.


>> But why is malloc and free not considered safe by default then?
>
> Well, because they aren't.

So that should change?


> The goal is to have human verified, compiler recognized memory-safety, when E allows for it.
>
> You can't:
> * mark nothing @safe/@trusted, because malloc/free are not safe;
> * mark the methods @trusted, because E may be unsafe.
>
> @trusted malloc/free is a hack, but it allows the compiler to infer @safe iff E is safe.

You mean outside RCArray, iff RCArray as a whole is manually verified?  But that would surely mean that the @trusted region is RCArray and neither the constructor or malloc/free?

And that assumes strong typing, which D currently does not provide. Without strong typing it will be very difficult for the compiler to infer anything across compilation units.

It's not only code itself to manage here, but also its evolution: http://forum.dlang.org/post/mbaksa$1ers$1@digitalmars.com
That's why it was proposed to change trusted so that it won't allow implicit usage of system code.

On Wednesday, 25 February 2015 at 09:49:41 UTC, Kagamin wrote:
> It's not only code itself to manage here, but also its evolution: http://forum.dlang.org/post/mbaksa$1ers$1@digitalmars.com
> That's why it was proposed to change trusted so that it won't allow implicit usage of system code.

But that will make things worse, not better, since the major issue is a lack of tools and process:

http://forum.dlang.org/thread/mb0uvr$2fdb$1@digitalmars.com?page=13#post-pqcattbwjsaqoekisrrc:40forum.dlang.org

It improves things without tools. Tools are always welcome, e.g. dfix already does something.

On Wednesday, 25 February 2015 at 07:07:00 UTC, Ola Fosheim Grøstad wrote:
> On Wednesday, 25 February 2015 at 00:12:41 UTC, anonymous wrote:
[...]
> That sounds more attractive than the provided example, but the right thing to do is to establish proper encapsulation. That means you need a protection level that is stronger than "private" that restricts "unsafe state" to a @trusted vetted construct. Like unique_ptr informally does in C++.

I'm not knowledgeable enough to agree or disagree here.

>>> But why is malloc and free not considered safe by default then?
>>
>> Well, because they aren't.
>
> So that should change?

We can't make malloc and free actually memory-safe, can we? We must not mark public unsafe functions @safe/@trusted.

> You mean outside RCArray, iff RCArray as a whole is manually verified?  But that would surely mean that the @trusted region is RCArray and neither the constructor or malloc/free?

RCArray as a whole is the actually trusted region, yes, since it must be manually verified that RCArray.array isn't leaked. But you can't mark it @trusted, because E may be unsafe.

The @trusted malloc/free dance is a mean hack to solve a problem. There may be other solutions that don't require breaking @trusted. Those may be better.

> And that assumes strong typing, which D currently does not provide. Without strong typing it will be very difficult for the compiler to infer anything across compilation units.

I don't follow.

On Wednesday, 25 February 2015 at 18:58:13 UTC, anonymous wrote:
> We can't make malloc and free actually memory-safe, can we? We must not mark public unsafe functions @safe/@trusted.

My point was that there is no conceptual difference between having a named function trusted_malloc!int() and trusted_free() and wrapping them up individually unnamed.

> RCArray as a whole is the actually trusted region, yes, since it must be manually verified that RCArray.array isn't leaked. But you can't mark it @trusted, because E may be unsafe.

But the semantic analysis should verify that code isn't injected unless it is also @trusted?

>> And that assumes strong typing, which D currently does not provide. Without strong typing it will be very difficult for the compiler to infer anything across compilation units.
>
> I don't follow.

C is not strongly typed, and neither is D. That means there are holes in the type system.