The article aims to explain how to use @safe, @system and importantly, @trusted, including all the hairy details of templates.

https://jakobovrum.github.io/d/2016/01/20/memory-safety.html

Any and all feedback appreciated.

On Wednesday, 20 January 2016 at 14:04:53 UTC, Jakob Ovrum wrote:
> The article aims to explain how to use @safe, @system and importantly, @trusted, including all the hairy details of templates.
>
> https://jakobovrum.github.io/d/2016/01/20/memory-safety.html
>
> Any and all feedback appreciated.

Nice article! Feeling like I have a much better grasp on the whole attribute system now that I read it, thanks! :-)

On Wednesday, 20 January 2016 at 14:04:53 UTC, Jakob Ovrum wrote:
> The article aims to explain how to use @safe, @system and importantly, @trusted, including all the hairy details of templates.
>
> https://jakobovrum.github.io/d/2016/01/20/memory-safety.html
>
> Any and all feedback appreciated.

I like the description of @trusted and template inference. Template inference, in particular, was not something that was obvious to me when first reading about D. I'm not sure how clear you make it that you can still mark templates @safe and what have you (you seem to just say don't make templates @trusted).

I wasn't aware of the point that "@trusted nested functions in templated functions do not have to have a memory safe interface as long as all calls to the function are memory safe". Interesting.

`auto p = () @trusted { return &t; } ();`

Huh, I thought Andrei was opposed to this idiom? Is it now considered reserved for templates or something has changed?

On Wed, Jan 20, 2016 at 07:25:43PM +0000, Dicebot via Digitalmars-d-announce wrote:
> `auto p = () @trusted { return &t; } ();`
> 
> Huh, I thought Andrei was opposed to this idiom? Is it now considered reserved for templates or something has changed?

Yeah, I thought this was exactly the case where some of us Phobos contributors got lambasted by Andrei and Walter for abusing @trusted.

The thing is, this is too easy to abuse, and too prone to careless mistakes with nasty consequences. Suppose you have a template function:

	auto func(T)(T t)
	{
		auto p = () @trusted { return &t; } ();
		...
		p.bar();
		...
	}

The problem is, the separation between p and p.bar() can be very large in a complicated function, and during maintenance, somebody accidentally introduces unsafe operations on p in the function body without realizing that it came from a function marked @trusted. However, the compiler won't catch this, because of the @trusted annotation.

Any exception to the strict usage of @trusted to me smells like a time bomb waiting to explode. It may not be today or tomorrow, but sooner or later somebody is going to slip up and the compiler won't help you. It's bad enough that every single change to a @trusted function must be vetted to ensure actual safety; now we have to also vet any modification to any function that contains @trusted anonymous functions? In a large template function, it's too easy to miss these @trusted sub-functions, because if the code change is far away enough, the @trusted annotation won't even show up in the diff. So reviewers may not even realize it's a change that may have broken @trusted.

I'm pretty sure somebody brought up a case where such a hack was actually necessary... but I'd still tread very, very carefully before recommending such a thing to the general D coder.


T

-- 
The best compiler is between your ears. -- Michael Abrash

On Wednesday, 20 January 2016 at 14:04:53 UTC, Jakob Ovrum wrote:
> The article aims to explain how to use @safe, @system and importantly, @trusted, including all the hairy details of templates.
>
> https://jakobovrum.github.io/d/2016/01/20/memory-safety.html
>
> Any and all feedback appreciated.

Nice article. I got a much better understanding reading it.

A small thing - I immediately tried adding @safe to one of my programs and hit an issue related to the first example in the article:

    void main() @safe {
        import std.stdio;
        writeln("hello, world");
    }

This is passes the @safe constraint, but 'stdout.writeln()' and 'stderr.writeln()' do not. (My program uses stderr.) stderr/stdout/stdin are __gshared and can't be referenced by safe code. The module level version of writeln, etc., access a trusted version of stdout to avoid this.

I don't have a specific suggestion for addressing this in the article. It's nicely written and delving into this may take away from the key points. But, I wanted to point this out in case others hit it.

(Note: This related to a recent thread on the learn forum: http://forum.dlang.org/thread/vkihzrwomhiwdzqelrxa@forum.dlang.org)

On Wednesday, 20 January 2016 at 19:55:45 UTC, H. S. Teoh wrote:
> On Wed, Jan 20, 2016 at 07:25:43PM +0000, Dicebot via Digitalmars-d-announce wrote:
>> `auto p = () @trusted { return &t; } ();`
>> 
>> Huh, I thought Andrei was opposed to this idiom? Is it now considered reserved for templates or something has changed?
>
> Yeah, I thought this was exactly the case where some of us Phobos contributors got lambasted by Andrei and Walter for abusing @trusted.

That was for non-templated functions where this approach makes no sense. Indeed it is counterproductive, because @trusted on the whole function is a better indication of what needs to be reviewed for memory safety (the whole function!).

> Any exception to the strict usage of @trusted to me smells like a time bomb waiting to explode. It may not be today or tomorrow, but sooner or later somebody is going to slip up and the compiler won't help you. It's bad enough that every single change to a @trusted function must be vetted to ensure actual safety; now we have to also vet any modification to any function that contains @trusted anonymous functions? In a large template function, it's too easy to miss these @trusted sub-functions, because if the code change is far away enough, the @trusted annotation won't even show up in the diff. So reviewers may not even realize it's a change that may have broken @trusted.

It is the only way to solve this problem.

On Wednesday, 20 January 2016 at 15:28:05 UTC, jmh530 wrote:
> I like the description of @trusted and template inference. Template inference, in particular, was not something that was obvious to me when first reading about D. I'm not sure how clear you make it that you can still mark templates @safe and what have you (you seem to just say don't make templates @trusted).

Templated functions can still be explicitly annotated with attributes, which disables inference for those attributes. This is often a good idea even for templated functions when template arguments do not inject code, so that every instantiation has the same, known set of attributes. Attribute inference can handle it, but explicit annotations provide documentation value. I might incorporate this into the article, but I'm wary of it losing focus.

> I wasn't aware of the point that "@trusted nested functions in templated functions do not have to have a memory safe interface as long as all calls to the function are memory safe". Interesting.

It is a necessary evil to propagate attributes correctly. Don't use it when you don't have to.

On Wednesday, 20 January 2016 at 20:28:03 UTC, Jon D wrote:
> This is passes the @safe constraint, but 'stdout.writeln()' and 'stderr.writeln()' do not. (My program uses stderr.) stderr/stdout/stdin are __gshared and can't be referenced by safe code. The module level version of writeln, etc., access a trusted version of stdout to avoid this.

Yeah, the standard library still has a ways to go even with @safe.

I always imagined that the standard pipes should use shared as opposed to __gshared. I don't think the current implementation is thread-safe, but I don't know how this affects in memory safety in this case.

On Thu, Jan 21, 2016 at 04:38:08AM +0000, Jakob Ovrum via Digitalmars-d-announce wrote:
> On Wednesday, 20 January 2016 at 15:28:05 UTC, jmh530 wrote:
> >I like the description of @trusted and template inference. Template inference, in particular, was not something that was obvious to me when first reading about D. I'm not sure how clear you make it that you can still mark templates @safe and what have you (you seem to just say don't make templates @trusted).
> 
> Templated functions can still be explicitly annotated with attributes, which disables inference for those attributes. This is often a good idea even for templated functions when template arguments do not inject code, so that every instantiation has the same, known set of attributes. Attribute inference can handle it, but explicit annotations provide documentation value. I might incorporate this into the article, but I'm wary of it losing focus.

A common idiom used in Phobos related to this is to use an attributed (pure, nothrow, @nogc, etc.) unittest that instantiates the template in question, to ensure that it does not accidentally become non-pure, throwing, etc. (the unittest will stop compiling if so). This way, we ensure that the template itself doesn't contain any impure / throwing / @system code, even though you can instantiate it with template parameters that may be impure, throwing, @system, etc..


> >I wasn't aware of the point that "@trusted nested functions in templated functions do not have to have a memory safe interface as long as all calls to the function are memory safe". Interesting.
> 
> It is a necessary evil to propagate attributes correctly. Don't use it when you don't have to.

Wouldn't it be better to refactor the code to separate out the part that needs to be @trusted into a separate place, with a safe API? Or are there cases for which this is impossible?


T

-- 
The early bird gets the worm. Moral: ewww...