June 17, 2021
On Thursday, 17 June 2021 at 01:07:05 UTC, Paul Backus wrote:
> On Thursday, 17 June 2021 at 00:34:12 UTC, Steven Schveighoffer wrote:
>> [...]
>
> It's impossible to guarantee, at the language level, that @safe code can never require manual review. The programmer is allowed to use any and all knowledge at their disposal to verify the memory safety of @trusted (or in your proposal, @system-block) code, including knowledge about @safe code.
>
> You might say, "the only thing a @trusted function can possibly know about a @safe function is its signature, so that doesn't matter," but that's not quite true. If the @trusted function and the @safe function are in the same module, the @trusted function can (in principle) rely on the inner workings of the safe function without invalidating its proof of memory safety, since the programmer knows that any given version of the @trusted function will only ever call the corresponding version of the @safe function.
>
> Of course, such a @trusted function should never pass code review. But the fact that it can exist in principle means that you cannot truly rely on @safe to mean "no manual checking required", even if @trusted lambdas and nested functions are forbidden.

I understand there is a big difference between "never need to check absent compiler error" and "only need to check if someone who wrote the code should find another line of work", but there is also a big difference between where we are now and where we could be, particularly since improvements in this area will yield compounding benefit.

June 16, 2021
On 6/16/2021 2:25 PM, Timon Gehr wrote:
> Yes. This.

At last, Timon, we agree on something! You've made my day!
June 17, 2021
On Thursday, 17 June 2021 at 01:33:39 UTC, Bruce Carneal wrote:
> On Thursday, 17 June 2021 at 01:07:05 UTC, Paul Backus wrote:
>> [...]
>
> I understand there is a big difference between "never need to check absent compiler error" and "only need to check if someone who wrote the code should find another line of work", but there is also a big difference between where we are now and where we could be, particularly since improvements in this area will yield compounding benefit.

More succinctly: I agree that the changes under discussion will not protect against arbitrarily devious coding practices, but I believe they would meaningfully reduce the likelihood of innocent mistakes.

I think that we should move towards safety unless it is believed that the change would inhibit future improvements or would put large burdens on current users.
June 16, 2021
I'm sure you can guess where this post is going.

Consider the following controversial language features:

1. implicit variable declaration
2. non-void default initialization
3. macros
4. operator overloading for non-arithmetic purposes
5. arithmetic in version declarations
6. transitive const
7. default constructors
8. direct access to shared values
9. emulation of hardware vector operations
10. recoverable assert failures
11. @trusted statements

Ok, I have successfully sold (1) and (2). But
the others are all uphill struggles for D.

What they all have in common is they're programming
language crack cocaine.

By that I mean that these features make programming faster
and easier. It's like that first hit of crack enables
a superpower where prodigious quantities of code can be
quickly churned out. Wow! What could be wrong with that?

On the other hand, there's dour old Walter dressed in his
severe nun's habit rapping the knuckles with a yardstick anyone
reaching for those treats.

The problem is those features come with a downside. Some
become apparent earlier, some later, even a decade later.
The downside is unreadable, unmaintainable code. Of course,
everyone personally believes that *they* can judiciously
and smartly snort the crack and there won't be downside.

Here's where I play the experience card. All those features
transform code into s**t after a while, no matter who uses them.
Including me. Earlier in my career, I would have happily implemented
all of them, because more power is better. Writing code faster is
better.

No it isn't. Writing code faster isn't better. More power isn't
always better. Having a language that pushes towards writing code
that is more readable is better, safer is better, more maintainable
is better.

Even C, (in)famous for never breaking backwards compatibility,
had implicit function declaratations for 30 years until quietly
taking it behind the woodshed and slitting its throat. (The verbiage
for it just disappeared from the C99 spec.)

That's for context. The rest will be about the @trusted proposal.

The question is: Why is @trusted at the function level rather than
at the statement level? It certainly seems more convenient to apply
it with statement granularity, and it will save 4 characters of typing
over the lambda approach. What could be wrong with that?
And indeed, that so far appears to be the general reaction.

The idea of putting it at the function level is to force (I know,
that sounds bad, but indulge me for a moment) the programmer
to think about the decomposition of programs into safe and unsafe
code. Ideally, untrusted code should be encapsulated and segregated
into separate sections of code, with clean, well-defined, well-considered,
and well thought through interfaces.

At statement level, one just schlepps @trusted in front and gives it
no more consideration. It is thoughtlessly applied, the compiler error
goes away, Mission Accomplished! It might as well be renamed
the @shaddup attribute. Zero thought is given to carefully crafting
a safe interface, because a safe interface to it is not required.
Of course that's tempting.

The ()@trusted{ ... }() is equally bad. Note that there are no parameters
to it, hence NO interface. This was discovered as a way to evade the
intent that @trusted was to be applied to a function interface. It
had never occurred to me to add semantics to disallow it. Even worse,
I myself have been seduced into using this trick.

But I consoled myself that at least it was ugly, and required an extra
4 characters to type. Unfortunately, that hasn't been enough of a
deterrence, as it appears to have been used with abandon.

I recall it was David Nadlinger who originally pointed out that @trusted
at the function level, even if only one statement was unsafe, would
hide safety issues in the rest of the function. Hence the appearance
of the lambda workaround. He is correct. But the real problem was in
failing to encapsulate the unsafe code, and place it behind a sound
and safe interface.

As for my own pre-safe code, I've been gradually upgrading it to be
fully @safe. It's a satisfying result.

Use of @trusted lambdas is a code smell, and making @trusted work at the
statement level is putting the stamp of approval on a stinky practice.
Enabling this is a one way street, it'd be very very hard to undo it.

P.S. A subtler aspect of this is D's semantic reliance on rich function
signatures. This passes critical semantic information to the compiler,
such as inputs, outputs, escapes, live ranges, who's zoomin' who, etc.
Having @trusted at the statement level, with no defined interface to it,
just torpedoes the compiler's ability to reason about the code.
What is the interface to an @trusted statement? Who knows.
An @trusted statement can do anything to the surrounding code in ways nearly
impossible for a compiler to infer. After all, that's why @system code
is called @system in the first place. It doesn't play by the rules.
June 17, 2021

On Wednesday, 16 June 2021 at 11:38:54 UTC, RazvanN wrote:

>

...

Cheers,
RazvanN

[1] https://issues.dlang.org/show_bug.cgi?id=17566
[2] https://github.com/dlang/dlang.org/pull/2260

Yes
@trusted lamdas are ugly but necessary in templates where @safe/@system is deducted.


struct Foo{
	
    int test()@safe{
    	return 1;
    }

}
struct Bar{
	
    int test()@system{
    	return 2;
    }

}

void system_fn()@system{

}

int template_fn(T)(T x){
    ()@trusted{
    	system_fn();
    }();

    return x.test();
}


void main()@safe{

	template_fn(Foo.init);   //ok
	template_fn(Bar.init);   //Error: `@safe` function `D main` cannot call `@system` function
}

June 17, 2021
On Thursday, 17 June 2021 at 01:07:05 UTC, Paul Backus wrote:
> Of course, such a @trusted function should never pass code review.

This isn't true. @trusted code may rely on invariants throughout the code base. As it should. And frequently does (or would if you tried to convert system code to safe). You can not make those invariants local to @trusted in the general case.

For instance algorithms that rely on sentinels. There is no reason to make all code that place sentinels @trusted.

There is a difference between preparing for danger and executing danger.


June 17, 2021
On Thursday, 17 June 2021 at 02:02:58 UTC, Walter Bright wrote:
> The idea of putting it at the function level is to force (I know,
> that sounds bad, but indulge me for a moment) the programmer
> to think about the decomposition of programs into safe and unsafe
> code. Ideally, untrusted code should be encapsulated and segregated
> into separate sections of code, with clean, well-defined, well-considered,
> and well thought through interfaces.

Then you have to make the "calls unsafe code" marker transitive. Which is completely unreasonable.

It is a mistake to think that unsafety is related to interfaces. That would only apply to the most trivial usage of unsafe code.

If @safe is meant to be useful then it should be done in a way that makes people want to use it. That includes people who currently slap @system all over their code base. That includes people who write kernels and device drivers.

If you want people to use safety features then you also have to design it in a way that makes people not hate their code base.

> At statement level, one just schlepps @trusted in front and gives it
> no more consideration. It is thoughtlessly applied, the compiler error
> goes away, Mission Accomplished!

Slapping one @trusted over the whole function is even easier than slapping two @trusted on 2 statements inside the function body.

So that argument makes no sense to me.

(Also "@trusted" looks far more innocent to newbies than "@unsafe", but that is another issue.)

> It might as well be renamed
> the @shaddup attribute. Zero thought is given to carefully crafting
> a safe interface, because a safe interface to it is not required.

Your lexer in DMD will obviously never be safe as it is written. It can be made trusted, but if @safe code overwrite sentinels then it will obviously break.

In actual system level code you cannot make the assumption that memory safety is related to interfaces. It is related to datastructures, events and timelines and their invariants.


> As for my own pre-safe code, I've been gradually upgrading it to be
> fully @safe. It's a satisfying result.

Well, all of dmd can be rewritten to be @safe with no problem. As is the case for most batch programs.

But a compiler is not really system level code.

> P.S. A subtler aspect of this is D's semantic reliance on rich function
> signatures. This passes critical semantic information to the compiler,
> such as inputs, outputs, escapes, live ranges, who's zoomin' who, etc.
> Having @trusted at the statement level, with no defined interface to it,
> just torpedoes the compiler's ability to reason about the code.

Easy solution: make @trusted inferred, and introduce @unsafe at the statement level.

Then let IDEs annotate call chains with some kind of syntax highlighting or other marker that lets people know that a call chain includes @unsafe code.

One thing that makes people dislike the look of D code is exactly those "rich" function signatures. It has a negative impact on programmers ability to interpret function signatures. Especially newbies.

It is easier to just slap @system on the whole code base. If you write system code. Which also is the kind of code that could benefit most from @safe...



June 17, 2021
On Wednesday, 16 June 2021 at 18:28:48 UTC, Timon Gehr wrote:
> On 16.06.21 13:38, RazvanN wrote:
>> 
>> 
>> What do you think?
>
> @trusted nested functions are an antipattern and this enshrines them in a language feature.

+1

Keep them only at function level.
June 17, 2021
On Thursday, 17 June 2021 at 02:02:58 UTC, Walter Bright wrote:

>
> The idea of putting it at the function level is to force (I know,
> that sounds bad, but indulge me for a moment) the programmer
> to think about the decomposition of programs into safe and unsafe
> code. Ideally, untrusted code should be encapsulated and segregated
> into separate sections of code, with clean, well-defined, well-considered,
> and well thought through interfaces.

No, the lambda approach will not lead to those. Just like every first D project out there abuses "static if" to hack around the limitations of "version", people will use something like "int a = () @trusted{ ... terrible hacks... }();" everywhere.

I think you are fighting a strawman here: we are not arguing that powerful features won't be abused. We are arguing that your safeguards will not prevent the abuse. By allowing immediately called nullary lambdas (sematic equivalent of blocks), you are not forcing people to think about anything but how to hack around the lack of trusted blocks.

I could play an experience card too, and say that the shittiest code I have ever seen was written to work around programming language limitations. So, your argument works both ways: yes, powerful features may and do lead to shitty code, and, yes, limitations may and do lead to shitty code.
June 17, 2021

Nice post! Just one thing I don't understand:

On Thursday, 17 June 2021 at 02:02:58 UTC, Walter Bright wrote:

>

Consider the following controversial language features:
...
2. non-void default initialization
...
6. transitive const
...
Ok, I have successfully sold (1) and (2). But
the others are all uphill struggles for D.

Are you trying to "sell" the idea that this list of features is bad? In that case, are you saying D's const and default initialization of T x = T.init; instead of T x = void; are bad? I get the other items on the list, but (2) and (6) confuse me.