July 11
On 7/11/24 18:23, Walter Bright wrote:
> mark everything trusted

:(

You can't mark a function @trusted if it does not have a @safe interface.
July 13
On Thursday, 11 July 2024 at 16:26:59 UTC, Walter Bright wrote:
> On 7/9/2024 8:58 AM, Nick Treleaven wrote:
>> [...]
>
> That's right. Converting an entire program to @safe can be done successfully by doing it incrementally, getting it to pass its test suite at each step. The way to do it incrementally is to start by labeling each function @trusted, then making them @safe one by one.
>
> I've done this, it works.
>
> And, as you observe, grepping for "@trusted" makes for a builtin-in TODO list!

This to me is the most sensible way to go about it.
July 13
On 7/13/24 14:37, aberba wrote:
> On Thursday, 11 July 2024 at 16:26:59 UTC, Walter Bright wrote:
>> On 7/9/2024 8:58 AM, Nick Treleaven wrote:
>>> [...]
>>
>> That's right. Converting an entire program to @safe can be done successfully by doing it incrementally, getting it to pass its test suite at each step. The way to do it incrementally is to start by labeling each function @trusted, then making them @safe one by one.
>>
>> I've done this, it works.
>>
>> And, as you observe, grepping for "@trusted" makes for a builtin-in TODO list!
> 
> This to me is the most sensible way to go about it.

It's not. You'll quickly lose track of the 3 kinds of `@trusted` functions:

- @trusted functions that should be @safe
- @trusted functions that should be @trusted
- @trusted functions that must be @system

This is plain safewashing.

If you want incremental safety in the current language, you have to start at the leaves, and put your TODO's in comments.

OpenD has an interesting approach towards default safety. They enabled a large fraction of safety checks in functions without a safety annotation. To disable them, explicit @system is required. This way you get linting benefits even without already ensuring full memory safety guarantees.

Anyway, there are many language-assisted ways to make incremental @safe migration work (some would require only minimal changes to the compiler), but I think slapping @trusted everywhere will not do anything to improve the perceptions about D's memory safety.
July 13
On 7/9/24 17:53, Nick Treleaven wrote:
> ...
> `@system:`
> ...

Has similar issues as `@safe:`.

July 13
On Saturday, 13 July 2024 at 13:09:14 UTC, Timon Gehr wrote:
> On 7/13/24 14:37, aberba wrote:
>> On Thursday, 11 July 2024 at 16:26:59 UTC, Walter Bright wrote:
>>> [...]
>> 
>> This to me is the most sensible way to go about it.
>
> It's not. You'll quickly lose track of the 3 kinds of `@trusted` functions:
>
> - @trusted functions that should be @safe
> - @trusted functions that should be @trusted
> - @trusted functions that must be @system
>
> This is plain safewashing.
>
> If you want incremental safety in the current language, you have to start at the leaves, and put your TODO's in comments.
>
> OpenD has an interesting approach towards default safety. They enabled a large fraction of safety checks in functions without a safety annotation. To disable them, explicit @system is required. This way you get linting benefits even without already ensuring full memory safety guarantees.
>
> Anyway, there are many language-assisted ways to make incremental @safe migration work (some would require only minimal changes to the compiler), but I think slapping @trusted everywhere will not do anything to improve the perceptions about D's memory safety.

I would add one point, Walter wrote in his original port:

"In fact, I've been doing just that. Adding @safe: at the top, and then everything that fails to compile gets marked @trusted. ****Eventually****, refactor the code as time permits."

Ouch!

July 13

On Thursday, 11 July 2024 at 16:26:59 UTC, Walter Bright wrote:

>

The way to do it incrementally is to start by labeling each function @trusted, then making them @safe one by one.

I've done this, it works.

I know you've done the first part, because there's now 1122 @trusted functions in dmd.backend. Many of them have an unsafe interface such as:

@trusted
void list_delete(list_t list) { free(list); }

Not much actual @safe code has come from it unfortunately.

July 13
On 7/13/2024 10:16 AM, Dennis wrote:
> Not much actual `@safe` code has come from it unfortunately.

That's because the design of it resists @safe requirements.

For a major one, there's `struct elem`:

https://github.com/dlang/dmd/blob/master/compiler/src/dmd/backend/el.d#L69

Note the union in it, which maps pointers on top of non-pointers. This is detected as unsafe. And it is unsafe! (But it works.)

Use of `struct elem` permeates the backend.

Then there is the `struct code`:

https://github.com/dlang/dmd/blob/master/compiler/src/dmd/backend/x86/code_x86.d#L299

which relies on `union evc`:

https://github.com/dlang/dmd/blob/master/compiler/src/dmd/backend/code.d#L43

which also "unionizes" pointers and values.

That pretty much covers the backend.

Nevertheless, I still have been creating PRs that replace other unsafe constructs, like replacing pointers to arrays with D arrays, and replacing pointers to single objects with references. (This is why I wanted the ref DIP implemented, but I have to delay using it until the bootstrap compiler supports it.)

The problem with the front end is different. It's problem is the lack of leaves! Due to the use of lazy semantic analysis, the compiler is very recursive. You can't make one function @safe without simultaneously making every function in the cycles @safe. Detouring through @trusted is the only practical path forward with that.

Another issue is the use of printf and its buddies. I've proposed ways of making printf @safe, but you've objected to them. There are several ways of making it @safe, and it's reasonable to do them, but I always run into protests :-)

(To clarify, I am not proposing to make printf itself safe, only making calls to printf safe by adjusting its format string, and flagging unsafe use of it.)
July 13

On Saturday, 13 July 2024 at 18:56:49 UTC, Walter Bright wrote:

>

Another issue is the use of printf and its buddies. I've proposed ways of making printf @safe, but you've objected to them. There are several ways of making it @safe, and it's reasonable to do them, but I always run into protests :-)

(To clarify, I am not proposing to make printf itself safe, only making calls to printf safe by adjusting its format string, and flagging unsafe use of it.)

I need a link to that proposal and the rejections.

July 13
On 7/13/2024 6:09 AM, Timon Gehr wrote:
>> This to me is the most sensible way to go about it.
> 
> It's not. You'll quickly lose track of the 3 kinds of `@trusted` functions:
> 
> - @trusted functions that should be @safe
> - @trusted functions that should be @trusted
> - @trusted functions that must be @system
> 
> This is plain safewashing.

Not if the @trusted functions have a safe interface (though the compiler cannot check the interface for safety, it is up to the programmer).


> If you want incremental safety in the current language, you have to start at the leaves, and put your TODO's in comments.

In my efforts to make the front end @safe, that approach does not work. The problem is that the front end flow graph is cyclical. There aren't many leaves, and a lot of those leaves in the front end have been marked as @safe.


> OpenD has an interesting approach towards default safety. They enabled a large fraction of safety checks in functions without a safety annotation. To disable them, explicit @system is required. This way you get linting benefits even without already ensuring full memory safety guarantees.

It's an interesting approach, but certainly half-baked. How does one decide which safety checks are to be included?


> Anyway, there are many language-assisted ways to make incremental @safe migration work (some would require only minimal changes to the compiler), but I think slapping @trusted everywhere will not do anything to improve the perceptions about D's memory safety.

You've often been quite adamant about memory safety being 100% (and I tend to appreciate that point of view and am in agreement with it). A half-baked approach won't help, either.

With the @trusted approach, I'll often change it to @safe, fix what I can, then put it back to @trusted.
July 13

On Thursday, 11 July 2024 at 16:26:59 UTC, Walter Bright wrote:

>

On 7/9/2024 8:58 AM, Nick Treleaven wrote:

>

Any pattern to subvert safe involves the @trusted attribute (except for compiler bugs, which editions help to fix). @trusted is an obvious flag in code review that memory-safety may be violated. Grepping for @trusted is an easy way to narrow down places where memory-safety is violated.

Arguing that @trusted can be abused is a tiny problem compared to the far harder to detect abuse in @system by default code.

That's right. Converting an entire program to @safe can be done successfully by doing it incrementally, getting it to pass its test suite at each step. The way to do it incrementally is to start by labeling each function @trusted, then making them @safe one by one.

I've done this, it works.

And, as you observe, grepping for "@trusted" makes for a builtin-in TODO list!

This is not the right approach. You start with marking things @safe that are leaves, and then move up. Leave things as @system until you can mark them as @safe.

And all templates should be unmarked.

Marking everything @trusted at first is just going to end in disaster.

When I converted mysql-native to have a @safe interface, I would start by marking the module @safe: at the top, and see what breaks. Then I would mark code @system that was not safe, and see if I could massage it into being @safe. I never marked anything @trusted unless I could prove that it had a safe interface.

In some cases, marking @safe: at the top does work! But it needs to be a module with no templates in it. In many cases, after finding all the safety problems, I had to mark selective pieces as @safe.

In my case, the largest problem with my code was using Variant which cannot be safe. I sure as hell could not mark all of variant trusted, as it's in Phobos, and it's also not safe.

Another problem with marking @safe: at the top, and bludgeoning your way through with @trusted marks is that you will end up marking actual @system code as @trusted, when it doesn't need to be. You only need the edge of the safety barrier to be marked as trusted.

And finally, if you mark everything trusted, and then find out later "oh, this actually cannot be made safe", then all of a sudden you might have to mark it as system, and now it cascades to a bunch of other things that you thought were already marked safe, and now those have to be marked system, and so on. Worst case scenario, you have external code depending on the trustedness, and now you break other people's code!

My recommendation is to use @safe: at the top to find where you need to address safety problems, and then handle one item at a time, marking @safe, @trusted or @system as appropriate. Then remove the top line (if it's not all safe).

-Steve