On Wednesday, 27 May 2020 at 13:47:46 UTC, Claude wrote:
> On Wednesday, 27 May 2020 at 13:42:08 UTC, Andrej Mitrovic wrote:
>> Is the actual problem those `@trusted:` declarations at the top of C headers?
>>
>> There could be a simple solution to that:
>>
>> Ban `@trusted:` and `@trusted { }` which apply to multiple symbols. Only allow `@trusted` to apply to a single symbol. For
>
> IMO, it makes things worse. Because the careless programmer will slap @trusted to every declaration (maybe with a script or find/replace macro of his editor). So now, we don't know if the annotation is greenwashing or careful examination of the definition.
>
> At least with "@trusted:" and "@trusted { }", the greenwashing is obvious.

the problem here ist: we are talking about careless programmers.
for those of us who actually care about safety it will be like:
1. ok lets write an extern(c) function.
2. forget to annotate it by accident.
3. hit compile
4. a) get an compiler error the hints that we missed that annotation
or b) no error at all and we now have silently added unsafe code to our codebase.

also, you can never trust that an @trusted function is actually memory safe.
even if it was evaluated in the past. the addition of new code (and even completely @safe code) can break any assumptions that @trusted fun was relying on.

so a competent QA team _has_ to either check all @trusted code fragments.
or with this dip check all @trusted and all extern functions.

On 5/27/20 9:42 AM, Andrej Mitrovic wrote:
> On Wednesday, 27 May 2020 at 09:50:50 UTC, Walter Bright wrote:
>> Un-annotated C declarations should be a red flag to any competent QA team. Recognizing a false @trusted is a whole lot harder.
> 
> Is the actual problem those `@trusted:` declarations at the top of C headers?
> 
> There could be a simple solution to that:
> 
> Ban `@trusted:` and `@trusted { }` which apply to multiple symbols. Only allow `@trusted` to apply to a single symbol. For example:

Oh wow what an interesting idea. Thanks.

On Friday, 22 May 2020 at 18:14:12 UTC, H. S. Teoh wrote:
> Then myFunc would become callable from @safe code, provided the passed-in argument is also @safe.
>
> The crucial point here is that while compiling myFunc, the compiler doesn't (need to) know the @safe-ty of `cb`, it can just treat it as an opaque object that it assumes the safety of, while it verifies the rest of the function body.
>
> This is parallel to C functions being of unverifiable safety, so if extern(C) functions were somehow marked and treated as opaque objects of unknown safety, then the compiler can still verify the rest of the code and produce a certificate of safety (modulo the C APIs used).
>
> If we had a way of expressing conditional safety, it could be a way to salvage @safe from this current situation.

I had this thought a million times, tried to explain it at two DConfs but no-one (as it appeared to me) got the point.

The main difference is, the function calling `myFunc` "knows" whether the delegate/function pointer it used as an argument is @safe or not. It's perfectly decidable.

Personally, I'd even go that far and say, that's how all the function attributes (also pure, nothrow, @nogc) are intended to work.

On Friday, May 22, 2020 9:29:03 AM MDT Timon Gehr via Digitalmars-d-announce wrote:
> On 22.05.20 16:49, bachmeier wrote:
> > I don't see that marking an extern(C) function @trusted buys you anything, at least not until you can provide a compiler guarantee for arbitrary C code.
>
> It buys you the ability to call that function from @safe code. Clearly you can't mark it @safe because the compiler has not checked it.

Yes, and @trusted in general buys you the ability to segregate and find code that is doing stuff that the compiler was unable to prove was @safe. Then when there's a bug related to memory safety, you know where to look.

C functions cannot be @safe, because they haven't been vetted by the compiler. So, either what they're doing is actually memory safe (and it's fine to mark whem with @trusted), or what it's doing is memory safe if used correctly, and the calling code needs to be marked @trusted when using it correctly. Either way, the code with potential memory safety issues is segregated, and so when inevitably you do run into a memory safety bug, you know that it's either the @trusted code or the @system code that it calls which needs to be fixed.

Of course, with this DIP, @safe no longer truly means that the code was vetted by @safety by the compiler but rather that all of the code that the compiler can see has been vetted for @safety, meaning that when there is an @safety bug, you must look not only for @trusted code, but you must look for non-extern(D) declarations which have been implictly treated as @trusted by the compiler. @safe still has value (and may even provide more value in that it will be used more often), but it provides much weaker guarantees in the process.

- Jonathan M Davis

On Wednesday, 27 May 2020 at 13:42:08 UTC, Andrej Mitrovic wrote:
> There could be a simple solution to that:
>
> Ban `@trusted:` and `@trusted { }` which apply to multiple symbols. Only allow `@trusted` to apply to a single symbol. For example:

That came to my mind immediately. I'm not entirely sure about `@trusted { }`, because it might be handy at times (I don't use `@attribute { }` often). But `@trusted:` is such that it is almost guaranteed that sooner or later it's applied to something it wasn't intended to. I have no idea if that's the case with braces.
Since `@trusted` is intended to be special, I'd go the full route and ban it for any non-direct use. Mass-`@trusted` makes no sense logically: If the programmer is required to investigate a function to determine it's `@safe`-ty, the time required for the investigation clearly outweighs the time marking the function `@trusted` (if the investigation has that result, of course), even if the judgment is fast.

If people go through a whole lot of `extern(C)` declarations and marks them @trusted individually because neither `@trusted:` nor `@trusted { }` did the job, it may occur to them it's probably not the right thing.

I really hope this makes it through.

On Friday, May 22, 2020 12:09:16 PM MDT rikki cattermole via Digitalmars-d- announce wrote:
> It kept being swapped about in the discussion thread, so I have been a little on edge over people using non-extern(D). Because linkage doesn't mean anything to anything but to cpu and linker in this discussion.

Except that the linker matters a great deal in this discussion with regards to extern(D) functions, because @safe and @trusted are part of the name mangling for extern(D) functions. That means that if an extern(D) function declaration's attributes do not match its definition, then you'll get a linker error. So, treating non-extern(D) function declarations as @safe by default isn't necessarily a problem (though it would certainly work to just treat all function declarations as @system by default rather than treating extern(D) function declarations differently). The cases where non-extern(D) function declarations weren't actually @safe would be caught during the linking process. Sure, it would be nice if it were caught sooner, but you don't end up with them being invisibly treated @safe when they're not like we're going to get with DIP 1028.

However, of course, with non-extern(D) declarations, even if the function definition is actually written in D, the fact that the function body was checked is not transmitted via the linking process, because it doesn't end up in the name mangling. So, the linkage used has a huge impact on whether you can rely on the @safe attribute on the function declaration actually meaning anything about whether the body was verified by the compiler.

- Jonathan M Davis

On Thursday, 28 May 2020 at 01:14:43 UTC, Jonathan M Davis wrote:
> On Friday, May 22, 2020 12:09:16 PM MDT rikki cattermole via Digitalmars-d- announce wrote:
>> [...]
>
> Except that the linker matters a great deal in this discussion with regards to extern(D) functions, because @safe and @trusted are part of the name mangling for extern(D) functions. That means that if an extern(D) function declaration's attributes do not match its definition, then you'll get a linker error. So, treating non-extern(D) function declarations as @safe by default isn't necessarily a problem (though it would certainly work to just treat all function declarations as @system by default rather than treating extern(D) function declarations differently). The cases where non-extern(D) function declarations weren't actually @safe would be caught during the linking process. Sure, it would be nice if it were caught sooner, but you don't end up with them being invisibly treated @safe when they're not like we're going to get with DIP 1028.
>
> [...]

I remember reading a suggestion that additional linker symbols be emitted to carry the attribute and possibly type information while leaving the ABI untouched.  Was this found to be impractical?

On Wednesday, May 27, 2020 6:24:25 PM MDT Andrei Alexandrescu via Digitalmars- d-announce wrote:
> On 5/27/20 9:42 AM, Andrej Mitrovic wrote:
> > On Wednesday, 27 May 2020 at 09:50:50 UTC, Walter Bright wrote:
> >> Un-annotated C declarations should be a red flag to any competent QA team. Recognizing a false @trusted is a whole lot harder.
> >
> > Is the actual problem those `@trusted:` declarations at the top of C headers?
> >
> > There could be a simple solution to that:
> >
> > Ban `@trusted:` and `@trusted { }` which apply to multiple symbols. Only
>
> > allow `@trusted` to apply to a single symbol. For example:
>
> Oh wow what an interesting idea. Thanks.

I've argued for years that mass-applying any attribute is bad practice. It makes it way too easy to accidentally apply an attribute to a function, and experience has shown that it makes it _far_ easier to not realize that a funtion already has a particular attribute (it's definitely happened in Phobos PRs that an attribute has been mass-applied and then someone comes along later and applies it to a specific function, because they didn't think that that function had that attribute). Unfortunately, plenty of people seem to love to mass apply attributes rather than marking each function individually (presumably, because they hate having to use attributes all over the place). And regardless of whether DIP 1028 is a good idea as-is, the sad reality of the matter is that it's not uncommon even in druntime for someone to slap @trusted: at the top of a module. It implies that the declarations in question were not actually checked, and it's incredibly error-prone when new declarations are added.

Personally, I normally only use : with access-level modifiers, and while I like doing that (especially since I think that public and private functions should normally be segregated anyway), I'm increasingly coming closer to the conclusion that it's actually a bad idea in practice, because it's not uncommon for people to not know which access-level modifier applies - especially when looking at diffs in PRs.

I would _love_ to see it become illegal to mass-apply @trusted (or even attributes in general), but I have no clue how easy it would be to get such a DIP accepted or how much screaming there would be over it if it were actually accepted.

- Jonathan M Davis

On Wednesday, May 27, 2020 8:13:52 PM MDT Bruce Carneal via Digitalmars-d- announce wrote:
> On Thursday, 28 May 2020 at 01:14:43 UTC, Jonathan M Davis wrote:
> > On Friday, May 22, 2020 12:09:16 PM MDT rikki cattermole via
> >
> > Digitalmars-d- announce wrote:
> >> [...]
> >
> > Except that the linker matters a great deal in this discussion with regards to extern(D) functions, because @safe and @trusted are part of the name mangling for extern(D) functions. That means that if an extern(D) function declaration's attributes do not match its definition, then you'll get a linker error. So, treating non-extern(D) function declarations as @safe by default isn't necessarily a problem (though it would certainly work to just treat all function declarations as @system by default rather than treating extern(D) function declarations differently). The cases where non-extern(D) function declarations weren't actually @safe would be caught during the linking process. Sure, it would be nice if it were caught sooner, but you don't end up with them being invisibly treated @safe when they're not like we're going to get with DIP 1028.
> >
> > [...]
>
> I remember reading a suggestion that additional linker symbols be emitted to carry the attribute and possibly type information while leaving the ABI untouched.  Was this found to be impractical?

Steven suggested something along those lines. I don't know how practical it would or wouldn't be, but I don't think that Walter even responded to the idea.

Regardless, even if it worked perfectly, all such a solution would deal with would be the cases where you have a non-extern(D) function which is actually written in D and had its function body marked as @safe (and thus was verified to be @safe). Such functions definitely exist, but the main problem is really non-extern(D) functions which are actually written in C or C++ or whatnot. They won't have been checked for @safety.

Right now, such declarations are treated by the compiler as @system by default, because it cannot check them for @safety, whereas with DIP 1028, it will then treat them as @safe in spite of the fact that it can't check them for @safety simply because Walter thinks that it would be too exceptional to treat them differently from the general rule of @safe by default and that if they're left as @system, too many people will just slap @trusted on their code as a whole to get the compiler to shut up.

But regardless of whether DIP 1028 is the correct decision, the problem remains that your typical extern(C) function cannot be checked for @safety by the compiler, because it was compiled by a C compiler. The question then is just how the D compiler should treat them, and that's the main point of contention. There may be solutions like Steven suggested which would deal with edge cases where the implementation is in D but the linkage isn't, but ultimately, they're just edge cases.

- Jonathan M Davis

On Tuesday, May 26, 2020 8:58:16 PM MDT Andrei Alexandrescu via Digitalmars-d- announce wrote:
> On 5/26/20 12:31 PM, Bruce Carneal wrote:
> > Currently a machine checked @safe function calling an unannotated extern C routine will error out during compilation. This is great as the C routine was not machine checked, and generally can not be checked.  Post 1028, IIUC, the compilation will go through without complaint.  This seems quite clear.  What am I missing?
>
> If that's the case, it's the death of DIP 1028.

Walter has acknowledged the problem and seems to think that because it's the programmer's responsibility to deal with extern(C) functions correctly (since it's not possible for the compiler to do it), it's up to the programmer to go and fix any existing code that should be marked @system and isn't and that having the compiler incorrectly mark extern(C) declarations as @safe isn't a big problem, because programmers need to be spending the time to check them anyway. He's already created some PRs to try to fix some issues with extern(C) declarations in druntime and explicitly markingthem as @system but doesn't seem to think that it's ultimately a big deal.

- Jonathan M Davis