On 2/6/15 8:43 PM, Andrei Alexandrescu wrote:
> On 2/6/15 3:21 PM, weaselcat wrote:
>> On Friday, 6 February 2015 at 23:02:54 UTC, Zach the Mystic wrote:
>>
>>> No, at least three of us, Steven, H.S. Teoh and myself have confirmed
>>> that we've moved beyond requesting @trusted blocks. We are no longer
>>> requesting them. We are requesting *@system* blocks, which can only
>>> appear in @trusted and @system functions. Any unsafe code appearing in
>>> a @trusted function which is not inside a @system block is an error.
>>> We've changed the name, but I think it will make a world of difference
>>> regarding how you will look at it. Marking '@system' code inside a
>>> @trusted function is exactly what is requested. Your message about
>>> '@trusted' being only acceptable as an interface has been heard. There
>>> will be no @trusted blocks, only @system blocks, which are the exact
>>> same thing, except they can only appear in @trusted and @system
>>> functions.
>>>
>>> This solution appeals to me greatly. It pinpoints precisely where
>>> unsafe code can generate; it catches unintended safety violations in
>>> all @trusted code outside @system blocks, as requested by many people
>>> so far; it makes systems programming highly visible, with redundancy
>>> at the function signature and at the unsafe code itself. I really
>>> think it's spot on!
>>
>> this sounds interesting, is anyone going to make a DIP for it?
>
> Consider the previous code:
>
> https://github.com/D-Programming-Language/phobos/blob/accb351b96bb04a6890bb7df018749337e55eccc/std/file.d#L194
>
>
> that got replaced with:
>
> https://github.com/D-Programming-Language/phobos/blob/master/std/file.d#L194
>
>
> With the system proposal we're looking at something like:

Please understand, Nobody is saying "let's replace incorrect code with the same incorrect code with different tags!"

The idea is to properly identify which code needs more scrutiny, and keep the mechanical checking of @safe where we can.

-Steve

On Friday, 6 February 2015 at 23:25:02 UTC, Walter Bright wrote:
> On 2/6/2015 3:02 PM, Zach the Mystic wrote:
>> This solution appeals to me greatly. It pinpoints precisely where unsafe code
>> can generate; it catches unintended safety violations in all @trusted code
>> outside @system blocks, as requested by many people so far; it makes systems
>> programming highly visible, with redundancy at the function signature and at the
>> unsafe code itself. I really think it's spot on!
>
> I suspect that such a feature would simply lull people into a false sense of security in that merely tagging an unsafe cast with @system and the compiler accepting it is good enough.
>
> My evidence for this is how @trusted was used in Phobos.

I'm also not saying phobos was written perfectly to begin with. I think that this whole @system blocks suggestion came up in a slightly different context than did your original discontent with std.file and std.array. I'm not sure you're ever going to stop careless programmers from getting their bad code to compile. But I think that's a different issue from giving good, careful programmers the tools they need. Right now, @trusted functions are the only tool they have to help people figure out where the unsafe code is, but it's not enough. Nested @trusted functions, and @trusted lambas are clearly being added as a workaround for not being able to "un@trust" large portions of code intended to be @safe.

On Friday, 6 February 2015 at 18:52:45 UTC, Andrei Alexandrescu wrote:
> I think the problem is overstated. -- Andrei

I think there could hardly be a more persuasive argument that this belief is wrong than Walter himself just having made the mistake several times, and not even immediately realizing what is wrong: https://github.com/D-Programming-Language/phobos/pull/2966 [1]

Sorry for singling out this one example here. While it is particularly demonstrative, I am certainly not intending to put Walter in a bad light. It's simply hard to get that stuff right, as templates can make it hard to accurately determine the complete public interface of a function. As pointed out in the PR, here are some more examples for this class of bugs from Phobos code, also written and reviewed by top D coders:

https://issues.dlang.org/show_bug.cgi?id=14135
https://issues.dlang.org/show_bug.cgi?id=14136
https://issues.dlang.org/show_bug.cgi?id=14137
https://issues.dlang.org/show_bug.cgi?id=14138

Neither of those issues would have been prevented by your new guidelines; the code in question is already written in that way. Quite to the contrary, consequent application of minimal @trusted blocks or even the workaround you reject so strongly would have prevented all of the bugs except for 14138.

David


[1] The diff in question, for when the PR is fixed: https://github.com/klickverbot/phobos/commit/db647f62cb5279ae42ad98665cd60cdcdb9b3dd5

On Saturday, 7 February 2015 at 12:40:26 UTC, David Nadlinger wrote:
> Neither of those issues would have been prevented by your new guidelines; the code in question is already written in that way. Quite to the contrary, consequent application of minimal @trusted blocks or even the workaround you reject so strongly would have prevented all of the bugs except for 14138.

This is an incredibly poor argument. The fact that there is no documentation for why the functions are @trusted and why they have to be @trusted is testament to a flawed process.

If you insist on relying on half-assed flawed verification you only catch those instances where it should not have been @trusted in the first place, and which would have been caught at an early stage with a decent process, but you will keep missing out on the hard to detect cases.

You will run into much more difficult problems if you don't do something about the safety review process.

Fix the weak typing rather than making the language more convoluted, the latter compounds the problem in the long run.

And I'll add this:

Please do not turn the compiler into a inadequate verification tool. The compiler should do what it can do well, but what it cannot do it should not do at all and leave to an external verification tool.

@trusted basically tells the compiler "this is beyond your capability so we left it to someone else".

Third parties should be able to provide incrementally improved verification tools, for various purposes, without mandating language or compiler changes.

So what you want from the language is simple clean semantics and reasonable best practice annotations for verification that can be extended based on the needs in a particular environment.

On 2/7/15 4:40 AM, David Nadlinger wrote:
> On Friday, 6 February 2015 at 18:52:45 UTC, Andrei Alexandrescu wrote:
>> I think the problem is overstated. -- Andrei
>
> I think there could hardly be a more persuasive argument that this
> belief is wrong than Walter himself just having made the mistake several
> times, and not even immediately realizing what is wrong:
> https://github.com/D-Programming-Language/phobos/pull/2966 [1]
>
> Sorry for singling out this one example here. While it is particularly
> demonstrative, I am certainly not intending to put Walter in a bad
> light. It's simply hard to get that stuff right, as templates can make
> it hard to accurately determine the complete public interface of a
> function. As pointed out in the PR, here are some more examples for this
> class of bugs from Phobos code, also written and reviewed by top D coders:
>
> https://issues.dlang.org/show_bug.cgi?id=14135
> https://issues.dlang.org/show_bug.cgi?id=14136
> https://issues.dlang.org/show_bug.cgi?id=14137
> https://issues.dlang.org/show_bug.cgi?id=14138
>
> Neither of those issues would have been prevented by your new
> guidelines; the code in question is already written in that way. Quite
> to the contrary, consequent application of minimal @trusted blocks or
> even the workaround you reject so strongly would have prevented all of
> the bugs except for 14138.
>
> David
>
>
> [1] The diff in question, for when the PR is fixed:
> https://github.com/klickverbot/phobos/commit/db647f62cb5279ae42ad98665cd60cdcdb9b3dd5

Nice, thanks for this work. One good guideline here is to almost always let generic code rely on deduction instead of ascribing safety attributes to it.

Andrei

>> https://github.com/klickverbot/phobos/commit/db647f62cb5279ae42ad98665cd60cdcdb9b3dd5
>
> Nice, thanks for this work. One good guideline here is to almost always let generic code rely on deduction instead of ascribing safety attributes to it.
>
> Andrei

And making this work in functions that already marked @trusted and need to be, but should have some parts inferred is the hole point of this thread and the @trusted-misuse in phobos.

On Friday, 6 February 2015 at 23:21:50 UTC, weaselcat wrote:
> On Friday, 6 February 2015 at 23:02:54 UTC, Zach the Mystic wrote:
>
>> No, at least three of us, Steven, H.S. Teoh and myself have confirmed that we've moved beyond requesting @trusted blocks. We are no longer requesting them. We are requesting *@system* blocks, which can only appear in @trusted and @system functions. Any unsafe code appearing in a @trusted function which is not inside a @system block is an error. We've changed the name, but I think it will make a world of difference regarding how you will look at it. Marking '@system' code inside a @trusted function is exactly what is requested. Your message about '@trusted' being only acceptable as an interface has been heard. There will be no @trusted blocks, only @system blocks, which are the exact same thing, except they can only appear in @trusted and @system functions.
>>
>> This solution appeals to me greatly. It pinpoints precisely where unsafe code can generate; it catches unintended safety violations in all @trusted code outside @system blocks, as requested by many people so far; it makes systems programming highly visible, with redundancy at the function signature and at the unsafe code itself. I really think it's spot on!
>
> this sounds interesting, is anyone going to make a DIP for it?

It was Teoh's idea. Maybe he should have the honors?

On 2/7/15 7:11 AM, "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= <ola.fosheim.grostad+dlang@gmail.com>" wrote:

> You are trying to do this:
>
> 1. mechanically verify the whole @trusted region
>
> 2. manually verify the whole @trusted region, but be sloppy about it
> here an there
>
> 3. Ooops, we were sloppy in the wrong spot...

No.

A @trusted function is manually verified, period.

But we also must tag potential points of leakage with @system. In fact, it probably could be a warning/error if you have a @trusted function without any @system escapes (it could just be marked @safe).

Think of it this way: the @system tags are the only places where issues can creep into the function. But then you have to apply the leaks to the whole function. It makes the problem of finding potential safety issues more tractable, because the compiler forces us to identify the root causes.

-Steve

On Monday, 9 February 2015 at 14:40:36 UTC, Steven Schveighoffer wrote:
> On 2/7/15 7:11 AM, "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= <ola.fosheim.grostad+dlang@gmail.com>" wrote:
>
>> You are trying to do this:
>>
>> 1. mechanically verify the whole @trusted region
>>
>> 2. manually verify the whole @trusted region, but be sloppy about it
>> here an there
>>
>> 3. Ooops, we were sloppy in the wrong spot...
>
> No.
>
> A @trusted function is manually verified, period.
>
> But we also must tag potential points of leakage with @system. In fact, it probably could be a warning/error if you have a @trusted function without any @system escapes (it could just be marked @safe).

That's a nice migration path, btw. First, warn about @trusted functions without @system blocks and don't enforce @safe-ty inside them, later disallow them and do enforce @safe-ty in the others.