On Thu, May 21, 2020 at 06:22:19PM -0700, Walter Bright via Digitalmars-d-announce wrote:
> I have made these points before, but I'll summarize them here for convenient referral.
[...]

Thank you.  This makes your position clear, even if I don't entirely agree with it.  On that note, though: this really should have been part of the DIP acceptance announcement, not added as an afterthought. Otherwise people will perceive (correctly or not) the DIP as being accepted without acknowledging any of the issues raised.  This summary is exactly what would have defused a lot of the frustrations raised here.  For the future may I recommend making it a point to include such a summary with the announcement of each DIP accepted/rejected.


T

-- 
Nobody is perfect.  I am Nobody. -- pepoluan, GKC forum

On 5/21/2020 8:36 PM, Paul Backus wrote:
> Something ought to be done to prevent this. It doesn't have to be the exact
> proposal from the discussion thread, but doing nothing and allowing widespread
> silent breakage cannot possibly be the best solution.

I can see that happening. A simple example would be:

    extern (C) void free(void* p);
    ...
    free(p);
    free(p);

The thing is, you are no worse off than now. If free() can be misused by calling it from system code, it can be misused by calling it from safe code.

There's no way the compiler can detect this. If you annotate it, you'll just have to annotate it correctly. Forcing an annotation just means slapping @safe: at the beginning of the file and moving on - it's not going to help.

On 5/21/2020 4:45 PM, H. S. Teoh wrote:
> This makes it sound like you think that those who disagree with you
> disagree with @safe by default.  That is not the case.

I'm sure you all know what I'm talking about.

On Friday, 22 May 2020 at 01:22:19 UTC, Walter Bright wrote:
> I have made these points before, but I'll summarize them here
> for convenient referral.
>
> [...]

Thank's for the reasoning, that should be added to the DIP acceptance since the beginning.

Stated that we need to live with that, I'm asking you a simple question:
What's the "official" definition of the @safe advantages that D offers as a language?

Sometime we need to explain that to customers asking for details about the code we wrote for them.

I would love to have a good insight detailing the promises that the language is committed to hold, maybe trying to give me a vision on the future ... since from now on I will stop using "mechanical verification" at all as selling argument.

On Friday, 22 May 2020 at 01:22:19 UTC, Walter Bright wrote:
> I have made these points before, but I'll summarize them here
> for convenient referral.
>

<big snip> of material indicating, among other things, that even really good programmers can screw up when it comes to language design now and then.  Ahem.
Continuing...

>
> How does this relate to safe by default?
>
> Consider the common (because that's how D started out) case of:
>
> ----- clibrary.d --------
>
>     T massage_data(... many parameters ...);
>     ... 200 more such declarations ...
>
> ----- app.d ----------
>
>     import clibrary;
>
>     void useClibrary( ... parameters ...) {
>         massage_data(parameters);
>     }
>
> ---------------------
>
> This code, today, does not use annotations and it works. It's been working
> for a long time. Now, we implement the amendment, and it stops compiling
> because useClibrary is @safe and massage_data is @system. The user is faced
> with the following alternatives:
>
> 1. Go through 200 functions in clibrary.d and determine which are @safe
> and which are @system. This is what we want them to do. We try to motivate
> this with compiler error messages. Unfortunately, this is both tedious and
> thoroughly impractical, as our poor user Will Not Know which are safe and
> which are system. We can correctly annotate core.stdc.stdio because I know
> those functions intimately. This is not true for other system C APIs, and
> even less true for some third party C library we're trying to interface to.

Agree completely.   Annotating C code correctly can be very difficult.  This argues against defaulting such code to @safe.

>
> 2. Annotate useClibrary() as @trusted or @system. While easier, this causes
> all benefits to @safe by default to be lost.

No.  The benefit of @safe is actually having machine checkable @safety.

>
> 3. Wrap the call to massage_data() with:
>
>     () @trusted { massage_data(parameters); } ();
>
> If there are a lot of calls to clibrary, this is going to look pretty awful.
> Nobody likes writing or reading such ugly code. It's ok here and there, but
> not as a general thing.

Yeah, this is ugly.

>
> 4. Edit clibrary.d and make the first line:
>
>     @safe:
>
> I submit that, just like with Java, Option 4 is what people will reach for,
> nearly every time. I've had some private conversations where people admitted
> this was what they'd do. People who knew it was wrong to do that.

The DIP gives us 4), the worst case, by default.  Anywhere and everywhere a C lib shows up it gives us 4), automatically.

>
> If it's @safe by default, and then someone chooses to annotate it with @system
> here and there, I'd feel a lot more confident about the accuracy of the code
> annotations than if it just had @safe: at the top. At least they tried.
>

So @safe is to be based on a feeling?  At least they *tried*?  This is a key point of disagreement.  Machine checked @safety is something.  Human hand waving @safety is not.

It is bizarre that no one has been able to get you to see the contradiction with your earlier writings on safety (big fan there btw). C's biggest blunder?  Yeah, well, all the bugs engendered by that blunder and many many more are now to be considered @safe by default. What the heck?

Finally, Atila, can you shed any light on this?  Where do you stand?  Why did it seem like a good idea to withhold your veto?

On 22.05.20 03:22, Walter Bright wrote:
> This is Obviously A Good Idea. Why would I oppose it?
> 
> 1. I've been hittin' the crack pipe again.
> 2. I was secretly convinced, but wanted to save face.
> 3. I make decisions based on consultation with my astrologer.
> 4. I am evil.

The sarcasm is not appreciated.

[...]> Now, we implement the amendment, and it stops compiling
> because useClibrary is @safe and massage_data is @system. The user is faced
> with the following alternatives:
> 
> 1. Go through 200 functions in clibrary.d and determine which are @safe
> and which are @system. This is what we want them to do. We try to motivate
> this with compiler error messages. Unfortunately, this is both tedious and
> thoroughly impractical, as our poor user Will Not Know which are safe and
> which are system. We can correctly annotate core.stdc.stdio because I know
> those functions intimately. This is not true for other system C APIs, and
> even less true for some third party C library we're trying to interface to.
> 
> 2. Annotate useClibrary() as @trusted or @system. While easier, this causes
> all benefits to @safe by default to be lost.

In the past you have said repeatedly that the point of @safe is to be "100% mechanically checkable, not correct by convention" [1]. In other words, it's supposed to be watertight.

Now you're saying that @safe can have "benefits" without being watertight. You're even widening an existing hole in @safe to make those benefits more accessible.

If you have actually given up on a watertight @safe, that's disappointing for those of us who were with you on a watertight @safe.

> 3. Wrap the call to massage_data() with:
> 
>      () @trusted { massage_data(parameters); } ();
> 
> If there are a lot of calls to clibrary, this is going to look pretty awful.
> Nobody likes writing or reading such ugly code. It's ok here and there, but
> not as a general thing.

Let me get this straight. As a general rule we have: Unsafe stuff is "supposed to be inconvenient" [2]. But unsafe calls to C functions must be convenient, because they're so common. So we add an exception to the rule. And that exception is cool. But making an exception in @safe-by-default for `extern (C)` prototypes is not cool. Because exceptions are bad.

Makes sense ... not.

> 4. Edit clibrary.d and make the first line:
> 
>      @safe:
> 
> I submit that, just like with Java, Option 4 is what people will reach for,
> nearly every time. I've had some private conversations where people admitted
> this was what they'd do. People who knew it was wrong to do that.

Unlike the catch-and-ignore of your Java analogy, `@safe:` does no harm. Or rather, it does the same amount of harm that your @safe-by-default `extern (C)` declarations do. But with the amendment, the idiot programmer at least has to pull the trigger to shoot their foot off. You're letting the compiler do it for them.

That @safe even applies to `extern (C)` prototypes is its own issue.

> If it's @safe by default, and then someone chooses to annotate it with @system
> here and there, I'd feel a lot more confident about the accuracy of the code
> annotations than if it just had @safe: at the top. At least they tried.

The morale of your Java analogy is that people will find and use the easy way out. No one is going to add @system to the declarations if the code already compiles with @safe-by-default.

> What is actually accomplished with this amendment if it was implemented?
> 
> 1. Adds a funky, special case rule. It's better to have simple, easily
> understood rules than ones with special cases offering little improvement.
> 
> 2. Existing, working code breaks.

You can't forgo breakage when making @safe the default. Any template that is inferred as @system will throw a wrench in your suddenly-@safe `main`.

> 3. The most likely code fixes are to just make it compile, absolutely
> nothing safety-wise is improved. The added annotations will be a fraud.

And your solution is to let the compiler add the frauds.

> D should not encourage "greenwashing" practices like the Java
> exception specification engendered. The compiler cannot vet the accuracy
> of bodyless C functions, and we'll just have to live with that. The proposed
> amendment does not fix that.

The proposed amendment at least makes the greenwashing visible. With the DIP as it is, the greenwashing is still there. The compiler just does it for you. Imagine Java doing catch-and-ignore by itself, just so that the idiot programmer doesn't have to do it.

It would be better if the greenwashing wasn't even possible. I.e., don't allow @safe on `extern (C)` prototypes. If that's what you're going for, great. But you're not, are you?

By the way, I don't think you have acknowledged Steven's idea [3] at all, yet. Maybe you've missed it. It's completely different from the "Obviously A Good Idea" amendment you've discussed here.




[1] https://issues.dlang.org/show_bug.cgi?id=14125#c26
[2] https://forum.dlang.org/post/r5hnjo$1j40$1@digitalmars.com
[3] https://forum.dlang.org/post/r6kvm4$1vq5$1@digitalmars.com

First, thank you for the explanation! I have a few observations though, mainly that the exception analogy feels like more an argument against the DIP than for it. And I also have an alternative proposal that might be considered?

On Friday, 22 May 2020 at 01:22:19 UTC, Walter Bright wrote:
> This, of course, was incredibly tedious and annoying. So what happened?
>
> 1. Functions would just do a try-catch(Throwable) and ignore the exceptions.
> Oops.
>
> 2. The Exception Specifications became simply `throws Throwable`. Oops.

This is what happens in swift these days:

func g() /*throws*/ {
  do {
    try f()
  } catch {}
}

But, it's very rare. And passing something like that in code review is very difficult and, almost always, the reviewer will ask for comments so other people will know why the laziness (if it even really was laziness, sometimes it's the right thing to do).

Of course the one big difference between exception specifications and @safe is the implied importance; @safe is compiler-enforced and has the user's trust, exception specifications are human guess work of what are the important exceptions and how varied they need to be - it's API design - very hard to get right.

The exception specification becoming "throws Exception", while highlights the failure of exception specifications, does so with a non-dangerous resolution. I.e. you still need to mark your function as throwing, and you still need to at least "try" to catch the exception. Even a forced "catch(Exception)" is better than nothing and shows where problems can be and encourages scrutiny.

> How does this relate to safe by default?

That people will use the most convenient solution to get their work to compile if the juice is not worth the squeeze?

>
> 1. Go through...
>
> 2. Annotate...safe by default to be lost.
>
> 3. Wrap the call...
>
> 4. Edit clibrary.d and make the first line:
>
>     @safe:

5. Edit clibrary.d and make the first line @trusted: ?

Which brings me to the proposal of making @safe illegal on function prototypes. Reason being that @safe stays compiler checked. @trusted maintains it's path and value in hunting down memory corruption. extern(Anything) functions cannot lie anymore about being compiler-checked.

>
> I submit that, just like with Java, Option 4 is what people will reach for,
> nearly every time. I've had some private conversations where people admitted
> this was what they'd do. People who knew it was wrong to do that.
>
> If it's @safe by default, and then someone chooses to annotate it with @system

Which is the rare case in this scenario right? And 1) there's no way to tell if someone made an effort in the case where none are marked @system. 2) this is also only for the case of C library bindings (reason being that most of my code, and also others' code I've seen, has extern(C) functions splattered around various D files and not in a dedicated clibrary.d file - is this just me?)

> here and there, I'd feel a lot more confident about the accuracy of the code
> annotations than if it just had @safe: at the top. At least they tried.
>
> What is actually accomplished with this amendment if it was implemented?
>
> 1. Adds a funky, special case rule. It's better to have simple, easily
> understood rules than ones with special cases offering little improvement.
>
> 2. Existing, working code breaks.
>
> 3. The most likely code fixes are to just make it compile, absolutely
> nothing safety-wise is improved. The added annotations will be a fraud.
>
> D should not encourage "greenwashing" practices like the Java
> exception specification engendered. The compiler cannot vet the accuracy
> of bodyless C functions, and we'll just have to live with that. The proposed
> amendment does not fix that.
>
> And so, I did not incorporate the proposed amendment to the Safe by Default
> DIP.

I think there's a failure to understand why explicit greenwashing via a deliberate (possibly code reviewed) addition of @safe: at the top is worse than a global implicit greenwashing by the compiler, ignoring the rare "feel-good-but-can't-be-sure" scenario above where some functions are marked as @system, and especially given the premise that programmers are lazy.

Also ... error on @safe prototypes?

> I can see that happening. A simple example would be:
>
>     extern (C) void free(void* p);
>     ...
>     free(p);
>     free(p);
>
> The thing is, you are no worse off than now. If free() can be misused by calling it from system code, it can be misused by calling it from safe code.

Wrong :-(. The scenario is this:

```
@safe void foo(int* p)
{   import customfreefunction.noannotations;
    p.free;
    p.free;
}
```

Now, this will not compile, because `free` is `@system`. But if `free` is in unannotated module, this will compile after the DIP implementation. Obviously not with the standard library `free` because it'll be annotated `@system` anyway, but with some custom `free` function this is an issue.

If the situation in using new features is indeed as dire as you fear, a better alternative would have been to just reject the DIP. Then there would be less `@safe` code, but at least you could trust `@safe` much more, due to the above phenomenon. I do think there would have been a way to have `@safe` by default even with the assumptions you made about it's abuse, but the DIP reviews are over so I don't think it's worth explaining anymore.

But at least you gave the reasonable rationale we wanted. Thank you.

On 5/22/20 4:24 AM, ag0aep6g wrote:
> By the way, I don't think you have acknowledged Steven's idea [3] at all, yet.

Yes, he did. With more nitpicking as a rebuttal, but I'm not really interested in having more discussions about this DIP or that proposal.

https://forum.dlang.org/post/ra7958$2r8v$1@digitalmars.com

-Steve

On Friday, 22 May 2020 at 01:22:19 UTC, Walter Bright wrote:
> I have made these points before, but I'll summarize them here
> for convenient referral.

Thanks Walter.  I really appreciate you taking the time to do this, as it's obviously no fun to be getting a big tide of negativity in this way.

> This is Obviously A Good Idea. Why would I oppose it?
>
> 1. I've been hittin' the crack pipe again.
> 2. I was secretly convinced, but wanted to save face.
> 3. I make decisions based on consultation with my astrologer.
> 4. I am evil.

... wait, we don't ALL make decision like that? :-)

In seriousness: I really, really wish that everyone throwing around personal attacks would take a moment to stop and think about the consequences of pursuing this kind of discourse: it distracts from the technical, practical and organizational issues at hand, it creates a worse mood than exists already, and generally makes it harder to have a constructive conversation around the core disagreements here.

It is perfectly possible to critique the decision taken, or the way in which it was taken, or to voice personal frustrations, without casting aspersions on the decision-maker.

With the rationale laid out clearly as it is here, I do have some responses in mind.  But before sharing them, I'd like to know whether that would be useful right now: I've no wish to just press for a re-hashing of conversations that have already been had.