On Thursday, 21 May 2020 at 21:41:53 UTC, Steven Schveighoffer wrote:
> The unfortunate end result of this change is that safety will be gutted with all C functions being trusted by default

I'm really sorry, Walter, but I have to agree with Steve on this point.  This was the one aspect of the DIP that I really wanted to get modified, I also proposed a number of options for how to do so, and I'm really worried about the consequences of this decision.

I recognize and respect your right to make decisions as you see fit for the language, but I really wish I could persuade you to change your mind about this one :-)

On Thursday, 21 May 2020 at 20:46:09 UTC, Walter Bright wrote:
> On 5/21/2020 9:14 AM, Seb wrote:
>> Why we can't we have a technical board where the community can vote in experts and potentially companies could even buy a seat for $$$ which would mean a lot more for them than the current very vague sponsorship options.
>> I'm aware that Walter doesn't like the idea of giving up ownership, but it makes all the other people question why they should still bother with this process and not simply fork and move to an open, transparent development...
>
> I expected flak from this decision. I'm prepared to take the flak because this is the right decision. I did not make it lightly.

That's interesting that you are saying it is the "right" decision. Let's analyze that for a second:

What this boils down to is

    Simplicity Vs. Memory Safety

Is the **correct** code actually simpler? A beginner that doesn't have the knowledge of D's safety system will ultimately create C bindings that are implicitly @safe without realizing the consequences of their actions.

This change introduces a risk to memory safety, extern(C) declarations that were @system will now be flipped to @safe. No, putting @system: at the top isn't a solution, extern(C) functions aren't all nicely put in their own modules. Druntime and Phobos both have extern(C) declarations throughout their source; I guarantee you missed them with your PR's trying to illustrate how simple it is. As others will most definitely miss them as well.

Reasons for Memory Safety:
- Less error prone and simpler for beginners, the default is what the *correct* behavior should be. They organically discover @safe writing C bindings.
- Less error prone for advanced users, won't accidentally call a C function that isn't explicitly verified as being @trusted.
- Switch to @safe by default won't introduce potential security bugs from extern(C) functions not being annotated @system in the switch.
- Proven implementation by other safe languages (Rust).

Reasons for Simplicity:
- simpler logic for the compiler
- simpler rules for the user that default to *incorrect* code
- Fear of the unknown, unforeseen complexity
- trust Walter


This is where the divide is, from my perspective. There's this fear of the unknown, that making it slightly more complicated will introduce these unforeseen complications that will far outweigh any benefit that is added from it.

I don't see what that can be. There are other languages that have safe by default, and then have C declarations be unsafe by default. They don't have any of these devastation complications you seem to think that there will be.

This is what doesn't sit right with me. If you could give an example of a complication that could occur, then I think you could get more people onboard. But it would have to be justifiable against the increased security risk of having extern(C) declarations be @safe by default. For a DIP that is trying to increase safety, it is a bit ironic that it will also be reducing safety at the same time.

Otherwise right now, as it stands, this basically boils down to Logic Vs. Fear.

> Please keep in mind that I've made other unpopular decisions that have proven their worth over time. I hope you'll reserve judgement until we all see how this change plays out.
>
> If it does turn out badly, it's on me and I'll take my lumps.

You've also done the opposite, you can't justify it with such a statement unless you've never made a mistake. This whole process is suppose to avoid exactly this; a single person taking a gamble.

On Thu, May 21, 2020 at 01:46:09PM -0700, Walter Bright via Digitalmars-d-announce wrote: [...]
> I expected flak from this decision. I'm prepared to take the flak because this is the right decision. I did not make it lightly.

This makes it sound like you think that those who disagree with you disagree with @safe by default.  That is not the case.  I think in general, (almost?) everyone here agrees that safety by default is the way to go.  We all agree that @safe by default ought to be in the language, in one form or another. That's not the dispute here.

The dispute concerns the specific implementation of @safe by default as set out in this DIP. Specifically, the weakening of the promise of @safe by implicitly trusting extern(C) declarations to be @safe, which is equivalent to programming by convention ("trust the programmer to remember to write @system on extern(C) prototypes"). Which, ironically, undermines the whole purpose of this DIP. (That is, unless I misunderstood, and the whole purpose of this DIP is to undermine @safety and thereby make it even more of a joke than it currently is right now.)


> Please keep in mind that I've made other unpopular decisions that have proven their worth over time. I hope you'll reserve judgement until we all see how this change plays out.
[...]

Oh I'm waiting to see this one plays out, you bet!  What will come of trusting the programmer to do the right thing when it comes to annotating extern(C) prototypes with @system?  Well, I'm sure it will all work out well in the end, since, after all, it isn't as if we didn't have ~50 years or so of experience with programming by convention, y'know, esp. in C code (that we should implicitly trust when calling from D, btw), which has led to beautiful security holes and exploits caused by memory corruption and other such wonderful things.

Or, since you like airplane analogies, there's absolutely nothing wrong with designing your cockpit controls such that the default behaviour, when the pilot does not indicate otherwise, is to nose-dive into the ground.  After all, if he didn't mean to do that, he should've remembered to override the default behaviour explicitly, right? That's what he was trained to do, anyway.  And besides, it's easier to implement the controls this way, since doing otherwise would add excessive complication to the design.


T

-- 
The early bird gets the worm. Moral: ewww...

On Thursday, 21 May 2020 at 16:32:32 UTC, Adam D. Ruppe wrote:
> On Thursday, 21 May 2020 at 16:14:02 UTC, Seb wrote:
>> Why we can't we have a technical board where the community can vote in experts and potentially companies could even buy a seat for $$$ which would mean a lot more for them than the current very vague sponsorship options.
>
> ditto, I think we should have like a seven person elected DIP committee who pass/fail things by majority vote. It is obvious to me that the current process is totally useless.

As noted earlier, I'm with Steve, Seb, Adam, and "everyone but Walter" as far as I can see.  If this stands we'll have gone from @safe meaning "the compiler is responsible" to "the compiler can't guarantee anything unless you're pure D all the way down".

Atila, what's your take on all this?  Is it fork time?

On 5/21/2020 2:44 PM, Joseph Rushton Wakeling wrote:
> One concern here is that these responses are scattered across different parts of a long discussion thread.  So it probably would be a good idea for the acceptance to be accompanied by an explanation of what the major objections were to the DIP, and why they were discounted.
> 
> It's not so different from the way that, at the end of a talk or presentation, it's a good idea to recap the key points that have already been covered.

Fair enough. I'll do so. Stay tuned.

On 5/21/2020 2:45 PM, bachmeier wrote:
> On Thursday, 21 May 2020 at 20:48:18 UTC, Walter Bright wrote:
>> On 5/21/2020 10:03 AM, bachmeier wrote:
>>> Walter makes decisions based on his mood on a particular day
>> That's uncalled for.
> 
> Regional variation in English? Translation: You make your decisions based on how you feel about the situation at a point in time. "Mood" is used because there's some subjectivity and "gut feel" involved in the decision. It's not intended to be negative, it's simply a description of how any human makes any decision. That's why I said the same about how a committee would make decisions.

As a native English speaker, telling someone they are making decisions based on moods is telling them they are making irrational decisions.

I accept that you did not intend it in this manner, but you should be aware that many people would be offended by it.

On Friday, 22 May 2020 at 00:50:00 UTC, Walter Bright wrote:
> On 5/21/2020 2:44 PM, Joseph Rushton Wakeling wrote:
>> One concern here is that these responses are scattered across different parts of a long discussion thread.  So it probably would be a good idea for the acceptance to be accompanied by an explanation of what the major objections were to the DIP, and why they were discounted.
>> 
>> It's not so different from the way that, at the end of a talk or presentation, it's a good idea to recap the key points that have already been covered.
>
> Fair enough. I'll do so. Stay tuned.

Thank you.

Having an explanation on why the DIP is accepted despite being highly controversial would be sufficient.

-Alex

On 5/21/2020 2:41 PM, Steven Schveighoffer wrote:
> I even put forth a completely ignored compromise solution that would have solved the problem and allowed extern(C) functions to be considered @safe by default: https://digitalmars.com/d/archives/digitalmars/D/Discussion_Thread_DIP_1028--Make_safe_the_Default--Final_Review_336354.html#N336968 

I did see it. I can't even get across the notion of what "undefined symbol xxxx" messages from the linker mean to a lot of programmers. I'll never get this one across. The linking process is already as complex as I dare.

BTW, two symbols resolving to the same address can be a bit awkward in some object formats and isn't really supported, like with COMDATs. In my experience, unless C or C++ compilers generate certain constructs, linkers (and debuggers) don't work for those constructs, whether they are documented to or not.


> This is why I consider the process a waste of time, and why I'm done participating.

The level of negativity in that thread was what caused me to stop responding, though I continued reading. Every reply I made produced 10 responses, an exponential explosion, and yet I was just repeating myself. Two sides to every story.

FWIW, I am going to try again with another post here, for those who want a convenient summary of the rationale.

I have made these points before, but I'll summarize them here
for convenient referral.

----

The proposed amendment to Safe by Default is:

"Extern C and C++ functions without bodies should be @system
by default."

The rationale is that since the compiler cannot check them for @safe,
they should be assumed unsafe.

This is Obviously A Good Idea. Why would I oppose it?

1. I've been hittin' the crack pipe again.
2. I was secretly convinced, but wanted to save face.
3. I make decisions based on consultation with my astrologer.
4. I am evil.

I'm going to play the experience card. Many Obviously Good Ideas
appear again and again. Why they aren't so hot only becomes
clear after (sometimes considerable) experience. Some are:

1. Implicit declaration of variables.
2. AST macros (debated in the n.g. multiple times).
3. Exception specifications.

I'll expound on Exception Specifications, as they are more similar
to this case than the others.

Exception Specifications appeared with Java back in the 90's. With it,
each function must list which exceptions it throws, such as (in D):

    T myFunction() throws Alpha, Gamma;

If myFunction() throws any exception that is not derived from Alpha
or Gamma, the compiler issues a compile-time error.

This is Obviously A Good Idea. Smart, experienced, well-meaning programmers
endorsed it wholeheartedly. (Even C++98 adopted the idea.) One such person,
Bruce Eckel (who incidentally authored the early Zortech C++ manual and
is a friend of mine) authored the well-regarded "Thinking In ..." series of
books, one of which was "Thinking in Java". He bought the idea, and preached
and promoted it heavily.

Then came the hangover. Bruce wrote a brilliant mea culpa (that has since
sadly vanished from the internet) about what went wrong. The troubles were:

1. It was tedious to go through every function called by myFunction() and
figure out what all the possible Exceptions thrown might be.

2. If a low level function started throwing a new type, every single caller
all the way up the call tree had to add it to its exception specification.

This, of course, was incredibly tedious and annoying. So what happened?

1. Functions would just do a try-catch(Throwable) and ignore the exceptions.
Oops.

2. The Exception Specifications became simply `throws Throwable`. Oops.

What was most damning was the Bruce Eckel himself was horrified to find that
this was his behavior when writing his own code. Other advocates of it also
admitted they did that too. Exception specifications became a farce, and
Java was quietly softened to not require them. Exception specifications were
just ignored by the C++ community.

How does this relate to safe by default?

Consider the common (because that's how D started out) case of:

----- clibrary.d --------

    T massage_data(... many parameters ...);
    ... 200 more such declarations ...

----- app.d ----------

    import clibrary;

    void useClibrary( ... parameters ...) {
        massage_data(parameters);
    }

---------------------

This code, today, does not use annotations and it works. It's been working
for a long time. Now, we implement the amendment, and it stops compiling
because useClibrary is @safe and massage_data is @system. The user is faced
with the following alternatives:

1. Go through 200 functions in clibrary.d and determine which are @safe
and which are @system. This is what we want them to do. We try to motivate
this with compiler error messages. Unfortunately, this is both tedious and
thoroughly impractical, as our poor user Will Not Know which are safe and
which are system. We can correctly annotate core.stdc.stdio because I know
those functions intimately. This is not true for other system C APIs, and
even less true for some third party C library we're trying to interface to.

2. Annotate useClibrary() as @trusted or @system. While easier, this causes
all benefits to @safe by default to be lost.

3. Wrap the call to massage_data() with:

    () @trusted { massage_data(parameters); } ();

If there are a lot of calls to clibrary, this is going to look pretty awful.
Nobody likes writing or reading such ugly code. It's ok here and there, but
not as a general thing.

4. Edit clibrary.d and make the first line:

    @safe:

I submit that, just like with Java, Option 4 is what people will reach for,
nearly every time. I've had some private conversations where people admitted
this was what they'd do. People who knew it was wrong to do that.

If it's @safe by default, and then someone chooses to annotate it with @system
here and there, I'd feel a lot more confident about the accuracy of the code
annotations than if it just had @safe: at the top. At least they tried.

What is actually accomplished with this amendment if it was implemented?

1. Adds a funky, special case rule. It's better to have simple, easily
understood rules than ones with special cases offering little improvement.

2. Existing, working code breaks.

3. The most likely code fixes are to just make it compile, absolutely
nothing safety-wise is improved. The added annotations will be a fraud.

D should not encourage "greenwashing" practices like the Java
exception specification engendered. The compiler cannot vet the accuracy
of bodyless C functions, and we'll just have to live with that. The proposed
amendment does not fix that.

And so, I did not incorporate the proposed amendment to the Safe by Default
DIP.

On Friday, 22 May 2020 at 01:22:19 UTC, Walter Bright wrote:
> I have made these points before, but I'll summarize them here
> for convenient referral.
>
> ----
>
> [...]

First, thank you for taking the time to summarize your position. I understand
your objection to the proposed amendment much better now than I did after
only reviewing the discussion thread.

Second, I'd like to present a potential failure mode of DIP 1028 that I think
still remains unaddressed. Consider the following scenario:

Someone has created bindings for a C library and published them on
code.dlang.org. Because they were written prior to DIP 1028, the author assumed
that @system was the default, and didn't bother to add explicit annotations to
@system functions. Their code looks like this:

--- clibrary.d

    void monkey_around(...); // assumed @system-by-default

---

Months or years later, I decide to write a D program that makes use of these
bindings. By then, @safe-by-default has been fully implemented. I add
`clibrary` as a dependency to my Dub project and write the following code:

--- app.d

    import clibrary;

    void main() // @safe-by-default
    {
        /* ... code ... */

        monkey_around(...);

        /* ... more code ... */
    }

---

My program compiles with no errors...and then corrupts memory at run-time, even
though every line of code I've written is @safe. Oops.

This is the nightmare scenario that people are worried about: safety violations
being introduced *silently* into existing, correct D code.

Of course, the author of clibrary.d *should* update their code to add explicit
@system annotations, but how will they know there's a problem if the compiler
never complains about it? Most likely, by the time they or one of their users
finally track down the root of the problem, the damage will already have been
done.

Something ought to be done to prevent this. It doesn't have to be the exact
proposal from the discussion thread, but doing nothing and allowing widespread
silent breakage cannot possibly be the best solution.