On Wednesday, May 27, 2020 3:30:32 AM MDT Andrei Alexandrescu via Digitalmars- d-announce wrote:
> On 5/27/20 1:49 AM, Walter Bright wrote:
> > On 5/26/2020 9:31 AM, Bruce Carneal wrote:
> >> Currently a machine checked @safe function calling an unannotated extern C routine will error out during compilation. This is great as the C routine was not machine checked, and generally can not be checked.  Post 1028, IIUC, the compilation will go through without complaint.  This seems quite clear.  What am I missing?
> >
> > Nothing at all.
>
> That means safe by default is effectively loosening D's notion of safety.
>
> This DIP must go.

Which is exactly what most of us have been arguing for weeks (months?). It either needs to go or be ammended so that non-extern(D) declarations continue to be treated as @system instead of automatically becoming @safe with the DIP.

The result of all of that arguing is that Walter accepted the DIP and then started this thread as his more detailed reply when there were a ton of complaints about the DIP's acceptance - and of course, you've already read and replied to his reasoning.

As far as I can tell, Walter understands the issues but fundamentally disagrees with pretty much everyone else on the issue. He seems to think that weakening @safe is worth doing, because it will ultimately mean that more code will be treated as @safe and mechnically checked by the compiler, whereas most everyone else thinks that weakening @safe is unacceptable. But since Walter managed to convince Atila, the DIP has been accepted.

- Jonathan M Davis

On Thursday, 28 May 2020 at 02:39:40 UTC, Jonathan M Davis wrote:
> On Wednesday, May 27, 2020 8:13:52 PM MDT Bruce Carneal via Digitalmars-d- announce wrote:
>> On Thursday, 28 May 2020 at 01:14:43 UTC, Jonathan M Davis wrote:
>> > [...]
>>
>> I remember reading a suggestion that additional linker symbols be emitted to carry the attribute and possibly type information while leaving the ABI untouched.  Was this found to be impractical?
>
> Steven suggested something along those lines. I don't know how practical it would or wouldn't be, but I don't think that Walter even responded to the idea.
>

Changing the DIP process to be 2 of 3 LMs required for acceptance could get us past all the "this is useless because Walter xyz..." road blocks.  I've a proposal in to Mike for that now.  We'll see how it goes.

[snip]

>
> But regardless of whether DIP 1028 is the correct decision, the problem remains that your typical extern(C) function cannot be checked for @safety by the compiler, because it was compiled by a C compiler. The question then is just how the D compiler should treat them, and that's the main point of contention. There may be solutions like Steven suggested which would deal with edge cases where the implementation is in D but the linkage isn't, but ultimately, they're just edge cases.
>
> - Jonathan M Davis

Yes.  I understand the practical limits of machine checking and the 1028 issues.  The edge case that I had in mind was piece wise replacement of C libraries with dlang reworkings and LCD FFI libraries written in dlang for Python and other languages.  With the symbol additions, ABI could be decoupled from D-specific capabilities.  Python and other C FFIs just work.  Dlang programs get broader guarantees.  Still, as you say, probably not a big concern.

On Thu, May 28, 2020 at 4:56 AM Jonathan M Davis via Digitalmars-d-announce <digitalmars-d-announce@puremagic.com> wrote:
>
> As far as I can tell, Walter understands the issues but fundamentally disagrees with pretty much everyone else on the issue.

I do not think so, the issue is, that there could be more people who
agree with Walter (like me),
but because we agree we do not participate.

> He seems to think
> that weakening @safe is worth doing, because it will ultimately mean that
> more code will be treated as @safe and mechnically checked by the compiler,

And I believe he is right.

> whereas most everyone else thinks that weakening @safe is unacceptable.

You make bad assumptions, who is "most everyone else"? I have only see few  members of dlang arguing about that. But there is much bigger community around there.

> But since Walter managed to convince Atila, the DIP has been accepted.
>
> - Jonathan M Davis
>

So everything is OK right?

On Thursday, May 28, 2020 2:50:44 AM MDT Daniel Kozak via Digitalmars-d- announce wrote:
> On Thu, May 28, 2020 at 4:56 AM Jonathan M Davis via
>
> Digitalmars-d-announce <digitalmars-d-announce@puremagic.com> wrote:
> > As far as I can tell, Walter understands the issues but fundamentally disagrees with pretty much everyone else on the issue.
>
> I do not think so, the issue is, that there could be more people who
> agree with Walter (like me),
> but because we agree we do not participate.

There may be some silent people who agree, but in all the discussions on this DIP, almost no one has agreed with Walter on this. It has not been a back and forth discussion with some on Walter's side and some against. It's been pretty much everyone against Walter. He did unfortunately manage to convince Atila, so the DIP has been accepted, but based on the discussions, I think that you may be the only person I've seen say anything positive about the DIP treating extern(C) functions as @safe. The fact that @safe becomes the default has garned some debate with some in favor and some against (with most seeming to be in favor), but the idea of making extern(C) declarations @safe by default has been almost universally considered a bad idea by anyone who has responded on the topic. Most DIPs do not get anywhere close to this level of negative feedback.

> > But since Walter managed to convince Atila, the DIP has been accepted.
>
> So everything is OK right?

Everything is okay? Because a bad DIP got accepted? No, most definitely not. Quite the opposite. With the DIP in its current state, @safe becomes a lie. The compiler no longer guarantees that @safe code is memory safe so long as it doesn't call any @trusted code where the programmer incorrectly marked it as @trusted. Instead, the compiler blindly treats non-extern(D) declarations as @safe and invisibly introduces memory safety bugs into @safe code. Nothing about that is "OK." From the way things currently look, we're going to have to deal with that hole in @safe in D code in the future, because the DIP has been accepted, but it adds yet another dark corner to the language of the sort that folks here tend to make fun of C++ for. Going forward, we're going to have to be explaining to people why @safe code doesn't actually guarantee that code is memory safe (in spite of the fact that that's what it claims to do) and why any and all non-extern(D) declarations have to be treated with extreme caution to avoid invisibly introducing memory safety bugs into your code.

Walter is very intelligent and has made many great decisions with D, but IMHO, this is definitely not one of them.

- Jonathan M Davis

On 28.05.20 10:50, Daniel Kozak wrote:
>> He seems to think
>> that weakening @safe is worth doing, because it will ultimately mean that
>> more code will be treated as @safe and mechnically checked by the compiler,
> And I believe he is right.
> 

No, it's a false dichotomy. Weakening @safe to allow more code to be @safe might have been sensible if there was no @trusted annotation. However, as things stand, @trusted is sufficient as a tool to introduce potentially wrong assumptions about memory safety, we don't need more, especially not implicit ones.

The reason why people are not using @safe is partly that it is not the default, but it is mostly that their library dependencies _including Phobos_ are not properly annotated. This needs actual work to fix.

If there is significant perceived value in performing @safety checks in @system code, we can add a new function attribute that causes non-transitive @safe checks but otherwise gets treated as @system. @safe does not have to take this role.

On Thu, May 28, 2020 at 03:21:09AM -0600, Jonathan M Davis via Digitalmars-d-announce wrote: [...]
> With the DIP in its current state, @safe becomes a lie.  The compiler no longer guarantees that @safe code is memory safe so long as it doesn't call any @trusted code where the programmer incorrectly marked it as @trusted. Instead, the compiler blindly treats non-extern(D) declarations as @safe and invisibly introduces memory safety bugs into @safe code.  Nothing about that is "OK."
[...]

I see it already.  The next time someone wants to make a codebase @safe but the compiler complains about some violation, just add `extern(C)` to the function and move on.


T

-- 
Claiming that your operating system is the best in the world because more people use it is like saying McDonalds makes the best food in the world. -- Carl B. Constantine

On Thursday, 28 May 2020 at 09:21:09 UTC, Jonathan M Davis wrote:
> He did unfortunately manage to convince Atila, so the DIP has been accepted, but based on the discussions, I think that you may be the only person I've seen say anything positive about the DIP treating extern(C) functions as @safe.
>
> - Jonathan M Davis

I think Walter had to make a tough call with many tradeoffs. The defining feature of engineering I would say.

Is he wrong? Maybe, I don't know. The obvious path is far from always being a winner.

If it does come back to haunt him, he can always add a DIP to make extern(!D) @system by default. It won't invalidate any work.

On Thursday, 28 May 2020 at 02:47:01 UTC, Jonathan M Davis wrote:
> Walter has acknowledged the problem and seems to think that because it's the programmer's responsibility to deal with extern(C) functions correctly (since it's not possible for the compiler to do it), it's up to the programmer to go and fix any existing code that should be marked @system and isn't and that having the compiler incorrectly mark extern(C) declarations as @safe isn't a big problem, because programmers need to be spending the time to check them anyway. He's already created some PRs to try to fix some issues with extern(C) declarations in druntime and explicitly markingthem as @system but doesn't seem to think that it's ultimately a big deal.
>
> - Jonathan M Davis

I've submitted a PR [1] to fix a whole bunch of these. It's currently blocked on what appear to be irrelevant CI failures (one of which is actually acknowledged as such in the discussion on Walter's PR). No one has given it any attention.

Is there something I'm doing wrong? Should I be splitting my PR by file, to make it easier to review? I have a similar PR for Phobos in the works, and it would be nice to have some feedback on this so that one doesn't get ignored too.

[1] https://github.com/dlang/druntime/pull/3117

Am Thu, 28 May 2020 10:50:44 +0200 schrieb Daniel Kozak:

> On Thu, May 28, 2020 at 4:56 AM Jonathan M Davis via Digitalmars-d-announce <digitalmars-d-announce@puremagic.com> wrote:
>>
>> As far as I can tell, Walter understands the issues but fundamentally disagrees with pretty much everyone else on the issue.
> 
> I do not think so, the issue is, that there could be more people who
> agree with Walter (like me),
> but because we agree we do not participate.

You can not really assume any opinion for people who did not participate, unless you can really prove why there's a bias. I did not participate either and I do not agree with Walter. So now we can say the opinions of those who did not participate in the discussion are split 50:50 ;-)

We could assume there's a slight bias of those agreeing with Walter not responding because they don't have to actively convince anyone, as the DIP has been accepted. But given how much negative feedbcak there is, it's also likely people would voice their opinion to support the decision. Really the best thing we can assume is that the opionions of those not participating are split in the same way as the ones of those who are participating. The strawpoll posted recently suggests that as well: https://www.strawpoll.me/20184671/r

-- 
Johannes

Am Thu, 28 May 2020 12:28:16 +0000 schrieb Sebastiaan Koppe:

> On Thursday, 28 May 2020 at 09:21:09 UTC, Jonathan M Davis wrote:
>> He did unfortunately manage to convince Atila, so the DIP has been accepted, but based on the discussions, I think that you may be the only person I've seen say anything positive about the DIP treating extern(C) functions as @safe.
>>
>> - Jonathan M Davis
> 
> I think Walter had to make a tough call with many tradeoffs. The defining feature of engineering I would say.
> 
> Is he wrong? Maybe, I don't know. The obvious path is far from always being a winner.
> 
> If it does come back to haunt him, he can always add a DIP to make extern(!D) @system by default. It won't invalidate any work.

This would be another round of massively breaking user code. And this is going to be exactly the argument that will be used to dismiss any DIP trying to change the defaults later on.


-- 
Johannes