On Friday, 22 May 2020 at 14:13:20 UTC, Paul Backus wrote:
> On Friday, 22 May 2020 at 13:58:14 UTC, bachmeier wrote:
>> On Friday, 22 May 2020 at 03:36:03 UTC, Paul Backus wrote:
>>> This is the nightmare scenario that people are worried about: safety violations
>>> being introduced *silently* into existing, correct D code.
>>
>> Honest question: What is the use case for an absolutely-positively-has-to-be-safe program that calls C code? Why would anyone ever do that? C is not and will never be a safe language. "Someone looked at that blob of horrendous C code and thinks it's safe" does not inspire confidence. Why not rewrite the code in D (or Rust or Haskell or whatever) if safety is that critical?
>
> The problem isn't that safety is critical, it's that the D compiler is lying to me about the safety of my code.

I understand your argument, especially since I wrote nearly identical words weeks ago.

For me, what it boils down to is this: currently the compiler isn't doing what it could because of the @system default. I work on a D codebase that can't be called from a @safe unittest and we could (probably are) be hiding bugs due to this.

Flipping the default will cause more incorrect code to fail to compile. Yes, there's a cost, which is carefully vetting extern(C) and extern(C++) declarations. The decision came down to finding this an acceptable trade-off.

On Friday, 22 May 2020 at 14:37:04 UTC, bachmeier wrote:
> On Friday, 22 May 2020 at 13:57:27 UTC, Mike Parker wrote:
>> [...]
>
> I think the source of the problem is that Walter's DIPs require the community to prove that Walter's proposal is so bad that he needs to reject it. Anyone else's proposal has to prove that it's worthy of being added to the language. There's a big perceived gap between those two. As I've said many times, it's odd for someone to judge his own DIPs, and as someone that is an academic administrator and runs a high-profile journal, I can say this type of practice is not the norm in those areas because it doesn't lead to good decisions.

I could have rejected it, and nearly did.

On 22.05.20 18:43, Adam D. Ruppe wrote:
> On Friday, 22 May 2020 at 16:39:42 UTC, jmh530 wrote:
>> Fortunately, the above point can be more easily fixed by making `free` @system
> 
> With the o/b system `free` might actually work out OK....

free(new int);

On Friday, 22 May 2020 at 17:07:37 UTC, Atila Neves wrote:
> And so I was convinced that everything being @safe is actually ok, especially because in real life, most C/C++ APIs aren't going to secretly corrupt your code.

I wrote up a thing about this earlier then deleted it because I didn't want to yell into the void, but now I kinda wish I kept it.

Short version of my thoughts is yes, most C functions are implicitly trusted, but this depends heavily on using correct arguments and that's what @trusted wrappers are supposed to ensure.

Consider:

char[16] buffer;
snprintf(&buffer[0], buffer.length, "%s", data);

btw fix this https://issues.dlang.org/show_bug.cgi?id=20853


But it is fair to say that usage is safe/trusted. However, you can just as well write:

snprintf(&buffer[12], buffer.length......

or

snprintf(&buffer[0], 156, ....); // oops my finger was misaligned on the keyboard and hit 56 without realizing it. or the size changed and this didn't etc



This is what a @trusted wrapper is supposed to be about - providing an interface that cannot be misused.

int snprintf(char[] buf, const char* fmt, ...) {
   return c.snprintf(buf.ptr, buf.length, fmt, ...);
}

Well, lol, that's not exactly *correct* thanks to the possibility of format string mismatch, but it bounds checks both sides of the slice and becomes more appropriately trusted due to the parameters being checked too.

We now actually have a trusted implementation with a safe interface - which is what @trusted is SUPPOSED to mean.

On 23/05/2020 5:07 AM, Atila Neves wrote:
> On Friday, 22 May 2020 at 12:28:56 UTC, rikki cattermole wrote:
>> On 23/05/2020 12:19 AM, Joseph Rushton Wakeling wrote:
>>> With the rationale laid out clearly as it is here, I do have some responses in mind.  But before sharing them, I'd like to know whether that would be useful right now: I've no wish to just press for a re-hashing of conversations that have already been had.
>>
>> No.
>>
>> This wasn't the first and certainly won't be the last time we as a community have been very unhappy with how Walter conducts himself with his DIP's.
>>
>> Although it seems an improvement has been made to how he needs to respond to the DIP assessment. It should also include a statement from Atila as well given his position.
> 
> I'm going through posts in order, so apologies if I'm "ignoring" something that shows up later in the discussion.
> 
> Personally and initially, I would have preferred it if non-extern(D) declarations were implicitly @system. I understood Walter's argument about special cases and how they're bad, but the thought of them being @safe made me feel, for lack of a better word, "icky".

This is one of the issues I had a problem with.

It is not about the linkage.

The problem is solely does the compiler have the source to the function body to verify it?

> Then I talked to Walter and he pointed out that if those declarations were @system, users would be prevented from calling them from now @safe code. A regular user would probably just slap `@safe:` at the top of the bindings module anyway. Then I realised that I did exactly that with my libclang bindings:
> 
> https://github.com/atilaneves/libclang/blob/5415707fa6072700bdbf21f827567ffa2fd5c424/source/clang/c/index.d#L254 

That is a failure of the language that should be resolved.

One of the arguments that has been brought up (although I don't remember if it made its way to the N.G.) is that if you don't have the body, can a function /even/ be @safe?

> "Worse", I made all functions `pure` and all parameters `in` as well for good measure. Why? I wanted to call them from my @safe pure code with `const` arguments. My reasoning at the time was "I trust libclang".

You might, but that doesn't give the compiler the right to do so by default. This a decision for a skilled programmer to make.

> And so I was convinced that everything being @safe is actually ok, especially because in real life, most C/C++ APIs aren't going to secretly corrupt your code.

What you did was give up some security in favor of freedom and now you can't get certified to work on top secret projects (analogy).

> Then there's the fact that, today, there's no safety anyway calling anything because everything is @system by default. And D source code won't be able to lie.
> 
> Does that clear it up?

No.

We simply do not agree, nor do I expect for us to come to terms on it anytime soon.

To me at least, this butchers @safe/trusted/system into a system that is near useless for guarantees for an entire program.

Sure its convenient for a single function, but absolutely useless in trying to solve the actual problem it was trying to solve in the first place.

On Friday, 22 May 2020 at 17:33:11 UTC, rikki cattermole wrote:
> On 23/05/2020 5:07 AM, Atila Neves wrote:
>> [...]
> It is not about the linkage.
>
> The problem is solely does the compiler have the source to the function body to verify it?

That's what I meant, sorry for not making it clearer.


>> [...]
>
> That is a failure of the language that should be resolved.

And how do you suggest we fix it?

> One of the arguments that has been brought up (although I don't remember if it made its way to the N.G.) is that if you don't have the body, can a function /even/ be @safe?

Yes.

>> [...]
>
> You might, but that doesn't give the compiler the right to do so by default. This a decision for a skilled programmer to make.

They still can (and should). The DIP is about changing the default.

>> [...]
>
> No.
>
> We simply do not agree, nor do I expect for us to come to terms on it anytime soon.

I meant "did I explain myself well enough that now you understand where I'm coming from, even though you might not ultimately agree?".

On 5/22/20 1:07 PM, Atila Neves wrote:
> And so I was convinced that everything being @safe is actually ok, especially because in real life, most C/C++ APIs aren't going to secretly corrupt your code.

Yes, it can, but not secretly. Just obviously and easily. Note this function:

https://github.com/atilaneves/libclang/blob/5415707fa6072700bdbf21f827567ffa2fd5c424/source/clang/c/index.d#L861

And so, you are free to pepper your @safe code with dangling pointers. Sure, you can claim that the C++ library didn't "corrupt your code", which is the case for ALL libraries if you use them properly. You did it, you created a dangling pointer, not the library.

The point of @safe is to protect you from corrupting your own code based on the guarantees that @safe provides. If we don't care about the guarantees of @safe as long as you are using C libraries, why are we bothering at all with any of this? I'd rather see a DIP that just removes safety completely if the language maintainers just trust all C libraries "in real life".

BTW, you should fix that invalid attribute, freeing a pointer is never @safe unless you can guarantee nobody else has a copy of that pointer (and considering it's passed by value, the CALLER still has that pointer!)

Will that break your code? It sure might! It just means YOUR code needs to be marked @trusted or @system as appropriate.

-Steve

On Friday, 22 May 2020 at 17:07:37 UTC, Atila Neves wrote:

> And so I was convinced that everything being @safe is actually ok, especially because in real life, most C/C++ APIs aren't going to secretly corrupt your code.

Uh? There's plenty of C/C++ code out there with api that when "used in the wrong way" will corrupt memory.

That's the _essence_ of @trusted code, the _programmer_ need to _correctly_ handle the usage, because the compiler can't do that job alone.

Perhaps not the ideal solution, but would a compiler flag, e.g. --strict-safe, that ensures the compiler errors on un-@trusted non-D function prototypes be appropriate? This way, those who do work on mission-critical stuff can feel a tad better.

On Friday, 22 May 2020 at 16:47:34 UTC, Steven Schveighoffer wrote:
> [snip]
>
> You can't, you don't control that code, someone else does (this is important, you can declare extern(C) functions anywhere, even locally).

You can make a separate module with one function that just calls the `free` function and has the @system attribute. Then, just change the import for the free function to that new module. Annoying, but potentially fix-able.

The bigger issue is that we know that we should check the `free` function, but what about all the other functions out there that might corrupt memory that we don't really know about. It's not scale-able.

>
> And from the experience of .save for forward ranges, people just aren't going to do this. They are going to do the easiest thing that works without complaint -- i.e. don't bother marking.

Probably true.