May 22, 2020
On Friday, 22 May 2020 at 17:41:38 UTC, Steven Schveighoffer wrote:
> On 5/22/20 1:07 PM, Atila Neves wrote:
>> And so I was convinced that everything being @safe is actually ok, especially because in real life, most C/C++ APIs aren't going to secretly corrupt your code.
>
> Yes, it can, but not secretly. Just obviously and easily. Note this function:
>
> https://github.com/atilaneves/libclang/blob/5415707fa6072700bdbf21f827567ffa2fd5c424/source/clang/c/index.d#L861
>
> And so, you are free to pepper your @safe code with dangling pointers. Sure, you can claim that the C++ library didn't "corrupt your code", which is the case for ALL libraries if you use them properly. You did it, you created a dangling pointer, not the library.

Right. And the point I was trying to make wasn't "look at what I did, it's cool". No, what I did was dumb. So dumb it took you no time at all to point out one of my mistakes. My point is that the result of making declarations implicity @system instead of @safe would make people just slap @safe on them without really thinking about it to get their code to compile. Like I did.

> The point of @safe is to protect you from corrupting your own code based on the guarantees that @safe provides.

I agree.

> If we don't care about the guarantees of @safe as long as you are using C libraries, why are we bothering at all with any of this?

We care. Annotations become explicit. Do I think this is ideal? No.

> BTW, you should fix that invalid attribute, freeing a pointer is never @safe unless you can guarantee nobody else has a copy of that pointer (and considering it's passed by value, the CALLER still has that pointer!)

You're completely right.


May 23, 2020
On 23/05/2020 5:40 AM, Atila Neves wrote:
> On Friday, 22 May 2020 at 17:33:11 UTC, rikki cattermole wrote:
>> On 23/05/2020 5:07 AM, Atila Neves wrote:
>>> [...]
>> It is not about the linkage.
>>
>> The problem is solely does the compiler have the source to the function body to verify it?
> 
> That's what I meant, sorry for not making it clearer.

It kept being swapped about in the discussion thread, so I have been a little on edge over people using non-extern(D). Because linkage doesn't mean anything to anything but to cpu and linker in this discussion.

>>> [...]
>>
>> That is a failure of the language that should be resolved.
> 
> And how do you suggest we fix it?

I don't know.

It is a problem that needs to be explored in more detail.

Adam had one idea that I convinced him to write up, but that didn't get very far when commented relating to propagation of attributes.

Major changes like this, should have been discussed and RFC'd instead of going to straight to a DIP. That is effectively what happened.

>>> [...]
>>
>> No.
>>
>> We simply do not agree, nor do I expect for us to come to terms on it anytime soon.
> 
> I meant "did I explain myself well enough that now you understand where I'm coming from, even though you might not ultimately agree?".

Yeah.

-------------

I didn't look at the binding, since you described it well previously, but of note is that you used @safe. What you should have used is @trusted instead.

If the programmer wants to slap @trusted: at the top of the file, I have no problems with that. You checked it (i.e. its LLVM, so of course its fine!).
May 22, 2020
On 22.05.20 19:07, Atila Neves wrote:
> Personally and initially, I would have preferred it if non-extern(D) declarations were implicitly @system. I understood Walter's argument about special cases and how they're bad, but the thought of them being @safe made me feel, for lack of a better word, "icky".

So the DIP itself wasn't good enough to convince you. Should have rejected it then. Walter could have made another, better argued DIP. That's how this is supposed to work, isn't it?

> Then I talked to Walter and he pointed out that if those declarations were @system, users would be prevented from calling them from now @safe code. A regular user would probably just slap `@safe:` at the top of the bindings module anyway. Then I realised that I did exactly that with my libclang bindings:
> 
> https://github.com/atilaneves/libclang/blob/5415707fa6072700bdbf21f827567ffa2fd5c424/source/clang/c/index.d#L254 
> 
> 
> "Worse", I made all functions `pure` and all parameters `in` as well for good measure. Why? I wanted to call them from my @safe pure code with `const` arguments. My reasoning at the time was "I trust libclang".
> 
> And so I was convinced that everything being @safe is actually ok, especially because in real life, most C/C++ APIs aren't going to secretly corrupt your code.

Your reasoning is fine when you're dealing with a function that has a safe interface. I.e., it can only corrupt your code when it's buggy. Then it makes sense so say "I trust the authors". D users do it with Phobos. Phobos authors do it with the C standard library. This kind of trust is perfectly ok.

Your reasoning is completely wrong when you're thinking about a function that doesn't have a safe interface. E.g., C's `memcpy` just isn't safe. It doesn't matter whether you trust the authors or not. The user has to make sure that the call is correct. Otherwise, `memcpy` will corrupt your memory. And `memcpy` is far from the only such function.

I'm positive that Walter is aware of the difference. Judging from this post, I'm afraid you might not be.
May 22, 2020
On Fri, May 22, 2020 at 05:26:03PM +0000, Adam D. Ruppe via Digitalmars-d-announce wrote: [...]
> int snprintf(char[] buf, const char* fmt, ...) {
>    return c.snprintf(buf.ptr, buf.length, fmt, ...);
> }
> 
> Well, lol, that's not exactly *correct* thanks to the possibility of format string mismatch, but it bounds checks both sides of the slice and becomes more appropriately trusted due to the parameters being checked too.
> 
> We now actually have a trusted implementation with a safe interface - which is what @trusted is SUPPOSED to mean.

Ironically, it was Walter himself who was a staunch proponent of this. And now we're basically saying, every C API is @safe by default unless you take the pains to realize that, oh noes it takes a pointer and a size and therefore passing the wrong size will cause buffer overflow yikes we better mark it @system.  Seems totally backwards to me.

But anyway, thinking about this a little more, it occurs to me that what we're trying to do here is really not 100% watertight @safe-ty, rather, what we're trying to do here is to maximize the surface area of compiler mechnical verification.  So @safe doesn't *really* mean "no memory corruption, guaranteed", it means "compiler will do its best to verify what it can, but what it can't, we just have to take on faith."  Or, to put it more nicely, "this code is @safe modulo the safety of calls to unverifiable C APIs".

Which leads to the idea of having some way of marking a function as @safe up to the safety of {a delegate parameter, a C interface, etc.}. A kind of conditional @safe-ty.  It has been a recurring need that keeps coming up, e.g., in relation to the recent DIP on lazy parameters. For example:

	// N.B.: hypothetical syntax
	int myFunc(int delegate(SomeObj obj) cb) @conditionalSafe {
		SomeObj obj = ...;
		// &obj++; // error: pointer arithmetic not allowed in @safe
		// systemFunc(); // error: cannot call systemFunc from @safe

		return cb(obj); // OK: we are @safe modulo safety of cb
	}
	void systemFunc() @system {...}

Then myFunc would become callable from @safe code, provided the passed-in argument is also @safe.

The crucial point here is that while compiling myFunc, the compiler doesn't (need to) know the @safe-ty of `cb`, it can just treat it as an opaque object that it assumes the safety of, while it verifies the rest of the function body.

This is parallel to C functions being of unverifiable safety, so if extern(C) functions were somehow marked and treated as opaque objects of unknown safety, then the compiler can still verify the rest of the code and produce a certificate of safety (modulo the C APIs used).

If we had a way of expressing conditional safety, it could be a way to salvage @safe from this current situation.


T

-- 
Doubtless it is a good thing to have an open mind, but a truly open mind should be open at both ends, like the food-pipe, with the capacity for excretion as well as absorption. -- Northrop Frye
May 22, 2020
On Friday, 22 May 2020 at 17:40:59 UTC, Atila Neves wrote:
> [snip]
>>
>> That is a failure of the language that should be resolved.
>
> And how do you suggest we fix it?
>

@safe as whitelist instead of blacklist.

When the compiler cannot verify that it is @safe, then it cannot be @safe.
May 22, 2020
On 22.05.20 19:54, Atila Neves wrote:
> Right. And the point I was trying to make wasn't "look at what I did, it's cool". No, what I did was dumb. So dumb it took you no time at all to point out one of my mistakes. My point is that the result of making declarations implicity @system instead of @safe would make people just slap @safe on them without really thinking about it to get their code to compile. Like I did.

Now you have accepted a DIP that does the dumb thing automatically. How is that any better?

[...]
>> If we don't care about the guarantees of @safe as long as you are using C libraries, why are we bothering at all with any of this?
> 
> We care. Annotations become explicit. Do I think this is ideal? No.

"Annotations become explicit." - What now? I probably misunderstand that sentence, but DIP 1028 does not require explicit annotations. That's why everyone is upset.
May 22, 2020
On 22.05.20 19:12, Atila Neves wrote:
> Flipping the default will cause more incorrect code to fail to compile. Yes, there's a cost, which is carefully vetting extern(C) and extern(C++) declarations. The decision came down to finding this an acceptable trade-off.

You can still carefully vet the extern declarations if DMD requires an explicit safety attribute on them. You can do it better even, because DMD will tell you where they are.
May 22, 2020
On Friday, 22 May 2020 at 18:11:28 UTC, ag0aep6g wrote:
> So the DIP itself wasn't good enough to convince you.

Had that been the case, I would have rejected it.

> Your reasoning is fine when you're dealing with a function that has a safe interface. I.e., it can only corrupt your code when it's buggy. Then it makes sense so say "I trust the authors". D users do it with Phobos. Phobos authors do it with the C standard library. This kind of trust is perfectly ok.
> Your reasoning is completely wrong when you're thinking about a function that doesn't have a safe interface. E.g., C's `memcpy` just isn't safe.

memcpy isn't a good example since it's explicitly @system:

https://dlang.org/phobos/core_stdc_string.html#.memcpy

> It doesn't matter whether you trust the authors or not. The user has to make sure that the call is correct. Otherwise, `memcpy` will corrupt your memory. And `memcpy` is far from the only such function.

Yes. But most of them aren't like memcpy. Most D code calls other D code, not C.

Am I saying nothing bad can happen if we implicitly trust extern(C) declarations? No. I'm saying we'll be no worse off if they're all implicitly @system.

This compiles with no warnings right *now*:

void main() {
    import core.stdc.stdlib: free;
    free(cast(void*) 42);
    free(new int);
    free(&main);
}



May 22, 2020
On Friday, 22 May 2020 at 18:17:29 UTC, ag0aep6g wrote:
> On 22.05.20 19:54, Atila Neves wrote:

>> We care. Annotations become explicit. Do I think this is ideal? No.
>
> "Annotations become explicit." - What now? I probably misunderstand that sentence, but DIP 1028 does not require explicit annotations. That's why everyone is upset.

Sorry, I didn't express myself well. I meant that the user can still mark functions as @system, they just have to do it explicitly.
May 22, 2020
On 5/22/20 1:54 PM, Atila Neves wrote:
> My point is that the result of making declarations implicity @system instead of @safe would make people just slap @safe on them without really thinking about it to get their code to compile. Like I did.

So the solution is -- make the compiler be dumb for you? If you are going to incorrectly put @safe on it, I want you to have to do it because YOU made a conscious decision to be careless, not have it done for you because you forgot.

How about we change the law so if you don't sign a contract, we assumed you signed it anyway. In order to reject the contract, you have to X out every paragraph and write "I don't agree to this" and sign that. Any paragraphs you forget are enforceable.

-Steve