On Thursday, 26 March 2020 at 15:02:15 UTC, Steven Schveighoffer wrote:
> [snip]
>
> At the top of your modules is not a big burden. If that drives you away from the language, then I'm sorry to say that you are missing out on the awesomeness of D, but I can't really help you.
>[snip]

I was talking about a hypothetical python person thinking about learning D, not about me personally.

On Thursday, 26 March 2020 at 16:19:44 UTC, Atila Neves wrote:
> On Thursday, 26 March 2020 at 14:24:24 UTC, Adam D. Ruppe wrote:
>> But D's runtime checks are also a valid solution.
>
> We can do better that at compile time.
>
But not always though. There are scenarios where you have to rely on runtime checks, such as user driven input for example.


Side note: There should be a significant push for runtime reflection if we are pushing for safe by default. I recall Andrei talking about it in one of his presentation on how the compile time reflection will lay ground work for runtime reflection. I have not seen any developments on that front. Reason being that classes can inherent from other classes.

-Alex

On Thursday, 26 March 2020 at 14:24:24 UTC, Adam D. Ruppe wrote:
> On Thursday, 26 March 2020 at 14:12:24 UTC, Steven Schveighoffer wrote:
>> I still think this is the appropriate path. We cannot continue to ignore memory safety as a secondary concern just because C code is by-default unsafe. Memory unsafe HAS to be opt-in for any new modern language to succeed.
>
> What frustrates me about these discussions is the facts that slices always check bounds by default. The GC prevents use-after-free bugs by default.
>
> C doesn't do those. So assuming C's problems apply to D is fallacious. Rust's complication is because they wanted to avoid the runtime checks. But D's runtime checks are also a valid solution.
>
> I suspect 95+% of C's problems already are extremely rare in D, yet the @safe advocates never seem to consider this at all.

This. Buffer overflows in D happen solely due to prejudice, when people abuse their C reflexes when writing in D, so compulsory safety may be useful to educate them to start using slices, but seriously, if it wasn't for C junkies the last buffer overflow would happen 30 years ago and not a second ago.

On 3/26/20 1:00 PM, jmh530 wrote:
> On Thursday, 26 March 2020 at 15:02:15 UTC, Steven Schveighoffer wrote:
>> [snip]
>>
>> At the top of your modules is not a big burden. If that drives you away from the language, then I'm sorry to say that you are missing out on the awesomeness of D, but I can't really help you.
>> [snip]
> 
> I was talking about a hypothetical python person thinking about learning D, not about me personally.
> 

So was I.

Python user: how do I call libX from D?

tutorial: Make sure you mark your functions as @system, or use @trusted escapes. To make things easy, just put @system: at the top of your module.

User: ok, that's not too bad. (probably)

-Steve

On Thursday, 26 March 2020 at 15:02:15 UTC, Steven Schveighoffer wrote:
> On 3/26/20 10:40 AM, jmh530 wrote:
>> On Thursday, 26 March 2020 at 14:12:24 UTC, Steven Schveighoffer wrote:
>>> [snip]
>>> And in actuality, most D code is @safe, so for most D code out there, this is not going to be a huge problem. You will have to mark few things. Wrappers/bindings for C libraries are going to be an exception, and that's just the pain we have to deal with. Either use trusted markings to make everything safe (after verification), or just punt to the user.
>>>
>> 
>> There is a lot of functionality that depends on C libraries. For instance, every or almost every function in lubeck calls at least one C function. If someone comes to D from python and wants to replace something from numpy/scipy with a lubeck equivalent, they will need to start slapping @trusted or @system on everything. That means they will need to understand the safety system and why stuff like that matters. For some people, that may be a big enough burden that they just throw up their hands and keep using python.
>> 
>
> Writing
>
> @system:
>
> At the top of your modules is not a big burden. If that drives you away from the language, then I'm sorry to say that you are missing out on the awesomeness of D, but I can't really help you.

I can, just write

@trusted:

This will be my goto solution as D lacks the means many other languages have to maintain compatibility. In C++ I can choose the standard I want to use, and avoid any removed or changed features if they affect me. Same goes for C#, Rust, and basically every other established language.

> How many people were driven away from windows development because they had to deal with __stdcall? You just googled for it (or whatever the hell was available at the time), said "oh this is how you do it", and did it. It wasn't a problem, you just did it.

You don't need to do that anymore, they changed it (for the better). It probably did drive people away. Especially when you deal with small issues consistently. They start piling up, and you can only deal with so many tiny issues (which D has many of; and will only grow with all the new changes coming forward).

> This is only even a discussion because of the current situation. If D started out this way, you would have no idea there was even a problem here.
>
> -Steve

Right, because people don't want to have to keep updating their code or have it be broken. Other languages ensure backwards compatibility, and if they don't they provide a way to keep the code working without having to modify the code to work.

D's solution to the problem: "this is the best practice now". That doesn't stop already written code from being broken.

On Thursday, 26 March 2020 at 16:19:44 UTC, Atila Neves wrote:
>> I suspect 95+% of C's problems already are extremely rare in D,
>
> Yes. The remaining 5% are all related to the stack and allocating on the C heap.

@safe doesn't deal with stack and C heap, it only suppresses C reflexes in the former 95%.

> If you allocate on the GC heap and use -preview=dip1000, then

Then you don't allocate on stack and C heap.

On 3/26/20 2:56 PM, Arine wrote:
> On Thursday, 26 March 2020 at 15:02:15 UTC, Steven Schveighoffer wrote:
>> Writing
>>
>> @system:
>>
>> At the top of your modules is not a big burden. If that drives you away from the language, then I'm sorry to say that you are missing out on the awesomeness of D, but I can't really help you.
> 
> I can, just write
> 
> @trusted:

This isn't a good idea. But yeah, you can do that at your own peril (and anyone who uses your library). I'd highly recommend using @system: instead.

> 
> This will be my goto solution as D lacks the means many other languages have to maintain compatibility.

The means are: use the correct markings. @system works now, and will after this change.

>> How many people were driven away from windows development because they had to deal with __stdcall? You just googled for it (or whatever the hell was available at the time), said "oh this is how you do it", and did it. It wasn't a problem, you just did it.
> 
> You don't need to do that anymore, they changed it (for the better). It probably did drive people away. Especially when you deal with small issues consistently. They start piling up, and you can only deal with so many tiny issues (which D has many of; and will only grow with all the new changes coming forward).

Meh, you just do it. Saying "why do I have to write @system when I don't care about safety" is like saying "why do I have to write void when I have no return value". Just do it, and your code works. For those who don't care.

> 
>> This is only even a discussion because of the current situation. If D started out this way, you would have no idea there was even a problem here.
> 
> Right, because people don't want to have to keep updating their code or have it be broken. Other languages ensure backwards compatibility, and if they don't they provide a way to keep the code working without having to modify the code to work.

I think the intention is to have an automated tool to mark things that are currently @system as @system explicitly. I would expect that feature in dfix, before this would become the default.

You are free to use other languages if you feel that way. IMO this change is for the better, and provides a much healthier default. Without it, most code is @safe-but-not-marked as that is the default. The new default will allow more usage of @safe.

> D's solution to the problem: "this is the best practice now". That doesn't stop already written code from being broken.

Or again, switch to a "sensible" language like Rust or C# or C++. Don't pick Swift though, they change stuff all the time, it probably is going to die soon, I doubt anyone will put up with that.

-Steve

On Thursday, March 26, 2020 10:02:22 AM MDT Atila Neves via Digitalmars-d wrote:
> On Thursday, 26 March 2020 at 11:40:41 UTC, IGotD- wrote:
> > FFI functions like extern(C) must be @system as the compiler
> > cannot check this. Should extern(C) automatically mean @system?
> > Well I think so because that's the only feasible possibility.
>
> The problem I see is this:
>
> extern(C) int add1(int i, int j) @safe {
>      return i + j + 1;
> }
>
> extern(C) doesn't necessarily mean the code is written in C and can't be verified by the compiler. It might just be written in D (or C++, or Rust, or...). I do however understand why one would want to say that an external C library has no @safe interface. But that can be done by applying @system manually. I'm not sure what the best solution is.

The issue is that the compiler should never treat anything as @safe unless it has verified that it's @safe, or the programmer has explicitly marked it as @trusted. Anything else introduces holes into the @safety system.

So, it should be fine to treat any and all function definitions as @safe by default, because the compiler can verify their @safety and provide the appropriate errors when the code isn't actually @safe. However, any function _declarations_ which are not extern(D) must not be treated as anything other than @system by default, or they introduce a hole into the @safety system. If the compiler treats them as @safe by default, then it's essentially marking them as @trusted for the programmer. They haven't been verified by the compiler, and they haven't been verified by the programmer. So, they silently introduce code that is potentially memory unsafe into your program. It then becomes impossible to find all memory safety issues by looking for @trusted code, and it becomes trivial to accidentally use a function which really isn't @safe without realizing it, because the compiler implicitly marked its declaration as @safe even though it wasn't verified for @safety.

extern(D) function declarations are of course fine, because the name mangling ensures that their definitions have actually been verified for @safety if they're marked as @safe, but the same is not true for any other function declarations.

Either all non-extern(D) function declarations should be treated as @system unless otherwise marked (as is currently the case), or they should require that the programmer explicitly mark them as @system or @trusted. Simply treating them as @system by default would of course be the simplest, and I don't really see a problem with that.

But no, it's not simply a question of extern(C) vs extern(D), because that's just a matter of linkage and name mangling. It's non-extern(D) function declarations specifically which are the problem. As long as the DIP only makes @safe the default for function definitions and extern(D) function declarations, then it should be fine, but it currently isn't very clear on the matter. It clearly indicates that extern(D) functions will be treated @safe by default, but it uses vague language about what's supposed to happen with non-extern(D) functions, and it makes no mention of function declaration vs definition.

- Jonathan M Davis

On Thursday, March 26, 2020 5:40:41 AM MDT IGotD- via Digitalmars-d wrote:
> On Thursday, 26 March 2020 at 05:14:44 UTC, Jonathan M Davis
>
> wrote:
> > Making it so that all code must be either verified by the compiler to be @safe or be marked by the programmer to be @trusted or @system means that all code which could contain memory safety issues will be segregated by @trusted or system, whereas right now, you can have large swathes of code which is not marked with anything and is unchecked. If the programmer is not using the compiler to verify @safety and is not verifying @system sections of code and marking it as @trusted, then there are no compiler guarantees about memory safety in that code. Sure, the programmer may have done a good enough job that there are no memory safety bugs in the code (and that's far more likely with D code than C/C++ code), but by making @safe the default, it makes it so that none of that will fall through the cracks unless the programmer explicitly tells the compiler to not check.
>
> FFI functions like extern(C) must be @system as the compiler
> cannot check this. Should extern(C) automatically mean @system?
> Well I think so because that's the only feasible possibility.
>
> I think we are heading into the @safe, @trusted, @system discussion and that's where I think the problem really lies, that @trusted just messes things up. If @safe code can call @system code directly then we are good and we can us extern(C) just as before (with or without @system block, another discussion) If we have this strange "human verified" @trusted nomenclature then things starts because fuzzy. What is an FFI function, trusted or system?
>
> I think that @trusted must go, then things start to clear up.

Without @trusted, @safe is completely broken. If you just think it through, that should be clear. Right now, if you have code such as

auto foo(int* ptr) @safe
{
    ...

    ++ptr;

    ...
}

the function will fail to compile, because you're attempting to do something that's @system inside an @safe function. You may or may not have realized that you were doing something @system, so it could very well be catching a bug for you. Either way, you then have four options:

1. Alter the code so that it doesn't need to do anything @system in order to
   do whatever it is that you're trying to do.

2. Mark the function as @trusted to indicate that you know that what you're
   doing is actually @safe.

3. Put the @system code in another function which you mark as @trusted, and
   call it from the @safe function.

4. Make it so that the function is @system instead, requiring that the
   caller verify that what they're doing is @safe and deal with @trusted.

Which solution is best depends on the code in question.

However, what happens if @trusted isn't a thing? Suddenly, there is no way to make this function @safe. The compiler isn't smart enough to understand your code to the point that it can determine that what you're doing is @safe, and you have no way to tell the compiler that you're sure that it's actually @safe. So, there are then two possibilities:

1. The function must be @system, and every function which calls it must be
   @system. You therefore have no clue where the code is that potentially
   has memory safety problems, and none of that code is being verified by
   the compiler for @safety. Ultimately, you end up with programs that might
   have pockets of @safe code, but in general, they won't be able to do much
   with @safe at all, because all it takes is a single @system operation
   (like calling a C function for I/O), and then nothing in the call stack
   can be @safe.

2. You allow @safe functions to do @system operations. This would then make
   @safe utterly meaningless, because it wouldn't be verifying anything at
   all. You could get partial verification by disallowing @system operations
   in @safe functions while still allowing an @safe function to call an
   @system functions, but that still hides that something @system is
   happening and makes it so that the compiler is neither checking those
   function calls nor flagging them as errors so that you know that you need
   to do something to actually use them in an @safe manner. It would
   introduce a massive hole in the @safety system.

Without @trusted, there is no bridge between @safe and @system code. @safe needs to be verifying code for memory safety, or it's utterly pointless, and a program that can't call any @system code is ultimately going to be useless. The downside to @trusted is of course that it's up to the programmer to verify it, and they could get it wrong, but since the compiler isn't smart enough to verify that code for memory safety, there really isn't an alternative if you want the compiler to be verifying much of your program for memory safety at all.

Ultimately, we do have to rely on the programmer to get @trusted right, but if they don't, then the rules of the @safety system make it so that when you have a memory safety issue, you only have to look at the code that's marked as @trusted and the @system code that it calls in order to track it down.

It can certainly be argued whether @safe or @system should be the default, and it can argued whether it really makes sense to have @trusted at the function level instead of having some kind of @trusted block within a function, but @trusted itself is vital to how the @safety system works.

- Jonathan M Davis

And then there is .di files which complicates the matters further.

But I agree with you.

If the compiler _cannot_ or has _not_ confirmed it is @safe, a function should not be marked as @safe and be callable.