On Wednesday, March 25, 2020 10:35:23 PM MDT Jonathan Marler via Digitalmars-d wrote:
> On Thursday, 26 March 2020 at 02:58:26 UTC, Walter Bright wrote:
> > On 3/25/2020 10:53 AM, Jonathan Marler wrote:
> >> I'm on the fence as to whether this is a good idea.  However, with all the D code I've written that would break from this I'd prefer not to do this;
> >
> > Are you sure there'd be that much that breaks? If much does break, I'd suggest re-evaluating the coding techniques you use.
>
> I'm constantly re-evaluating and improving my coding techniques. Something I love doing actually.  I'm not sure how this feature breaking code is related to poor coding techniques though?  The point is that to fix code after this feature comes in, I'll need to go through thousands of source files and tag code appropriately.  What does this have to do with the quality of the code itself?

Making it so that all code must be either verified by the compiler to be @safe or be marked by the programmer to be @trusted or @system means that all code which could contain memory safety issues will be segregated by @trusted or system, whereas right now, you can have large swathes of code which is not marked with anything and is unchecked. If the programmer is not using the compiler to verify @safety and is not verifying @system sections of code and marking it as @trusted, then there are no compiler guarantees about memory safety in that code. Sure, the programmer may have done a good enough job that there are no memory safety bugs in the code (and that's far more likely with D code than C/C++ code), but by making @safe the default, it makes it so that none of that will fall through the cracks unless the programmer explicitly tells the compiler to not check.

In general, it should result in either far more code being checked for memory safety by the compiler or programmers just telling the compiler to shut up by using some combination of @trusted and @system. So, in the cases where the programmer allows the compiler to check, there is a real possibility that memory saftey bugs will be found (thus presumably resulting in those bugs being fixed), and in the cases where the programmer does not allow the compiler to check, it's much easier for other programmers to see that that's what's happening (e.g. simply grepping for @system and @trusted will show wherever it happens when templates are not involved, which is not currently the case).

It's certainly possible that you will find very few bugs in your code as a result of this change, but I think that it should be pretty clear that code in general will benefit, and you could be surprised by what you find in your own code.

The only real downsides to this change that I'm aware of are

1. You could have to add a bunch of annotations to existing code, which could be very annoying (which is what appears to be your main problem with the DIP).

2. It will make it a bit more annoying to throw together small programs, because you'll have to either actually make your code @safe or slap @system on it.

However, in both cases, if you don't care about @safety, you can just slap @system at the top of all of your modules, much as that isn't great practice. So, it might be briefly annoying, but it shouldn't be a big deal long term, and having @safe be the default should result in far more D code in general being verified for memory safety, which will result in fewer bugs in D code in general.

- Jonathan M Davis

On 3/25/2020 9:35 PM, Jonathan Marler wrote:
> I assume there's a command-line switch to enable it so I could see for myself?

https://github.com/dlang/dmd/pull/10709/files#diff-8248195efc96675f6bf930674f8fcf6eR744

On 3/25/2020 9:14 PM, Jonathan M Davis wrote:
> In general, a D wrapper library around a C library should be presenting an
> @safe API

A D wrapper should be as thin as possible, which means if the C function being wrapped is safe then the D wrapper should be @safe, and if is not safe then the D wrapper should be @system.

For the D wrapper developer, since he's providing a service to the D user, part of the job will be identifying which of the C interfaces are safe and which are system (of course, he can just mark them all as @system just to get things done and move on, and it'll be up to the users of said library if that is acceptable or not).

> D code does make it much easier to integrate with C code than is the case
> with many other languages, but it also loses a lot of its value if the
> compiler treats C code as if it were @safe even though its memory safety was
> not verified by the compiler, and the programmer gave no indication to the
> compiler that they had verified it. Having the compiler treat C bindings as
> @safe by default would be a huge whole in @safe and make it much harder to
> track down bugs related to memory safety when they occur.

I seriously doubt that would be any harder than it is now.

For those D programmers interfacing with C, they are more sophisticated than raw beginners, and it is reasonable to expect them to be capable of adding @system: at the start of the module before they go through and check which ones can be @safe.

P.S. I started looking through druntime/src/core/stdc/*.d. They all use @system: or @trusted: at the start. Unfortunately, little thought seems to have been put into it. For example:

  https://github.com/dlang/druntime/blob/master/src/core/stdc/limits.d#L29

declares everything as @trusted, yet should be @safe. It isn't technically broken, but it isn't right.

On Wednesday, March 25, 2020 11:43:41 PM MDT Walter Bright via Digitalmars-d wrote:
> On 3/25/2020 9:14 PM, Jonathan M Davis wrote:
> > In general, a D wrapper library around a C library should be presenting an @safe API
>
> A D wrapper should be as thin as possible, which means if the C function being wrapped is safe then the D wrapper should be @safe, and if is not safe then the D wrapper should be @system.
>
> For the D wrapper developer, since he's providing a service to the D user, part of the job will be identifying which of the C interfaces are safe and which are system (of course, he can just mark them all as @system just to get things done and move on, and it'll be up to the users of said library if that is acceptable or not).

There is a huge difference between providing bindings for a C library and providing a D wrapper library for a C library. IMHO, how thick the D wrapper should be very much depends on the C code in question, but in general, I would expect a D wrapper library to be trying to present a D API with the niceties that go with that, which does not necessarily mean that the wrapper is thin. Sometimes, a thin wrapper works just fine, but in general, if a wrapper is thin, then IMHO, it's providing very little value over simply using the C bindings directly and thus is of questionable utility.

> > D code does make it much easier to integrate with C code than is the
> > case
> > with many other languages, but it also loses a lot of its value if the
> > compiler treats C code as if it were @safe even though its memory safety
> > was not verified by the compiler, and the programmer gave no indication
> > to the compiler that they had verified it. Having the compiler treat C
> > bindings as @safe by default would be a huge whole in @safe and make it
> > much harder to track down bugs related to memory safety when they
> > occur.
>
> I seriously doubt that would be any harder than it is now.
>
> For those D programmers interfacing with C, they are more sophisticated than raw beginners, and it is reasonable to expect them to be capable of adding @system: at the start of the module before they go through and check which ones can be @safe.

The problem is not that it's hard to mark extern(C) declarations as @system. The problem is that if they're automatically @safe, then it's harder to track them down when there's an @safety bug. It should always be possible to segregate memory safety bugs in @safe code by looking for @trusted code that it's calling. I don't think that _anything_ should be considered @safe unless the compiler has actually verified that it is, with @trusted of course being used to indicate that the programmer claims to have verified it. extern(C) declarations cannot be verified by the compiler and thus should never be considered @safe. At most, the programmer should be marking them with @trusted.

> P.S. I started looking through druntime/src/core/stdc/*.d. They all use @system: or @trusted: at the start. Unfortunately, little thought seems to have been put into it. For example:
>
>
> https://github.com/dlang/druntime/blob/master/src/core/stdc/limits.d#L29
>
> declares everything as @trusted, yet should be @safe. It isn't technically broken, but it isn't right.

That's concerning, since it implies that whoever did it was just slapping @trusted on the various C declarations. Honestly, I'm inclined to argue that using @trusted with : or {} is just plain bad practice. Non-extern(D) functions need to be individually verified and thus should be marked individually.

- Jonathan M Davis

On Thursday, 26 March 2020 at 02:58:26 UTC, Walter Bright wrote:
> Are you sure there'd be that much that breaks? If much does break, I'd suggest re-evaluating the coding techniques you use.

Safety checks are quite conservative to the point that you even proposed https://issues.dlang.org/show_bug.cgi?id=20691 and they will be more conservative to plug more holes.

On Wednesday, 25 March 2020 at 22:40:10 UTC, H. S. Teoh wrote:
> On Wed, Mar 25, 2020 at 09:58:40PM +0000, Jonathan Marler via Digitalmars-d wrote: [...]
>> That being said, in practice I'm not sure how much benefit @safe actually provides. In theory it sounds nice.  It could help audit code, but for me, I audit all my code the same whether or not it's safe.  So for me the whole feature seems kinda pointless. Maybe this is different for others?  Does anyone have any real life examples/experience where @safe has helped?  Has the benefit warranted the cost to manage these tags throughout your code? Do we have any projects that are already using this behavior by putting "@safe:" at the top of every file?  Does anyone have any pointers to projects that have done this?  Have they seen any benefits from doing so?
>
> Some of the latest new features like DIP1000 are in full force only inside @safe code.  I've run into a couple of escaping reference bugs that were not caught because I didn't tag my code @safe, but once I added @safe I immediately got a compiler error pinpointing the code that leaked a scoped reference.
>
> I wouldn't say this is a big impact, but it did catch a couple of bugs that would've been a pain to track down.  From this perspective, it makes sense to make @safe the default: most users would not bother with the pain of manually tagging everything @safe just to get a few minor benefits.

Even to get those minor benefits your code should be annotated with `scope`, `return` and whatnot. And even then dip1000 is not the default yet, so even those minor benefits are out. If people are lazy to add 6 characters `@safe:` why do you think people aren't lazy enough to keep their codebase dip1000 compliant?

On Thursday, 26 March 2020 at 05:16:23 UTC, Walter Bright wrote:
> On 3/25/2020 9:35 PM, Jonathan Marler wrote:
>> I assume there's a command-line switch to enable it so I could see for myself?
>
> https://github.com/dlang/dmd/pull/10709/files#diff-8248195efc96675f6bf930674f8fcf6eR744

Hmmm, unfortunately the first project I tried it on exposed a compiler bug.  I've seen this bug before in recent compilers.  Symbols for certain templates aren't getting emitted.

That being said, I ran it on a few projects and didn't find a single issue.  However, I'm pretty sure I was compiling unsafe code a none of my code is tagged with @safe/@system/@trusted.  I was compiling my custom druntime/standard library and rootfs which contains assembly and all sorts of crazy stuff. There might be a bug in your PR?  I rebased it and I've pushed the rebased version I compiled here: https://github.com/marler8997/dmd/tree/safeDefaultRebased

NOTE: I rebased it so I could use druntime/phobos off of master so I wouldn't have to cross-reference which versions of those repos to use to get everything working

On Thursday, 26 March 2020 at 05:43:41 UTC, Walter Bright wrote:
> https://github.com/dlang/druntime/blob/master/src/core/stdc/limits.d#L29
>
> declares everything as @trusted, yet should be @safe. It isn't technically broken, but it isn't right.

That one should be @system, like any other C header. @safe is supposed to be checked by the compiler, so C headers can't possibly have @safe declarations.

On Thu, Mar 26, 2020 at 12:15 AM Steven Schveighoffer via Digitalmars-d < digitalmars-d@puremagic.com> wrote:

> In response to Walter's response to ag*, I would say that there is a fatal problem with automatically allowing extern(C) function prototypes (and really, anything that does not mangle @safe) to be default @safe.
>
> The reason is simple -- the change is silent and automatically marks everything @safe that has not been checked.
>
> I would argue that if the compiler is going to make things @safe by default, then things that are not marked and are not @safe should not compile AT ALL COSTS. Otherwise the value of @safe is completely lost.
>
> The DIP should be rejected IMO unless all functions with no mechanism to
> mangle @safe into the name (e.g. extern(C), extern(C++), etc) that have
> no implementation are either:
>
> a) required to be marked, or
> b) default @system.
>
> Everything else in the DIP is possibly annoying to deal with but at least doesn't silently destroy the meaning of @safe.
>
> I will note that I support the notion of @safe by default. I would be in favor of the DIP as long as this fatal flaw is not included.
>
> -Steve
>

I'm really on board with this. My feeling though is that it should be b;
extern(Language) should be @system by default, except maybe extern(Rust),
which nobody has ever asked for.

On 3/25/2020 11:14 PM, Jonathan M Davis wrote:
> On Wednesday, March 25, 2020 11:43:41 PM MDT Walter Bright via Digitalmars-d
> wrote:
>> On 3/25/2020 9:14 PM, Jonathan M Davis wrote:
>>> In general, a D wrapper library around a C library should be presenting
>>> an @safe API
>>
>> A D wrapper should be as thin as possible, which means if the C function
>> being wrapped is safe then the D wrapper should be @safe, and if is not
>> safe then the D wrapper should be @system.
>>
>> For the D wrapper developer, since he's providing a service to the D user,
>> part of the job will be identifying which of the C interfaces are safe
>> and which are system (of course, he can just mark them all as @system
>> just to get things done and move on, and it'll be up to the users of said
>> library if that is acceptable or not).
> 
> There is a huge difference between providing bindings for a C library and
> providing a D wrapper library for a C library.

Yes, you're right.


> That's concerning, since it implies that whoever did it was just slapping
> @trusted on the various C declarations.

Yup.