June 09, 2022

On Thursday, 9 June 2022 at 15:30:47 UTC, Steven Schveighoffer wrote:

>

What is not OK is the compiler turning actual requests for GC allocation into stack allocations based on that. At this point, it's a literal, but if it did this for e.g. new T, we are in for a lot of trouble.

scope has to have a strong guarantee to be meaningful. Why use scope in @system code if it was ignored by the compiler? The optimization should be allowed.

>

I'll ask, is it undefined behavior to return a scope pointer as a non-scope pointer? If so, should we make UB so easy to do?

We shouldn't, but we can't detect all cases of wrong use of scope in general in @system code. We do need a warning in the scope docs though.

...

>

In any case, I filed a bugzilla issue: https://issues.dlang.org/show_bug.cgi?id=23175

Thanks for finding this. I think the problem is changing the meaning of in in @system code. If in is to mean scope too then a deprecation period is needed to weed out any uses of in in @system code. (in meaning scope too in @safe code is fine).

June 09, 2022

On Thursday, 9 June 2022 at 15:38:10 UTC, jmh530 wrote:

>

On Thursday, 9 June 2022 at 15:23:35 UTC, Timon Gehr wrote:

>

[snip]
There is no upside to allowing this scope annotation.

[snip]

Below is also unintuitive as the compiler has all the information needed to verify that the reference is escaping. Also, since the escape analysis occurs only in @safe functions, it is assumed that no escape happens in the @trusted one called by the @safe one.

The situation reminds me a little of restrict in C. restrict tells the compiler to make certain assumptions about the code, but it is up the programmer to ensure that those assumptions are upheld.

int* a;

@safe
void foo(scope int* x) {
    //a = x; //error
    bar(x);
}

@trusted
void bar(scope int* x) {
    a = x;
}

void main() {
    int x = 1;
    foo(&x);
    assert(*a == 1);
}
June 09, 2022
On 6/9/2022 7:51 AM, Timon Gehr wrote:
> Your are predicting that some people will explicitly do the wrong and lazy thing,

My experience is that the vast bulk of people will do the least amount of effort. It's why software is always larded up with technical debt. I do it, too. Yes, sometimes I've used duct tape and baling wire. Anyone who claims they haven't, I don't believe :-)

> hence the compiler should do the wrong and lazy thing implicitly by default. This just makes no sense. What's the big harm in annoying lazy people slightly more? It's not like they won't complain loudly about `@safe` by default in any case.

I'm the recipient of all the complaints that I'm breaking their existing code.

> May as well do it right or not at all.

This entire thread is what happens with "not at all".

At some point all C functions have to be trusted in some form or other because the D compiler has NO way to check them, and neither does the D programmer. Putting `@trusted` on the C declarations accomplishes nothing, it's safety theater.

In druntime, we've gone through many (certainly not all) of the C declarations and appropriately added correct annotations to them. But realistically, this is not scalable.
June 09, 2022
On 6/9/2022 5:58 AM, Timon Gehr wrote:
> But if the compiler can easily tell that something makes no sense, it should still be an error in both `@safe` and `@system` code!

Sometimes it makes sense for a function to return the address of a local. For example, if you want to detect how large the stack has gotten. I use this in, for example, the garbage collector to see how much stack needs to be scanned. It can also be used to "step" on the stack after a function returns, as one might want to do for security software.

I've also done things like write 0xDEADBEEF all over memory in order to flush out memory bugs. This involves using pointers in UB ways that don't make sense as far as the language is concerned.

In @safe code it is nonsense to write specific numbers into a pointer. But in @system code, it does make sense.

I don't think one could write a symbolic debugger with @safe code. Like writing instruction bytes into a buffer, and then calling it? How unsafe can one get? :-)

And so on.
June 09, 2022

On 6/9/22 11:07 PM, Walter Bright wrote:

>

On 6/9/2022 7:51 AM, Timon Gehr wrote:

>

Your are predicting that some people will explicitly do the wrong and lazy thing,

My experience is that the vast bulk of people will do the least amount of effort. It's why software is always larded up with technical debt. I do it, too. Yes, sometimes I've used duct tape and baling wire. Anyone who claims they haven't, I don't believe :-)

In order for safe-by-default extern(C) to actually prevent code breakage, their code:

  1. Has to be marked @system or unmarked
  2. Has to call C functions that are unmarked
  3. Has to all be actually @safe code (as checked by the D compiler) that they didn't mark as @safe.

1, and 2, I can see being true. 3 not so much. Especially if dip1000 is enabled.

They aren't going to bother with @trusted:, they'll just apply @system: to all their code, not just the extern(C) functions. Which is fine. That's actually the correct way to mark "I don't care about safety", and it's not lying.

Not to mention that we have lots of C modules like this: https://github.com/dlang/druntime/blob/ae0724769e3808398b3efdaed4ebdb59c676100d/src/core/stdc/stdio.d#L52

So even if you make extern(C) safe by default most C modules are ALREADY MARKED WITH @system:!! Will they not complain when printf can't be called from their hello world program? Hey, maybe they'll just declare a new printf prototype because that will now make it @safe!

>

At some point all C functions have to be trusted in some form or other because the D compiler has NO way to check them, and neither does the D programmer. Putting @trusted on the C declarations accomplishes nothing, it's safety theater.

Then you don't understand what @trusted or @safe means. Marking a C function @safe doesn't mean it's bug free. It means that it obeys the rules of D @safe.

I trust that the libc authors implemented free correctly. I don't trust that they completely disregarded the C spec and made it valid for D @safe. This means extern(C) calls that takes a char * must only read at most one byte from that pointer. This means that all arrays only can read one value from the array (because a C array is a pointer). This means that memory can never be freed, because it would leave dangling pointers. This means that size_t parameters accompanying pointers can't be used to judge how many elements of the pointer to read. This is not an exhaustive list.

>

In druntime, we've gone through many (certainly not all) of the C declarations and appropriately added correct annotations to them. But realistically, this is not scalable.

I am completely lost here. How is it not scalable to go into every libc module of druntime and mark them with @system: at the top? I can do it if you want, it probably will take 10 minutes. Most time will be spent searching for already existing @system: attributes to skip having to attribute that module.

-Steve

June 09, 2022
On 6/9/2022 8:49 PM, Steven Schveighoffer wrote:
>> In druntime, we've gone through many (certainly not all) of the C declarations and appropriately added correct annotations to them. But realistically, this is not scalable.
> 
> I am completely lost here. How is it not scalable to go into every libc module of druntime and mark them with `@system:` at the top? I can do it if you want, it probably will take 10 minutes. Most time will be spent searching for already existing @system: attributes to skip having to attribute that module.

I did say "correct annotations". Granted, just slapping @system on them is easier.

My experience is that asking people to make *any* edits to existing files is asking a lot, especially if it is code that is considered tested and working, and especially if it is code they don't have rights to the repository to change.

I'm the one who gets the earful when these changes are needed.
June 10, 2022

On Friday, 10 June 2022 at 05:14:21 UTC, Walter Bright wrote:

>

My experience is that asking people to make any edits to existing files is asking a lot, especially if it is code that is considered tested and working, and especially if it is code they don't have rights to the repository to change.

I'm the one who gets the earful when these changes are needed.

Just have a compiler option that gives the old behaviour.

June 10, 2022

On 6/10/22 1:14 AM, Walter Bright wrote:

>

On 6/9/2022 8:49 PM, Steven Schveighoffer wrote:

> >

In druntime, we've gone through many (certainly not all) of the C declarations and appropriately added correct annotations to them. But realistically, this is not scalable.

I am completely lost here. How is it not scalable to go into every libc module of druntime and mark them with @system: at the top? I can do it if you want, it probably will take 10 minutes. Most time will be spent searching for already existing @system: attributes to skip having to attribute that module.

I did say "correct annotations". Granted, just slapping @system on them is easier.

I'm trying to parse this. You mean to say, there are enough unmarked extern(C) functions inside druntime, that fixing them all as they come up is not scalable? That seems unlikely. Note that modules like core.stdc.math has @trusted: at the top already.

My point above is that unmarked extern(C) calls are @system now, and marking them as @system will not change anything.

I will volunteer to mark any druntime extern(C) functions within a 2-day turnaround if they are posted on bugzilla and assigned to me. Start with @system: at the top, and mark them as the errors occur.

>

My experience is that asking people to make any edits to existing files is asking a lot, especially if it is code that is considered tested and working, and especially if it is code they don't have rights to the repository to change.

I'm the one who gets the earful when these changes are needed.

The vast majority of compiler errors that will come from a safe-by-default DIP are functions that are actually @system and now implicitly get marked @safe -- not because they call unmarked extern(C) functions, but because they do @system things (like casting). If you think the "no-edits" bar is only cleared if extern(C) functions are assumed @safe, you are 100% wrong.

You can point the complaints about extern(C) functions at my ear, and deal with the significant majority of complaints that are about @safe by default D code.

I would love to see a viable safe-by-default DIP get added.

-Steve

June 10, 2022

On Friday, 10 June 2022 at 03:07:23 UTC, Walter Bright wrote:

>

On 6/9/2022 7:51 AM, Timon Gehr wrote:

>

Your are predicting that some people will explicitly do the wrong and lazy thing,

My experience is that the vast bulk of people will do the least amount of effort. It's why software is always larded up with technical debt. I do it, too. Yes, sometimes I've used duct tape and baling wire. Anyone who claims they haven't, I don't believe :-)

>

hence the compiler should do the wrong and lazy thing implicitly by default. This just makes no sense. What's the big harm in annoying lazy people slightly more? It's not like they won't complain loudly about @safe by default in any case.

I'm the recipient of all the complaints that I'm breaking their existing code.

>

May as well do it right or not at all.

This entire thread is what happens with "not at all".

At some point all C functions have to be trusted in some form or other because the D compiler has NO way to check them, and neither does the D programmer. Putting @trusted on the C declarations accomplishes nothing, it's safety theater.

In druntime, we've gone through many (certainly not all) of the C declarations and appropriately added correct annotations to them. But realistically, this is not scalable.

That's why it's a mistake to even mark C declarations with @safe or @trusted. Rust treats all C declarations as unsafe. That mistake could have been fixed, slightly by having C/C++ declarations be unsafe by default.

The word used for the keyword of @trusted is kind of wrong, C code should never be "trusted", it is always unsafe. So when you say at some point all C code is trusted, that statement is just wrong. Think of it as safe and unsafe. You are basically saying all C code is safe, when that's wrong. All C code is unsafe.

June 11, 2022
On 6/10/22 05:07, Walter Bright wrote:
> 
>> hence the compiler should do the wrong and lazy thing implicitly by default. This just makes no sense. What's the big harm in annoying lazy people slightly more? It's not like they won't complain loudly about `@safe` by default in any case.
> 
> I'm the recipient of all the complaints that I'm breaking their existing code.
> ..

I am aware. But if that's your concern it kills the DIP on its own. If anything, you'll receive more complaints with the broken behavior, because then nobody is happy, not even @safe-by-default advocates.

>  > May as well do it right or not at all.
> 
> This entire thread is what happens with "not at all".

Well, I contest that. It's not even closely related. This is an issue concerning `@system` code. What's the default has not much at all to do with it.