On Wednesday, 10 June 2020 at 14:20:37 UTC, Dennis wrote:
> On Wednesday, 10 June 2020 at 12:36:13 UTC, Stanislav Blinov wrote:

>> As before, that last line is not memory corruption, it's UB.
>
> Each example has two consecutive moments:
> A) Memory gets in a state that violates the programmer's intended invariants.
> B) Because of that, immutable memory is mutated, or memory outside a pointer's allocated block is accessed.
>
> If I understand correctly, you claim that A) should be called memory corruption and B) is undefined behavior, while I call B) memory corruption. In the D specification, corrupting memory is undefined behavior, and undefined behavior can manifest in memory corruption, so when talking about @safe the terms are sometimes used interchangeably.

Memory corruption is undefined behavior, but undefined behavior is not memory corruption. The former is a subset of the latter, they're not equivalent, and so should not be used interchangeably. BUT, so far as those comments in examples are concerned, I do stand corrected regardless (should've consulted a dictionary beforehand), since "corruption" can refer both to corrupting and being corrupt(ed). Apologies.

On 10.06.20 17:56, Stanislav Blinov wrote:
> undefined behavior is not memory corruption.
It is legal for an implementation to cause memory corruption whenever it encounters undefined behavior.

On Wednesday, 10 June 2020 at 17:02:57 UTC, Timon Gehr wrote:
> On 10.06.20 17:56, Stanislav Blinov wrote:
>> undefined behavior is not memory corruption.
> It is legal for an implementation to cause memory corruption whenever it encounters undefined behavior.

Just as it is legal to not. Therefore, one is not equivalent to (interchangeable with) the other.

On Wednesday, 10 June 2020 at 17:11:02 UTC, Stanislav Blinov wrote:
> Just as it is legal to not. Therefore, one is not equivalent to (interchangeable with) the other.

Please, let's not discuss at length the nuances in the definition/semantics of the terms "memory corruption" and "undefined behavior", unless there is an important distinction relevant for the DIP.

On Wednesday, 10 June 2020 at 08:38:31 UTC, Mike Parker wrote:
> This is the discussion thread for the first round of Community Review of DIP 1035, "@system Variables":
>
> https://github.com/dlang/DIPs/blob/148c78e6e7eeb5609715cb31a45c0aa5c8ebdce7/DIPs/DIP1035.md

I don't have any specific feedback yet on the contents of the DIP or their correctness, but I do want to commend the author on how systematic it is in covering the different issues or concerns, and providing concrete examples to illustrate them.

This is a standard that all DIPs should aspire to :-)

On Sunday, 14 June 2020 at 10:46:17 UTC, Joseph Rushton Wakeling wrote:

> I don't have any specific feedback yet on the contents of the DIP or their correctness, but I do want to commend the author on how systematic it is in covering the different issues or concerns, and providing concrete examples to illustrate them.
>
> This is a standard that all DIPs should aspire to :-)

Indeed. Problem well defined, solution well presented.

Bump (we don't want to forget a DIP discussion now do we?)

On Tuesday, 16 June 2020 at 10:44:46 UTC, Dukc wrote:
> Bump (we don't want to forget a DIP discussion now do we?)

Seems the feature is not controversal - nobody complains, so it seems everybody is pleased with it. Let's move it forward!

Synopsis: this DIP should be best replaced with a series of bug reports related to @safe. We can't define new features whenever the ones we have have bugs in them. (Note that accepting the DIP would still allow all of its egregious examples to compile.)

A few notes:

* The background section should kickstart the narrative, but the general remarks it makes leave the reader wanting. For example: "This can only work if the language in which such types are defined has the needed encapsulation capabilities." to which the obvious question is, "what's wrong with a struct that puts that thing in a private member and brokers all access to it?"

* The second example uses:

__traits(getMember, sum, "tag") = 1; // ! ruin integrity of tagged union

The obvious conclusion here is: using __traits(getMember) breaks encapsulation, therefore it should not be allowed in @safe code. That is a bug in @safe that needs fixing (regardless of the introduction of another feature).

The explanation that follows fails to convince:

* "the tag having private visibility only encapsulates at the module level, while @trusted is applied at the function level, so it can still be violated by functions in the same module" -> the D protection system is fundamentally module level. Construing that in an objection for just this one feature creates the odd precedent that all other encapsulation primitives should be changed or complemented accordingly. That's well outside the scope of this one feature.

* "even outside the module, __traits(getMember, ) bypasses private" -> that's an obvious bug that needs to be fixed.

* <<The tagged union is just one example; there are many other situations where data should only be modified by code that "knows what it's doing">> - they call that private.

* "As long as the reference count can be meddled from @safe code, freeing the memory cannot be @trusted, making @safe reference counting unsupported in D." -> That means there are bugs in @safe, not a need for yet another feature with its own liabilities.

* The section title "Existing holes in @safe" contains its own conclusion. The obvious answer is, great work on documenting them, they need to be fixed!

* And indeed the example with getPtr() illustrates an obvious bug. Safe code has no business calling into @system code. It's worth noting, again, that even if this DIP is approved, that code would continue being problematic.

* The paragraph about bool's problems even has a reference to an existing issue. So how does it motivate the introduction of a new feature?

* The quoted Rust discussion is the best argument in the DIP. I don't know Rust enough to figure whether they have a solution similar to our @safe/@trusted/@system troika, i.e. I'm not sure they'd have a workaround just as satisfying as ours. The DIP should discuss that to establish applicability. (Also, that Rust proposal is quite controversial.)

Speaking of workarounds, aside from fixing the bugs in @safe, I think someone in need for a system variable can use a little module with:

struct SysVar(T) {
    private T value;
    ref T get() @system { return value; }
    void set(ref T rhs) @system { value = rhs; }
}

Embellishments are immediate (ctors, allowing void init, opAssign, etc). To the extent this type can be pried open in @safe code, these are definitely bugs in the existing language.

* The "Alternatives" section should be moved before the bulk of the proposal. The narrative should go: here's the problem we have, here are the solution within the existing language, here are the problems with them, here is the new feature that solves them better.

* "First of all, disallowing the bypassing of private in @safe code is not sufficient for ensuring struct invariants. As mentioned in the quote, sometimes invariants need to hold onto types that are not unsafe, such as int. When there are no pointer members, then private fields can still be indirectly written to using overlap in a union, void-initialization, or array casting." -> all of these are unsafe operations. What am I missing?

* "Finally, it would mean that circumventing visibility constraints using __traits(getMember, ...) must become @system or deprecated entirely, similarly to .tupleof. This would break all (@safe) code that uses this feature, and reintroduces the problems of Issue #15371. All things considered, making private work with @trusted appears to be a bigger hassle than introducing checks for @system variables and fields." -> I don't understand this argument. Yes you should be able to bypass, just not in @safe code. Yes fixing that would break code, as it should. Again it's a problem with the feature that won't get solved by adding another one. I think the best value that can be derived from this DIP might be the necessity of __traits(getPublicMember, ...).

To conclude, it seems this DIP has great value at documenting, arguing, and illustrating the failings of @safe, but should at best conclude that those bugs need fixing.

On 17.06.20 03:12, Andrei Alexandrescu wrote:
> * The background section should kickstart the narrative, but the general remarks it makes leave the reader wanting. For example: "This can only work if the language in which such types are defined has the needed encapsulation capabilities."

It's a bad formulation. This is not about encapsulation.

> to which the obvious question is, "what's wrong with a struct that puts that thing in a private member and brokers all access to it?"
> ...

It does not. The module would. Also, you could add `@safe` code to it that would then in turn break memory safety. If you diff looks like this:

- @system int x;
+ int x;

That is fishy.

If you diff looks like this:

+ @safe void foo(){ x=3; }

That's not supposed to be fishy.

Furthermore, your suggestion does not solve the issue with unsafe initializers.

> * The second example uses:
> 
> __traits(getMember, sum, "tag") = 1; // ! ruin integrity of tagged union
> 
> The obvious conclusion here is: using __traits(getMember) breaks encapsulation, therefore it should not be allowed in @safe code. ...

@safe means memory safe, not encapsulated. See reflection in Java.
For variables that satisfy invariants, the compiler cannot always figure out whether accesses are permissible without some annotations.

> The explanation that follows fails to convince:
> 
> * "the tag having private visibility only encapsulates at the module level, while @trusted is applied at the function level, so it can still be violated by functions in the same module" -> the D protection system is fundamentally module level.

The documentation says @trusted code has to be audited, not all code in modules that contain a bit of @trusted code or no @trusted but both @system and @safe code.

> Construing that in an objection for just this one feature creates the odd precedent that all other encapsulation primitives should be changed or complemented accordingly. That's well outside the scope of this one feature.
> 

@safe means memory safe, not encapsulated. private variables are variables that have to respect some invariant. @system variables are variables that have unsafe values.

> 
> * "even outside the module, __traits(getMember, ) bypasses private" -> that's an obvious bug that needs to be fixed.

It's not obvious that it is a bug (it would need to be defined to be a bug) nor is it obvious that making bypassing of `private` unsafe obviates the need for @system variables.

> 
> * <<The tagged union is just one example; there are many other situations where data should only be modified by code that "knows what it's doing">> - they call that private.

So your suggestion is @safe code cannot access any private members?

@safe code is code written by someone who potentially does not know what they are doing.

> 
> * And indeed the example with getPtr() illustrates an obvious bug. Safe code has no business calling into @system code.

Under current language rules, it's not @safe code. That's the problem. Variable initializers have no safety annotations.

> It's worth noting, again, that even if this DIP is approved, that code would continue being problematic.

No, that's only true for your supposedly equivalent workaround, the DIP solves that problem. It's proposed change (3).

However, I think accessing `extern` variables would also need special consideration.

> 
> * The paragraph about bool's problems even has a reference to an existing issue.

The issue is there is undefined behavior in @safe code. Fixing it requires design work as there are multiple ways to fix it.

> So how does it motivate the introduction of a new feature?

The DIP introduces a couple concepts related to @system variables and defines them.

> 
> * The quoted Rust discussion is the best argument in the DIP.

Not really. I don't think you will find additional arguments there.

> I don't know Rust enough to figure whether they have a solution similar to our @safe/@trusted/@system troika, i.e. I'm not sure they'd have a workaround just as satisfying as ours.

The Rust workaround would be to create some sort of cell struct with a private member and unsafe accessors and use that.

> The DIP should discuss that to establish applicability. (Also, that Rust proposal is quite controversial.)

It's similar to some extent but not fully. Rust has unsafe functions and unannotated functions, the equivalent of a trusted function is any function that has at least one `unsafe` block in it. Additionally, functions that can access certain state that unsafe functions can access have to be treated as trusted, but the language does not make it easy to track this.

That's why there is a desire for unsafe fields in Rust; to eliminate that last category. This is also why we want it in D, as we can't trust @safe code.


> 
> Speaking of workarounds, aside from fixing the bugs in @safe, I think someone in need for a system variable can use a little module with:
> 
> struct SysVar(T) {
>     private T value;
>     ref T get() @system { return value; }
>     void set(ref T rhs) @system { value = rhs; }
> }
> ...

You don't need the setter and calling it with an rvalue calls postblit and destructor more times than necessary.

> Embellishments are immediate (ctors, allowing void init, opAssign, etc). To the extent this type can be pried open in @safe code, these are definitely bugs in the existing language.

Not all of them, though the fact that it has a default constructor `this(T value){ this.value = value; }` is a bug. Maybe you can make it work the way you envision, but what is to stop someone from coming along and adding some more @safe code to that module? You won't even find the problem by grepping for @trusted and nobody wrote any unsafe code.

> 
> 
> * "First of all, disallowing the bypassing of private in @safe code is not sufficient for ensuring struct invariants. As mentioned in the quote, sometimes invariants need to hold onto types that are not unsafe, such as int. When there are no pointer members, then private fields can still be indirectly written to using overlap in a union, void-initialization, or array casting." -> all of these are unsafe operations. What am I missing? 

Simply that those are not inherently unsafe operations.

---
union U{
    int a;
    private int b;
}

void main()@safe{
    U u;
    u.a=3; // change b
}
---
void main()@safe{
	int x=void;
}
---
struct S{
    private int x;
}
void main()@safe{
    S s;
    (cast(int[])cast(int[1])s)[]=2;
}
---

Walter has expressed a desire to keep it that way, particularly adamantly for `void` initialization.