April 10, 2022

On Sunday, 10 April 2022 at 04:12:08 UTC, Zach Tollen wrote:

>

On Sunday, 10 April 2022 at 03:37:03 UTC, Bruce Carneal wrote:

>

[...]

Well thanks for your interest, at any rate. And your honesty.

Out of curiosity, do you have an opinion of DIP1035? At least intuitively, I feel like the essence of it should run: "Some variables are dangerous, but the compiler can't figure that out. Therefore we allow those variables to be marked @system so they are treated as unsafe."

I view 1035 as a mechanism to extend the reach of @safe, to reduce the load on conscientious code reviewers.

>

But there are other parts of DIP1035 (initializing globals to unsafe values, and also, an entire aggregate is unsafe if a single member is unsafe) which seem to be about inferring certain things to be unsafe as opposed to helping the compiler to determine what is unsafe. My interest in merging the two DIPS came almost entirely from seeing the value in the "help the compiler learn what is unsafe" side, and I thought the "inferring things unsafe" side could be handled separately. (The helping-the-compiler side is eloquently illustrated in the one of the DIP's linked videos by Steven Schveighoffer. That's where I got my understanding of what I thought the essence of DIP1035 should be about. Of course, that's no reason to arbitrarily merge the two features, @system and __metadata. But it might make it easier to explain why I thought it was possible.)

Thanks for the explanation. I think Steven is a good one to listen to. He talks sense habitually.

If you're available do join us and others at the next beerconf to talk about this and other topics. Paul, another of the good ones and DIP 1035 co-author, sometimes joins in as well.

April 10, 2022

On Sunday, 10 April 2022 at 05:41:36 UTC, Bruce Carneal wrote:

>

Thanks for the explanation. I think Steven is a good one to listen to. He talks sense habitually.

If you're available do join us and others at the next beerconf to talk about this and other topics. Paul, another of the good ones and DIP 1035 co-author, sometimes joins in as well.

Wow, thanks. I think I will.

The only time-relevant issue I have is that if adopted as written now, DIP1035 may preclude the investigation of what I'm proposing, since it uses a single compiler bit for all @system variables, whereas I realized in my prior post that I needed to expand my proposal to two bits — one for @system-initialized, and one for @system-variable.

Even if adopted, however, it should be rather effortless to expand the underlying implementation of DIP1035 if what I'm saying ever proves compelling to actually consider. So it's really not that urgent, but I at least wanted to voice my concern now to establish precedent. (Again, I realize that my proposal is either nuts or misunderstood, depending on who's looking at it.)

April 10, 2022

On Sunday, 10 April 2022 at 05:41:36 UTC, Bruce Carneal wrote:

>

I view 1035 as a mechanism to extend the reach of @safe, to reduce the load on conscientious code reviewers.

One aspect of DIP1035 I'm confused about in this regard is item (1) in the proposed changes: "An aggregate with at least one @system field is an unsafe type."

This concerns me because it seems like it would extend the reach of @system, rather than @safe. It seems like having one @system variable would contaminate the whole structure so that you always had to use @trusted just to do anything with it.

Maybe I'm thinking too much...

April 10, 2022

On Sunday, 10 April 2022 at 13:02:07 UTC, Zach Tollen wrote:

>

On Sunday, 10 April 2022 at 05:41:36 UTC, Bruce Carneal wrote:

>

I view 1035 as a mechanism to extend the reach of @safe, to reduce the load on conscientious code reviewers.

One aspect of DIP1035 I'm confused about in this regard is item (1) in the proposed changes: "An aggregate with at least one @system field is an unsafe type."

This concerns me because it seems like it would extend the reach of @system, rather than @safe. It seems like having one @system variable would contaminate the whole structure so that you always had to use @trusted just to do anything with it.

Yes, at what cost do we achieve additional (automated) safety? I have no 1035 improvements to offer, and support it as is, but I think that your concern here is on point.

April 10, 2022
On 10.04.22 02:04, Zach Tollen wrote:
> On Saturday, 9 April 2022 at 17:22:38 UTC, Timon Gehr wrote:
>> Your proposal is not simpler.
> 
> It is simpler.

If `@system` and `__metadata` are separate annotations, you get:

- `@system` rules
- `__metadata` rules

If you conflate them, you get:

- `@system` rules
- `__metadata` rules
- Weird interactions between the rules
- Additional arbitrary rules to be able to pretend the rules for the different annotations are compatible anyway

The additional rules and weird interactions are what's causing complexity, in addition to the misnamed qualifier and the significantly enhanced area of contact between D users and inherent `__metadata` weirdness.

> The only question is whether that simplicity comes at too high a cost.
> ...

If "simplicity" is causing costs, you are probably doing it wrong. Don't try to make things "simpler" than they are. It's a source of complexity and further ad-hoc patches.

>> Your proposal fails to meet your own standards.
> 
> I don't know if it meets my standards or not. I would need an example where a variable which was either explicitly marked, or inferred `@system`, and implemented as desired with that label, then changed its behavior as a result of also implicitly having the privileges of `__mutable`.
> 

It's not a privilege if it's undesirable. Also, you have been given multiple examples by now.

(You removed relevant context here.)
>> I see what you did there. I can do that too: Clearly you think that simplicity and expressiveness are both monotone functions of the number of attributes that are in the language. If what you say were true, we should conflate @safe and pure, as well as @nogc and nothrow because that would be "simpler". That's ridiculous!
> 
> No I'm saying that simplicity and expressiveness are tradeoffs.

I was objecting to your setting up of a straw man that you then proceeded to attack. I then simply demonstrated my ability to do the same, expecting it might sway you as you seem to enjoy that kind of reasoning yourself. Unfortunately this has not worked out.

In any case, there are very simple systems that are very expressive. You want to be in that quadrant.

> The Go language, for example, traded a lot of expressiveness for simplicity. That doesn't make it bad language design.

I don't think this is entirely the case. They made some miscalculations early on and are now left with a language that is neither as simple nor as expressive as it should have been, nor as orthogonal as they claim.

> It means they weighed the pros and the cons, and they made a decision.
> ...

It's probably one level further away from that. They didn't spend a lot of effort to make a good language. They just spent vastly more time writing the compiler and libraries and my understanding is they did a pretty decent job there. Also, perhaps they actively wanted to avoid having a cerebral community that cares about abstract things such as PL theory. (There was a year-long active hostility towards potentially simple things such as generics in the Go community. You can draw your own conclusions.)

>>> The use cases of DIP1035, for the most part, are a strict subset of the uses of `__mutable`/`__metadata`.
>>
>> ???
> 
> All `__metadata` variables qualify as DIP1035 `@system`. If you added `@system` to any instance of `__metadata`, it would have no effect.

Granted. Now you have shown "var is `__metadata`" => "var is `@system`", but that's not some sort of magical relationship between the qualifiers. There are multiple reasons why variables might by `@system` as it just collects every source of unsafety. `__metadata` is a new such source that we are considering adding to the language.

However, your claim above was the exact opposite: "var is @system" => "var is __metadata". As you said it is even a _strict subset_, you additionally claim that there are use cases where variables should be `__metadata` but not `@system`.

You seem to be contradicting yourself. It's either bad logic or bad communication.

> Therefore, if only `__metadata` were implemented, users could utilize it to achieve the intended goals of DIP1035.

No, because DIP1035 does not alter variable types in any way. You could similarly say: "Just initialize your variables with unsafe values and get rid of the need for a `@system` annotation." This is however not a simplification. The compiler still needs to track all that and making it part of the notation makes it simpler to understand.

> (Just allow them to mark it wherever they wanted instead of confining it to aggregate members, even though most of the use cases are in fact aggregate members.)

This is not a complete specification...

> The question is simply whether additionally being able to violate the type system would cause intractable problems.

No, not at all. Given that you apparently dismissed all the evidence in this thread as "tractable problems", your standard is way off in my opinion. E.g., I would similarly not run into _intractable_ problems if I chopped one of my arms off. That does not make it any less stupid even though one arm is clearly simpler than two arms. It would however not simplify my life.

You are basically stating that you will not move away from your position, no matter what. Digging in like that is not healthy, and this proposal is not actually worth spending time on, neither yours nor mine.

> I don't know. I suspect that almost all use cases of `@system` are mutable to begin with. And if they weren't, I would have to look the workarounds, to see if issues of having undesired `__metadata` functionality caused more pain than the simplicity of using only one keyword merited.
> ...

You keep claiming it's simpler. It's not. You are chasing after a prize that's not actually worth anything because you are using some useless, tunnel-vision notion of simplicity that only considers one (very unimportant) aspect of what potentially makes a language simple.

I would not chop off my arm even if someone offered to buy my house for one dollar in exchange, even though one dollar is a very simple amount. Avoiding that kind of "trade-off" should not be controversial.

>> So your proposal is...
> 
> My proposal is to take everything DIP1035 marks or infers as `@system` and extend `__metadata` functionality to it.

I see, so if a module-scope variable is inferred `@system`, you will get an error because `__metadata` has been applied to a non-member variable. (I get that you state at other places you don't want it to work that way, but given that you seem to be unable to simply state what your proposal is, it's unlikely to actually be all that simple.)

> This may sound dangerous. 

Primarily it sounds utterly pointless. All the reasoning about why it's "dangerous" is only required because you refuse to accept that it's pointless.

> The logic is simply that this is `@system` code, and `@system` code is dangerous.

Given your own standards of evidence, you will have to do much better than that. Please tell me how having separate `@system` and `__metadata` will cause intractable pain, suffering and ultimately end civilization as we know it. If you can't do that in a way that satisfies me, I will dismiss everything you say.

> But it is also powerful. That's the overarching idea. Both dangerous and powerful. (D should also be working towards `@safe` by default, of course.)
> 
> Again, the only question is whether having `__metadata` functionality will cause intractable pain to the use cases of `@system` which don't require it.
> ...

I really don't understand why you are willing to suffer everything short of "intractable pain" for this issue.

>> In any case, even if your proposal was simple to state formally, it would still lead to really bad outcomes.
> 
> The difference between us is that you are sure of that, and I am not.
> ...

I wouldn't say that's the whole difference.

> ...
> 
> The solution to our debate — insofar as my claims deserve any merit at all — is to implement both DIP1035 and the `__metadata` DIP, with the additional requirement that all uses of `__metadata` be marked `@system` as well.

Not much point in that, it's just more line noise.

> Semantically, it would change nothing, and it would actually be easier to understand for newcomers.
> ...

Newcomers will not have to learn about `__metadata` or `@system` variables for a pretty long time. Then they may come across `@system` variables at some point, and finally, some further minority of D programmers will actually need to significantly concern themselves with `__metadata` rules.

If you keep the qualifiers separate, the meaning of `@system` is simple and intuitive and nobody who would not otherwise need to will have to concern themselves with `__metadata`. That's a much better proposition for newcomers than arbitrary conflation.

> Then, five years later, to test my theory, add a compiler switch `preview=deprecate__metadata`, put the `__metadata` functionality into the `@system`, and make `__metadata` a no-op.
> 
> If nothing breaks, I was right. If something breaks, and the workarounds aren't seamless, then you were right.
> 
...

In other words, you will just move the goalposts until you can claim you were right. Maybe you should care less about having been right in the past and more about being right in the future. Note that in my very first debate on this forum, I admitted that I had been wrong and that Andrei was right. For me, this is not about being right, it's about not having to go through the hassle of switching programming languages. I don't enjoy pointless debate, but weird language rules make it harder to justify why I am even using D. I need it to look like things are getting better (which generally they have been over the past 10 years), not worse.
April 10, 2022

On Saturday, 9 April 2022 at 06:25:10 UTC, vit wrote:

>

RAII in D is bad: move ctor are not implemented, overloading copy ctors sometimes crash dmd, copy ctors are not called from GC slices, dtors have bad interaction with stack tuples/value sequences and opCast, emplace...

Imho these should be prioritized first before, doing yet another language change. If they are fixed, then there might be more options available on how to design the rc functionality.

April 10, 2022
On 10.04.22 16:33, Bruce Carneal wrote:
> On Sunday, 10 April 2022 at 13:02:07 UTC, Zach Tollen wrote:
>> On Sunday, 10 April 2022 at 05:41:36 UTC, Bruce Carneal wrote:
>>> I view 1035 as a mechanism to extend the reach of @safe, to reduce the load on conscientious code reviewers.
>>
>> One aspect of DIP1035 I'm confused about in this regard is item (1) in the [proposed changes](https://github.com/dlang/DIPs/blob/72f41cffe68ff1f2d4c033b5728ef37e282461dd/DIPs/DIP1035.md#proposed-changes): "An aggregate with at least one @system field is an unsafe type."
>>
>> This concerns me because it seems like it would extend the reach of *@system*, rather than *@safe*. It seems like having one @system variable would contaminate the whole structure so that you always had to use @trusted just to do anything with it.
>>
> 
> Yes, at what cost do we achieve additional (automated) safety?  I have no 1035 improvements to offer, and support it as is, but I think that your concern here is on point.
> 
> 

Here, "unsafe type" has a very specific meaning that is actually elaborated on by the DIP. Even unsafe types can typically be used from @safe code just fine. I think this particular concern grew out of a reading of the document that was overly cursory. This "contamination" concern has nothing in the DIP to back it up.
April 10, 2022
On Friday, 8 April 2022 at 18:00:14 UTC, rikki cattermole wrote:
> Here is my test code that I used to determine what I needed to do to make const RC types useless (and hopefully safe).
> ...

Isn't better then to try and find design patterns or rules for use with mutable RC structures?

All the talk across the years was basically about trying to eat the pancake and also at same time trying to keep it for later, and neither is done after all deliberation across the years. Perhaps it would be best to just break the existing immutable system, and redefine it to allow rc features found in C++ or other languages, or just have a limited version of rcs in D (compared to C++ or other languages employing them), and workaround those restrictions through some well defined rules and design patterns.

And when mentioning breaking the immutable system, I really mean breaking it, by removing the transitiveness it has, or some other major change, like not being really immutable, and therefore safe to put in readonly memory.


April 10, 2022
On 10.04.22 16:48, Alexandru Ermicioi wrote:
> On Saturday, 9 April 2022 at 06:25:10 UTC, vit wrote:
>> RAII in D is bad: move ctor are not implemented, overloading copy ctors sometimes crash dmd, copy ctors are not called from GC slices, dtors have bad interaction with stack tuples/value sequences and opCast, emplace...
> 
> Imho these should be prioritized first before, doing yet another language change. ...

I agree that lifetime semantics that actually work should be a much higher priority than hiding reference counts in immutable structures. (But maybe the people who are willing to fix either thing are not the same people.)
April 10, 2022

On Sunday, 10 April 2022 at 14:52:43 UTC, Timon Gehr wrote:

>

On 10.04.22 16:33, Bruce Carneal wrote:

>

On Sunday, 10 April 2022 at 13:02:07 UTC, Zach Tollen wrote:

>

On Sunday, 10 April 2022 at 05:41:36 UTC, Bruce Carneal wrote:

>

Here, "unsafe type" has a very specific meaning that is actually elaborated on by the DIP. Even unsafe types can typically be used from @safe code just fine. I think this particular concern grew out of a reading of the document that was overly cursory. This "contamination" concern has nothing in the DIP to back it up.

Very glad to hear that from a careful reader.