April 08, 2022
On Friday, 8 April 2022 at 14:23:01 UTC, H. S. Teoh wrote:
> On Fri, Apr 08, 2022 at 01:50:01PM +0000, Alexandru Ermicioi via Digitalmars-d wrote:
>> On Wednesday, 6 April 2022 at 09:41:52 UTC, RazvanN wrote:
>> > ...
>> It may be a dumb and already discussed question, but what's the problem of making rc counted structs mutable only?
>> 
>> In this way, you won't have the dilemma of circumventing the immutable/const system, to just increment the counter, or any metadata.
>> 
>> Note: the payload of rc struct, may be const or immutable if needed.
> [...]
>
> In theory, this would IMO be the right approach.  However, it does hamper usability. For example, you could no longer write:
>
> 	auto myFunc(in RC!Data data) { ... }
>
> but you'd have to write instead:
>
> 	auto myFunc(RC!(const(Data)) data) { ... }
>
> because RC!... has to remain always mutable.

Nicer version: RC!(const T)
Though that is true.

>
> Some of the nice implicit conversions would also no longer work as
> desired, e.g., RC!Data would not implicitly convert to RC!(const(Data)),
> where with GC'd data, Data* implicit converts to const(Data)*. This
> asymmetry makes metaprogramming harder to work with RC.

Can't alias this be employed for this?

I.e. in Rc struct have this:
alias this = asQualified;

Where asQualified returns version with suitable modifier.

>
> Things get worse once you have nested structures, e.g.:
>
> 	struct SubData {...}
> 	struct S {
> 		RC!SubData data;
> 	}
>
> Now you couldn't pass S to anything that takes const, because transitivity of const would force RC!SubData to be const(RC!SubData), which breaks the refcounting mechanism.
> Basically, once you have RC!xxx in any subpart of your data structure, it "reverse-infects" the entire data structure with mutable, because you can no longer use const on any containing part of the structure, since it would break RC.

True, it won't be easy to use RC as fields, but could we ignore this problem for initial implementation, by prohibiting immutable versions?

You can always extend RC structs to support immutable/const qualifiers on itself.

I keep reading this forum for a long time, and always see only discussions about rc structs and etc, but no viable solutions. Perhaps it's better to try implement a restricted version of it first? i.e. only mutable rcs?


April 08, 2022
On 08.04.22 17:07, Zach Tollen wrote:
> On Friday, 8 April 2022 at 08:23:34 UTC, Timon Gehr wrote:
>> Yes. I do not understand why so many people are so keen to conflate entirely different things into the same concepts. That's just bad language design.
> 
> No it's not.

Of course it is. It hampers expressiveness while adding incidental complexity. Lose-lose.

> Bad language design is when you *arbitrarily* conflate different things without first examining the pros and cons.

The language should allow me to state what I mean without adding random additional baggage.

> But it's also bad language design not to be open to the possibility of a conflation of different things which ends up being harmonious and easy to explain.

That's just never how it works out. You should only conflate things that are actually the same thing.
April 09, 2022
Here is my test code that I used to determine what I needed to do to make const RC types useless (and hopefully safe).

```d
void main() {
    auto r1 = /+const+/ Reference(2);
    const r2 = r1;
    const r3 = Reference(2);
    const(Reference) r4 = Reference(2);
    //auto r5 = cast(const)r1;
    auto r6 = cast(void*)&r1;
    auto r7 = cast(const)&r1;
}

struct Reference {
    int x;

@safe:

    this(int value) {
        this.x = value;
    }

    this(ref Reference other) {
        this.x = other.x;
    }


    ~this() {
        this.x = 0;
        writeln("destructor");
    }

    void opAssign(ref Reference other) {
        __ctor(other);
    }

    void opAssign(Reference other) {
        __ctor(other);
    }

    // bool isNull() { ... }

    @disable this(int value) const;

    //@disable this(ref Reference other) const;
    @disable this(ref const Reference other) const;
    @disable this(this);

    //@disable ~this() const;

    @disable void opAssign(ref Reference other) const;
    @disable void opAssign(Reference other) const;

    @disable auto opCast(T)();
}
```
April 09, 2022

On Friday, 8 April 2022 at 17:28:30 UTC, Timon Gehr wrote:

>

On 08.04.22 17:07, Zach Tollen wrote:

>

On Friday, 8 April 2022 at 08:23:34 UTC, Timon Gehr wrote:

>

Yes. I do not understand why so many people are so keen to conflate entirely different things into the same concepts. That's just bad language design.

No it's not.

Of course it is. It hampers expressiveness while adding incidental complexity. Lose-lose.

Wrong. Simplicity is fine when it allows you to do what you want without forcing you to do what you don't want. If what you say were true, then D should have adopted every single built-in attribute and every type qualifier which has ever been suggested, because the lack of them would "hamper expressiveness."

The incidental complexity of a given multi-purpose feature only arises when the one purpose of the feature simultaneously makes another purpose of the same feature difficult to achieve. I do not consider this an inevitable outcome. I want to examine the specific case instead.

> >

But it's also bad language design not to be open to the possibility of a conflation of different things which ends up being harmonious and easy to explain.

That's just never how it works out. You should only conflate things that are actually the same thing.

The use cases of DIP1035, for the most part, are a strict subset of the uses of __mutable/__metadata. It is agreed that you can only access __metadata variables from @system code. Therefore by implementing __metadata you basically get DIP1035 for free. My proposal is simply to use DIP1035's word @system two encompass the two ideas.

The purpose of DIP1035 is to be able to force a variable to be considered unsafe, because it is already being used in an unsafe way. If you're trying to decide whether merging @system and __metadata is a good idea, you would first need to come up with a use case proving a possible conflict of purposes as follows:

You would need to show how marking a variable @system because it is already being used dangerously might then lead to it being used dangerously differently, but now in an unintended and undesirable way, solely because before, when it was used dangerously, it was immutable.

It seems like a really tall order to me. And if you can't provide an example like that, then on what basis can you say that these two features can't be combined?

My main point is that mutability, in nearly all cases, is precisely what would motivate a variable to require being marked @system. As to global data which is initialized to unsafe values and therefore inferred @system, I don't even know what these would be for. They seem like they would always be bugs to me.

April 09, 2022

On Friday, 8 April 2022 at 18:00:14 UTC, rikki cattermole wrote:

>

Here is my test code that I used to determine what I needed to do to make const RC types useless (and hopefully safe).

...

I implemented shared_ptr/rc_ptr with this functionality (non copyable const rc ptr):
https://github.com/submada/btl/blob/master/source/btl/autoptr/shared_ptr.d
https://github.com/submada/btl/blob/master/source/btl/autoptr/rc_ptr.d

It is in package btl:autoptr (https://code.dlang.org/packages/btl).

Example:

SharedPtr!(const int) sp = SharedPtr!int.make(42);

const SharedPtr!(const int) csp1 = sp;  //OK
const SharedPtr!(int) csp2 = sp;  //OK

sp = csp1;   //ERROR
sp = csp2;   //ERROR

//RcPtr is same.

Ref counted pointers in D have problem:

RAII in D is bad: move ctor are not implemented, overloading copy ctors sometimes crash dmd, copy ctors are not called from GC slices, dtors have bad interaction with stack tuples/value sequences and opCast, emplace...

Simple ref counted pointers have also this problem:

Rc!(const int) rc1 = Rc!(int).make(42);
Rc!(const int) rc2 = Rc!(immutable int).make(42);

Has Rc!(const int) atomic counting or not? :)

April 09, 2022
On 4/9/22 02:56, Zach Tollen wrote:
> On Friday, 8 April 2022 at 17:28:30 UTC, Timon Gehr wrote:
>> On 08.04.22 17:07, Zach Tollen wrote:
>>> On Friday, 8 April 2022 at 08:23:34 UTC, Timon Gehr wrote:
>>>> Yes. I do not understand why so many people are so keen to conflate entirely different things into the same concepts. That's just bad language design.
>>>
>>> No it's not.
>>
>> Of course it is. It hampers expressiveness while adding incidental complexity. Lose-lose.
> 
> Wrong. Simplicity is fine

Your proposal is not simpler.

> when it allows you to do what you want without forcing you to do what you don't want.

Your proposal forces me to do what I don't want without allowing me to do what I want. I don't want that.

Your proposal fails to meet your own standards.

> If what you say were true, then D should have adopted every single built-in attribute and every type qualifier which has ever been suggested, because the lack of them would "hamper expressiveness."
> ...

I see what you did there. I can do that too: Clearly you think that simplicity and expressiveness are both monotone functions of the number of attributes that are in the language. If what you say were true, we should conflate @safe and pure, as well as @nogc and nothrow because that would be "simpler". That's ridiculous!

> The incidental complexity of a given multi-purpose feature only arises when the one purpose of the feature simultaneously makes another purpose of the same feature difficult to achieve. I do not consider this an inevitable outcome. I want to examine the specific case instead.
> 
>>> But it's also bad language design not to be open to the possibility of a conflation of different things which ends up being harmonious and easy to explain.
>>
>> That's just never how it works out. You should only conflate things that are actually the same thing.
> 
> The use cases of DIP1035, for the most part, are a strict subset of the uses of `__mutable`/`__metadata`.

???

`__metadata` is _much more niche_ and _much more restricted_. Almost no D programmer will have to use it directly.

> It is agreed that you can only access `__metadata` variables from `@system` code. Therefore by implementing `__metadata` you basically get DIP1035 for free.

No. I suggest you go read DIP1035 and find all the aspects not covered by `__metadata`.

> My proposal is simply to use DIP1035's word `@system` two encompass the two ideas.
> ...

So your proposal is to take the `@system` and `__metadata` proposal, concatenate them and change all occurrences of `__metadata` to `@system`? That would be even less sensible than what I thought you were arguing for, so I guess your proposal is actually not "simply" this. One does not simply conflate `@system` and `__metadata`.

In any case, even if your proposal was simple to state formally, it would still lead to really bad outcomes.

> The purpose of DIP1035 is to be able to force a variable to be considered unsafe, because it is already being used in an unsafe way. If you're trying to decide whether merging `@system` and `__metadata` is a good idea, you would first need to come up with a use case proving a possible conflict of purposes as follows:
> 
> You would need to show

You are making a proposal.

> how marking a variable `@system` because it is *already* being used dangerously might then lead to it being used dangerously *differently*, but now in an unintended and undesirable way, solely because before, when it was used dangerously, it was immutable.
> ...

So you want to argue that everything that's potentially dangerous is actually the same thing? Can as well just turn off type checking and bounds checks in `@system` code completely at that point. It's such fallacious reasoning...

> It seems like a really tall order to me.

To the extent that I can make sense of your paragraph, it's very easy and Paul has already done it.

NB: I really dislike it when some people go from "there are no counterexamples" to "there is exactly one counterexample and I don't care about that particular one" after they have been presented a counterexample to their claims. It makes me not want to engage with their posts anymore. It's so disrespectful to other people's time. If you thought there were no counterexamples and were presented one, chances are that you also missed other things.

> And if you can't provide an example like that, then on what basis can you say that these two features can't be combined?
> ...

I am not interested in that particular scenario as I can provide multiple examples that immediately kill your proposal. However, it seems it's not worth the effort as you will just move the goalposts with informal nonsense such as "for the most part" and "in nearly all cases" or "used dangerously". The qualifiers have precise formal meanings.

> My main point is that mutability, in nearly all cases, is precisely what would motivate a variable to require being marked `@system`.

Even in cases where mutability is the issue you should not simply be able to bypass `const` implicitly/by accident, even in `@system` code...

> As to global data which is initialized to unsafe values and therefore inferred `@system`, I don't even know what these would be for. They seem like they would always be bugs to me.

I agree with the point that it seems that you don't know existing language rules and use cases well enough.

Anyway, for anyone who's actually interested, here are some of the problems with the proposal (very likely not exhaustive, it's such an arbitrary and badly motivated conflation that we'd probably find new and horrible consequences for years to come. I have already spent more than two hours on this post and don't want to think about this any more, it's such a pointless waste of time):

- There is nothing about `@system` that screams "this is mutable", it's a very unintuitive language rule. You would literally state `@system immutable` and it would be ignored.

- `@system` variables will be a reasonably common occurrence in low-level code. Explicit `__mutable` is much more specialized and restricted in how it can be applied. It does not make sense to extend `@system` to encompass this much more narrow use case as most `@system` users won't want it.

- `@system` can be inferred from an initializer. It is then _very confusing_, if `const`/`immutable` are just stripped away from the variable declaration, _especially_ if it does not happen often.

- By conflating `__metadata` and `@system`, the compiler seems to engage in implicit, unsafe and unnecessary type-punning. `immutable` would be stripped away _transitively_ from an _existing value_ upon initialization. There is possibly another design where `@system t = foo()` would just not compile because you can't assign the `immutable` result of `foo()` to the mutable `t`, but that makes similarly little sense.

- Templated code may have logic that differs based on the mutability of the argument (e.g., copy the values of all mutable fields from one instance to another).

- Variables inside templates may not always be inferred `@system` the same way for different template arguments. Now suddenly you sometimes get completely unsafe, completely implicit type-punning, potentially you can't even tell from the outside. If you are "lucky", your code might not compile because you can't assign a `T v = init();` back to a `T w` or something.

- Memory safety may actually depend on a `@system` field keeping its value, so it's useful to have `immutable @system` fields.

- There can be mutable and immutable instances of the same type, some of them static. If a type has `@system` fields, the proposal would prevent immutable instances from being stored in the read-only data segment.

- Other optimizations based on `immutable` similarly still work even with `@system` values. (Unless `@system` is arbitrarily conflated with `__metadata`.) You should not have to choose between fast code and soundness of `@safe` checks.
April 10, 2022

On Saturday, 9 April 2022 at 17:22:38 UTC, Timon Gehr wrote:

>

Your proposal is not simpler.

It is simpler. The only question is whether that simplicity comes at too high a cost.

>

Your proposal fails to meet your own standards.

I don't know if it meets my standards or not. I would need an example where a variable which was either explicitly marked, or inferred @system, and implemented as desired with that label, then changed its behavior as a result of also implicitly having the privileges of __mutable.

>

I see what you did there. I can do that too: Clearly you think that simplicity and expressiveness are both monotone functions of the number of attributes that are in the language. If what you say were true, we should conflate @safe and pure, as well as @nogc and nothrow because that would be "simpler". That's ridiculous!

No I'm saying that simplicity and expressiveness are tradeoffs. The Go language, for example, traded a lot of expressiveness for simplicity. That doesn't make it bad language design. It means they weighed the pros and the cons, and they made a decision.

> >

The use cases of DIP1035, for the most part, are a strict subset of the uses of __mutable/__metadata.

???

All __metadata variables qualify as DIP1035 @system. If you added @system to any instance of __metadata, it would have no effect. Therefore, if only __metadata were implemented, users could utilize it to achieve the intended goals of DIP1035. (Just allow them to mark it wherever they wanted instead of confining it to aggregate members, even though most of the use cases are in fact aggregate members.) The question is simply whether additionally being able to violate the type system would cause intractable problems. I don't know. I suspect that almost all use cases of @system are mutable to begin with. And if they weren't, I would have to look the workarounds, to see if issues of having undesired __metadata functionality caused more pain than the simplicity of using only one keyword merited.

>

So your proposal is...

My proposal is to take everything DIP1035 marks or infers as @system and extend __metadata functionality to it. This may sound dangerous. The logic is simply that this is @system code, and @system code is dangerous. But it is also powerful. That's the overarching idea. Both dangerous and powerful. (D should also be working towards @safe by default, of course.)

Again, the only question is whether having __metadata functionality will cause intractable pain to the use cases of @system which don't require it.

>

In any case, even if your proposal was simple to state formally, it would still lead to really bad outcomes.

The difference between us is that you are sure of that, and I am not.

>

To the extent that I can make sense of your paragraph, it's very easy and Paul has already done it.

NB: I really dislike it when some people go from "there are no counterexamples" to "there is exactly one counterexample and I don't care about that particular one" after they have been presented a counterexample to their claims. It makes me not want to engage with their posts anymore. It's so disrespectful to other people's time. If you thought there were no counterexamples and were presented one, chances are that you also missed other things.

Give me an example of a global variable which is intended to be immutable, and which is also initialized to point to undefined memory, which isn't a bug. assuming it's not a bug, since it is meant to be immutable, then this is one case where the conflation of @system and __metadata might be risky. The case could be ameliorated by the fact that you can't access it in @safe code. What I'm saying is that I suspect actual cases of this will be rare, that they may not outweigh the benefits of having a very simple system, with one keyword to encompass all unsafe activity.

>

...

The solution to our debate — insofar as my claims deserve any merit at all — is to implement both DIP1035 and the __metadata DIP, with the additional requirement that all uses of __metadata be marked @system as well. Semantically, it would change nothing, and it would actually be easier to understand for newcomers.

Then, five years later, to test my theory, add a compiler switch preview=deprecate__metadata, put the __metadata functionality into the @system, and make __metadata a no-op.

If nothing breaks, I was right. If something breaks, and the workarounds aren't seamless, then you were right.

Zach

April 10, 2022

On Sunday, 10 April 2022 at 00:04:43 UTC, Zach Tollen wrote:

>

On Saturday, 9 April 2022 at 17:22:38 UTC, Timon Gehr wrote:

I've read your posts, and the replies from Timon. Since Timon's careful and clear posts did not reach you, I'll summarize my take on your posts below rather than attempt a similarly detailed correction/illumination of your latest (which would, most probably, be a poor use of time for both of us, certainly for me)

Summary response:

I find your posts to be very enthusiastic but also (very) poorly reasoned. I'd suggest working on that with someone you look up to. It's very hard to improve on your own if you don't, evidently, understand that you could and should do much better.

April 10, 2022

On Sunday, 10 April 2022 at 00:04:43 UTC, Zach Tollen wrote:

>

On Saturday, 9 April 2022 at 17:22:38 UTC, Timon Gehr wrote:

>

So your proposal is...

My proposal is to take everything DIP1035 marks or infers as @system and extend __metadata functionality to it.

I should amend my proposal to say I don't really think anything should be inferred to have __metadata functionality. (That's going too far even for my bold mind!) You should definitely have to say that a specific variable is __metadata/enhanced @system. So I'd have to find a different solution as to how DIP1035 attempts to resolve global variables which are initialized to unsafe data, e.g.

// global scope
@system:
int* x = cast(int*) 0xDEADBEEF;

First of all, I don't remember anyone actually complaining about this kind of thing. So it's not really bothering me. But this is how I'd solve it with my proposal, if I had to:

  • Have the compiler make a subtle distinction between a @system variable and a @system initialization. This is solely to avoid having a separate keyword (e.g. @systemVar) for empowered @system variables as opposed to just ones which are initialized unsafely. It's a little messy, but the result would be
@system:
int* x = cast(int*) 0xDEADBEEF; // detected as initialized unsafely
@system int* y = 0xDEADBEEF; // specifically marked @system and therefore empowered with `__mutable` too

void main() @safe {
    *x = 10; // error: variable `x` was detected to have been initialized
             //   unsafely, so its reference cannot be modified in safe code
    *y = 10; // error: `@system` variable `y` cannot be modified in safe code
}

I'm pretty sure that would solve that problem.

--

DIP1035 also suggests that an "aggregate with at least one @system field is an unsafe type." I think this might be going too far. If you have to be in @system code to touch even the safe members of the type, I suspect you will lose the opportunity to catch a lot of unsafe bugs. I think it should be "any action which read or writes a @system variable is a @system action." But I would seek more clarification from the DIP authors on what they really meant by making the whole type unsafe.

--

None of this matters, of course. I don't think I can convince anybody that my proposal is worth trying, especially when you can just as easily adopt both DIP1035 and __metadata separately and see how things go. I just wanted to amend my previous claim to say that at minimum, you should have to mark empowered @system/__metadata specifically and that they shouldn't be inferred.

Zach

April 10, 2022

On Sunday, 10 April 2022 at 03:37:03 UTC, Bruce Carneal wrote:

>

On Sunday, 10 April 2022 at 00:04:43 UTC, Zach Tollen wrote:

>

On Saturday, 9 April 2022 at 17:22:38 UTC, Timon Gehr wrote:

I've read your posts, and the replies from Timon. Since Timon's careful and clear posts did not reach you, I'll summarize my take on your posts below rather than attempt a similarly detailed correction/illumination of your latest (which would, most probably, be a poor use of time for both of us, certainly for me)

Summary response:

I find your posts to be very enthusiastic but also (very) poorly reasoned. I'd suggest working on that with someone you look up to. It's very hard to improve on your own if you don't, evidently, understand that you could and should do much better.

Well thanks for your interest, at any rate. And your honesty.

Out of curiosity, do you have an opinion of DIP1035? At least intuitively, I feel like the essence of it should run: "Some variables are dangerous, but the compiler can't figure that out. Therefore we allow those variables to be marked @system so they are treated as unsafe."

But there are other parts of DIP1035 (initializing globals to unsafe values, and also, an entire aggregate is unsafe if a single member is unsafe) which seem to be about inferring certain things to be unsafe as opposed to helping the compiler to determine what is unsafe. My interest in merging the two DIPS came almost entirely from seeing the value in the "help the compiler learn what is unsafe" side, and I thought the "inferring things unsafe" side could be handled separately. (The helping-the-compiler side is eloquently illustrated in the one of the DIP's linked videos by Steven Schveighoffer. That's where I got my understanding of what I thought the essence of DIP1035 should be about. Of course, that's no reason to arbitrarily merge the two features, @system and __metadata. But it might make it easier to explain why I thought it was possible.)