November 09, 2021
On 09/11/2021 2:14 PM, tsbockman wrote:
> What you seem to be asking for instead is a way to trick the type system into agreeing that mutating a reference count doesn't actually mutate anything, which is nonsense. If that's really necessary for some reason, it needs to be special cased into the language spec, like how `pure` explicitly permits memory allocation.

Imagine saying to someone:

Yes you have made this struct immutable.

Yes you have set this bit of memory containing that immutable struct to read only.

Yes you ended up with a crashed program because that immutable struct went ahead and tried to write to that read only memory.

And yes, I understand that you couldn't have known that a field that you didn't write the implementation of used an escape hatch to write to const data.

It doesn't make sense.
November 08, 2021
On Tue, Nov 09, 2021 at 02:22:26PM +1300, rikki cattermole via Digitalmars-d wrote:
> 
> On 09/11/2021 2:14 PM, tsbockman wrote:
> > What you seem to be asking for instead is a way to trick the type system into agreeing that mutating a reference count doesn't actually mutate anything, which is nonsense. If that's really necessary for some reason, it needs to be special cased into the language spec, like how `pure` explicitly permits memory allocation.
> 
> Imagine saying to someone:
> 
> Yes you have made this struct immutable.
> 
> Yes you have set this bit of memory containing that immutable struct to read only.
> 
> Yes you ended up with a crashed program because that immutable struct went ahead and tried to write to that read only memory.
> 
> And yes, I understand that you couldn't have known that a field that you didn't write the implementation of used an escape hatch to write to const data.
> 
> It doesn't make sense.

Yes, subverting the type system with __mutable doesn't make sense. It just adds all kinds of loopholes and other unwanted interactions to the language.  A better approach is to have a nice way of converting const(Wrapper!T) -> Wrapper!(const T), so that we can implement a refcount in Wrapper without violating the type system.


T

-- 
Freedom of speech: the whole world has no right *not* to hear my spouting off!
November 08, 2021

On 11/8/21 8:04 PM, deadalnix wrote:

>

On Monday, 8 November 2021 at 22:40:03 UTC, Andrei Alexandrescu wrote:

>

On 2021-11-08 17:26, rikki cattermole wrote:

>

a reference counted struct should never be const

So then we're toast. If that's the case we're looking at the most important challenge to the D language right now.

Not necessarily, but this is in fact the same problem as the head mutable problem.

If const RCSlice!(const T) can convert to RCSlice!(const T), which it should to have the same semantic as slice.

The reference counting problem goes away if you can mutate the head.

The reference count cannot be in the head, it has to be in the block. So this conversion is actually unsound.

-Steve

November 09, 2021

On Tuesday, 9 November 2021 at 01:14:24 UTC, tsbockman wrote:

>

Borrowing is required for a general-purpose RC type, so that the payload can actually be used without a reference to the payload escaping outside the lifetime of the counting reference. But, the effective lifetime of the counting reference is not visible to the scope borrow checker, because at any point the reference's destructor may be manually called, potentially freeing the payload while there is still an extant borrowed reference.

With current language semantics, the destructor (and any other similar operations, such as reassignment) of the reference type must be @system to prevent misuse of the destructor in @safe code.
https://issues.dlang.org/show_bug.cgi?id=21981

The solution to this problem is to introduce some way of telling the compiler, "this destructor is @safe to call automatically at the end of the object's scope, but @system to call early or manually."

I believe it is also possible to make this @safe by doing borrow checking at runtime, although it would introduce some overhead, and make the API less ergonomic.

Maybe the best compromise we can reach in the current language is to offer a @safe borrow-checked interface alongside an unchecked @system interface.

November 08, 2021
On Mon, Nov 08, 2021 at 10:38:59PM -0500, Steven Schveighoffer via Digitalmars-d wrote:
> On 11/8/21 8:04 PM, deadalnix wrote:
> > On Monday, 8 November 2021 at 22:40:03 UTC, Andrei Alexandrescu wrote:
> > > On 2021-11-08 17:26, rikki cattermole wrote:
> > > > a reference counted struct should never be const
> > > 
> > > So then we're toast. If that's the case we're looking at the most important challenge to the D language right now.
> > 
> > Not necessarily, but this is in fact the same problem as the head mutable problem.
> > 
> > If `const RCSlice!(const T)` can convert to `RCSlice!(const T)`,
> > which it should to have the same semantic as slice.
> > 
> > The reference counting problem goes away if you can mutate the head.
> 
> The reference count cannot be in the head, it has to be in the block. So this conversion is actually unsound.
[...]

It could work if the block has a mutable counter at the top and the rest is immutable/const.  But the current type system cannot express such a thing; if the head is const, then it's const all the way down.


T

-- 
Two wrongs don't make a right; but three rights do make a left...
November 09, 2021

On Tuesday, 9 November 2021 at 03:43:01 UTC, Paul Backus wrote:

>

On Tuesday, 9 November 2021 at 01:14:24 UTC, tsbockman wrote:

>

With current language semantics, the destructor (and any other similar operations, such as reassignment) of the reference type must be @system to prevent misuse of the destructor in @safe code.
https://issues.dlang.org/show_bug.cgi?id=21981

The solution to this problem is to introduce some way of telling the compiler, "this destructor is @safe to call automatically at the end of the object's scope, but @system to call early or manually."

I believe it is also possible to make this @safe by doing borrow checking at runtime, although it would introduce some overhead, and make the API less ergonomic.

How? The general purpose runtime borrow checking schemes I've considered all have the same fundamental problems as reference counting in general, when examined closely.

November 09, 2021

On Tuesday, 9 November 2021 at 03:43:01 UTC, Paul Backus wrote:

>

On Tuesday, 9 November 2021 at 01:14:24 UTC, tsbockman wrote:

>

With current language semantics, the destructor (and any other similar operations, such as reassignment) of the reference type must be @system to prevent misuse of the destructor in @safe code.
https://issues.dlang.org/show_bug.cgi?id=21981

The solution to this problem is to introduce some way of telling the compiler, "this destructor is @safe to call automatically at the end of the object's scope, but @system to call early or manually."

I believe it is also possible to make this @safe by doing borrow checking at runtime, although it would introduce some overhead, and make the API less ergonomic.

How? All of the runtime borrow checking schemes that I have considered turn out to have the same fundamental problems as reference counting, when examined closely.

November 09, 2021
On Monday, 8 November 2021 at 22:07:46 UTC, Adam Ruppe wrote:
> On Monday, 8 November 2021 at 21:42:12 UTC, Andrei Alexandrescu wrote:
>> - work with qualifiers like T[] does, both RCSlice!(qual T) and qual(RCSlice!T)
>
> I think this is an incorrect requirement. A const(T) is now allowed to increase a reference count by definition.

This is technically true, but he wrote "built in", so then you don't have to heed the same constraints. You only have to require that acquire()/release() match up, so how that is accounted for is an implementation detail since you don't provide any access to the count.

Same goes for pure functions. You can imagine that all pure function calls have a hidden parameter with "ref count storage", which it is allowed to modify.

So you might be able do this without breaking the type system, if it is builtin, but you have to prove it.

(Basic implementation is to use fat pointers and keep the counter in mutable memory, like shared_ptr. Anything beyond that is an optimization.)

November 09, 2021
On Tuesday, 9 November 2021 at 09:02:32 UTC, Ola Fosheim Grøstad wrote:
> On Monday, 8 November 2021 at 22:07:46 UTC, Adam Ruppe wrote:
>> On Monday, 8 November 2021 at 21:42:12 UTC, Andrei Alexandrescu wrote:
>>> - work with qualifiers like T[] does, both RCSlice!(qual T) and qual(RCSlice!T)
>>
>> I think this is an incorrect requirement. A const(T) is now allowed to increase a reference count by definition.
>
> This is technically true, but he wrote "built in", so then you don't have to heed the same constraints. You only have to require that acquire()/release() match up, so how that is accounted for is an implementation detail since you don't provide any access to the count.
>

Yes. In addition, it is worth mentioning that many functional programming languages are using reference counting internally. It is so that they can do copy on write instead of copying everything all the time for no reason.
November 09, 2021

On Monday, 8 November 2021 at 21:42:12 UTC, Andrei Alexandrescu wrote:

>

This was prompted by this exchange with Paul Backus that went like this:

Me: "We never got reference counting to work in safe pure nogc code. And we don't know how. If we did, we could write a slice-like type that does everything a slice does PLUS manages its own memory. This is the real problem with containers."

Paul: "I'm pretty sure at least some of us (including Atila) know how."

I'll hijack this thread with a slide from my DConf Online 2020 talk, because with the current language it's impossible for a library writer to write a useful @safe vector struct:

scope v = vector(1, 2, 3, 4);
scope s = v[];
v ~= 5;  // could reallocate here
s[3] = 42;  // oops

I initialy "solved" this with @trusted, but fortunately somebody on the forums pointed out I was being an idiot.

It might just be my biases but I think this is a more useful nut to crack. And the only way to crack it is to some sort of ownership system, since there's no way currently in D to have a compile-time guarantee that there's only one alias left.

"We can have @safe vectors, unless you actually want to append" isn't a great slogan. And vectors are clearly simpler than ref counted arrays.