On 11/14/2021 10:25 AM, Timon Gehr wrote:
> How do you manually deallocate an immutable payload?

The same way it is done now. Call free().

Calling free() on an object ends its lifetime. As I mentioned to Steven, lifetime and mutability are independent attributes.

On Sun, Nov 14, 2021 at 10:08:29AM -0800, Walter Bright via Digitalmars-d wrote:
> On 11/14/2021 10:02 AM, Timon Gehr wrote:
> > On 14.11.21 08:16, Walter Bright wrote:
> > > On 11/12/2021 4:31 AM, Steven Schveighoffer wrote:
> > > > One of the prerequisites to doing reference counting is to have a mutable piece of data inside an immutable piece of data.
> > > 
> > > Or maybe just give up on having immutable ref counted objects. Ref counted objects are mutable.
> > 
> > Yes, this is definitely a reasonable way to resolve this debate.
> 
> Sometimes the obvious way is the best way!
> 
> Note that one still can have an immutable *payload* for a ref counted object.

This seems to be a reasonable way to resolve this issue.

However, it does have far-reaching implications. For example, it implies that refcounted objects cannot be used inside any type that may get passed to a const-receiving interface.

As a hypothetical example, say we have this code:

	class SomePayload { ... }

	struct DataContainer {
		string metadata;
		SomePayload payload;
		SomeOtherPayload payload2;
	}

	void updateData(ref DataContainer data) {
		// presumably update data
	}

	void analyzeData(in DataContainer data) {
		// read-only access of data
	}

	void main() {
		DataContainer data = getData(...);
		updateData(data);
		analyzeData(data);
	}

Using `in` for analyzeData is reasonable, since the function does not wish to modify the payload.

Now, suppose after some development we decide that we want to make SomePayload a ref-counted resource instead of a GC-allocated class instance. So we attempt to modify DataContainer thus:

	struct DataContainer {
		string metadata;
		RefCounted!SomePayload payload;
		SomeOtherPayload payload2;
	}

Now we have a problem: we can no longer call analyzeData with the data, because it requires mutation of the ref count.

I deliberately nested the refcounted object inside another type, to illustrate my point: if refcounted objects require mutation (which makes sense since the ref count needs to be updated), that means *no* function that might possibly receive a refcounted object (as a nested part of a larger data structure) will be able to take it as a const argument.

IOW, const becomes unusable with any object that could potentially contain a ref-counted sub-object.

This is probably not a problem in non-template code, but when you're dealing with generic code, this will exclude ref-counted objects from any generic function that uses const in any way. Generic code does not know (nor care) whether some subobject of an argument might possibly be refcounted, but it seems reasonable (and likely) that a generic function that does not plan to modify an argument would use `in` or `const` to qualify its parameters.  Such a function would not be usable with any argument that might possibly have ref-counted objects nested somewhere within it.

So, either (1) we cannot use const/in in generic functions, or (2)
ref-counted objects cannot be used in generic code.

Given D's emphasis on generic code, (1) seems like the only viable option.  Which makes const in D even narrower in scope than ever.

The same applies for wrapper objects: if any part of the object may contain a ref-counted object, the entire object becomes unusable with const.

So, const is turtles all the way down, and ref-counting must be mutable all the way *up*.


T

-- 
Change is inevitable, except from a vending machine.

On 14.11.21 20:51, Walter Bright wrote:
> On 11/14/2021 10:25 AM, Timon Gehr wrote:
>> How do you manually deallocate an immutable payload?
> 
> The same way it is done now. Call free().
> 
> Calling free() on an object ends its lifetime. As I mentioned to Steven, lifetime and mutability are independent attributes.
> 

I agree that they are independent attributes, but how does the compiler know that something is an allocation/deallocation function? (My suggestion was to annotate such functions __mutable.)

On 14.11.21 21:04, H. S. Teoh wrote:
> 
> So, either (1) we cannot use const/in in generic functions, or (2)
> ref-counted objects cannot be used in generic code.
> 
> Given D's emphasis on generic code, (1) seems like the only viable
> option.  Which makes const in D even narrower in scope than ever.

I don't agree that it changes anything for generic code. `const` already prevents useful patterns such as lazy initialization.

The issue is that despite all that, people still think D const is similar to C++ const. It's really not. Notions such as "const correctness" are misguided in D.

On Sun, Nov 14, 2021 at 09:22:39PM +0100, Timon Gehr via Digitalmars-d wrote:
> On 14.11.21 21:04, H. S. Teoh wrote:
> > 
> > So, either (1) we cannot use const/in in generic functions, or (2)
> > ref-counted objects cannot be used in generic code.
> > 
> > Given D's emphasis on generic code, (1) seems like the only viable option.  Which makes const in D even narrower in scope than ever.
> 
> I don't agree that it changes anything for generic code. `const` already prevents useful patterns such as lazy initialization.

Yes, which already confines const to a very narrow scope indeed. And if we now make ref-counting require mutable, then const is basically relegated to the niche of niches, usable only for very narrow and specific use cases like const(char)[] or immutable(char)[] aka string.

As I've mentioned before, while const/immutable do provide very strong guarantees, they are also very narrow in scope, such that IME they are rarely useful outside of bottom-level types (i.e., leaf nodes in the graph of type dependencies). Outside of that, they're so restrictive that they aren't really applicable most of the time.  They are useful only 1 or 2 levels above PODs, anything else and you start running into problems.

It seems wasteful to allocate a large part of the type system to something so limited in applicability in real-life code.


> The issue is that despite all that, people still think D const is similar to C++ const. It's really not. Notions such as "const correctness" are misguided in D.

C++ const is a joke, I'm not even thinking about it. :-D

The thing is that `in` is touted (in tutorials, documentation, and sometimes in advice to D newbies) as a good practice for function parameters when the function is not modifying data. In practice, however, this is only applicable to PODs and aggregates 1 or 2 levels above that. Beyond that `in` basically adds no value, since as you mentioned, it excludes a lot of useful idioms like lazy initialization, caching, and now ref-counting.

An entire keyword, spent on a feature that can only be used in code of limited complexity.


T

-- 
Государство делает вид, что платит нам зарплату, а мы делаем вид, что работаем.

On 14.11.21 21:46, H. S. Teoh wrote:
> On Sun, Nov 14, 2021 at 09:22:39PM +0100, Timon Gehr via Digitalmars-d wrote:
>> On 14.11.21 21:04, H. S. Teoh wrote:
>>>
>>> So, either (1) we cannot use const/in in generic functions, or (2)
>>> ref-counted objects cannot be used in generic code.
>>>
>>> Given D's emphasis on generic code, (1) seems like the only viable
>>> option.  Which makes const in D even narrower in scope than ever.
>>
>> I don't agree that it changes anything for generic code. `const`
>> already prevents useful patterns such as lazy initialization.
> 
> Yes, which already confines const to a very narrow scope indeed. And if
> we now make ref-counting require mutable, then const is basically
> relegated to the niche of niches, usable only for very narrow and
> specific use cases like const(char)[] or immutable(char)[] aka string.
> ...

Well, my point was, _this is the status quo_. we are not now "making" it anything. Of course, it's more likely that those who are unaware will notice it if reference counting is popularized as a memory management strategy. I guess this was your point.

Interestingly, std.typecons.RefCounted "works" with const objects, but that's only because postblit is unsound.

On 14.11.21 21:46, H. S. Teoh wrote:
> The thing is that `in` is touted (in tutorials, documentation, and
> sometimes in advice to D newbies) as a good practice for function
> parameters when the function is not modifying data. In practice,
> however, this is only applicable to PODs and aggregates 1 or 2 levels
> above that. Beyond that `in` basically adds no value, since as you
> mentioned, it excludes a lot of useful idioms like lazy initialization,
> caching, and now ref-counting.
> 
> An entire keyword, spent on a feature that can only be used in code of
> limited complexity.

True, that's not optimal.

On Sunday, 14 November 2021 at 18:24:00 UTC, Timon Gehr wrote:
>> 1. ref counted objects are mutable
> Works, but you may have to extend it to "anything that's not allocated with the GC is mutable".

Static data can be immutable too.


>> 2. ref counted objects are outside of the type system
> For this to work, there still needs to be a careful definition what @trusted functions are allowed to do with data of certain qualification. I think adding some type system support that is strictly @system, like __mutable variables and functions is better than having no language support at all, because otherwise, in order to support reference counting, you may end up penalizing code that does not actually use it.

I assumed that 'outside of the type system' meant 'language-level support for reference counting'; a a type with opaque representation, like a hash table.


>> PS. I suspect that ref counted shared mutable objects are an indicator of insanity.
> Possibly, though it's hard to know everyone's use cases in advance.

GC really makes more sense for multithreaded programs.  Travis downs mentions at https://travisdowns.github.io/blog/2020/07/06/concurrency-costs.html.  IMO if you are sharing mutable data between threads, and you can't stand gc for some reason, then it's perfectly legitimate to make you manage the lifetime yourself.