On Monday, 8 April 2024 at 19:59:55 UTC, Richard (Rikki) Andrew Cattermole wrote:

void thread1() {
	shared(Type) var = new shared Type;
	sendToThread2(var);

	for(;;) {
		// I know about var!
	}
}

void sentToThread2(shared(Type) var) {
	thread2(var);
}

void thread2(shared(Type) var) {
	for(;;) {
		// I know about var!
	}
}

I see. I thought you were talking about violating the assumptions of the data structure, but you're worried about sharing it between threads. Well, if you don't want to share something between the threads, that's exactly when you don't make it shared, and mission accomplished!

However, I think you're advocating for something like @live. That is, a type that's destroyed in RAII fashion but that can be passed from one function to another, without risking dangling references, and you also want it to work between threads.

This is where DIP1040, or if it fails then DIP1014 steps in.

You can have a RAII data structure by disabling the copy constructor, but ownership can to still be passed around with .move. With either of these two DIPs you can insert code to be executed each time the struct is moved, if you have to.

As for doing this between two threads, you want a data struct that both handles shared data and is RAII. The struct handle itself can be shared or thread local. Because it's RAII, only one thread can own it. Because it handles shared data, it can be moved between threads. It still needs locking when borrowed, because the ownership could be passed to another thread before the borrowing ends.

Yes, ref counting is required AFAIK if borrowing is allowed as it probably is. Going to write a separate reply to this.

You're partially right: It's possible to design a @safe shared data structure entirely without shared. What shared enables is that bigger part of implementation of the structure can be @safe, if still very low-level and bug-prone.

On Monday, 8 April 2024 at 19:59:55 UTC, Richard (Rikki) Andrew Cattermole wrote:

I think you are losing sight of the big picture.

The big picture is we already have a fine safety mechanism for majority of the cases, the garbage collector! It also just works with graphs.

Ownership and/or reference counting are needed for some cases. I suspect they're more important than "system programming is niche" arguments suggest because you don't always use them for RAM memory, but also things like file or thread handles. However, managing access to other resources than RAM memory is a lot easier since you don't have to deal with the possibility of escaped pointers to the resource you're protecting, and already well supported by the present language.

What we're left with are the cases where you need to manage RAM but the GC won't do. This is a niche area. D needs to support it since systems programming is in it's domain, but it isn't a case worth optimising much.

Since forever D has supported doing these by simply marking the manual memory handling code @system. This is already a reasonable standard considering how rarely you need to do it. D where it's currently going to does better with DIP1000, @system variables and move constructors, enabling, in principle, safely doing most of what Rust can do. It's true that ref counting and borrowing are more ergonomic to use in Rust than in D since the language is built around those, and probably it enables doing something safely that D doesn't.

But so what? Rust does better in a case which is far from your worst problems in D either even if you always dabble at the low level. If D implements a type system addition to match Rust, it complicates the whole language but improves it only in a very narrow case.

Also, Rust is a first-and-foremost systems programming language. D is a hybrid systems and application programming language. What is worthwhile for one language isn't always worthwhile for the other.

As for "ref counting doesn't work because of graphs" - I'm not sure, but probably this can be worked around somehow without forcing client to use @system the GC. Rust manages it somehow after all. But even if that isn't the case, it's a small corner case that doesn't affect the big picture.

On Tuesday, 9 April 2024 at 08:16:22 UTC, Dukc wrote:

One way to frame the "big picture" of language design is as an optimization problem across multiple concerns {safety, performance, testability, scalability, deployability, ease-of-use}.

Traditionally, language designers have weighted some of these concerns much much more heavily than others. The D community at large has not, leading to, IMO, a very enjoyable and improving language.

We're not in an either/or situation here. We can certainly improve GC and we can explore the space between our, improving, GC and malloc()...

I'm not an expert in this area but I do think that there are big wins for D to be had here.

On a side note, I find the Mojo approach to memory-mgmt appealing. They've certainly embraced the {performance X ease-of-use X safety} trifecta.

On Tuesday, 9 April 2024 at 14:20:22 UTC, Bruce Carneal wrote:

Yes we can. My point is, since each language feature has a complexity cost, and since deteministic memory management is so rarely needed, any improvement to it need to have a much better benefit-to-complexity ratio than something that improves the whole language.

Any large overhaul of the type system to improve memory management by RAII or ref counting will have hard time justifying itself, unless it also provides a lot of other benefits.

On Wednesday, 10 April 2024 at 12:31:03 UTC, Dukc wrote:

By whose count? I've found there are plenty of places where they make a difference.

On Wednesday, 10 April 2024 at 19:59:46 UTC, Sebastiaan Koppe wrote:

One clarification: I meant heap memory management. Stack memory objects obviously get freed deterministically but I didn't mean those.

Another clarification: I meant rarely needed compared to the GC. Not rarely as in "unlikely that you personally should ever do it".

Both my own count and of what others have written in this forum. On my own code, I almost always use the GC. While I can imagine doing manual freeing on a few critical places if I want to optimise, it's far far less often needed than the GC (and would be even less needed if we had a generational GC that performs better).

Well, on my old Spasm code where I don't have the GC, I do resort to deterministic heap management fairly much, but a better answer in the long run would be having some sort of portable GC instead of managing the heap with other means. Even if ref counted memory was as simple to use as the GC (which it cannot be), you'd want the GC just to be able to use existing D code out of the box. This also applies to a bare-metal project: a scheduler or low level device driver code might need or want to use other means of memory management for latency or performance reasons, but in all likelihood it's going to be only a minority of the code. Someone just recently linked a blog post demonstrating how a GC is extremely relevant even in systems programming.

Even if it's a hard real-time system, it can be done by keeping the critical real-time thread outside the world the GC stops, but delegate everything possible to normal GC-controlled threads.