On Sunday, 8 November 2020 at 14:04:31 UTC, IGotD- wrote:
> On Sunday, 8 November 2020 at 13:39:17 UTC, Ola Fosheim Grøstad wrote:
>>
>> The problem is that raw C pointers can be owning pointers. So it would be a breaking change.
>
> We need to break this change because if we don't D will not progress. The alternative is some pointer container as if we would litter C++ std::shared_ptr all over the place, not too happy about that though. This will of course be a library change but I think we should have a language classifier for it. This doesn't really need to be a breaking change but an additional "fat pointer".

Make all raw pointers borrowing, then add owning pointer types. Make all  class pointers work like shared_ptr.

Simple solution: unique_ptr for structs and embedded shared_ptr for classes and dynamic arrays. Owning slices will need a new pointer to the refcount head.

On Sunday, 8 November 2020 at 15:09:55 UTC, Ola Fosheim Grøstax wrote:
>
> Make all raw pointers borrowing, then add owning pointer types. Make all  class pointers work like shared_ptr.
>
> Simple solution: unique_ptr for structs and embedded shared_ptr for classes and dynamic arrays. Owning slices will need a new pointer to the refcount head.

No, then you put restrictions on raw pointers which is something we don't need. Forced single ownership is something that does not belong in D. Let's just keep the raw pointers as they are and add other pointer types instead.

This is something not many languages has thought of which is generic pointer types, the ability to mix pointer types in a program that are not pre-determined. C++ has its unique_ptr and shared_ptr but I was think more in line with A_ptr or B_ptr and what those pointers are supposed to use is configurable. This is of course possible as library additions but I've never seen languages to offer this out of the box and it would be nice to have a language framework for this. Nim offers to replace its "ref" types with any type of GC. However, why not go a step further and allow several types of "ref" types.

On Sunday, 8 November 2020 at 13:39:17 UTC, Ola Fosheim Grøstad wrote:
> On Sunday, 8 November 2020 at 13:28:02 UTC, random wrote:
>> If I understand it right you also can't have a GC with heap compaction in D, because this would invalidate pointers. Most high performance GCs use heap compaction...
>
> It is possible, but expensive..

How would you do this? Track all pointers (type system knows them) and if you move the object update all pointers to that object. What is with something evil like C++ iterator::end(), which is often a pointer one past the last valid element in the buffer? I think you would have to track for all pointers to which GC allocated object they belong...

>> The language relies on GC (you can't use all features without it), but at the same time it is designed in a way you can't have a competitive GC (compared to JVM/CLR).
>
> The problem is that raw C pointers can be owning pointers. So it would be a breaking change.

This is not completely clear to me. Do you mean (like you said previously) that an other pointer type is needed so GC can differentiate between them? Is there something else I'm missing about ownership?

On Sunday, 8 November 2020 at 17:00:07 UTC, IGotD- wrote:
> No, then you put restrictions on raw pointers which is something we don't need. Forced single ownership is something that does not belong in D. Let's just keep the raw pointers as they are and add other pointer types instead.

That is what I would do too, but then you can use a library. And it breaks all existing GC based libraries with no help from the type checker when porting.

You cannot convert GC code this way, to do that you have to make current pointer types owning and introduce a new nonowning type.

> This is something not many languages has thought of which is generic pointer types, the ability to mix pointer types in a program that are not pre-determined. C++ has its unique_ptr and shared_ptr but I was think more in line with A_ptr or B_ptr and what those pointers are supposed to use is configurable. This is of course possible as library additions but I've never seen languages to offer this out of the box and it would be nice to have a language framework for this.

I have suggested it before in the forums. In C++ you can template the smart pointer type, I think. Although manual borrowing is the common way...

On Sunday, 8 November 2020 at 15:04:33 UTC, Ola Fosheim Grøstad wrote:
> Or just ARC, which I would prefer if done with global static analysis.

+1 I learned Swift a while ago, mostly because I like the idea of ARC. (I came to the conclusion that I don't like Swift btw^^).

People often dismiss it because it's slower than tracing GC, but there is more to it.
Obviously lower memory usage and deterministic destruction.

The real killer feature of ARC is imho that you can use it to implement value semantics with copy on write.
Swift does it for arrays. Semantically you pass by value, but it will only duplicate the array if you write to it and the refcount is greater one.
You can also use this to implement classes with value semantics (with a little bit boilerplate).

On Sunday, 8 November 2020 at 17:38:01 UTC, random wrote:
> On Sunday, 8 November 2020 at 13:39:17 UTC, Ola Fosheim Grøstad wrote:
>> On Sunday, 8 November 2020 at 13:28:02 UTC, random wrote:
>>> If I understand it right you also can't have a GC with heap compaction in D, because this would invalidate pointers. Most high performance GCs use heap compaction...
>>
>> It is possible, but expensive..
>
> How would you do this? Track all pointers (type system knows them) and if you move the object update all pointers to that object. What is with something evil like C++ iterator::end(), which is often a pointer one past the last valid element in the buffer? I think you would have to track for all pointers to which GC allocated object they belong...

I wouldnt... But yes you could have many generation buckets and move 2 old near empty buckets to a new one if the static analysis tells you it is safe... I wouldnt compact all buckets every collection cycle.

I would probably change the semantics so that you need to hold array ownership through a slice, then you can compact arrays. Keep only elements reachable from active slices.
 it would be a breaking change.

> This is not completely clear to me. Do you mean (like you said previously) that an other pointer type is needed so GC can differentiate between them?

Yes, basically.

On Sunday, 8 November 2020 at 17:38:01 UTC, random wrote:
> On Sunday, 8 November 2020 at 13:39:17 UTC, Ola Fosheim Grøstad wrote:
>> On Sunday, 8 November 2020 at 13:28:02 UTC, random wrote:
>>> [...]
>>
>> It is possible, but expensive..
>
> How would you do this? Track all pointers (type system knows them) and if you move the object update all pointers to that object. What is with something evil like C++ iterator::end(), which is often a pointer one past the last valid element in the buffer? I think you would have to track for all pointers to which GC allocated object they belong...
>
>>> [...]
>>
>> The problem is that raw C pointers can be owning pointers. So it would be a breaking change.
>
> This is not completely clear to me. Do you mean (like you said previously) that an other pointer type is needed so GC can differentiate between them? Is there something else I'm missing about ownership?

Yes, in systems languages like Modula-3, Active Oberon, or if you prefer something more modern, .NET Native, Swift or Go, the pointers are safe by default and you have an explicit unsafe pointer for "do any kind of tricks pointer".

On Sunday, 8 November 2020 at 19:12:06 UTC, Paulo Pinto wrote:
> pointers are safe by default and you have an explicit unsafe pointer for "do any kind of tricks pointer".

By having borrowing pointers for GC you could also potentially reduce the graph that has to be scanned, when following a borrowing pointer can never reach an owning pointer. Requires global analysis.

On Sunday, 8 November 2020 at 17:00:07 UTC, IGotD- wrote:
> This is something not many languages has thought of which is generic pointer types, the ability to mix pointer types in a program that are not pre-determined. C++ has its unique_ptr and shared_ptr but I was think more in line with A_ptr or B_ptr and what those pointers are supposed to use is configurable. This is of course possible as library additions but I've never seen languages to offer this out of the box and it would be nice to have a language framework for this. Nim offers to replace its "ref" types with any type of GC. However, why not go a step further and allow several types of "ref" types.

Nim does go this step further and allows for custom pointer types. Not that good of an idea, interop between libraries doesn't improve when they cannot even agree on the used pointer type, but it is supported in Nim.

On Wednesday, 11 November 2020 at 13:30:32 UTC, Araq wrote:
>
> Nim does go this step further and allows for custom pointer types. Not that good of an idea, interop between libraries doesn't improve when they cannot even agree on the used pointer type, but it is supported in Nim.

Isn't that the case as well when you change the GC algorithm? Sure shared libraries that are compiled with one type GC and another with a different one will cause problems. Is there a possibility to detect this, with name mangling for example?