On Thursday, 21 November 2019 at 00:59:01 UTC, Timon Gehr wrote:
> allocation strategy. Therefore, allowing built-in pointers to own their memory is vastly less useful than allowing library-defined smart pointers to do so.

Yes, but if you could define all builtin pointers as defaulting to borrowing and introduce a syntax for marking a builtin pointer as owning. Then owning library structs could use that syntax to tell the compiler that that a pointer field is owning the resource.

If you also introduce a syntax for marking other types as borrowing/owning then you could use the same analysis for file handles, texture handles etc.

On Thursday, 21 November 2019 at 08:18:56 UTC, Johannes Pfau wrote:
> TypeScript does that. In addition, it disallows dereferencing if the pointer is in the undefined state. So it forces you to check a pointer before you dereference it. I always found that to be quite useful.

Yes, but TypeScript is using flow-typing.

I guess D could switch to flow-typing somehow, but I am not sure if it is a good idea to do so without doing it in a more wholesome manner.

On 11/20/2019 7:47 PM, Doc Andrew wrote:
> And honestly, knowing whether a pointer is in an undefined state is still a very
> useful piece of information that the flow analysis would provide.
> Dereferencing a pointer of unknown provenance should be an error in @live code,
> no different than null.

That's what I considered; there would be so many "cry wolf" false positives it would be useless.

Besides, the only way a null pointer dereference could result in memory corruption is if it comes with an offset that is larger than the protected memory zone at address 0. I.e. it's very unlikely. It's so unlikely I've literally never had one (but I've had lots and lots of null pointer seg faults).

On 11/21/2019 12:18 AM, Johannes Pfau wrote:
> TypeScript does that. In addition, it disallows dereferencing if the
> pointer is in the undefined state. So it forces you to check a pointer
> before you dereference it. I always found that to be quite useful.

I would find that to be annoying, as the CPU hardware checks it before dereferencing it, too, for free.

On 11/20/2019 4:59 PM, Timon Gehr wrote:
> On 20.11.19 23:45, Walter Bright wrote:
>> On 11/20/2019 4:16 AM, Timon Gehr wrote:
>>> - What do you want to achieve with borrowing/ownership in D?
>>
>> I want to prevent the following common issues with pointer code:
>>
>> 1. use after free
>> 2. neglecting to free
>> 3. double free
>> ...
> 
> GC prevents those,

That's right. The GC is memory safe.

> and those problems cannot appear in @safe code.

@safe code has to call free() some time when manually managing memory.

> @live doesn't prevent them at the interface between @live and non-@live code.

@live relies on any function it calls obeying @live conventions for its interface. This allows incremental adoption of @live code.


> What about user-defined types? What about allowing internal pointers into manually-managed memory to be exposed in @safe code?

Exposing an internal pointer in @live code is considered "borrowing" from the root of its container.


>>> - How will I write a compiler-checked memory safe program that uses varied allocation strategies, including plain malloc,
>>
>> I'm not sure what clarification you want about plain malloc/free, although there are limitations outlined in ob.md.
>> ...
> 
> I.e., it is not planned that we will be able to write such programs?

I believe I covered that in ob.md. What am I missing?


> The worry is that @live _removes_ value from tracing GC. If every pointer is owns its data, how do I express a pointer to GC-owned memory? Do I need to write a "smart" pointer data type that's just a shallow wrapper for a GC pointer? Also, if I do that, how do I make sure different GC-backed pointers don't lend out the same owning pointer at the same time?

@live does not distinguish a GC-allocated raw pointer from a malloc-allocated raw pointer. This means you'll be able to write generic @live code that can handle both equally. Of course, if all you're using is the GC, you won't need to bother with @live at all.


>> There's been a lot of progress with this with the addition of DIP25, DIP1000, and DIP1012. This further improves it by making the protections transitive.
> As far as I can tell, @live doesn't bring us closer to @safe RC, because it applies to built-in pointers instead of library-defined smart pointers. I think this is completely backwards. Every owning pointer also needs to know the allocation strategy. Therefore, allowing built-in pointers to own their memory is vastly less useful than allowing library-defined smart pointers to do so.

Nothing about @live stops programmers from using library-defined smart pointers. The smart pointer would be the owning pointer, and if it exposed an internal pointer that internal pointer would be treated as "borrowing" from the owner and further access to the smart pointer would be denied until the borrower's last use.


>> and conflating different allocators, which I don't have a good idea on.
> Do the checks for library-defined smart pointers instead of built-in pointers. Built-in pointers shouldn't care about lifetime nor allocator.

People use raw pointers, and that isn't going away (because performance). Telling people "just use smart pointers" is like telling C++ people to do that. It doesn't work reliably.

The checks on smart pointers can be done with RAII and reference counting, and the dips already implemented.


> The point of adding restrictions is to gain expressiveness. It's why type systems are a good idea. In this case, the point of borrowing restrictions should be to enable @safe code to manipulate interior pointers into manually-managed data structures.

They can do that now as long as the container only exposes interior pointers as 'ref'.

On Thursday, 21 November 2019 at 11:10:28 UTC, Walter Bright wrote:

> I would find that to be annoying, as the CPU hardware checks it before dereferencing it, too, for free.

Why would you wait until runtime when the compiler can do it at compile time?

--
/Jacob Carlborg

On Wednesday, 20 November 2019 at 22:15:18 UTC, Walter Bright wrote:

> In order to make non-null checking actually work, the language semantics would likely need to change to make:
>
>    T*    a non-null pointer
>    T?*   an optionally null pointer
>
> or something like that. Two different pointer types would need to exist.

That would be really nice. But instead it would be just one pointer type and arbitrary  optional types:

T*           a regular pointer (can't be null)
T*?         an optional pointer (can point to something or can be empty)
int           a regular int
int?         an optional int (can contain an int or can be empty)
Object    a regular reference type (can't be null)
Object?  an optional reference (can point to something or can be empty)

Basically there wouldn't be "null" anymore.

--
/Jacob Carlborg

On 22/11/2019 1:35 AM, Jacob Carlborg wrote:
> On Wednesday, 20 November 2019 at 22:15:18 UTC, Walter Bright wrote:
> 
>> In order to make non-null checking actually work, the language semantics would likely need to change to make:
>>
>>    T*    a non-null pointer
>>    T?*   an optionally null pointer
>>
>> or something like that. Two different pointer types would need to exist.
> 
> That would be really nice. But instead it would be just one pointer type and arbitrary  optional types:
> 
> T*           a regular pointer (can't be null)
> T*?         an optional pointer (can point to something or can be empty)
> int           a regular int
> int?         an optional int (can contain an int or can be empty)
> Object    a regular reference type (can't be null)
> Object?  an optional reference (can point to something or can be empty)
> 
> Basically there wouldn't be "null" anymore.
> 
> -- 
> /Jacob Carlborg

If we look at @live as a superset of @safe (basically @safe on steroids) which is the way I'm looking at it, all pointers in @live code would have to be non-null if they are alive (so viewable / owned).

It simplifies the language some what, and allows us to use something like Nullable for optional values.

While it is good that we are thinking about it, it can be put off while Walter gets the rest working (as he has been saying in essence).

On Thursday, 21 November 2019 at 12:35:03 UTC, Jacob Carlborg wrote:
> [snip]
>
> That would be really nice. But instead it would be just one pointer type and arbitrary  optional types:
> [snip]

What you describe is like Kotlin's approach. The part where you say there would be no null, just empty, is kind of like just renaming null as empty, correct? Kotlin still for null in T? types.

On 2019-11-21 14:35, jmh530 wrote:

> What you describe is like Kotlin's approach. The part where you say there would be no null, just empty, is kind of like just renaming null as empty, correct?

Yes, basically.

> Kotlin still for null in T? types.

Sure, you could have "null" as syntax sugar for representing and empty type.

-- 
/Jacob Carlborg