On Tuesday, 12 November 2019 at 03:08:28 UTC, Walter Bright wrote:
> On 11/11/2019 7:24 AM, jmh530 wrote:
>> malloc is also only allowed in @system code.
>
> Although malloc() is marked as @system, it actually is @safe. It's free() that needs to be @system.

You can only really consider malloc @safe cause "void*" is completely useless in @safe, and casting it to a different type is what is unsafe. By those constraints free() could also be considered "safe". If you do cast the pointer to another type, then both are not safe.

On 11/11/2019 7:35 PM, Jab wrote:
> On Tuesday, 12 November 2019 at 03:08:28 UTC, Walter Bright wrote:
>> On 11/11/2019 7:24 AM, jmh530 wrote:
>>> malloc is also only allowed in @system code.
>>
>> Although malloc() is marked as @system, it actually is @safe. It's free() that needs to be @system.
> 
> You can only really consider malloc @safe cause "void*" is completely useless in @safe, and casting it to a different type is what is unsafe. By those constraints free() could also be considered "safe". If you do cast the pointer to another type, then both are not safe.

The cast of the pointer returned from malloc() is indeed unsafe, but the malloc() call is safe.

On 11/11/2019 7:03 PM, Jab wrote:
> How have you not had this problem before?

Because I was careful.


> DMD's code base is kind of horrible. Pointers to arrays that end up having to be accessed by (*p)[x]. A simple wrapper could easily have alleviated that entirely. It's kind of ironic that the D reference compiler showcases exactly how you shouldn't write D code.

DMD was rather recently converted to D from a "C with Classes" codebase. Its structure very much reflects that heritage.


> Of all the things people would be willing to have such a big breaking changes for. Auto-decoding to name one. This is now where the line is drawn? Some of these decisions are being made too hastily IMO.

No decision has been made about DIP 1025. That's what this process is for. Everyone gets their say.

On Monday, 11 November 2019 at 23:20:31 UTC, Walter Bright wrote:
> On 11/11/2019 12:50 PM, Meta wrote:
>> Would it be enough to put an assert in GC.realloc/extend/free that asserts the memory is GC-owned, maybe tied to a compiler switch?
>
> Currently, for `a~=b` if `a` is not owned by the GC, it is copied to a new GC location by the runtime.

Yes, my point being, if the concern is over manually-allocated pointers being invalidated by accidental reallocation by the GC, wouldn't the absolute least intrusive change possible be to put asserts in the relevant GC methods to make sure it's only reallocating/extending/freeing memory *it* owns? It would mean a crash at runtime only, and only in debug mode, but you get a semi-nice stack trace showing you the exact point you're trying to do something bad (assuming I properly understand the GC implementation, which I may not).

On Monday, 11 November 2019 at 21:41:14 UTC, Walter Bright wrote:
> On 11/11/2019 5:13 AM, Dennis wrote:
>> On Monday, 11 November 2019 at 11:26:49 UTC, Nicholas Wilson wrote:
>>> is functional identical, thus any issues present in 2 are also present in 1. The suggestion that doing so in a loop would create more garbage is false as it is functionally identical.
>> 
>> There is a difference: a = a ~ b is guaranteed to make a copy while a ~= b might just use spare capacity of the memory that slice `a` refers to.
>> 
>> https://dlang.org/spec/arrays.html#array-concatenation
>
> That's right. I had overlooked that.
>
>
>> This is what the dead/alive example is supposed to express.
>> It is still true however that the rationale is not very convincing.
>> The first example indeed doesn't show any benefits of the DIP:
>> 
>> ```
>> int[] slice = cast(int*)malloc(10 * int.sizeof)[0 .. 10];
>> slice = slice ~ 1; // now guaranteed to make a copy
>> free(slice.ptr); // Still oops
>> ```
>
> Imagine these 3 lines are spread out over a large code base.
>
>
>> The following claim in the DIP is also unsubstantiated:
>> 
>>> This change is a necessary part of D evolving towards being memory safe without using
>>> a GC.
>
> Memory safety cannot be achieved without control over who is the owner of memory.
I thought the GC knows if the memory is in its control or not? Why not prohibit the concatenation after the GC checked if it is memory it has no view over. That slice was allocated outside or points on the stack so stop with an error. It's not compile time, but better than nothing, but doesn't break all the apps that want to live a simple life of GC memory handling. If I had wanted to make my programmers life difficult I would have chosen Rust or C++.

>
>
>> I would like to see an example of memory corruption in `@safe` code that can happen because of slice appending.
>
>   @trusted void myfree(void* p) { free(p); }
>   free(slice.ptr);

cheating doesn't count. @trusted in the same vein ad casting, is explicit security overriding, you get what you called for.

On Monday, 11 November 2019 at 22:28:05 UTC, Walter Bright wrote:
> The idea is to be able to guarantee memory safety. I've spoken about this in my presentations and DConf, and that's clearly what DIP25 and DIP1000 were targeted at, and my proposals for an Ownership/Borrowing system for D.
>
> This is where the industry is moving. Even C++ is heading that direction:
>
>   http://open-std.org/JTC1/SC22/WG21/docs/papers/2018/p1179r0.pdf
>
> This proposal is written by Herb Sutter, and is no joke.
>
> We either get on the bus or get run over by the bus.

By the short bus.
Sorry, couldn't resist.

On Monday, 11 November 2019 at 10:27:26 UTC, Mike Parker wrote:
> This is the feedback thread for the first round of Community Review for DIP 1025, "Dynamic Arrays Only Shrink, Never Grow":
>
> https://github.com/dlang/DIPs/blob/1b525ec4c914c06bc286c1a6dc93bf1533ee56e4/DIPs/DIP1025.md
>
> All review-related feedback on and discussion of the DIP should occur in this thread. The review period will end at 11:59 PM ET on November 25, or when I make a post declaring it complete.
>
> At the end of Round 1, if further review is deemed necessary, the DIP will be scheduled for another round of Community Review. Otherwise, it will be queued for the Final Review and Formal Assessment.
>
> Anyone intending to post feedback in this thread is expected to be familiar with the reviewer guidelines:
>
> https://github.com/dlang/DIPs/blob/master/docs/guidelines-reviewers.md
>
> *Please stay on topic!*
>
> Thanks in advance to all who participate.

I understand where this proposal comes from, but it would certainly cause a lot of breakage and it would cripple the universal feature common to all programming languages, arrays. Without filling the void this proposal leaves behind, I don't see any support for it.

I would personally look at the conflation of dynamic arrays and slices. If you could somehow separate the 2 concepts and THEN constrain slices to only be able to shrink, that might work. (or keep them conflated but make borrowed slices non appendable.) (or make the implicit explicit by somehow requiring `.array` before appending).

On the other hand, the fact that they are conflated also brings benefits, and you certainly don't want to loose them. It is a tough nut to crack.

OT:

In general I agree with the sentiment D's language features oftentimes depend too much on an all-knowing garbage collector. Maybe there is a way that behaviour could be made customisable for people who need that, instead of having everyone bear the costs. (e.g. I have done a lot of betterC programming lately, and while I can use my own custom allocators, things like closures still depend on a GC.)

On Tuesday, 12 November 2019 at 03:08:28 UTC, Walter Bright wrote:
> Although malloc() is marked as @system, it actually is @safe. It's free() that needs to be @system.

I think that is a mute point.
malloc gives you a void*, you can't really do anything with that void*
in @safe code.
You can't even cast it to int*, and it shouldn't.

I know I sound like a broken record, but IMO mmm and @safe is the wrong direction.

On Monday, 11 November 2019 at 22:28:05 UTC, Walter Bright wrote:
> The idea is to be able to guarantee memory safety. I've spoken about this in my presentations and DConf, and that's clearly what DIP25 and DIP1000 were targeted at, and my proposals for an Ownership/Borrowing system for D.
>
> This is where the industry is moving. Even C++ is heading that direction:
>
>   http://open-std.org/JTC1/SC22/WG21/docs/papers/2018/p1179r0.pdf
>
> This proposal is written by Herb Sutter, and is no joke.
>
> We either get on the bus or get run over by the bus.

golang is not going in that direction. They are not @safe but mostly because
they have chosen performance over safety.

To me D is great because it is not doing what everybody else is doing.
If D was only doing what c++ or rust are doing, D can only become as
good as those IMO. But D should be better.
Generative programming in combination with functional programming, is
awesome. Hardly anybody is doing that.
I'm not saying we should steal other peoples good ideas.
I'm trying to say we should steal interesting ideas/mistakes and improve on them.
For instance taking the c++ template meta-programming mistake and
making it a feature was a huge win.

Additionally, I think there is an elephant in the room. Multi-threading.
I just watched a talk on rust async/await.
https://www.youtube.com/watch?v=lJ3NC-R3gSI
It took them ~10 years to get the borrow-checker + multi threading thing
done/right, and compared to us they have infinite resources.
Is there a plan for D and the borrow-checker in place/planning?

On Tuesday, 12 November 2019 at 08:41:52 UTC, Sebastiaan Koppe wrote:
> I would personally look at the conflation of dynamic arrays and slices. If you could somehow separate the 2 concepts and THEN constrain slices to only be able to shrink, that might work. (or keep them conflated but make borrowed slices non appendable.) (or make the implicit explicit by somehow requiring `.array` before appending).
>
> On the other hand, the fact that they are conflated also brings benefits, and you certainly don't want to loose them. It is a tough nut to crack.

I'm starting to think that the issue is not that they are conflated, but that it isn't clear whether "dynamic" GC arrays  are values/object, high-level/low-level or language-constructs/runtime-constructs.

If concatenation expressions work with high-level values conceptually, and slices are views on those values, and GC-arrays are high-level-nonruntime-constucts, then things might become more clear.

Then leave to the optimizer to estimate and preallocate a buffer size (or with hints).

I don't think there is anything wrong with having high level concepts, but then they should be truly high level so that the optimizer can do the work and there should be some clear delineation between high-level and  low-level types. (You could still allow low level operations temporarily, but the life-time of that would have to be made formal, so that the optimizer has free hands when the low-level work is done).

But then you also need another low-level concept for views on an address-bound object/buffer/memory.

So it is a tough nut, yes.


> In general I agree with the sentiment D's language features oftentimes depend too much on an all-knowing garbage collector.

Yes, but is also cultural, in the sense that some people rely 100% on the garbage collector. So how can  the type system ensure that the high quality libraries they write are usable in the more general programming context?

If all programmers only relied on the GC by 30%, then maybe this would be less of an issue, as they would think differently about the use context of the library.  The rather weakly typed C++ ecosystem works because there is some shared understanding of the use context and conventions.

So that is an extra challenge for D. Maybe static analysis could help. Some innovation that can prove properties down the call chain fromt he call site (rather than all those attributes on the function signatures).