On 3/18/2019 7:28 PM, jmh530 wrote:
> BTW, did you see this:
> https://atilaoncode.blog/2019/03/13/issues-dip1000-cant-yet-catch/

C++ has no protection for its library types, except by convention. Translating them by rote into D leaves those problems intact. They need to be redesigned with D's mechanisms in mind for them to be memory safe. Leaking pointers to its internals is exactly that sort of issue.

Okay, one more question. I might be misunderstanding how DIP1000 works, but I cannot figure out how to copy data owned by an inner scope to an outer scope. I know that `scope` is supposed to prevent this, but in my case it's perfectly fine, as I am transferring ownership of the data from the inner scope to a data store in the outer scope (e.g., copying some data in a deeper-nested scope into an outer array). However, I can't seem to get the compiler to understand this transfer of ownership (I also tried moving values with std.algorithm.move).

My example code is a bit too long to post here, but it can be found at https://run.dlang.io/is/e6Lc15. The main crux of this issue is:

@safe
Queue!Data copyToQueue(DataRange data)
{
    Queue!Data output;
    while (!data.empty)
    {
        auto d = data.front;
        data.popFront();
        import std.random: uniform;
        if (uniform(0, 100) < 50)
        {
            output.push(d);
        }
    }
    return output;
}

And the definition of Queue!T.push is:

    @safe
    void push(return scope T val) scope
    {
        ptr += 1;
        if (ptr >= store.length && ptr < size_t.max)
        {
            store.length = store.length == 0
                ? 8
                : store.length * growthFactor;
        }
        //FIXME: either I don't understand DIP1000
        //well enough, or this should compile
        //Workaround: surround it with an @trusted lambda
        store[ptr] = val;
    }

From my understanding of DIP1000, annotating `val` with `return scope` in `Queue.push` tells the compiler "this value can only escape by being returned from the function, or being copied into the first parameter" (in this case, the first parameter being the `this` reference). Then, marking `Queue.push` as `scope` also tells the compiler "no references to `this` may escape from this function".

In this case, the Queue I am copying data into is in an outer scope, and the data I'm copying is `scope` (see the code at my link). The problem is that the compiler doesn't like this; it's telling me "Error: scope variable val assigned to non-scope this.store[this.ptr]". This is confusing, though, because I thought that's exactly why there's this special proviso for `return` and `return scope`:

"If the function returns void, and the first parameter is ref or out, then all subsequent return ref parameters are considered as being assigned to the first parameter for lifetime checking. The this reference parameter to a struct non-static member function is considered the first parameter."

As long as I'm not escaping the `this` reference from `Queue.push`, shouldn't the lifetime analysis all check out?

On Tuesday, 19 March 2019 at 03:16:04 UTC, Walter Bright wrote:
> On 3/18/2019 7:28 PM, jmh530 wrote:
>> BTW, did you see this:
>> https://atilaoncode.blog/2019/03/13/issues-dip1000-cant-yet-catch/
>
> C++ has no protection for its library types, except by convention. Translating them by rote into D leaves those problems intact. They need to be redesigned with D's mechanisms in mind for them to be memory safe. Leaking pointers to its internals is exactly that sort of issue.

I didn't translate anything, I wrote it from scratch. C++ doesn't even have slices!

The fact is that DIP1000 didn't prevent me from writing @safe code where a pointer dangled. The point of my blog was "how can we improve D to disallow anyone else from writing code like this by making it fail to compile with -dip1000?". It's got nothing to do with C++, the comparison was with Rust, where one can't write the faulty code.

Given the focus on memory safety in D without having to rely on the GC, I think this should be given serious consideration.

On 19.03.19 11:19, Atila Neves wrote:
> On Tuesday, 19 March 2019 at 03:16:04 UTC, Walter Bright wrote:
>> On 3/18/2019 7:28 PM, jmh530 wrote:
>>> BTW, did you see this:
>>> https://atilaoncode.blog/2019/03/13/issues-dip1000-cant-yet-catch/
[...]
> The fact is that DIP1000 didn't prevent me from writing @safe code where a pointer dangled.
I don't think that's what happens. As far as I see, you get a dangling pointer, because you've got bad `@trusted` code in automem [1]:

    () @trusted { _allocator.expandArray(mutableElements, delta.toSizeT); }();

By default, `_allocator` is `GCAllocator` (via some obfuscations). `expandArray` calls the allocator's `reallocate` method. `GCAllocator.reallocate` is not memory safe. The documentation says [2]:

    "The deallocate and reallocate methods are @system because they
    may move memory around, leaving dangling pointers in user code"


[1] https://github.com/atilaneves/automem/blob/4d8e8800b27ac7e92ed066237fd1359f59116fc5/source/automem/vector.d#L441
[2] https://dlang.org/phobos/std_experimental_allocator_gc_allocator.html#.GCAllocator.reallocate

On 19.03.19 04:48, Meta wrote:
> My example code is a bit too long to post here, but it can be found at https://run.dlang.io/is/e6Lc15. The main crux of this issue is:
> 
> @safe
> Queue!Data copyToQueue(DataRange data)
> {
>      Queue!Data output;
>      while (!data.empty)
>      {
>          auto d = data.front;
>          data.popFront();
>          import std.random: uniform;
>          if (uniform(0, 100) < 50)
>          {
>              output.push(d);
>          }
>      }
>      return output;
> }
> 
> And the definition of Queue!T.push is:
> 
>      @safe
>      void push(return scope T val) scope
>      {
>          ptr += 1;
>          if (ptr >= store.length && ptr < size_t.max)
>          {
>              store.length = store.length == 0
>                  ? 8
>                  : store.length * growthFactor;
>          }
>          //FIXME: either I don't understand DIP1000
>          //well enough, or this should compile
>          //Workaround: surround it with an @trusted lambda
>          store[ptr] = val;
>      }

I think it boils down to the following.

This works:

    struct Q
    {
        string s;
        void push(return scope string v) scope @safe
        {
            this.s = v;
        }
    }

This doesn't:

    struct Q
    {
        string[] s;
        void push(return scope string v) scope @safe
        {
            this.s.length = this.s.length + 1;
            this.s[$ - 1] = v;
        }
    }

The reason is that `scope` is not transitive. The array `this.s` is `scope`, but an element like `this.s[$ - 1]` isn't. The compiler considers the elements to have infinite lifetime.

Here's another example that illustrates this. When you've got a `scope Q`, the compiler won't let you return `q.s`, but it doesn't mind you returning an element of it:

    struct Q
    {
        string[] s;
    }
    string[] f(scope Q q)
    {
        return q.s; /* Error: scope variable q may not be returned */
    }
    string g(scope Q q)
    {
        return q.s[0]; /* no error, the element is not `scope` */
    }

On Tuesday, 19 March 2019 at 14:28:00 UTC, ag0aep6g wrote:
> On 19.03.19 11:19, Atila Neves wrote:
>>>> [...]
> [...]
>> [...]
> I don't think that's what happens. As far as I see, you get a dangling pointer, because you've got bad `@trusted` code in automem [1]:
>
> [...]

Very good point.

I'm going to have to stroke my beard whilst looking upwards for a bit now.

On Tuesday, 19 March 2019 at 10:19:49 UTC, Atila Neves wrote:
> The fact is that DIP1000 didn't prevent me from writing @safe code where a pointer dangled. The point of my blog was "how can we improve D to disallow anyone else from writing code like this by making it fail to compile with -dip1000?". It's got nothing to do with C++, the comparison was with Rust, where one can't write the faulty code.

Hum, that's a little inaccurate. The part of your code that creates the dangling is marked as @trusted, which makes... aaaaand I've been ninja'ed.

But yeah, the problem isn't that -dip1000 allows unsafe code, it's that there is no way to do what automem does with -dip1000 without using @trusted at some point. Language-level reference counting could fix that and is on W&A's roadmap, but that's another can of worms.

On Tuesday, 19 March 2019 at 14:28:00 UTC, ag0aep6g wrote:
> 
> [snip]
> By default, `_allocator` is `GCAllocator` (via some obfuscations). `expandArray` calls the allocator's `reallocate` method. `GCAllocator.reallocate` is not memory safe. The documentation says [2]:
>

So I suppose the question then becomes can something like `reallocate` be made @safe with DIP1000?

On Tuesday, 19 March 2019 at 20:10:50 UTC, jmh530 wrote:
> So I suppose the question then becomes can something like `reallocate` be made @safe with DIP1000?

Probably not.