On Wednesday, 13 March 2019 at 11:35:02 UTC, Olivier FAURE wrote:
> On Wednesday, 13 March 2019 at 08:49:37 UTC, Atila Neves wrote:
>> [...]
>
> One possibility might be to implement at run time what rust does at compile time, eg make it impossible to mutate vector as long as one or more readable view to its content exists.
>
> So for instance, make opSlice and opIndex return a struct that bumps a reference counter up inside the vector; the counter is bumped down once that struct is destructed. As long as the number of "live" slices is non-zero, the vector isn't allowed to re-allocate.
>
>> [...]
>
>> [...]
>
> I don't think that's the point.
>
> It's supposed to be *impossible* to get a memory corruption in @safe code. Not "convoluted and wouldn't pass code review", impossible. When you write a @trusted function, getting a memory corruption with that function is supposed to be impossible as well.
>
> Other people have gone over why memory safety is necessary before. Not all code goes through code review, sometimes review miss errors even for critical applications, etc.

I just tagged v0.6.0.

I made `opSlice` `@system` given what has been discussed here.

I introduced a new member function `range` which return a forward range that keeps a pointer to the vector inside of it. Coupled with dip1000 it seems to do the trick.

I also made it so `Vector` is no longer an input range itself.

On Wednesday, 13 March 2019 at 11:35:02 UTC, Olivier FAURE wrote:
>
> One possibility might be to implement at run time what rust does at compile time, eg make it impossible to mutate vector as long as one or more readable view to its content exists.
>
> So for instance, make opSlice and opIndex return a struct that bumps a reference counter up inside the vector; the counter is bumped down once that struct is destructed. As long as the number of "live" slices is non-zero, the vector isn't allowed to re-allocate.

I'm not 100% sure, but it kinda seeems like this solution requires, but is obviated by scope.

> It's supposed to be *impossible* to get a memory corruption in @safe code. Not "convoluted and wouldn't pass code review", impossible. When you write a @trusted function, getting a memory corruption with that function is supposed to be impossible as well.

I agree that @safety should be a strict guarantee.

On Wednesday, 13 March 2019 at 08:49:37 UTC, Atila Neves wrote:
> On Tuesday, 12 March 2019 at 21:06:17 UTC, bitwise wrote:
>> [...]
>>    Bit
>
> After thinking about what @bitwise said, I was actually going to post back similar code.
>
> The code in the github issue is problematic, but it's a far cry from how such errors are usually introduced in C++. Normally it's because another thread has a reference to the now dangling pointer, or calling a function with the slice means it gets escaped somewhere.

I think that in C++, iterator validity is a hard concept to overlook. I would bet that even relatively new C++ programmers are familiar with the idea. I think most intro C++ classes teach this type of thing relatively early on. This awareness prevents/mitigates a lot of potential problems, resulting in the issue being underrepresented in peoples negative views of C++ (or so it would seem, I've seen no actual data on this). In any case, it would be great if callers could just ignore the issue completely.

> void oops(ref Vector!int vec, int[] slice);

Yeah, I remember Walter's post about how this will mess with ref-counted objects as well. IIRC, the solution was compiler support double-incrementing the ref-count when it finds this case. I wonder if this is actually necessary though, because in C++, shared_ptr has the aliasing constructor, and with D's templates, something like RefCounted could be modified to expose the members of it's contained type as properties that worked like shared_ptr's aliasing constructor, wrapping the returned field value in a ref-counted object that shares the ref-count of its parent object. If this was done, the member exposing that slice would do the extra increment without compiler support.

> Either way, thanks for pointing out the possible issue. I'm going to have to think long and hard about whether it's possible to fix it with the language we have now.

Thanks for exploring this issue with me. I agree its tough to build a proper container right now, but I do believe I have more options than when I started.

I just found this as well, which I'm pretty excited about:
https://github.com/dlang/DIPs/blob/master/DIPs/accepted/DIP1014.md

The version of std::vector from Visual Studio makes use of this for iterator validation in debug mode. The vector heap-allocates a "proxy" object and stores it's own pointer in it. When the vector moves (and it's move ctor is called), it updates the pointer in the proxy object. It also gives this proxy out to iterators so they can track the original vector. Without any type of move callback in D, such an implementation would have been impossible, so this is awesome news.

On Wednesday, 13 March 2019 at 13:29:51 UTC, Atila Neves wrote:
> [...]
>
> I introduced a new member function `range` which return a forward range that keeps a pointer to the vector inside of it. Coupled with dip1000 it seems to do the trick.

Awesome. I'm trying to test the same thing, but -dip1000 seems to have no effect, and -scope is unrecognized. Do I have to download the beta or GIT HEAD to get a functional DIP1000 implementation?

Thanks again,
    Bit

On Wednesday, 13 March 2019 at 13:29:51 UTC, Atila Neves wrote:
> I made `opSlice` `@system` given what has been discussed here.
>
> I introduced a new member function `range` which return a forward range that keeps a pointer to the vector inside of it. Coupled with dip1000 it seems to do the trick.

Have you tested it with opIndex? You'd probably get the same problem, since it returns a reference to data that might be reallocated later.

On Friday, 15 March 2019 at 16:11:43 UTC, bitwise wrote:
> On Wednesday, 13 March 2019 at 13:29:51 UTC, Atila Neves wrote:
>> [...]
>>
>> I introduced a new member function `range` which return a forward range that keeps a pointer to the vector inside of it. Coupled with dip1000 it seems to do the trick.
>
> Awesome. I'm trying to test the same thing, but -dip1000 seems to have no effect, and -scope is unrecognized. Do I have to download the beta or GIT HEAD to get a functional DIP1000 implementation?
>
> Thanks again,
>     Bit

It definitely has an effect. This code compiles with -dip1000, doesn't without:

void main() @safe {
    int i;
    fun(&i);
}

void fun(scope int* i) @safe {

}


Works as intended on dmd 2.085.0 and 2.084.1.

On Saturday, 16 March 2019 at 12:17:20 UTC, Olivier FAURE wrote:
> On Wednesday, 13 March 2019 at 13:29:51 UTC, Atila Neves wrote:
>> I made `opSlice` `@system` given what has been discussed here.
>>
>> I introduced a new member function `range` which return a forward range that keeps a pointer to the vector inside of it. Coupled with dip1000 it seems to do the trick.
>
> Have you tested it with opIndex? You'd probably get the same problem, since it returns a reference to data that might be reallocated later.

Ugh, you're right. Not sure what to do about that one, but thanks for bringing it up!