On 2/7/20 6:30 PM, ag0aep6g wrote:
> On 08.02.20 00:10, nullptr wrote:
>> ```
>> import std;
>>
>> struct SomeRange
>> {
>>      int[3] val;
>>
>>      enum empty = false;
>>
>>      auto popFront() @safe {}
>>
>>      ref auto front() @safe
>>      {
>>          return val;
>>      }
>> }
>>
>> void main() @safe
>> {
>>      SomeRange().take(10).map!((return ref x) => x[]).joiner.writeln;
>> }
>> ```
>>
>> I don't know how applicable this is to your use case, but this code will compile and run under -dip1000.
> 
> That shouldn't compile. You have found a hole in DIP 1000.
> 
> ----
> struct SomeRange
> {
>      int[3] val = [10, 20, 30];
>      ref auto front() @safe { return val; }
> }
> 
> int[] f() @safe
> {
>      SomeRange sr;
> //     return sr.val[]; /* error, as expected */
>      return sr.front[]; /* no error, but escapes reference to local */
> }
> 
> void main() @safe
> {
>      auto x = f();
>      import std.stdio;
>      writeln(x); /* Prints garbage. */
> }
> ----
> 
> I'm too lazy right now to check if it's already in Bugzilla.

The original code is not invalid though. f is not valid, and that is a bug, but the original code posted by nullptr should be fine by memory safety standards.

If there is no possible way to do the original code with dip1000 attributes, then that's a bug with dip1000's design (would not be surprised).

-Steve

On 08.02.20 01:17, Steven Schveighoffer wrote:
> The original code is not invalid though. f is not valid, and that is a bug, but the original code posted by nullptr should be fine by memory safety standards.

Maybe. But then it should also work with `int[3] front;`.

> If there is no possible way to do the original code with dip1000 attributes, then that's a bug with dip1000's design (would not be surprised).

Or DIP 1000 just doesn't allow that kind of code. @safe (including DIP 1000) is not supposed to allow all de-facto safe programs.

Simplified, we're looking at this:

----
struct Joiner
{
    int[3] _items;
    int[] _current;
}
void main() @safe
{
    Joiner j;
    j._current = j._items[];
}
----

I.e., a self-referential struct. Or most fundamentally:

----
struct Joiner
{
    Joiner* p;
}
void main() @safe
{
    Joiner j;
    j.p = &j; /* error */
}
----

`dmd -dip1000` complains about the marked line:

    Error: reference to local variable j assigned to non-scope j.p

What if I mark `j` as `scope`? Then I should be able to assign a reference-to-local to `j.p`. Indeed, the error goes away, but another takes its place:

    Error: cannot take address of scope local j in @safe function main

Right. That can't be allowed, because `scope` gives only one level of protection, and `&j` would need two (one for `j` itself and one for the thing it points at).

If that code were allowed, you could do this:

----
struct Joiner
{
    Joiner* p;
}
Joiner g;
void main() @safe
{
    scope Joiner j;
    () @trusted { j.p = &j; } (); /* pretend it's allowed */
    g = *j.p; /* dereference and copy */
}
----

Returning a copy of a dereferenced `scope` pointer is always allowed, because `scope` only provides one level of protection.

On 08.02.20 02:38, ag0aep6g wrote:
> Simplified, we're looking at this:
> 
> ----
> struct Joiner
> {
>      int[3] _items;
>      int[] _current;
> }
> void main() @safe
> {
>      Joiner j;
>      j._current = j._items[];
> }
> ----
> 
> I.e., a self-referential struct. Or most fundamentally:
> 
> ----
> struct Joiner
> {
>      Joiner* p;
> }
> void main() @safe
> {
>      Joiner j;
>      j.p = &j; /* error */
> }
> ----
> 
> `dmd -dip1000` complains about the marked line:
> 
>      Error: reference to local variable j assigned to non-scope j.p
> 
> What if I mark `j` as `scope`? Then I should be able to assign a reference-to-local to `j.p`. Indeed, the error goes away, but another takes its place:
> 
>      Error: cannot take address of scope local j in @safe function main
> 
> Right. That can't be allowed, because `scope` gives only one level of protection, and `&j` would need two (one for `j` itself and one for the thing it points at).

I went a bit overboard with that second reduction. The original struct is self-referential, but it's not recursive. The errors are the same for the first reduction. But it's not as clear that they're necessary.

In the first reduction, `j` might be `scope`, but `j._items` has no indirections. `scope` doesn't actually mean anything for it. It's just an `int[3]` on the stack. So taking its address could be allowed.

But (the current implementation of) DIP 1000 is apparently too conservative for that. It seems to treat pointers into a struct the same as pointers to the whole thing.

On 2/7/20 8:38 PM, ag0aep6g wrote:

> If that code were allowed, you could do this:
> 
> ----
> struct Joiner
> {
>      Joiner* p;
> }
> Joiner g;
> void main() @safe
> {
>      scope Joiner j;
>      () @trusted { j.p = &j; } (); /* pretend it's allowed */
>      g = *j.p; /* dereference and copy */
> }
> ----
> 
> Returning a copy of a dereferenced `scope` pointer is always allowed, because `scope` only provides one level of protection.

Again, the limitations of dip1000 are apparent. Why can't I mark p as only pointing at scope data? And in which cases do I need to do this (in the true Joiner code)?

This kind of stuff is so difficult to reason about and develop as a library that people will just end up removing dip1000 from their compilation.

-Steve

On 08.02.20 15:57, Steven Schveighoffer wrote:
> This kind of stuff is so difficult to reason about and develop as a library that people will just end up removing dip1000 from their compilation.

I 100% agree that DIP 1000 is hard to reason about. It's pretty limited by design, and the implementation has so many bugs. If anyone has a better design (and implementation), I'd be all for that.

About just ditching the compiler switch: Then you can't even take the address of a local. Also, it's going to become the default eventually.

On 08.02.20 07:27, ag0aep6g wrote:
> On 08.02.20 02:38, ag0aep6g wrote:
>> Simplified, we're looking at this:
>>
>> ----
>> struct Joiner
>> {
>>      int[3] _items;
>>      int[] _current;
>> }
>> void main() @safe
>> {
>>      Joiner j;
>>      j._current = j._items[];
>> }
>> ----
[...]
> In the first reduction, `j` might be `scope`, but `j._items` has no indirections. `scope` doesn't actually mean anything for it. It's just an `int[3]` on the stack. So taking its address could be allowed.
> 
> But (the current implementation of) DIP 1000 is apparently too conservative for that. It seems to treat pointers into a struct the same as pointers to the whole thing.

My attempt at lifting this limitation:

https://github.com/dlang/dmd/pull/10773

On 2/8/20 1:20 PM, ag0aep6g wrote:
> On 08.02.20 15:57, Steven Schveighoffer wrote:
>> This kind of stuff is so difficult to reason about and develop as a library that people will just end up removing dip1000 from their compilation.
> 
> I 100% agree that DIP 1000 is hard to reason about. It's pretty limited by design, and the implementation has so many bugs. If anyone has a better design (and implementation), I'd be all for that.

I unfortunately don't. I suspect that making scope a type constructor might help, but I don't know enough about how dip1000 works to know for sure.

I like the idea of dip1000 inferring most things. But the end result of joiner just getting inferred @system is so difficult to figure out why.

I think before dip1000 is the default, we need some way to say, I expect inference to infer these calls as safe. And it should tell you why it can't.

> 
> About just ditching the compiler switch: Then you can't even take the address of a local. Also, it's going to become the default eventually.

Then the user will just switch to @trusted. I mean to have to figure out what joiner is actually doing, in conjunction with map, like you did, is so convoluted, nobody is going to spend that time. The easier thing is to give up and switch to system where you have to.

I don't feel dip1000 is ready for the default mode until we have better ways to diagnose its issues, and concrete responses to things that really should be safe but aren't.

-Steve