On Thursday, May 16, 2019 11:22:30 PM MDT Mike Franklin via Digitalmars-d- announce wrote:
> I consider it a bug that the compiler doesn't emit an error when using attributes on types for which they are not intended.

As in you think that something like

auto foo(scope int i) {...}

should be illegal, because scope makes no sense on an int? That's nice in theory, but templates make such an approach a serious problem. It needs to work to do something like

auto foo(T)(scope T t) {...}

without having to have separate overloads for types where scope makes sense and types where it doesn't. Similarly, you don't want to have to use static ifs whenever you declare a variable that you want to be scope in the cases where the template argument is a type where scope does work.

In general, D ignores attributes when they don't apply rather than making it an error, because making it an error causes serious problems for generic code. This does unfortunately mean that some people are bound to sometimes end up using an attribute when it doesn't apply, thinking that it does, and that's unfortunate, but overall, it just works better for the compiler not to complain about such cases.

- Jonathan M Davis

On Friday, 17 May 2019 at 05:22:31 UTC, ag0aep6g wrote:
> On 17.05.19 06:50, Meta wrote:
>> Walter, can I get you to take a look at this post I made a few months ago, and the contained example? I feel that this is a case that *should* definitely work, but I'm not sure if it can *currently* work - and so far, nobody else seems to be either, save for you.
>> 
>> https://forum.dlang.org/post/laqjadtwrsdhdrqokryx@forum.dlang.org
>
> You don't like my explanation?
>
> https://forum.dlang.org/post/q6r4bf$2hu4$1@digitalmars.com (same thread)

Your explanation was fine, but I need a good solution, other than wrapping the array assignment in @trusted.

On Friday, 17 May 2019 at 05:32:42 UTC, Mike Franklin wrote:
> On Friday, 17 May 2019 at 05:22:30 UTC, Mike Franklin wrote:
>
>> My assessment (which could be wrong):
>> `scope` and `return` only apply to pointers and `ref`s.  If you remove all `scope` and `return` attributes from the function `push`, it works fine.
>>
>> I consider it a bug that the compiler doesn't emit an error when using attributes on types for which they are not intended.
>>
>> Mike
>
> Working example:  https://run.dlang.io/is/TCP0td

That does compile, but I don't think that it's working the way I want it to. I believe it only works because a GC-managed string is used for the backing storage. If you change that to a static array on the stack:

@safe
void main()
{
    immutable(char)[16] rawData = "2 6 4 1 0 2 9 4 5";
    auto dataRange = makeDataRange(rawData);
    auto result = dataRange.copyToQueue();
    import std.stdio;
    writeln("The result of data processing is: ", result);
}

It will refuse to compile with this message:
Error: reference to local variable rawData assigned to non-scope parameter input calling makeDataRange

`makeDataRange` is defined like this:
@safe
DataRange makeDataRange(string input)
{
    auto range = DataRange(input);
    return range;
}

So that static array is getting implicitly sliced, i.e., its address is being taken. It's pretty obvious why `input` is not being inferred as scope - it's being returned from `makeDataRange`. However, when I try to manually annotate it with return or return scope, I run into further errors:

DataRange makeDataRange(return scope string input)
{ ...etc. }

Error: scope variable input assigned to non-scope parameter rawData calling DataRange.this
Error: scope variable dataRange assigned to non-scope parameter data calling copyToQueue

So I continue annotating things with scope or return or return scope whenever the compiler complains about it, going up through the call chain until I arrive back at my original problem mentioned in the post I linked.

(My original example with changes made going through this exercise: https://run.dlang.io/is/uQDXG6)

This is why I say that I'm not sure that I quite understand dip1000. I *thought* I did, but an example that seems like it should clearly work (at least to me), does not.

If you look at `main` above, `rawData` has the same lifetime as the `dataRange` struct returned from `makeDataRange` and the queue returned from `copyToQueue`. True, there is some traditionally unsafe stuff happening in between; however, I thought that the point of adding all these annotations is to tell the compiler how the lifetimes of these objects propagate up and down the call stack, so that it can check that there will be no memory corruption. I'm not doing anything here that will result in a pointer to an expired stack frame, or otherwise cause memory corruption or use after free, or anything like that (*unless* I allow either `dataRange` or `result` to escape from the main function - which dip1000 correctly disallows).

On 17.05.19 14:10, Meta wrote:
> Your explanation was fine, but I need a good solution, other than wrapping the array assignment in @trusted.

I see. As far as I understand DIP 1000, it's not supposed to enable your use case without having to use `@trusted`.

DIP 1000 stops at heap allocations. It just assumes infinite lifetime for them. If you want to restrict the lifetime of a heap allocation (in your case: tie it to the lifetime of a struct), you have to do it manually.

On Friday, 17 May 2019 at 17:05:21 UTC, ag0aep6g wrote:
> On 17.05.19 14:10, Meta wrote:
>> Your explanation was fine, but I need a good solution, other than wrapping the array assignment in @trusted.
>
> I see. As far as I understand DIP 1000, it's not supposed to enable your use case without having to use `@trusted`.

If this is true, then I have a big problem with DIP1000. This is an extremely common use case (copying memory from an inner scope with a limited lifetime to some store in an outer scope with a longer or infinite lifetime).

> DIP 1000 stops at heap allocations. It just assumes infinite lifetime for them.

Yes, as per the DIP.

> If you want to restrict the lifetime of a heap allocation (in your case: tie it to the lifetime of a struct), you have to do it manually.

I don't want to *restrict* the lifetime of a heap allocation. I want the compiler to recognize that the lifetime of my original data is the same as the processed output, and thus allow my code to compile.

On Friday, 17 May 2019 at 05:27:02 UTC, Walter Bright wrote:
> On 5/16/2019 9:50 PM, Meta wrote:
>> Walter, can I get you to take a look at this post I made a few months ago, and the contained example? I feel that this is a case that *should* definitely work, but I'm not sure if it can *currently* work - and so far, nobody else seems to be either, save for you.
>> 
>> https://forum.dlang.org/post/laqjadtwrsdhdrqokryx@forum.dlang.org
>
> As always, I recommend drastically reducing the example. It nearly always makes the actual problem emerge from all the noise.

I'll try to reduce it further, but this example is already as reduced as I could make it while still having the same structure as my actual code.

On Thu, May 16, 2019 at 10:35:27AM +0000, Seb via Digitalmars-d-announce wrote:
> On Thursday, 16 May 2019 at 10:03:42 UTC, Kagamin wrote:
> > On Thursday, 16 May 2019 at 05:22:42 UTC, Seb wrote:
> > > Yes that sounds like the culprit. Btw as mentioned on DConf, the dip1000 switch contains a few other breaking changes which will make it even harder to adopt too.
> > 
> > Well, it's an inherent property of DIP1000 to not compile code that previously compiled. Though safety of tupleof shouldn't depend on DIP1000.
> 
> Well, here's the full discussion:
> 
> https://github.com/dlang/dmd/pull/8035

Finally got round to skimming through that discussion.

Looks like in this case, what we need is for toHash to be declared @trusted when .tupleof includes private members (because toHash is not supposed to modify any private members, and I assume hashing over a private member shouldn't violate @safe -- right?).

Either that, or RedBlackTree needs to be changed so that @safe-ty doesn't depend on random user types having private fields breaking compilation.  It's pretty ridiculous, from a user's POV, for a standard container to fail to compile just because the user had the audacity to declare private members in his object! And the fact that this root problem is masked under a totally obscure compile error only adds salt to the wound.


T

-- 
IBM = I Blame Microsoft

On 17.05.19 19:25, Meta wrote:
> I don't want to *restrict* the lifetime of a heap allocation. I want the compiler to recognize that the lifetime of my original data is the same as the processed output, and thus allow my code to compile.

You have a heap allocation that references your original data, which might be on the stack (Queue.store[0] points into rawData). The compiler would have to make sure (or recognize) that the pointers on the heap don't outlive the data to which they point.

But the lifetime of the heap allocation is infinite, and the lifetime of the stack data is not. So that fails. To make it work you'd need some way to have a shorter-than-infinite lifetime for the heap.

Whether you want to restrict the lifetime explicitly or have the compiler figure it out on its own ... it doesn't really matter when we can't do either.

On Friday, May 17, 2019 11:25:40 AM MDT Meta via Digitalmars-d-announce wrote:
> I don't want to *restrict* the lifetime of a heap allocation. I want the compiler to recognize that the lifetime of my original data is the same as the processed output, and thus allow my code to compile.

It is my understanding that DIP 1000 really doesn't track lifetimes at all. It just ensures that no references to the data escape. So, you can't do something like take a scope variable and put any references to it or what it refers to in a container. Honestly, from what I've seen, what you can ultimately do with scope is pretty limited. It definitely helps in simple cases, but it quickly gets to the point that it's unable to be used in more complex cases - at least not without casting and needing to use @trusted. So, it's an improvement for some kinds of code, but I suspect that in general, it's just going to be more annoying than it's worth. Time will tell though.

- Jonathan M Davis

On 5/17/2019 10:26 AM, Meta wrote:
> I'll try to reduce it further, but this example is already as reduced as I could make it while still having the same structure as my actual code.

It doesn't need to have the same structure. It just needs to exhibit the problem.