On 4/14/20 4:14 PM, Walter Bright wrote:
> On 4/14/2020 10:16 AM, Steven Schveighoffer wrote:
>> That being said, I agree with the simple case of returning a pointer from a stack variable directly from a function being disallowed. That also can be easily worked around, which should probably be required, but is never correct anyway.
> 
> Oh, it can be correct, when one wants to examine the stack pointer value. I've used it for that purpose myself. (Examining the stack pointer is valuable when determining whether other pointers are pointing into the stack or not.)

Do you need to call a function to do that? Won't just &somevariable work?

Also, this compiles just fine, and gives you the stack pointer value:

size_t stackPtr() @system
{
  int i;
  return cast(size_t)&i;
}

> 
> As I replied to Timon, you're drawing a subjective (not objective) line at what is acceptable or not.
> 

Yes, it's subjective. But so is common sense.

The objective line is that all should be allowed. I'm fine with that too if that means you can write useful memory safe @system code. It's an entirely valid and *consistent* proposal to say "@system just allows anything, including obvious memory errors. Use @safe if you want compiler restrictions."

Or, we could say, for the 0.00000001% of the time you want to get a stack pointer value, use a cast, which means the other 99.9999999% of the cases which are actually memory problems are helpfully caught. You must be able to see that there is a difference in common utility between needing to capture the stack pointer value, and wanting to utilize heap space.

It doesn't even make sense that array appending is somehow "the one thing" that causes problems, when you can pass stack pointers to functions which you don't control, slice static arrays, create stack pointers at will and *then* do whatever you want, etc. If there is a subjective story for this "feature" it has quite a few holes in it. It's like putting up a fence post instead of a fence to protect a space. It just ends up being more annoying than functional.

I can do this too, apparently since it's a TLS array and not a heap array, totes ok!

int*[1] arr;

void bar()
{
    int i;
    arr[0] = &i;
}

-Steve

On 4/14/20 4:40 PM, Steven Schveighoffer wrote:
> It's an entirely valid and *consistent* proposal to say "@system just allows anything, including obvious memory errors. Use @safe if you want compiler restrictions."

Note that this is especially palatable if @safe is the default and you have to opt-in to @system.

-Steve

On Tuesday, 14 April 2020 at 20:10:20 UTC, Walter Bright wrote:
> So, Adam, how do you feel about:
>
>   @system int* pumpkin(int i) { return &i);
>
> Timon says that should be an error. What is your opinion?

I could go either way on it. That does catch a real problem in real world Phobos use and it is very rarely what you actually want:

string s() {
        return hexDigest!MD5("test");
}

So I see the value in that error.

But the array example is different anyway because the above thing is almost always wrong while the array is far more ambiguous. Is it temporary scratch? Is it being passed to a different thread but you then wait until the other thread is done before returning? (That's what my crazy code that prompted this did btw.)

If it is an error 99% of the time, the error seems reasonable. But even then, what if you are one of the 1% of exceptions?

Or with the array, I'd guess it is an error 60% of the time. Maybe still fair to have it an error there... but what if you are in the other 40%? What do you do?

How do I tell the compiler "I know this looks wrong at first glance, but trust me"?

Again, that's why my subject line for this thread was "and trust". I don't mind this error most the time, we can see from experience that this kind of mistake can be made often enough to take note of it.

I just want some way to tell the compiler to trust me. OK, I confess, it'd probably still annoy me, like the annoying integral promotion casts kinda drive me nuts. But I can live with it, since at least there's a way to cast it away.

But here, @trusted didn't work. cast didn't work. So it just stopped me. I changed the code to use a static TLS instance instead of a stack item and got it compiling again, but I'm also concerned that D is going down an ugly path here. What if my workaround hack now gets plugged later? I want something, akin to `cast`, that is formally defined to function as an escape hatch.


(Maybe a double cast through void* might work, there seems to be potential in tricking the compiler with that, but those can be even harder to get right than the original lifetime problem, so we do need to make sure the cure isn't worse than the disease in allowing the exceptions.)

On 4/14/2020 3:58 PM, Adam D. Ruppe wrote:
> but what if you are in the other 40%? What do you do?

I proposed a direct way you can proceed with your method, it works.

I also proposed a more meta @safe rewrite that produced smaller and faster code than the @system example rejected by the compiler.

This makes it hard to discern the value in loosening up that particular restriction.

On Saturday, 11 April 2020 at 01:21:56 UTC, Adam D. Ruppe wrote:
> ```
> void main() {
>         int a;
>         b ~= &a;
> }
>
> int*[] b;
> ```
>
> trust.d(3): Error: copying & a into allocated memory escapes a reference to local variable a
>
>
> (Interestingly, `b = [&a]` instead of ~= passes muster. What's the difference? Just another bug in this?)
>
>
> But the inconsistency isn't why I'm posting right now, it is my fear that D is losing faith in me. There seems to be no way to say "trust me" to the compiler. Note that I'm not even using any switches to dmd, and adding @trusted had no effect.

Meanwhile, in Rust:

static mut b: Vec<*mut i32> = Vec::new();

fn bad() {
    let mut a = 0;
    unsafe { b.push(&mut a as *mut i32) };
}

fn main() {
    bad();
    println!("{:?}", unsafe { &b });
}

which outputs (for a particular run): [0x7ffc6e4b23b8]

So you have to say "trust me" a couple of times (or more than
twice, if you count the muts and the cast), but you *can* say
that. You can faithfully implement this bug in Rust.

Part of why you can do that, is that this code is using
pointers. The same code with only references fails despite the
unsafe{} blocks:

5 |     unsafe { b.push(&mut a) };
  |              -------^^^^^^-
  |              |      |
  |              |      borrowed value does not live long enough
  |              argument requires that `a` is borrowed for `'static`
6 | }
  | - `a` dropped here while still borrowed

Pointers in Rust lack safety and liveness guarantees. That
stuff is for references. D has references too...

| Rust     | D             |
|----------+---------------|
| &i32     | const ref int |
| &mut i32 | ref int       |
| *i32     | const(int)*   |
| *mut i32 | int*          |

... but not really. You can't declare them ("only parameters or
foreach declarations can be ref"), and you can't take a
reference of a variable, and you can't cast a pointer to a ref.

So it's not enough to say "Rust has normal vs. unsafe{} code,
and D has @safe vs. normal code, and the difference is only a
matter of defaults." It's also the case that Rust has unsafe
pointers and safe references, but D only has pointers, so any
memory protection that D gets must fall on its pointers.

With @live the protection only applies to pointers within
@safe code, and with @safe-as-default it might be even easier
to do things that way, or even to remove protections that
formerly applied to @system code. So the trend is not
necessarily towards it one day being more appropriate to say
that Rust has pointers and references, but D only has
references which it calls pointers.

On 4/14/2020 1:40 PM, Steven Schveighoffer wrote:
> Also, this compiles just fine, and gives you the stack pointer value:
> 
> size_t stackPtr() @system
> {
>    int i;
>    return cast(size_t)&i;
> }

Yup. I also gave workarounds to Adam's example.


> I can do this too, apparently since it's a TLS array and not a heap array, totes ok!
> 
> int*[1] arr;
> 
> void bar()
> {
>      int i;
>      arr[0] = &i;
> }

Ironically, I argue for consistency with "safe by default" and you argue for special cases, and here the reverse.

But be careful, you may talk me into disallowing these cases, too :-/

On Wednesday, 15 April 2020 at 06:12:22 UTC, Walter Bright wrote:

> Ironically, I argue for consistency with "safe by default" and you argue for special cases, and here the reverse.
>
> But be careful, you may talk me into disallowing these cases, too :-/

A good reference point would be C. If it is allowed in C, it should be allowed in @system D. If I put @system on a D function, i tell the compiler not to bug me with incorrect assumptions about my code's safety.

On Wednesday, 15 April 2020 at 02:19:51 UTC, Walter Bright wrote:
> I proposed a direct way you can proceed with your method, it works.

Is that formally defined and guaranteed to continue working in the future? Or is it just exploiting a temporary hole in the system that could be patched at any time without notice?

Maybe document it something like "assumeUnique" with "allowEscape" to give that peace of mind and I'll be happy with it.

Note that my original problem was already solved before I wrote this opening message. I'm not asking for a solution to this particular case; I have plenty of those (for now at least, who knows until you arbitrarily break them too).

I'm asking for confidence in D's future.

On 4/15/20 2:12 AM, Walter Bright wrote:
> On 4/14/2020 1:40 PM, Steven Schveighoffer wrote:
>> Also, this compiles just fine, and gives you the stack pointer value:
>>
>> size_t stackPtr() @system
>> {
>>    int i;
>>    return cast(size_t)&i;
>> }
> 
> Yup. I also gave workarounds to Adam's example.

Of course. There are workarounds to all these issues. The question then becomes, is the inconsistency for this particular restriction worth the annoyance it causes to work around it.

The answer is subjective, as always. IMO, fetching the stack pointer is something that just isn't ever a requirement for code. It's such a specialized need that the annoyance it causes is worth all the memory problems it prevents when people accidentally return stack references. Maybe that's just my code, maybe everyone else needs to fetch the stack pointer in their code. But I didn't think that was the case.

On the other hand, using heap data is pretty common in my code, and occasionally I might stick a reference to stack data in there. If I am doing it responsibly, there is no harm. The compiler might make me jump through some annoyance hoops, but it's possible to do.

So we have to draw a line, an arbitrary line, that says "everything like this is going to require some workaround, everything else, well it's @system code, you got what you signed up for". My line is that returning stack pointers, if provable, is more often than not an error, and I would like to be reminded of it from the compiler. Putting stack data into heap blocks, is already not consistently flagged, so I don't think it's saving much to have this annoyance available.

But if the compiler is going to be annoying, be annoying according to a plan, not just random annoyance.

>> I can do this too, apparently since it's a TLS array and not a heap array, totes ok!
>>
>> int*[1] arr;
>>
>> void bar()
>> {
>>      int i;
>>      arr[0] = &i;
>> }
> 
> Ironically, I argue for consistency with "safe by default" and you argue for special cases, and here the reverse.

There are different levels of consistency. If you have a reasonable rule, then be consistent in that rule. It might make you have to do some extra workarounds to use the language, but the result is that you should be a better programmer, and you have a rationale to point at for why things are the way they are.

I'll take for example the rule that if(cond); is disallowed. It's a wonderful rule, but totally not consistent with the language (I can write empty statements in most other places). BUT it's such a common mistake that it's worth the annoyance. It's caught bugs for me many many times.

However, what if while(cond); was allowed? That kind of inconsistency doesn't make sense. It's inconsistent to disallow empty statements for an if block, but allow them for while blocks. Arguing that people would want consistency with while statements but not if statements is hard to justify.

In the safe debate, the problem is simply that code that compiles today and is @system will compile tomorrow and be @safe, without ANY checking or any deprecation. This destroys everything that @safe has been built upon.

Ironically, what I'm asking for is consistency with existing code, and you are arguing to make an exception that extern(C) @system functions compile should silently convert to @safe functions without warning. We just cannot change the meaning of all code without warning people or causing an error.

If we didn't have existing code that's extern(C) then the debate would be entirely different. It would be a new thing, and it would be a legitimate position to have extern(C) prototypes safe by default.

I'll note that I proposed an alteration that has received zero responses that I think would solve all those concerns:

https://forum.dlang.org/post/r6kvm4$1vq5$1@digitalmars.com

> 
> But be careful, you may talk me into disallowing these cases, too :-/
> 

If you did, it would at LEAST be reasonable! At least the rule would be consistent. So I would be fine with it! As Adam says, there is no rhyme or reason for this one rule vs. the allowances for other code, so there's no predictability as to when D might somehow decide what you did was not correct.

Ruling by whim is not as understandable as providing clear and consistent rules to base changes on.

-Steve

On 14.04.20 22:10, Walter Bright wrote:
> On 4/14/2020 6:34 AM, Adam D. Ruppe wrote:
>> Indeed, but isn't that what @safe is for? There should be a way do it intentionally, to tell the compiler "trust me". That's my main point with this thread - D is being overbearing now.
>>
>> "Copy the pointer to that array please, D."
> 
> 
> So, Adam, how do you feel about:
> 
>    @system int* pumpkin(int i) { return &i);
> 
> Timon says that should be an error.

That's actually not what I said. I said I would not be opposed to it being an error.