Menu
Search

Forums

Forum Index

June 11, 2022
Gravatar of Timon GehrPosted by Timon Gehr
in reply to Walter Bright		Reply
Timon Gehr
Gravatar of Timon Gehr
Posted in reply to Walter Bright


Reply
On 6/10/22 05:15, Walter Bright wrote:
> On 6/9/2022 5:58 AM, Timon Gehr wrote:
>> But if the compiler can easily tell that something makes no sense, it should still be an error in both `@safe` and `@system` code!
> 
> Sometimes it makes sense for a function to return the address of a local. For example, if you want to detect how large the stack has gotten. I use this in, for example, the garbage collector to see how much stack needs to be scanned. It can also be used to "step" on the stack after a function returns, as one might want to do for security software.
> 
> I've also done things like write 0xDEADBEEF all over memory in order to flush out memory bugs. This involves using pointers in UB ways that don't make sense as far as the language is concerned.
> 
> In @safe code it is nonsense to write specific numbers into a pointer. But in @system code, it does make sense.
> 
> I don't think one could write a symbolic debugger with @safe code. Like writing instruction bytes into a buffer, and then calling it? How unsafe can one get? :-)
> 
> And so on.

Well, you can do that knowing what the backend will or will not do with code that the spec says could mean anything at all. Others may be a bit less privileged. ;) If it's needed, I think it's better to have explicit support for such use cases, not involving any UB.
June 13, 2022
Gravatar of deadalnixPosted by deadalnix
in reply to John Colvin		Reply
deadalnix
Gravatar of deadalnix
Posted in reply to John Colvin


Reply

On Wednesday, 8 June 2022 at 17:50:18 UTC, John Colvin wrote:

>

The compiler is going “you told me foo doesn’t leak references to the string passed to it, I believe you. Based on that, this temporary array is safe to put on the stack”. I think it’s reasonable for the compiler to lean on scope like this.

The problem is foo and whether the compiler should somehow prevent the inconsistency between the signature and implementation. Obviously the answer is “yes, ideally”, but in practice with @safe, @system, dip1000, @live and so on it’s all a mess.

So I gave it some time, and I think I am now convinced that doing this optimization is simply not a good idea.

If the value stays on stack - which is all that DIP1000 can check for anyways, then modern backend can track it. LLVM for instance, will annotate function parameter to indicate if they escape or not and do so recursively through the callgraph.

LDC is already able to do stack promotion when escape analysis proves something doesn't escape.

This is WAY preferable because:

  • It works regardless of annotations from the dev.
  • It is always correct, it will not fubar existing @system code.
  • Inlining is likely to uncover more opportunity to do this, there is no point doing it before.

Doing this type of optimization to explicitly free elements on heap is worth it. But DIP1000 doesn't allow to track this reliably.

June 13, 2022
Gravatar of John ColvinPosted by John Colvin
in reply to deadalnix		Reply
John Colvin
Gravatar of John Colvin
Posted in reply to deadalnix


Reply

On Monday, 13 June 2022 at 11:14:36 UTC, deadalnix wrote:

>

On Wednesday, 8 June 2022 at 17:50:18 UTC, John Colvin wrote:

>

The compiler is going “you told me foo doesn’t leak references to the string passed to it, I believe you. Based on that, this temporary array is safe to put on the stack”. I think it’s reasonable for the compiler to lean on scope like this.

The problem is foo and whether the compiler should somehow prevent the inconsistency between the signature and implementation. Obviously the answer is “yes, ideally”, but in practice with @safe, @system, dip1000, @live and so on it’s all a mess.

So I gave it some time, and I think I am now convinced that doing this optimization is simply not a good idea.

If the value stays on stack - which is all that DIP1000 can check for anyways, then modern backend can track it. LLVM for instance, will annotate function parameter to indicate if they escape or not and do so recursively through the callgraph.

LDC is already able to do stack promotion when escape analysis proves something doesn't escape.

This is WAY preferable because:

  • It works regardless of annotations from the dev.
  • It is always correct, it will not fubar existing @system code.
  • Inlining is likely to uncover more opportunity to do this, there is no point doing it before.

Doing this type of optimization to explicitly free elements on heap is worth it. But DIP1000 doesn't allow to track this reliably.

Without expressing an opinion either way, I want to note that this has implications for @nogc. If you remove scope from the parameter of foo, the compiler won't let main be @nogc due to the slice literal allocation.

string foo(scope string s) @nogc {
    return s;
}

void main() @nogc {
    foo(['a']);
}
June 13, 2022
Gravatar of deadalnixPosted by deadalnix
in reply to John Colvin		Reply
deadalnix
Gravatar of deadalnix
Posted in reply to John Colvin


Reply

On Monday, 13 June 2022 at 12:02:31 UTC, John Colvin wrote:

>

Without expressing an opinion either way, I want to note that this has implications for @nogc. If you remove scope from the parameter of foo, the compiler won't let main be @nogc due to the slice literal allocation.

string foo(scope string s) @nogc {
    return s;
}

void main() @nogc {
    foo(['a']);
}

It's as if @nogc should track leaks and not allocations...

June 13, 2022
Gravatar of Nick TreleavenPosted by Nick Treleaven
in reply to deadalnix		Reply
Nick Treleaven
Gravatar of Nick Treleaven
Posted in reply to deadalnix


Reply

On Monday, 13 June 2022 at 11:14:36 UTC, deadalnix wrote:

>

If the value stays on stack - which is all that DIP1000 can check for anyways, then modern backend can track it. LLVM for instance, will annotate function parameter to indicate if they escape or not and do so recursively through the callgraph.

LLVM might not have the library source code.

>

LDC is already able to do stack promotion when escape analysis proves something doesn't escape.

This is WAY preferable because:

  • It works regardless of annotations from the dev.

So does scope inference (when it's done).

>
  • It is always correct, it will not fubar existing @system code.

I'm not sure why someone would have written scope in @system code unless they didn't understand it's only checked in @safe code. The stack promotion optimization is the only reason to use it in @system code AIUI.

>
  • Inlining is likely to uncover more opportunity to do this, there is no point doing it before.

Then the LLVM optimization and the scope optimisation are both needed for the best code in all cases.

>

Doing this type of optimization to explicitly free elements on heap is worth it. But DIP1000 doesn't allow to track this reliably.

Please can you provide an example? (There's one Timon came up with in your scope thread recently, Walter's working on that).

June 13, 2022
Gravatar of Steven SchveighofferPosted by Steven Schveighoffer
in reply to deadalnix		Reply
Steven Schveighoffer
Gravatar of Steven Schveighoffer
Posted in reply to deadalnix


Reply

On 6/13/22 9:25 AM, deadalnix wrote:

>

On Monday, 13 June 2022 at 12:02:31 UTC, John Colvin wrote:

>

Without expressing an opinion either way, I want to note that this has implications for @nogc. If you remove scope from the parameter of foo, the compiler won't let main be @nogc due to the slice literal allocation.

    string foo(scope string s) @nogc {
        return s;
    }

    void main() @nogc {
        foo(['a']);
    }

It's as if @nogc should track leaks and not allocations...

Wouldn't that be a leak if ['a'] was GC allocated?

-Steve

June 13, 2022
Gravatar of Walter BrightPosted by Walter Bright
in reply to Steven Schveighoffer		Reply
Walter Bright
Gravatar of Walter Bright
Posted in reply to Steven Schveighoffer


Reply
On 6/10/2022 5:46 AM, Steven Schveighoffer wrote:
> I'm trying to parse this. You mean to say, there are enough unmarked extern(C) functions inside druntime, that fixing them all *as they come up* is not scalable? That seems unlikely. Note that modules like core.stdc.math has @trusted: at the top already.

It is not scalable for the reasons mentioned. Nobody has ever gone through windows.d to see what to mark things as, and nobody ever will.


> I will volunteer to mark any druntime extern(C) functions within a 2-day turnaround if they are posted on bugzilla and assigned to me. Start with @system: at the top, and mark them as the errors occur.

They aren't even done *now*, after 15+ years. See windows.d and all the the others.


> If you think the "no-edits" bar is only cleared if extern(C) functions are assumed @safe, you are 100% wrong.

I'm not seeing how I am wrong.


> You can point the complaints about extern(C) functions at my ear, and deal with the significant majority of complaints that are about @safe by default D code.

That's a very nice offer, but that won't change that I get the complaints and people want me to fix it, not brush it off on you.


> I would love to see a viable safe-by-default DIP get added.

At least we can agree on that!
June 13, 2022
Gravatar of Walter BrightPosted by Walter Bright
in reply to Timon Gehr		Reply
Walter Bright
Gravatar of Walter Bright
Posted in reply to Timon Gehr


Reply
On 6/10/2022 4:59 PM, Timon Gehr wrote:
> On 6/10/22 05:07, Walter Bright wrote:
>> This entire thread is what happens with "not at all".
> Well, I contest that. It's not even closely related. This is an issue concerning `@system` code. What's the default has not much at all to do with it.

Steven is compiling ordinary code with the default, when there is no obvious reason why it should be system code.

@system code should be relatively rare.
June 13, 2022
Gravatar of Walter BrightPosted by Walter Bright
in reply to John Colvin		Reply
Walter Bright
Gravatar of Walter Bright
Posted in reply to John Colvin


Reply
On 6/9/2022 5:44 AM, John Colvin wrote:
> That is their fault

If the language encourages it, it's the language's fault.
June 13, 2022
Gravatar of Walter BrightPosted by Walter Bright
in reply to deadalnix		Reply
Walter Bright
Gravatar of Walter Bright
Posted in reply to deadalnix


Reply
On 6/13/2022 4:14 AM, deadalnix wrote:
> On Wednesday, 8 June 2022 at 17:50:18 UTC, John Colvin wrote:
>> The compiler is going “you told me `foo` doesn’t leak references to the string passed to it, I believe you. Based on that, this temporary array is safe to put on the stack”. I think it’s reasonable for the compiler to lean on `scope` like this.
>>
>> The problem is `foo` and whether the compiler should somehow prevent the inconsistency between the signature and implementation. Obviously the answer is “yes, ideally”, but in practice with @safe, @system, dip1000, @live and so on it’s all a mess.
> 
> So I gave it some time, and I think I am now convinced that doing this optimization is simply not a good idea.
> 
> If the value stays on stack - which is all that DIP1000 can check for anyways, then modern backend can track it. LLVM for instance, will annotate function parameter to indicate if they escape or not and do so recursively through the callgraph.
> 
> LDC is already able to do stack promotion when escape analysis proves something doesn't escape.
> 
> This is WAY preferable because:
>   - It works regardless of annotations from the dev.
>   - It is always correct, it will not fubar existing @system code.
>   - Inlining is likely to uncover more opportunity to do this, there is no point doing it before.
> 
> Doing this type of optimization to explicitly free elements on heap is worth it.

The D compiler *does* keep track of this when it does attribute inference. Attribute inference is not currently done for regular functions because of the risk of mismatch between a function declaration and a function definition that may arise because of inference.

> But DIP1000 doesn't allow to track this reliably.

Yes, it does. Any instances where it doesn't are a bug and are fixed.
« First   ‹ Prev
1 2 3 4 5 6 7 8 9 10 11 12 13 14
Next ›   Last »