June 08, 2022

On Wednesday, 8 June 2022 at 16:32:25 UTC, John Colvin wrote:

> >

There is no frame of reference in which this result is in any way reasonable.

My guess is that technically foo has undefined behaviour.

Sure, but that also mean it could format your hard drive, and it'd be hard to argue this is reasonable.

If the compiler understands of what's going on to decide it can recycle the memory, it understands enough to tell you you are using it after freeing and if it cannot, then it shouldn't do it.

In this case specifically, assuming the compiler see the memory doesn't escape and promoting [c] on stack, it should still do the right thing. That means the compiler is somehow getting out of its way to break the code.

That doesn't sound reasonable, no matter how you slice it.

June 08, 2022

On Wednesday, 8 June 2022 at 14:52:53 UTC, Steven Schveighoffer wrote:

>
string foo(in string s)
{
    return s;
}

void main()
{
    import std.stdio;
    string[] result;
    foreach(c; "hello")
    {
        result ~= foo([c]);
    }
    writeln(result);
}

This has nothing to do with -preview=in.
Change foo's signature to:

>

string foo(scope string s)

And you'll see the bug, even without -preview=dip1000.

Why is this happening ? You correctly guessed, because the frontend wrongfully lets the string go on the stack instead of allocating with it.

Some of the changes for DIP1000 made it to releases even without the switch, that's one example.

June 08, 2022

On Wednesday, 8 June 2022 at 17:09:49 UTC, Mathias LANG wrote:

>

And you'll see the bug, even without -preview=dip1000.

Why is this happening ? You correctly guessed, because the frontend wrongfully lets the string go on the stack instead of allocating with it.

Some of the changes for DIP1000 made it to releases even without the switch, that's one example.

No, promoting the array on stack is not sufficient to explain the behavior - thought it is certainly part of it.

The compiler is going out of his way in some other way to break the code.

June 08, 2022

On Wednesday, 8 June 2022 at 16:58:41 UTC, deadalnix wrote:

>

On Wednesday, 8 June 2022 at 16:32:25 UTC, John Colvin wrote:

> >

There is no frame of reference in which this result is in any way reasonable.

My guess is that technically foo has undefined behaviour.

Sure, but that also mean it could format your hard drive, and it'd be hard to argue this is reasonable.

If the compiler understands of what's going on to decide it can recycle the memory, it understands enough to tell you you are using it after freeing and if it cannot, then it shouldn't do it.

In this case specifically, assuming the compiler see the memory doesn't escape and promoting [c] on stack, it should still do the right thing. That means the compiler is somehow getting out of its way to break the code.

That doesn't sound reasonable, no matter how you slice it.

The compiler is going “you told me foo doesn’t leak references to the string passed to it, I believe you. Based on that, this temporary array is safe to put on the stack”. I think it’s reasonable for the compiler to lean on scope like this.

The problem is foo and whether the compiler should somehow prevent the inconsistency between the signature and implementation. Obviously the answer is “yes, ideally”, but in practice with @safe, @system, dip1000, @live and so on it’s all a mess.

June 08, 2022

On Wednesday, 8 June 2022 at 17:50:18 UTC, John Colvin wrote:

>

The compiler is going “you told me foo doesn’t leak references to the string passed to it, I believe you. Based on that, this temporary array is safe to put on the stack”. I think it’s reasonable for the compiler to lean on scope like this.

Never mind, result is a string[] so yes, it's somewhat expected.

June 08, 2022
On 6/8/22 19:22, deadalnix wrote:
> On Wednesday, 8 June 2022 at 17:09:49 UTC, Mathias LANG wrote:
>> And you'll see the bug, even without `-preview=dip1000`.
>>
>> Why is this happening ? You correctly guessed, because the frontend wrongfully lets the `string` go on the stack instead of allocating with it.
>> ...

Your code is literally calling this function:

```d
string foo(scope string s){ return s; }
```

This causes UB, therefore you can't blame the compiler frontend here. I guess you can complain about the language specification, but what else are you expecting `scope` to do? There could be some more diagnostics I guess, like for the case where a stack variable is escaped directly.


>> Some of the changes for DIP1000 made it to releases even without the switch, that's one example.
> 
> No, promoting the array on stack is not sufficient to explain the behavior - thought it is certainly part of it.
> 
> The compiler is going out of his way in some other way to break the code.

It's reusing the same location on the stack for all instances of `[c]`. I think that's a pretty complete and straightforward explanation of the behavior. What is missing?

Anyway, this kind of issue is why one should never rely on undefined behavior giving a specific result; the compiler may get smart about it later.
June 08, 2022
On Wednesday, 8 June 2022 at 18:32:41 UTC, Timon Gehr wrote:
> On 6/8/22 19:22, deadalnix wrote:
>> On Wednesday, 8 June 2022 at 17:09:49 UTC, Mathias LANG wrote:
>>> And you'll see the bug, even without `-preview=dip1000`.
>>>
>>> Why is this happening ? You correctly guessed, because the frontend wrongfully lets the `string` go on the stack instead of allocating with it.
>>> ...
>
> Your code is literally calling this function:
>
> ```d
> string foo(scope string s){ return s; }
> ```
>
> This causes UB, therefore you can't blame the compiler frontend here.

I got to say here, you shouldn't be able to compile that code at all if it is going to shoot you in the foot unintentionally.

- Alex
June 08, 2022
On 6/8/22 20:44, 12345swordy wrote:
> On Wednesday, 8 June 2022 at 18:32:41 UTC, Timon Gehr wrote:
>> On 6/8/22 19:22, deadalnix wrote:
>>> On Wednesday, 8 June 2022 at 17:09:49 UTC, Mathias LANG wrote:
>>>> And you'll see the bug, even without `-preview=dip1000`.
>>>>
>>>> Why is this happening ? You correctly guessed, because the frontend wrongfully lets the `string` go on the stack instead of allocating with it.
>>>> ...
>>
>> Your code is literally calling this function:
>>
>> ```d
>> string foo(scope string s){ return s; }
>> ```
>>
>> This causes UB, therefore you can't blame the compiler frontend here.
> 
> I got to say here, you shouldn't be able to compile that code at all if it is going to shoot you in the foot unintentionally.
> 
> - Alex

Well, I agree that in simple cases like this one, the compiler should just complain. In general though, it won't understand what's going on. If you want to catch everything, you'll have to use @safe, but that will also reject some things that are actually fine.
June 08, 2022

On 6/8/22 1:09 PM, Mathias LANG wrote:

>

On Wednesday, 8 June 2022 at 14:52:53 UTC, Steven Schveighoffer wrote:

>
string foo(in string s)
{
    return s;
}

void main()
{
    import std.stdio;
    string[] result;
    foreach(c; "hello")
    {
        result ~= foo([c]);
    }
    writeln(result);
}

This has nothing to do with -preview=in.
Change foo's signature to:

>

string foo(scope string s)

And you'll see the bug, even without -preview=dip1000.

Yes, it has been noted by John.

But for some reason, this specific code doesn't fail with -preview=dip1000 or -preview=in, but only when both are specified.

Apparently in under preview in doesn't really mean the same thing as scope const. So does it have nothing to do with preview in? simple experimentation says it does.

Note that scope arrays started being allocated on the stack in 2.092.0, coincidentally the same release that added -preview=in.

>

Why is this happening ? You correctly guessed, because the frontend wrongfully lets the string go on the stack instead of allocating with it.

Whether it's right or wrong, it's a change that silently introduces memory corruption. It ought to produce a warning for such code. I'm not blaming anybody here for behavior that is probably correct per the spec. But is there no mechanism to be had for warning about this? D already doesn't allow returning a pointer to stack data, even in @system code. Doesn't this also qualify?

-Steve

June 08, 2022
On Wednesday, 8 June 2022 at 18:44:28 UTC, 12345swordy wrote:
> On Wednesday, 8 June 2022 at 18:32:41 UTC, Timon Gehr wrote:
>> On 6/8/22 19:22, deadalnix wrote:
>>> On Wednesday, 8 June 2022 at 17:09:49 UTC, Mathias LANG wrote:
>>>> And you'll see the bug, even without `-preview=dip1000`.
>>>>
>>>> Why is this happening ? You correctly guessed, because the frontend wrongfully lets the `string` go on the stack instead of allocating with it.
>>>> ...
>>
>> Your code is literally calling this function:
>>
>> ```d
>> string foo(scope string s){ return s; }
>> ```
>>
>> This causes UB, therefore you can't blame the compiler frontend here.
>
> I got to say here, you shouldn't be able to compile that code at all if it is going to shoot you in the foot unintentionally.
>
> - Alex

I believe this is because foo is not annotated with @safe, thus it's @system by default and you're allowed to do all kinds of unsafe things. Mark it @safe and the compiler will correctly complain:

```
@safe
string foo(in string s)
{
    return s; // Error: scope variable `s` may not be returned
}

void main()
{
    import std.stdio;
    string[] result;
    foreach(c; "hello")
    {
        result ~= foo([c]);
    }
    writeln(result);
}
```

In addition, changing `in` to `const return scope` makes the compiler aware that you intend to return the value, and thus it seems to somehow know not to re-use that stack space, and correctly prints ["h", "e", "l", "l", "o"].