On Thursday, March 01, 2018 16:01:08 Steven Schveighoffer via Digitalmars-d wrote:
> On 3/1/18 3:24 PM, ag0aep6g wrote:
> > On Thursday, 1 March 2018 at 20:14:07 UTC, Steven Schveighoffer wrote:
> >> dmd -version=dosafe -noboundscheck -run testarrayoverlap.d => no
> >> error, undefined behavior
> >> dmd -run testarrayoverlap.d => error
> >>
> >> @safe has nothing to do with it.
> >
> > @safe has everything to do with. @safe guarantees that there's no undefined behavior. If you can trigger UB in @safe code, that's a bug. The guarantee must hold whether asserts are enabled or not. -noboundscheck does not just disable asserts, it undermines @safe, rendering it meaningless.
>
> Yeah, it seems like -noboundscheck should never be used.

IMHO, it was a mistake to ever add it.

> > For just disabling asserts (without affecting @safe), there's -release. With your same code, `dmd -release` => UB, but `dmd -release -version=dosafe` => Error.
>
> I was unaware that it would work this way. I thought it would be the same whether you used @safe or not with -release.

That depends on the feature. It's true for array bounds checking, but it's not true for stuff like final switch statements - or apparently, this situation. I don't know if that should be changed or not. Probably.

- Jonathan M Davis

On Thursday, 1 March 2018 at 21:01:08 UTC, Steven Schveighoffer wrote:
> Yeah, it seems like -noboundscheck should never be used.

Agreed.

> It's undefined behavior if the check is disabled. How you get the check disabled may be affected by @safe, but whether it's UB or not has nothing to do with @safe. It has to do with "I turned off the checks". Basically, in D, if your program *would have* thrown an Error, but didn't because you turned it off, the compiler is free to assume UB.

You're looking at the behavior of the compiled executable. Then it makes sense to say that a program compiled with the checks has defined behavior (throwing Errors) and a program without the checks does something undefined (because the compiler manual doesn't say that anything in particular is supposed to happen).

That's not how I understand/use "undefined behavior". To me, it's the D source code that can have "undefined behavior". When it does, the program (as given in source code) is invalid and the compiler can do whatever it wants with it. It may:

1) reject the program statically, or it may
2) generate code that throws an Error at run time, or it may
3) generate code that does something arbitrary.

With that meaning of UB, out-of-bounds accesses and overlapping copies have undefined behavior in @system code, but they have defined behavior in @safe code.

For @system code, DMD chooses option #2 or option #3 depending on whether you give -release or not. DMD is free to do this, because the source code has UB.

For @safe code, DMD goes with option #2 either way, because the spec demands it.

If it were capable, DMD could also go with option #1 for both @system and @safe code. The spec allows/encourages that.

In this interpetation, -noboundscheck switches DMD to a different dialect of D. In that dialect, out-of-bounds accesses (and overlapping copies, apparently) always have UB, in both @system and @safe code. That defeats the purpose of @safe. Which is why I don't really care for that dialect.

On Thursday, 1 March 2018 at 21:01:08 UTC, Steven Schveighoffer wrote:
> Yeah, it seems like -noboundscheck should never be used.

How good is DMD at omitting redundant bounds checks? I assume not much engineering effort has been put towards that due to "-boundscheck=off" being available.

I do not know, but from my experience it is good at it. I have done many benchmarks for plenty of code, and in recent D compilers -boundscheck=off does not improve speed. To be fair using -boundscheck=off make D code slower in many cases, which is wierd but true.

On Fri, Mar 2, 2018 at 8:48 AM, Nathan S. via Digitalmars-d < digitalmars-d@puremagic.com> wrote:

> On Thursday, 1 March 2018 at 21:01:08 UTC, Steven Schveighoffer wrote:
>
>> Yeah, it seems like -noboundscheck should never be used.
>>
>
> How good is DMD at omitting redundant bounds checks? I assume not much engineering effort has been put towards that due to "-boundscheck=off" being available.
>

On 3/2/18 3:36 AM, Daniel Kozak wrote:
> I do not know, but from my experience it is good at it. I have done many benchmarks for plenty of code, and in recent D compilers -boundscheck=off does not improve speed. To be fair using -boundscheck=off make D code slower in many cases, which is wierd but true.

If you are doing -release, then -boundscheck=off is redundant except for @safe code.

-Steve

On 3/1/18 5:27 PM, ag0aep6g wrote:

> You're looking at the behavior of the compiled executable. Then it makes sense to say that a program compiled with the checks has defined behavior (throwing Errors) and a program without the checks does something undefined (because the compiler manual doesn't say that anything in particular is supposed to happen).

No, I'm looking at the source code.

At the very basic level, you have this:

assert(foo == 0);

Or whatever other condition you have. What this does is gives the compiler leeway to ASSUME foo is 0 at this point. It can make any number of optimizations assuming this. If foo is NOT equal to 0 at this point, then it is a program error, and the assumptions may cause bad things to happen.

When you compile in normal mode, this assert causes an Error to be thrown, before your code can do any damage.

When you compile without asserts, this causes undefined behavior.

The various switches and modes can say whether asserts or other checks are present or not, and if they are not, the compiler is going to execute your UB code with those wrong assumptions. But ONLY if the assert would have been false.

I extend this same treatment and logic to all checks: contracts, bounds checks, overlapping slice assign checks, etc. To me, they aren't any different than asserts. They aren't any different if they are in @safe code or @system code.

> That's not how I understand/use "undefined behavior". To me, it's the D source code that can have "undefined behavior". When it does, the program (as given in source code) is invalid and the compiler can do whatever it wants with it. It may:
> 
> 1) reject the program statically, or it may
> 2) generate code that throws an Error at run time, or it may
> 3) generate code that does something arbitrary.
> 
> With that meaning of UB, out-of-bounds accesses and overlapping copies have undefined behavior in @system code, but they have defined behavior in @safe code.

By default, both @system and @safe code have bounds checks. So by default, it is defined behavior (an Error is thrown) regardless of the safety.

If you turn the checks off (through any means) then it becomes UB.

> In this interpetation, -noboundscheck switches DMD to a different dialect of D. In that dialect, out-of-bounds accesses (and overlapping copies, apparently) always have UB, in both @system and @safe code. That defeats the purpose of @safe. Which is why I don't really care for that dialect.

I agree, I think we should remove the option to disable bounds checks on @safe code, in any way. It's too dangerous. If you want performance that comes without bounds checks, use a trusted escape, or write system code.

-Steve

On 02.03.2018 15:39, Steven Schveighoffer wrote:
> 
> 
>> In this interpetation, -noboundscheck switches DMD to a different dialect of D. In that dialect, out-of-bounds accesses (and overlapping copies, apparently) always have UB, in both @system and @safe code. That defeats the purpose of @safe. Which is why I don't really care for that dialect.
> 
> I agree, I think we should remove the option to disable bounds checks on @safe code, in any way. It's too dangerous. If you want performance that comes without bounds checks, use a trusted escape, or write system code.

I.e., the -release flag should not remove assertions in @safe code, or at the very least it should not turn them into sources of UB.

On 3/2/18 10:00 AM, Timon Gehr wrote:
> On 02.03.2018 15:39, Steven Schveighoffer wrote:
>>
>>
>>> In this interpetation, -noboundscheck switches DMD to a different dialect of D. In that dialect, out-of-bounds accesses (and overlapping copies, apparently) always have UB, in both @system and @safe code. That defeats the purpose of @safe. Which is why I don't really care for that dialect.
>>
>> I agree, I think we should remove the option to disable bounds checks on @safe code, in any way. It's too dangerous. If you want performance that comes without bounds checks, use a trusted escape, or write system code.
> 
> I.e., the -release flag should not remove assertions in @safe code, or at the very least it should not turn them into sources of UB.

-release flag already operates this way. It's the -noboundscheck or -boundscheck=off that causes problems.

-Steve

On 02.03.2018 16:05, Steven Schveighoffer wrote:
> On 3/2/18 10:00 AM, Timon Gehr wrote:
>> On 02.03.2018 15:39, Steven Schveighoffer wrote:
>>>
>>>
>>>> In this interpetation, -noboundscheck switches DMD to a different dialect of D. In that dialect, out-of-bounds accesses (and overlapping copies, apparently) always have UB, in both @system and @safe code. That defeats the purpose of @safe. Which is why I don't really care for that dialect.
>>>
>>> I agree, I think we should remove the option to disable bounds checks on @safe code, in any way. It's too dangerous. If you want performance that comes without bounds checks, use a trusted escape, or write system code.
>>
>> I.e., the -release flag should not remove assertions in @safe code, or at the very least it should not turn them into sources of UB.
> 
> -release flag already operates this way.

That's not what the spec says:

https://dlang.org/dmd-linux.html#switch-release

"Compile release version, which means not emitting run-time checks for contracts and asserts. Array bounds checking is not done for system and trusted functions, and assertion failures are undefined behaviour."

Note that this is the only way to stop checking assertions. There is no option to just ignore them.

> It's the -noboundscheck or -boundscheck=off that causes problems.
> 
> -Steve

At least the specification advises to use caution:

"off: Bounds checks are disabled completely (even in @safe code). This option should be used with caution and as a last resort to improve performance. Confirm turning off @safe bounds checks is worthwhile by benchmarking."

On 03/02/2018 03:39 PM, Steven Schveighoffer wrote:
> On 3/1/18 5:27 PM, ag0aep6g wrote:
[...]
> No, I'm looking at the source code.
> 
> At the very basic level, you have this:
> 
> assert(foo == 0);
> 
> Or whatever other condition you have. What this does is gives the compiler leeway to ASSUME foo is 0 at this point. It can make any number of optimizations assuming this. If foo is NOT equal to 0 at this point, then it is a program error, and the assumptions may cause bad things to happen.
> 
> When you compile in normal mode, this assert causes an Error to be thrown, before your code can do any damage.
> 
> When you compile without asserts, this causes undefined behavior.

Now you're looking beyond the source code. There's no "normal mode" or "without asserts mode" in D source code. What you've got in the D code is @safe and @system.

Looking at it your way makes sense when the question is how the actual generated executable will behave. But I think it's also valuable to look at the language in isolation, without compiler settings.

I'd put it this way:

1) @system code may have UB. For example, out-of-bounds accesses have UB in @system code.

2) A compiler may give guarantees on what it does with specific instances of UB. For example, DMD guarantees that it throws an Error on out-of-bounds accesses, unless you compile with -release. This makes it effectively implementation defined behavior. You can rely on this. But when considering the source code in isolation, it still has UB.

3) @safe code cannot have UB. This is a strong guarantee. This cannot be affected by compiler switches without violating the spec.

[...]
> I extend this same treatment and logic to all checks: contracts, bounds checks, overlapping slice assign checks, etc. To me, they aren't any different than asserts. They aren't any different if they are in @safe code or @system code.

I don't agree with the @safe/@system part. @safe code must not exhibit UB. That's the point of the attribute.

Checks that would fail may cause UB in @system code, but they can't be allowed to cause UB in @safe code. Because nothing can be allowed to cause UB in @safe code.

[...]
> I agree, I think we should remove the option to disable bounds checks on @safe code, in any way. It's too dangerous. If you want performance that comes without bounds checks, use a trusted escape, or write system code.

I'm not going to argue against that, obviously.