On Monday, March 05, 2018 02:30:12 Walter Bright via Digitalmars-d wrote:
> The idea behind removal of the runtime checks is as a performance optimization done on a debugged program. It's like turning on or off array bounds checking. Many leave asserts and array bounds checking on even in released code to ensure memory safety.
>
> At a minimum, turning it off and on will illuminate just what the checks are costing you.
>
> It's at the option of the programmer.

Except that disabling array bounds checking negates the guarantees of @safe, and you have to go to the extra effort of supplying a compiler flag to tell the compiler that you don't care about the compiler guaranteeing @safe.

And your suggesting that with assertions, @safe's guarantees can be removed. You never catch everything when debugging or running your program prior to releasing it. Assertions can help considerably in catching problems, but there is zero guarantee that that they've caught all of the problems. They're disabled in release mode on the theory that the program has been tested enough that that assertions are unlikely to fail, but they could still have failed if they remained.

It's one thing to remove the assertions and let your program behave badly as a result whenever you didn't catch all of the problems. It's quite another to have the compiler make your code @safe code @system on the assumption that the assertion would have passed. Personally, I don't mind if the compiler is able to make optimization decisions based on assertions - the code is buggy anyway if the assertion would have failed - but I very much mind if those optimizations introduce behavior that could violate @safe. Because then, instead of just having a bug, you have memory corruption problem, which could be really hard to detect and track down, and it would be essentially invisible, because most programmers who understand @safe and @trusted, would assume that they needed to check the @system code and @trusted code for the problem, when it could now be sitting in @safe code just because one of the developers added an assertion to help catch a bug during testing. If assertions make it possible for @safe code to actually be @system, that seems like a serious bug to me.

IIRC, Andrei made a huge deal with you several years ago about how array bounds checking needed to be kept in @safe code or @safe meant nothing. I don't see how this situation is any different. @safe code needs to guaranteed to be @safe so long as @trusted is used correctly, otherwise, @safe is arguably meaningless - certainly, it then doesn't restrict the code that you have to look through for memory safety problems, and that's one of the main purposes of @safe.

If you want to introduce a compiler flag like -boundscheck=off that makes it so that compiling out assertions can introduce @system behavior, then fine. But removing assertions with -release and removing @safety guarantees are two completely separate issues. I sure don't want assertions in my production builds; I'd be forced to use them a lot less if they were going to be in my production builds. But I also don't want to introduce @safety issues into my production builds by using assertions.

You keep talking about how important @safety is to D and how much better it makes it than C, and then you go and talk about invisibly removing @safety guarantees with common, built-in features like assertions. If assertions removed @safety guaranatees, I would be forced to never use them in my code, which IMHO is ridiculous and is certainly going to make my code worse. And IMHO, suggesting that someone should then use a user-defined assertion isn't a good solution either, because then the built-in default that everyone is going to use unless someone more knowledgable tells them not to is an @safety problem.

If you want to be able to optimize based on assertions then fine, but please, please, please do not allow the compiler to introduce @safety problems when adding optimizations - preferably not even in @system code, because realistically, almost no one is going to have a clue what @safety problems could be introduced by an optimization based on an assertion, making it impossible for the programmer to verify that @system code can be marked @trusted. And most programmers won't know that assertions could even introduce optimizations that weren't memory safe.

I honestly don't see how we can claim that @safe code is guaranteed to be memory safe if assertions - or any other D feature other than the programmer using @trusted incorrectly - could introduce anything that is not guaranteed to be memory safe.

- Jonathan M Davis

On 3/6/2018 1:58 AM, Jonathan M Davis wrote:
> [...]

The entire point of making assert a core language feature was so that the compiler could take advantage of it to generate better code. It's been like that for D since day 1. It has always been documented to do that. It has been discussed many times in this n.g. over the years with loooong threads. I designed it that way so that D could potentially produce better code than other languages in ways they could not match.

There is no other purpose to making it a core language feature.

It's fine if you want an assert-like feature to have other semantics - just define one called 'check', give it the semantics you want, and put it in the library. As I mentioned to Timon, support for that sort of thing is why D has special support for __LINE__ and __FILE__.

Or just don't turn off assert checking. Personally, I use asserts in a way that they add little overhead so they can remain active in the release build.

It's entirely under your control.

On Tuesday, March 06, 2018 02:17:42 Walter Bright via Digitalmars-d wrote:
> On 3/6/2018 1:58 AM, Jonathan M Davis wrote:
> > [...]
>
> The entire point of making assert a core language feature was so that the compiler could take advantage of it to generate better code. It's been like that for D since day 1. It has always been documented to do that. It has been discussed many times in this n.g. over the years with loooong threads. I designed it that way so that D could potentially produce better code than other languages in ways they could not match.
>
> There is no other purpose to making it a core language feature.
>
> It's fine if you want an assert-like feature to have other semantics - just define one called 'check', give it the semantics you want, and put it in the library. As I mentioned to Timon, support for that sort of thing is why D has special support for __LINE__ and __FILE__.
>
> Or just don't turn off assert checking. Personally, I use asserts in a way that they add little overhead so they can remain active in the release build.
>
> It's entirely under your control.

I don't mind if assertions allow the compiler to better optimize. What I object to quite strongly is about the compiler being allowed to introduce optimizations that violate @safety guarantees. Built-in features should not result in @safe code invisibly becoming @system. That violates the entire purpose of @safe.

I don't see how this situation is any different from the array bounds checks, and those aren't compiled out in @safe code - even when assertions are compiled out.

- Jonathan M Davis

On Tuesday, 6 March 2018 at 02:05:58 UTC, Walter Bright wrote:
> On 3/5/2018 2:30 PM, John Colvin wrote:
>> This just feels bad. Adding extra failsafes for my debug program shouldn't make my release program less safe.
>
> Then use `enforce()`.

So, to clarify, adding asserts to my code makes my release builds violate @safe?

On Tuesday, 6 March 2018 at 16:30:09 UTC, John Colvin wrote:
> On Tuesday, 6 March 2018 at 02:05:58 UTC, Walter Bright wrote:
>> On 3/5/2018 2:30 PM, John Colvin wrote:
>>> This just feels bad. Adding extra failsafes for my debug program shouldn't make my release program less safe.
>>
>> Then use `enforce()`.
>
> So, to clarify, adding asserts to my code makes my release builds violate @safe?

Only if the assert does not hold, you have _not_ tested it, and a future optimiser will use the assert expression for some hints that generated broken code.

At the end, I'm with Walter: you should have tested the code with the assert enabled, and you should have noticed that the assert it's not holding _before_ running the code in release without them.

/Paolo

On Tuesday, March 06, 2018 17:01:04 Paolo Invernizzi via Digitalmars-d wrote:
> On Tuesday, 6 March 2018 at 16:30:09 UTC, John Colvin wrote:
> > On Tuesday, 6 March 2018 at 02:05:58 UTC, Walter Bright wrote:
> >> On 3/5/2018 2:30 PM, John Colvin wrote:
> >>> This just feels bad. Adding extra failsafes for my debug program shouldn't make my release program less safe.
> >>
> >> Then use `enforce()`.
> >
> > So, to clarify, adding asserts to my code makes my release builds violate @safe?
>
> Only if the assert does not hold, you have _not_ tested it, and a future optimiser will use the assert expression for some hints that generated broken code.
>
> At the end, I'm with Walter: you should have tested the code with the assert enabled, and you should have noticed that the assert it's not holding _before_ running the code in release without them.

The problem with this approach is that even if you have tested your code well, sometimes you miss things, and then you have bugs. And that's life, but with @safe, you're not supposed to then up with memory safety problems. That's why array bounds checking is not removed in @safe code even with -release. Yes, well-tested code would hopefully not need the checks anymore, but bugs still creep in, and if the array bounds checking is removed, then you can end up with memory corruption.

This issue is really no different from array bounds checking, and I don't see any reason to treat it any differently. @safe could should be guaranteed to be @safe so long as @trusted is used correctly. The compiler should not be assuming that the programmer has done a perfect job of testing and caught every possible bug and that there's no way that an assertion would have failed if it were still compiled in and that it is thus safe to introduce optimizations which would not be memory safe if the assertion would have failed. If the compiler does that, then @safe is being violated, which the compiler should never be doing.

- Jonathan M Davis

On 03/06/2018 06:01 PM, Paolo Invernizzi wrote:
> Only if the assert does not hold, you have _not_ tested it,

In other words: only if you have a bug in your code.

If @safe is only safe as long you don't have bugs, it's no different from @system. So -release turns @safe code into @system code, if it contains asserts.

> and a future optimiser will use the assert expression for some hints that generated broken code

On Tuesday, March 06, 2018 16:30:09 John Colvin via Digitalmars-d wrote:
> On Tuesday, 6 March 2018 at 02:05:58 UTC, Walter Bright wrote:
> > On 3/5/2018 2:30 PM, John Colvin wrote:
> >> This just feels bad. Adding extra failsafes for my debug program shouldn't make my release program less safe.
> >
> > Then use `enforce()`.
>
> So, to clarify, adding asserts to my code makes my release builds violate @safe?

If the compiler actually optimized based on assertions, then yes, but not right now. As I understand it, the problem is theoretical at the moment, because the compiler does not yet optimize based on assertions, but once it does, if it's allowed to introduce optimizations that would be not be memory safe if the assertion would have failed if it hadn't been compiled out, then @safe will be violated, and at that point, I would be telling everyone to never use assertions, because they're too dangerous.

If we can restrict the compiler to optimizations that are memory safe, then I don't see a problem, but clearly, Walter is not in agreement that the optimizations should be restricted in that manner.

- Jonathan M Davis

On Tuesday, 6 March 2018 at 17:24:35 UTC, Jonathan M Davis wrote:
> On Tuesday, March 06, 2018 16:30:09 John Colvin via Digitalmars-d wrote:
>> On Tuesday, 6 March 2018 at 02:05:58 UTC, Walter Bright wrote:
>> > On 3/5/2018 2:30 PM, John Colvin wrote:
>> >> This just feels bad. Adding extra failsafes for my debug program shouldn't make my release program less safe.
>> >
>> > Then use `enforce()`.
>>
>> So, to clarify, adding asserts to my code makes my release builds violate @safe?
>
> If the compiler actually optimized based on assertions, then yes, but not right now. As I understand it, the problem is theoretical at the moment, because the compiler does not yet optimize based on assertions, but once it does, if it's allowed to introduce optimizations that would be not be memory safe if the assertion would have failed if it hadn't been compiled out, then @safe will be violated, and at that point, I would be telling everyone to never use assertions, because they're too dangerous.
>
> If we can restrict the compiler to optimizations that are memory safe, then I don't see a problem, but clearly, Walter is not in agreement that the optimizations should be restricted in that manner.
>
> - Jonathan M Davis

I think a reasonable compromise is to introduce a new system attribute such as @unsafeoptimize to tell the programmer that this function may have it's @safe attribute removed when making optimizations based on the asserts. We have @trusted attribute for a good reason here.

On Tuesday, 6 March 2018 at 17:34:08 UTC, 12345swordy wrote:
> On Tuesday, 6 March 2018 at 17:24:35 UTC, Jonathan M Davis wrote:
>> On Tuesday, March 06, 2018 16:30:09 John Colvin via Digitalmars-d wrote:
>>> On Tuesday, 6 March 2018 at 02:05:58 UTC, Walter Bright wrote:
>>> > [...]
>>>
>>> So, to clarify, adding asserts to my code makes my release builds violate @safe?
>>
>> If the compiler actually optimized based on assertions, then yes, but not right now. As I understand it, the problem is theoretical at the moment, because the compiler does not yet optimize based on assertions, but once it does, if it's allowed to introduce optimizations that would be not be memory safe if the assertion would have failed if it hadn't been compiled out, then @safe will be violated, and at that point, I would be telling everyone to never use assertions, because they're too dangerous.
>>
>> If we can restrict the compiler to optimizations that are memory safe, then I don't see a problem, but clearly, Walter is not in agreement that the optimizations should be restricted in that manner.
>>
>> - Jonathan M Davis
>
> I think a reasonable compromise is to introduce a new system attribute such as @unsafeoptimize to tell the programmer that this function may have it's @safe attribute removed when making optimizations based on the asserts. We have @trusted attribute for a good reason here.

I simply don't understand why enforce or a custom check can't be used @safe code, if you want that behaviour.

If the program must HALT on this or that, well, what is better than an explicit piece of unremovable code that do that? Instead of writing 'assert', one should write 'enforce'.

/Paolo