On Wednesday, 21 June 2017 at 18:04:07 UTC, Moritz Maxeiner wrote:
> On Wednesday, 21 June 2017 at 17:55:05 UTC, MysticZach wrote:
>> Question: If `assert` itself allowed a user-defined hook, what would the remaining justification be for decoupling `in` and `out` contracts from the `assert` logic?
>
> Because then you won't have normal asserts and contracts be subject to different semantics?
> If I use a library, I may very well want to disable the library's internal assert checks (because I have enough experience that it's working properly), but keep it's contracts alive, because my code is still shiny new and riddled with bugs.

Timon appears to think that the checking logic is more monolithic. From his reply above [1]:

"If [an alternative checking system is utilized], there should be a way to hook into the checking logic. This has nothing at all to do with contract syntax. asserts and contracts are coupled already, as in-contracts form a disjunction on override by catching AssertErrors."

So I'm hoping more people will weigh in on this issue. For example, how easy is it to separate a library's internal contracts from its external ones? Would you have to write internal contracts in a different way from the ones facing the user? How often is this distinction the one causing problems with productivity?

The way I'm thinking about it would be that if there were a pragma to turn off contracts in a particularly hot code path, then the rest of the program could remain safe, while the fast part was allowed to go as fast as possible, addressing the performance issue.

But regarding the information issue, what kind of error information is better delivered specifically through compiler knowledge of `in` and `out` contracts, versus what it would deliver in the same way via regular `assert`s? Or are all contracts basically just fancy sugar for asserts at the beginning and end of a function body? What can the compiler do with the extra information? What can it say to the user that the user wouldn't already be able to figure out if it were a regular assert?

[1] http://forum.dlang.org/post/oie2nt$emf$1@digitalmars.com

On Wed, Jun 21, 2017 at 07:18:18PM +0000, MysticZach via Digitalmars-d wrote:
> [...] Or are all contracts basically just fancy sugar for asserts at the beginning and end of a function body?
[...]

This is a sticky point about D's current DbC implementation that myself and several others feel is a design flaw. In particular, that in-contracts are executed as part of the *callee*, when the intent of DbC is really that it is the obligation of the *caller* to fulfill its stipulations, and therefore the contract verification should happen at *caller* site rather than at the beginning of the callee.

This particular implementation detail causes problems with binary-only libraries: most library vendors would prefer to ship the library compiled with -release rather than not, but in -release, the asserts in any in-contracts would be elided, making them essentially non-existent by the user uses the library.  So you either have to dispense with DbC altogether, or be forced to ship two versions of your library, one with contracts compiled in and one without, in order for your users to benefit from DbC *and* not have to suffer performance penalties in their own release builds.

Had in-contracts been implemented on the caller's side instead, this would no longer be a problem: the contracts will still be part of the library API, so the user can benefit from them when not compiling with -release, but now the library itself can be shipped only with the binaries compiled with -release for best performance.

This is probably something outside the scope of this DIP, however.


T

-- 
Political correctness: socially-sanctioned hypocrisy.

On 21.06.2017 19:39, MysticZach wrote:
> On Wednesday, 21 June 2017 at 15:18:21 UTC, Timon Gehr wrote:
>> On 21.06.2017 02:51, MysticZach wrote:
>>>
>>> I think people could get used to the cognitive dissonance.
>>
>> That's really not what D is about.
> 
> My counterargument to that is that it's possible that the cognitive dissonance only occurs because of what people are used to, rather than what is inherent to the syntax.
> ...

This is a purely philosophical distinction without empirical basis.

>>> I've already gotten used to it just by writing this DIP.
>>
>> I think it is likely that you are an outlier.
> 
> Well my impression was that Walter liked it too, although I could have misinterpreted his comment here:
> 
> http://forum.dlang.org/post/ogvt66$1bcp$1@digitalmars.com
> ...

He is saying it is good that a DIP to improve contract syntax /exists/. I agree with that.

>>> If such an alternative checking system is utilized,
>>
>> If so, there should be a way to hook into the checking logic. This has nothing at all to do with contract syntax. asserts and contracts are coupled already, as in-contracts form a disjunction on override by catching AssertErrors.
> 
> Improving the checking logic interface may solve at a higher level the problem I'm trying to solve at a very low one, with mere syntax changes, and it might be (is probably?) the best way forward.
> ...

Your proposal does not solve this problem, and there is no need for this DIP to do that.

>>> the syntax for  writing contracts should be as easy
>>> for them as for those using `assert`.
>>
>> Maybe, but your DIP does not pull its own weight as long as the latter syntax is not a notable improvement over what we have now.
> 
> Well, my view is that my DIP is actually a very lightweight syntax change, and an equally lightweight improvement in contract syntax, so it's a lost-cost, mild improvement .

We are looking for a significant improvement. Otherwise, what's the point? We need to justify the cost.

> The cognitive dissonance argument 

(Just to be clear: I think framing it as a psychological issue does not have much merit.)

> is the main one against it, and I really don't know if that dissonance is based on fundamental flaws in the way I'm thinking about it,

The point of contracts is assigning blame by documenting assumptions and guarantees. If something within the function body crashes, it's ideally the fault of the function implementation.

> or just the "Shock of the New" [1].

I like new as long as it is an improvement. This is not. Having syntax subtrees that do not actually logically belong to their parent in the grammar is awkward language design, especially if they affect the parent's signature.

> ...
> As far as Teoh's proposal, I would say that its quality is highly correlated to the percentage of projects that find built-in `assert` adequate to their needs,

The two issues might need to be decoupled, but that is not the job of the contract syntax overhaul.

> which is hard to assess precisely - the better `assert` is, or can be made to be, the better Teoh's proposal is, I'd say.

Projects that want to do fancy things within contracts can use the existing contract syntax, but all they can legitimately do is throw AssertErrors, possibly with a message, just as assert does; the new syntax does not change this fact.

(BTW: There should be an optional message, as in: in(test(), "test failed").)

On Thursday, 22 June 2017 at 00:27:38 UTC, Timon Gehr wrote:
> On 21.06.2017 19:39, MysticZach wrote:
>> My counterargument to that is that it's possible that the cognitive dissonance only occurs because of what people are used to, rather than what is inherent to the syntax.
>
> This is a purely philosophical distinction without empirical basis.

Well I never experienced the dissonance myself, and was surprised to find that others did experience it. I just thought my proposal was a better syntax for `in` and `out` contracts.

>> Well my impression was that Walter liked it too, although I could have misinterpreted his comment here:
>> 
>> http://forum.dlang.org/post/ogvt66$1bcp$1@digitalmars.com
>
> He is saying it is good that a DIP to improve contract syntax /exists/. I agree with that.

It's really not clear what he meant. I guessed that the highest probability of what he meant was that he actually liked the DIP, as opposed to the mere fact that it _was_ a DIP. But as I already said, I could have misinterpreted his comment. In the face of lack of knowledge, it's a sign of wisdom to admit that one might be wrong, I think.

>>>> If such an alternative checking system is utilized,
>>>
>>> there should be a way to hook into the checking logic.
>>> 
>> Improving the checking logic interface may solve at a higher level the problem I'm trying to solve at a very low one, with mere syntax changes, and it might be (is probably?) the best way forward.
>
> Your proposal does not solve this problem, and there is no need for this DIP to do that.

The goal of this DIP, and the problem I'm trying to solve, is "Improve Contract Usability." And for this, H.S. Teoh's proposal is a very good one. But it still has a sticking point, which I want to resolve, namely that it elevates the existing `assert` functionality beyond the current requirement that one must explicitly write `assert`, going so far as to imply it in the grammar itself. But because of many complaints about the limitations of the assert mechanism as it currently exists, I would hesitate to install it into the grammar as the "One Chosen Way" to bail out of contracts with the new syntax.

However, if the functioning of the `assert` mechanism were more customizable, then it would be easier to entrust it, in my opinion, with the "sacred responsibility" of being installed into the grammar. H.S. Teoh's proposal would then stand on a firmer foundation, and my initial proposed syntax would become the inferior optionl. At that point, this DIP could be rewritten to advocate his syntax instead. (I think it would be better to just retain the number and title "DIP1009: Improve Contract Usability" than to make a new one for the same issue. Other DIPs have followed this pattern, where it was the goal and title that remained the same, while the specifics changed.)

> We are looking for a significant improvement. Otherwise, what's the point? We need to justify the cost.

The intent of my proposal was to make a small improvement. The cost (or so I thought, and may still believe) was also small. Small improvements are still improvements. DIP1003 is an example of this.

https://github.com/dlang/DIPs/blob/master/DIPs/DIP1003.md

> The point of contracts is assigning blame by documenting assumptions and guarantees. If something within the function body crashes, it's ideally the fault of the function implementation.
> ...
> I like new as long as it is an improvement. This is not. Having syntax subtrees that do not actually logically belong to their parent in the grammar is awkward language design, especially if they affect the parent's signature.

I'm still wondering what, in practice, the difference really is. With existing syntax:

int fun(int a)
in { assert(a); } // 1
do {
    assert(a); // 2
    ...
}

What will the compiler or programmer actually learn from 1 if it violates that they won't learn from 2 if it violates? What is the practical incentive for `in` contracts at all? All my new syntax does is assume that there is in fact a difference between 1 and 2, and makes 1 easier to write, as:

int fun(int a) {
    in assert(a); // 1
    assert(a); // 2
    ...
}

As far as syntax subtrees not belonging to their parent, I can see where the cognitive dissonance comes from. But it just doesn't seem that bad to me, since contracts are always executed as if they are sequential statements anyway. I would imagine that new programmers who only ever encountered the new proposed syntax would be surprised that the old syntax ever existed in the first place, as it's so unnecessarily awkward.

But at this point, we might as well wait for more feedback from other people.

On Wednesday, 21 June 2017 at 19:34:53 UTC, H. S. Teoh wrote:
> This is a sticky point about D's current DbC implementation that myself and several others feel is a design flaw. In particular, that in-contracts are executed as part of the *callee*, when the intent of DbC is really that it is the obligation of the *caller* to fulfill its stipulations, and therefore the contract verification should happen at *caller* site rather than at the beginning of the callee.
>
> This particular implementation detail causes problems with binary-only libraries: most library vendors would prefer to ship the library compiled with -release rather than not, but in -release, the asserts in any in-contracts would be elided, making them essentially non-existent by the user uses the library.  So you either have to dispense with DbC altogether, or be forced to ship two versions of your library, one with contracts compiled in and one without, in order for your users to benefit from DbC *and* not have to suffer performance penalties in their own release builds.
>
> Had in-contracts been implemented on the caller's side instead, this would no longer be a problem: the contracts will still be part of the library API, so the user can benefit from them when not compiling with -release, but now the library itself can be shipped only with the binaries compiled with -release for best performance.
>
> This is probably something outside the scope of this DIP, however.

It's related. A design flaw in D's DbC means that contracts on the whole are less important. Which means, unfortunately, that improving them is less important. Which could affect the final decision. But on the other side, binary-only libraries published in release mode are actually rare in D, right?

On Thu, Jun 22, 2017 at 05:46:06AM +0000, MysticZach via Digitalmars-d wrote: [...]
> As far as syntax subtrees not belonging to their parent, I can see where the cognitive dissonance comes from. But it just doesn't seem that bad to me, since contracts are always executed as if they are sequential statements anyway.
[...]

Then possibly you're missing the point behind DbC.  The idea of "executing" a contract seems to indicate that you're taking the current D implementation of it as normative. However, that is not the case.  The concept behind DbC is that the function specifies a set of conditions the caller must satisfy before calling it (the in-contract), and in return it promises to satisfy another set of conditions (the out-contract).  Conceptually speaking, the contracts are not "executed" as if they were part of the code in the function's body; rather, they are checks that are made by the runtime (as a conceptual entity -- one might think of it as a circuit breaker or some such safeguarding device) such that if they are violated, the program is aborted because it has entered an invalid state from which further execution would lead to UB.

Of course, how you implement the DbC concept is a different (albeit related) issue. In languages targeted for a VM like Java, one could conceivably implement contract verification as part of the VM itself, so that it is done "transparently" to user code.  In languages like D, however, because we're targeting the machine directly, no such intermediate layer exists, and hence the natural choice of implementing contracts as part of the code. (Though, as I've said in another post, D's implementation leaves some things to be desired, such as in-contracts being part of the callee rather than the caller, which IMO would have been a better choice.)

But in any case, the so-called "cognitive dissonance" comes from conflating contracts, which conceptually is part of the function's user-facing API, as opposed to the implementation details in the function body.  It's like saying that the locking mechanism of the safety cap of a medicine bottle is part of the medicine's chemistry. The locking mechanism is intended to protect the medicine, e.g., from children who would suffer unintended consequences of the medicine's chemistry by their incorrect usage of it. But that hardly makes the safety mechanism the same thing as the medicine itself.  Similarly, the in-contract of a function is intended to protect it from incorrect usage by buggy callers, but that hardly makes it a part of the function's body.


T

-- 
Береги платье снову, а здоровье смолоду.

On Thu, Jun 22, 2017 at 06:17:54AM +0000, MysticZach via Digitalmars-d wrote:
> On Wednesday, 21 June 2017 at 19:34:53 UTC, H. S. Teoh wrote:
> > This is a sticky point about D's current DbC implementation that myself and several others feel is a design flaw. In particular, that in-contracts are executed as part of the *callee*, when the intent of DbC is really that it is the obligation of the *caller* to fulfill its stipulations, and therefore the contract verification should happen at *caller* site rather than at the beginning of the callee.
> > 
> > This particular implementation detail causes problems with binary-only libraries: most library vendors would prefer to ship the library compiled with -release rather than not, but in -release, the asserts in any in-contracts would be elided, making them essentially non-existent by the user uses the library.  So you either have to dispense with DbC altogether, or be forced to ship two versions of your library, one with contracts compiled in and one without, in order for your users to benefit from DbC *and* not have to suffer performance penalties in their own release builds.
[...]
> But on the other side, binary-only libraries published in release mode are actually rare in D, right?

Rare or not, if we want D to grow, we cannot ignore proprietary usage, which would most likely involve binary-only libraries.  Precluding this (or at the very least discouraging it) just because of a design flaw in D's implementation of DbC would be a grave strategic error on our part.


T

-- 
Debian GNU/Linux: Cray on your desktop.

On Thursday, 22 June 2017 at 05:46:06 UTC, MysticZach wrote:
> On Thursday, 22 June 2017 at 00:27:38 UTC, Timon Gehr wrote:
>
> But it still has a sticking point, which I want to resolve, namely that it elevates the existing `assert` functionality beyond the current requirement that one must explicitly write `assert`, going so far as to imply it in the grammar itself.

What happens with H.S. Teoh's proposal is not that `assert` is implied by the grammar, but that a function can *finally* specify its contracts *independent* of how those contracts are actually implemented. That's a good thing, because
a) it allows the contract implementation for that new syntax to be swapped out transparently (no changes need to be made by the programmer to get a better contract implementation)
b) it makes contracts more compact (and thus easier to read) without losing relevant information (contract implementation is not something you should have to deal with for every single library independently)

> But because of many complaints about the limitations of the assert mechanism as it currently exists, I would hesitate to install it into the grammar as the "One Chosen Way" to bail out of contracts with the new syntax.

Again, that's not what H.S. Teoh's proposal would do. All it does is install an *implementation agnostic*, *abtract* way to specify contracts into the grammar. Whether that is lowered to assert, or anything else is an implementation detail and it certainly isn't fixed to asserts.
Simply document the behaviour one can expect when using it in debug and release mode and leave the implementation details (assert or something else) unspecified.

>
> However, if the functioning of the `assert` mechanism were more customizable, then it would be easier to entrust it, in my opinion, with the "sacred responsibility" of being installed into the grammar. H.S. Teoh's proposal would then stand on a firmer foundation, and my initial proposed syntax would become the inferior optionl. At that point, this DIP could be rewritten to advocate his syntax instead.

A more customizable assert would not change anything for the DIP imo, since I do not want implementation details to be part of contract writing.

>
> The intent of my proposal was to make a small improvement. The cost (or so I thought, and may still believe) was also small. Small improvements are still improvements. DIP1003 is an example of this.

DIP1003 did not introduce an entirely new system, it merely slightly changed an existing sytem.
DIP1009 *does* introduce several new syntax forms, i.e. adding a whole new system, which means it *also* introduces the responsibility of maintaining backwards compatibility when someone tries to improve contracts again (and then we would have three systems to specify contracts).
Both DIP1003 and DIP1009 need to have a positive benefit/cost balance, which is easy for DIP1003 (since the costs are insignificant), but not for DIP1009: The costs are high down the line and it needs to justify them. The only way I can see those costs being justified is by separating constract specification from contract implementation and that requires syntax similar to H.S. Teoh's proposal.

On Thursday, 22 June 2017 at 06:43:38 UTC, H. S. Teoh wrote:
> On Thu, Jun 22, 2017 at 05:46:06AM +0000, MysticZach via Digitalmars-d wrote: [...]
>> As far as syntax subtrees not belonging to their parent, I can see where the cognitive dissonance comes from. But it just doesn't seem that bad to me, since contracts are always executed as if they are sequential statements anyway.
> [...]
>
> Then possibly you're missing the point behind DbC.  The idea of "executing" a contract seems to indicate that you're taking the current D implementation of it as normative.

I understand. Thus, a better DIP would suggest reimplementing D's DbC system altogether. Because as it is, they are little more than syntax dressing, which happens to be less convenient that just writing out the asserts in the first place. They're more like syntax _vinegar_ than sugar. Which explains why hardly anyone uses them. Why write this:

int fun(int a)
in { assert (a); }
do {
    return a;
}

...when you could just do this:

int fun(int a) {
   assert(a);
   return a;
}

With your improvement, it's a little better,

int fun(int a)
in (a)
{
    return a;
}

...but the cost is that `assert` is implied, and no other system of checking is allowed, which seems like a sufficient flaw to make it not decisively better. To solve this, you'd have to decouple `assert` from the contract, requiring this:

int fun(int a)
in (assert(a))
{
   return a;
}

This also looks good. But now the grammar is weird because it's just an expression in there without a statement, which doesn't happen anywhere else in D.

I start to get the nagging feeling that your point about the limitation in D's DbC implementation is actually the fatal flaw here. It's currently _illegal_ (outside of interface declarations) to expose the signature separately from the body, which, as you point out,  is more or less the whole point of DbC. <shrug> The existing system is itself distinctly _worse_ than just writing out asserts manually... and as far as I can tell,  none of the alternative syntaxes is a decisive improvement over it. Yours looks the best, but implying `assert` in the grammar seems to be going too far, especially for a DbC system that is only half functional. If the `assert` system were made more user-definable, it would make more sense. Do you agree with me on this?

Plans to improve `assert` are already in the air. Maybe we should wait on them before promoting your suggestion?

On Thursday, 22 June 2017 at 12:21:29 UTC, MysticZach wrote:
> I start to get the nagging feeling that your point about the limitation in D's DbC implementation is actually the fatal flaw here. It's currently _illegal_ (outside of interface declarations) to expose the signature separately from the body, which, as you point out,  is more or less the whole point of DbC. <shrug> The existing system is itself distinctly _worse_ than just writing out asserts manually... and as far as I can tell,  none of the alternative syntaxes is a decisive improvement over it. Yours looks the best, but implying `assert` in the grammar seems to be going too far, especially for a DbC system that is only half functional. If the `assert` system were made more user-definable, it would make more sense. Do you agree with me on this?
>
> Plans to improve `assert` are already in the air. Maybe we should wait on them before promoting your suggestion?

Conversely, as Moritz suggests, contracts could have their own separate logic, with optional user-defined hooks. Do you have a suggestion on how to implement this behind the scenes?