On Wednesday, 15 June 2016 at 18:34:15 UTC, tsbockman wrote:
> On Wednesday, 15 June 2016 at 16:40:19 UTC, Andrei Alexandrescu wrote:
>> Getting to the design: the root of the problem is a byzantine design that is closed to extension.
>
> The design was closed deliberately because of (8). Template bloat is a major concern, even with the current finite design.
>
> I want `checkedint` to be usable in public APIs, and that requires some standardization of error handling and base types to be enforced upon the users. Otherwise, everyone will choose something different and all template instantiations involving integer types will become practically single-use.
>
>> Looking at the IntFlagPolicy, it offers three canned behavior: throws, asserts, and noex.
>
> The choice of policies is motivated by the natural incompatibility of (2), (4), (6), and (7). I built in enough variety to allow people to choose their own priorities among those goals, and no more because of (8).

Re-reading what I wrote there, I makes it sound like the closed design was motivated solely by template bloat concerns. But, that's not true either.

Standardizing the error handling methods is also important for other interoperability-related reasons:

    * If every third-party library designs a different error handling method,
      people writing applications that depend on many libraries will have to
      study the `checkedint` error signaling of each one, and make sure that
      their code which interacts with each library knows how to detect and
      respond to its particular signaling mechanism.

      My design for `checkedint` reduces the number of `checkedint`-specific
      signaling methods that need to be studied down to one (the sticky flags
      policy). The other two policies are just using D's standard error-
      handling facilities, and don't require any user intervention anyway,
      unless you want to catch the exception for some reason.

      Even the sticky flags policy is only of concern for `nothrow @nogc` APIs,
      which I suspect in practice will mostly mean game libraries.

    * The relative simplicity of my policy set has allowed me to arrange them
      into a strict linear hierarchy, so that the results of mixing policies
      is predictable and safe.

On Wednesday, 15 June 2016 at 16:40:19 UTC, Andrei Alexandrescu wrote:
> One of the first things I looked for was establishing bounds for numbers, like Smart!(int, 0, 100) for percentage. For all its might, this package does not offer this basic facility, and from what I can tell does not allow users to enforce it via policies.

Another angle on why `checkedint` does not currently include a bounded type: although some of the internals might be similar, conceptually a `BoundInt` type is actually the *opposite* of the ideal which `SmartInt` and `SafeInt` strive toward.

The problem which this package was written to solve, is that when programming people frequently want to use "integers" as a data type. A real "integer" is, by definition, UNbounded, with an infinite range of values available. This makes designing algorithms simpler, but of course leads to problems when a value strays outside the narrow range in which the behaviour of machine integers correctly emulates that of mathematical integers.

The whole point of `checkedint` is to be able to write algorithms based on the simplifying assumption that your variables behave like real, UNbounded mathematical integers, secure in the knowledge that an error message will be generated if that assumption is invalidated.

Intentionally limiting values to a specific range is a different problem, and never the one that I was trying to solve - although I agree that the `checkedint` package would be a good namespace in which to park a `BoundInt` type, if someone wants to submit one.

On 06/15/2016 07:34 PM, tsbockman wrote:
> The whole point of `checkedint` is to be able to write algorithms based
> on the simplifying assumption that your variables behave like real,
> UNbounded mathematical integers, secure in the knowledge that an error
> message will be generated if that assumption is invalidated.

Fair point, though one could make the same claim for integers within other limits than the machine's. -- Andrei

On 06/15/2016 07:13 PM, tsbockman wrote:
>
> Standardizing the error handling methods is also important for other
> interoperability-related reasons:
>
>      * If every third-party library designs a different error handling
> method,
>        people writing applications that depend on many libraries will
> have to
>        study the `checkedint` error signaling of each one, and make sure
> that
>        their code which interacts with each library knows how to detect and
>        respond to its particular signaling mechanism.
>
>        My design for `checkedint` reduces the number of
> `checkedint`-specific
>        signaling methods that need to be studied down to one (the sticky
> flags
>        policy). The other two policies are just using D's standard error-
>        handling facilities, and don't require any user intervention anyway,
>        unless you want to catch the exception for some reason.
>
>        Even the sticky flags policy is only of concern for `nothrow
> @nogc` APIs,
>        which I suspect in practice will mostly mean game libraries.
>
>      * The relative simplicity of my policy set has allowed me to
> arrange them
>        into a strict linear hierarchy, so that the results of mixing
> policies
>        is predictable and safe.

I don't agree with this. On the face of it, three built-in policies solve the matter of interoperability: if you want interop, use one of those. Done. But interop is not a reason to preclude any flexibility. -- Andrei

On Thursday, 16 June 2016 at 00:03:26 UTC, Andrei Alexandrescu wrote:
> On 06/15/2016 07:34 PM, tsbockman wrote:
>> The whole point of `checkedint` is to be able to write algorithms based
>> on the simplifying assumption that your variables behave like real,
>> UNbounded mathematical integers, secure in the knowledge that an error
>> message will be generated if that assumption is invalidated.
>
> Fair point, though one could make the same claim for integers within other limits than the machine's. -- Andrei

Sure. I'm just trying to explain my design goals and philosophy - not to argue that no other approach could be valid.

But, I would appreciate it if comments distinguished between:

1) My goals and philosophy differ from yours, versus
2) My implementation is bad.

I think a lot of the high-level criticism you brought is really rooted in (1), even though you present it as (2).

On Thursday, 16 June 2016 at 00:06:13 UTC, Andrei Alexandrescu wrote:
> On 06/15/2016 07:13 PM, tsbockman wrote:
>>
>> Standardizing the error handling methods is also important for other interoperability-related reasons:
>> [...]
>
> I don't agree with this. On the face of it, three built-in policies solve the matter of interoperability: if you want interop, use one of those. Done. But interop is not a reason to preclude any flexibility. -- Andrei

Well, I'm satisfied with my design in this respect, and completely lack motivation, time, emotional energy, etc. to rewrite `checkedint` again from the ground up to properly account for arbitrary pluggable error handling policies.

So, the community will have to decide how important this is, and find someone else to take over the project if you all demand a rewrite.

On 06/15/2016 02:50 PM, tsbockman wrote:
> On Wednesday, 15 June 2016 at 16:40:19 UTC, Andrei Alexandrescu wrote:
>> I think there are a few considerable issues with the proposal, but
>> also that all are fixable.
>
> I already sent a much longer message detailing some of the reasons why I
> believe my design is sensible. But, before we continue this discussion
> much further, I need to stop and make one thing clear:
>
> What you are proposing is *not* "fixing" my design - it is basically
> scrapping it and replacing it with a ground-up rewrite, with perhaps
> some bits and pieces and general inspiration taken from my work.

I'd say that's a fair assessment on the face of it. But the pieces are there and the grit and expertise to assemble them are already there. The real question is what's the "right" design. Ideally we should optimize for that.

> I'm OK with the community rejecting my design if that's what people
> really want to do, but I will not be implementing your proposal myself.
> I will simply leave `checkedint` on DUB, and move on with my life.

That is sensible, and a course of action I thought would transpire. But I suggest you to reconsider. Your PR is a solid implementation of the wrong idea. Before anyone gets offended, let me add that I think there's a lot of good in that. For this kind of work (not algorithm-intensive) ideas are cheap; the solid implementation matters a lot more than the idea - the same expertise can be put to use on another idea.

I tried to offer a careful review by a competent peer, with an eye for taking an okay design toward a great design. The kind of review I wish to get now as much as at any point of my career. They are difficult to receive but often have a lot of potential in them.

> So, I'm putting out a general call now for the community to download the
> DUB package, try actually *using* it for something, and speak up as to
> whether the design seems good, or not.
>
> If the decision is made to accept the high-level design, then we can go
> back to bikeshedding about names, fixing typos, tweaking/trimming the
> API, etc.

I don't think checkedint in a form close to its current one is for Phobos. I would argue it's not for D. Starting with the problem statement "we want to check integral operations for overflow and other surprising behaviors" and ending with 4.5 KLOC defining a bunch of names in 6 modules makes it highly suspicious there is an overgrown underbelly somewhere. The language features are not orchestrated properly. Even if it were the case that there's no smaller design that conforms with the requirements, that means requirements have a problem.


Andrei

On 06/15/2016 08:07 PM, tsbockman wrote:
> On Thursday, 16 June 2016 at 00:03:26 UTC, Andrei Alexandrescu wrote:
>> On 06/15/2016 07:34 PM, tsbockman wrote:
>>> The whole point of `checkedint` is to be able to write algorithms based
>>> on the simplifying assumption that your variables behave like real,
>>> UNbounded mathematical integers, secure in the knowledge that an error
>>> message will be generated if that assumption is invalidated.
>>
>> Fair point, though one could make the same claim for integers within
>> other limits than the machine's. -- Andrei
>
> Sure. I'm just trying to explain my design goals and philosophy - not to
> argue that no other approach could be valid.
>
> But, I would appreciate it if comments distinguished between:
>
> 1) My goals and philosophy differ from yours, versus
> 2) My implementation is bad.
>
> I think a lot of the high-level criticism you brought is really rooted
> in (1), even though you present it as (2).

My perception is your implementation is professional. -- Andrei

On Thursday, 16 June 2016 at 00:31:38 UTC, Andrei Alexandrescu wrote:
> On 06/15/2016 02:50 PM, tsbockman wrote:
>> What you are proposing is *not* "fixing" my design - it is basically
>> scrapping it and replacing it with a ground-up rewrite, with perhaps
>> some bits and pieces and general inspiration taken from my work.
>
> I'd say that's a fair assessment on the face of it. But the pieces are there and the grit and expertise to assemble them are already there. The real question is what's the "right" design. Ideally we should optimize for that.

I set out to solve a specific problem with `checkedint`, and I believe my design succeeds in the areas that it is intended to, to the extent that is reasonably possible with D today.

In order to convince me to rewrite this, you would need to convince me that *my* goals can be met significantly better by a different library design. So far, I see no evidence of this.

You are of course free to set different goals than I have, but from my perspective this is *major* scope creep, and not what I volunteered for.

>> I'm OK with the community rejecting my design if that's what people
>> really want to do, but I will not be implementing your proposal myself.
>> I will simply leave `checkedint` on DUB, and move on with my life.
>
> That is sensible, and a course of action I thought would transpire. But I suggest you to reconsider. Your PR is a solid implementation of the wrong idea. Before anyone gets offended, let me add that I think there's a lot of good in that.

There is only one thing about all this that really offends me:

`checkedint` (and @burner's `SafeInt` before it) have been under development in the open for over a year now. There have been several discussions in the forums, with feedback being actively solicited. Significant design changes were made to address various people's needs. `SafeInt` was an open pull request for many months with 100+ comments accumulating in that time.

Why didn't you make your design requirements known at any earlier point in this process? If you are ultimate gate keeper for Phobos (as you seem to be), you ought to make your requirements known *before* the implementation is finished.

Even so, I'm not particularly upset. I've learned a great deal through this project, and `checkedint` will still be available for anyone who wants to use it through DUB.

The code is Boost Licensed, so anyone who wants to is free to fork it and try to fulfill your vision.

> For this kind of work (not algorithm-intensive) ideas are cheap; the solid implementation matters a lot more than the idea - the same expertise can be put to use on another idea.
>
> I tried to offer a careful review by a competent peer, with an eye for taking an okay design toward a great design. The kind of review I wish to get now as much as at any point of my career. They are difficult to receive but often have a lot of potential in them.

I don't mind a scathing review, so long as it has been properly researched and thought through. I do object to being criticized for not including features that no one asked me for until now, and which have little to do with the original purpose of the project, though.

>> So, I'm putting out a general call now for the community to download the
>> DUB package, try actually *using* it for something, and speak up as to
>> whether the design seems good, or not.
>>
>> If the decision is made to accept the high-level design, then we can go
>> back to bikeshedding about names, fixing typos, tweaking/trimming the
>> API, etc.
>
> I don't think checkedint in a form close to its current one is for Phobos. I would argue it's not for D. Starting with the problem statement "we want to check integral operations for overflow and other surprising behaviors" and ending with 4.5 KLOC defining a bunch of names in 6 modules makes it highly suspicious there is an overgrown underbelly somewhere. The language features are not orchestrated properly. Even if it were the case that there's no smaller design that conforms with the requirements, that means requirements have a problem.
>
>
> Andrei

I believe a fundamental part of the problem, is that something like `SmartInt` should be built in to the language as the default integer type, not tacked on as a library. (Stuff like bound types would still be implemented in the library.)

However, my understanding is that this approach was reject by Walter Bright about two years ago (and obviously actually making it default would involve unacceptable breakage at this point). So, I tried to work with what we have.

On Thursday, 16 June 2016 at 00:33:38 UTC, Andrei Alexandrescu wrote:
> On 06/15/2016 08:07 PM, tsbockman wrote:
>> But, I would appreciate it if comments distinguished between:
>>
>> 1) My goals and philosophy differ from yours, versus
>> 2) My implementation is bad.
>>
>> I think a lot of the high-level criticism you brought is really rooted
>> in (1), even though you present it as (2).
>
> My perception is your implementation is professional. -- Andrei

Thank you.