On Thursday, 13 August 2020 at 23:28:43 UTC, H. S. Teoh wrote:
> And before somebody tells me this is too verbose: we already have to do this for short ints, no thanks to the recent change that arithmetic involving anything smaller than int will implicitly promote to int first:

That's not actually a recent change; it has always been like that, inherited from C.

D's difference is that C will implicitly convert back to the small type and D won't without an explicit cast. Drives me nuts and very rarely actually catches real mistakes.

The most recent change here is even like `-a` will trigger the error. It always did the promotion but it used to cast right back and now will error instead. Ugh.

On Thursday, 13 August 2020 at 23:33:27 UTC, James Blachly wrote:
> On 8/13/20 5:33 PM, Walter Bright wrote:
> You make a very good case for the choice overall but not the lack of warnings. Part of the issue (IMO) is that D is /more/ than just a systems language (side note, it sometimes seems it tries to be everything to everyone), hence the original complaint of "silent" or surprising errors cropping up in the context of e.g.
>
> Array.length
>
> returning ulong. Especially in cases like this, compiler warnings would be helpful.

Yes, I want the warning messages, so I know where are the potential problems are located, and *explicitly* write intentional cast to fix them; instead of spending nights debugging the application.

On 8/13/2020 3:27 PM, mw wrote:
> For users / applications that do value correctness more than performance, can we have a compiler switch which turn all the types & operations (e.g. in modules, that users also specified on command-line) into core_checkedint or std_experimental_checkedint *automatically*?


The thing about checkedint is there are several different behaviors one might choose as responses to overflow. There is no one-size-fits-all, if we did pick one we'll inevitably have complaints.

checkedint is really a nice package. I encourage you to check it out.

On 8/13/2020 4:33 PM, James Blachly wrote:
> Especially in cases like this, compiler warnings would be helpful.

Warnings are inherently problematic, as the poor user is always faced with "is this a real problem or not?" I've been very resistant to adding warnings in the past because it balkanizes the language into dialects.

The core language should decide what's an error and what isn't.

On Thu, Aug 13, 2020 at 11:38:03PM +0000, Adam D. Ruppe via Digitalmars-d wrote:
> On Thursday, 13 August 2020 at 23:28:43 UTC, H. S. Teoh wrote:
> > And before somebody tells me this is too verbose: we already have to do this for short ints, no thanks to the recent change that arithmetic involving anything smaller than int will implicitly promote to int first:
> 
> That's not actually a recent change; it has always been like that, inherited from C.
> 
> D's difference is that C will implicitly convert back to the small type and D won't without an explicit cast. Drives me nuts and very rarely actually catches real mistakes.
> 
> The most recent change here is even like `-a` will trigger the error. It always did the promotion but it used to cast right back and now will error instead. Ugh.

Ah, yeah, that last one was what really annoyed me, and it was pretty recent.  I never understood the logic of that. I mean the starting value is already `byte`, why does its negation have to auto-promote to int?! But thus sez the rulez, so who am I to question it amirite? :-(

It annoyed me so much that I wrote a wrapper module just to work around it. Basically you append a `.np` to any of your narrow int values, and it wraps it in an infectious struct that auto-truncates arithmetic operations. The presence of the `.np` is a kind of self-documentation that we're flouting the usual int promotion rules, so I'm reasonably happy with this hack. :-P

Now somebody just has to write a wrapper for signed/unsigned conversions... ;-)


T

-- 
It is not the employer who pays the wages. Employers only handle the money. It is the customer who pays the wages. -- Henry Ford


-----------------------------------snip---------------------------------------
module nopromote;

enum isNarrowInt(T) = is(T : int) || is(T : uint);

/**
 * A wrapper around a built-in narrow int that truncates the result of
 * arithmetic operations to the narrow type, overriding built-in int promotion
 * rules.
 */
struct Np(T)
    if (isNarrowInt!T)
{
    T impl;
    alias impl this;

    /**
     * Truncating binary operator.
     */
    Np opBinary(string op, U)(U u)
        if (is(typeof((T x, U y) => mixin("x " ~ op ~ " y"))))
    {
        return Np(cast(T) mixin("this.impl " ~ op ~ " u"));
    }

    /**
     * Truncating unary operator.
     */
    Np opUnary(string op)()
        if (is(typeof((T x) => mixin(op ~ "cast(int) x"))))
    {
        return Np(cast(T) mixin(op ~ " cast(int) this.impl"));
    }

    /**
     * Infectiousness: any expression containing Np should automatically use Np
     * operator semantics.
     */
    Np opBinaryRight(string op, U)(U u)
        if (is(typeof((T x, U y) => mixin("x " ~ op ~ " y"))))
    {
        return Np(cast(T) mixin("u " ~ op ~ " this.impl"));
    }
}

/**
 * Returns: A lightweight wrapped type that overrides built-in arithmetic
 * operators to always truncate to the given type without promoting to int or
 * uint.
 */
auto np(T)(T t)
    if (isNarrowInt!T)
{
    return Np!T(t);
}

// Test binary ops
@safe unittest
{
    ubyte x = 1;
    ubyte y = 2;
    auto z = x.np + y;
    static assert(is(typeof(z) : ubyte));
    assert(z == 3);

    byte zz = x.np + y;
    assert(zz == 3);

    x = 255;
    z = x.np + y;
    assert(z == 1);
}

@safe unittest
{
    byte x = 123;
    byte y = 5;
    auto z = x.np + y;
    static assert(is(typeof(z) : byte));
    assert(z == byte.min);

    byte zz = x.np + y;
    assert(zz == byte.min);
}

@safe unittest
{
    import std.random;
    short x = cast(short) uniform(0, 10);
    short y = 10;
    auto z = x.np + y;
    static assert(is(typeof(z) : short));
    assert(z == x + 10);

    short s = x.np + y;
    assert(s == x + 10);
}

// Test unary ops
@safe unittest
{
    byte b = 10;
    auto c = -b.np;
    static assert(is(typeof(c) : byte));
    assert(c == -10);

    ubyte ub = 16;
    auto uc = -ub.np;
    static assert(is(typeof(uc) : ubyte));
    assert(uc == 0xF0);
}

version(unittest)
{
    // These tests are put here as actual module functions, to force optimizer
    // not to discard calls to these functions, so that we can see the actual
    // generated code.
    byte byteNegate(byte b) { return -b.np; }
    ubyte ubyteNegate(ubyte b) { return -b.np; }

    byte byteTest1(int choice, byte a, byte b)
    {
        if (choice == 1)
            return a.np + b;
        if (choice == 2)
            return a.np / b;
        assert(0);
    }

    short shortAdd(short a, short b) { return a.np + b; }

    // Test opBinaryRight
    byte byteRightTest(byte a, byte c)
    {
        auto result = a + c.np;
        static assert(is(typeof(result) : byte));
        return result;
    }

    unittest
    {
        assert(byteRightTest(127, 1) == byte.min);
    }

    short multiTest1(short x, short y)
    {
        return short(2) + 2*(x - y.np);
    }

    unittest
    {
        // Test wraparound semantics.
        assert(multiTest1(32767, 16384) == short.min);
    }

    short multiTest2(short a, short b)
    {
        short x = a;
        short y = b;
        return (2*x + 1) * (y.np/2 - 1);
    }

    unittest
    {
        assert(multiTest2(1, 4) == 3);
    }
}
-----------------------------------snip---------------------------------------

On Thursday, 13 August 2020 at 23:47:30 UTC, Walter Bright wrote:
> On 8/13/2020 3:27 PM, mw wrote:
>> For users / applications that do value correctness more than performance, can we have a compiler switch which turn all the types & operations (e.g. in modules, that users also specified on command-line) into core_checkedint or std_experimental_checkedint *automatically*?
>
>
> The thing about checkedint is there are several different behaviors one might choose as responses to overflow. There is no one-size-fits-all, if we did pick one we'll inevitably have complaints.

I’ve seen the different hooks, and was just about to add:

If we choose this approach, the hook can also have its own command line option to let the user in explicit full control of what the integer operation behavior s/he really wants for his/her application.

On Thursday, 13 August 2020 at 23:50:08 UTC, Walter Bright wrote:
> On 8/13/2020 4:33 PM, James Blachly wrote:
> Warnings are inherently problematic, as the poor user is always faced with "is this a real problem or not?" I've been very resistant to adding warnings in the past because it balkanizes the language into dialects.

It can be default-to-off warnings.

So for users who really care, or when their program run into trouble, they can turn the compiler option on explicitly to find the potential breaks.

Personally, I always turn on compiler warning to its max level.

On Thu, Aug 13, 2020 at 11:54:57PM +0000, mw via Digitalmars-d wrote:
> On Thursday, 13 August 2020 at 23:47:30 UTC, Walter Bright wrote:
[...]
> > The thing about checkedint is there are several different behaviors one might choose as responses to overflow. There is no one-size-fits-all, if we did pick one we'll inevitably have complaints.
[...]
> If we choose this approach, the hook can also have its own command line option to let the user in explicit full control of what the integer operation behavior s/he really wants for his/her application.
[...]

The application can just use CheckedInt instantiated with whatever preferences it wants to. Define an alias to that in a common module and import that, and you're good to go:

	alias Int = CheckInt!(... /* whatever settings you want here */);
	...
	Int i, j;
	Int k = i + j; // etc.

What we do *not* want is a compiler switch that will silently change the meaning of existing code. Imagine the chaos if you compiled your application with -int-behaviour=blahblah, and suddenly all of your dub dependencies start misbehaving because their code was written with the standard rules in mind, and subtly changing that causes massive breakage of their internal logic. Or worse, silent, subtle breakage in rare corner cases, of the kind that introduces security holes without a single warning, the kind that cyber-criminals love to exploit.


T

-- 
Political correctness: socially-sanctioned hypocrisy.

On Friday, 14 August 2020 at 00:04:55 UTC, H. S. Teoh wrote:
> On Thu, Aug 13, 2020 at 11:54:57PM +0000, mw via Digitalmars-d wrote:
>> On Thursday, 13 August 2020 at 23:47:30 UTC, Walter Bright wrote:
> [...]
>> > The thing about checkedint is there are several different behaviors one might choose as responses to overflow. There is no one-size-fits-all, if we did pick one we'll inevitably have complaints.
> [...]
>> If we choose this approach, the hook can also have its own command line option to let the user in explicit full control of what the integer operation behavior s/he really wants for his/her application.
> [...]
>
> The application can just use CheckedInt instantiated with whatever preferences it wants to. Define an alias to that in a common module and import that, and you're good to go:
>
> 	alias Int = CheckInt!(... /* whatever settings you want here */);

This doesn’t work for language buildins array.length.

>
> What we do *not* want is a compiler switch that will silently change the meaning of existing code. Imagine the chaos if you compiled your application with -int-behaviour=blahblah, and suddenly all of your dub dependencies start misbehaving because their code was written with the standard rules in mind, and

I saw that problem already, that’s why I also said :  “in modules, that users also specified on command-line”.

Now on a 2nd thought, on the module file level , let user specify at the top.

On Thursday, 13 August 2020 at 23:47:30 UTC, Walter Bright wrote:

> checkedint is really a nice package. I encourage you to check it out.

Yes, it works: https://github.com/mingwugmail/dlang_tour/blob/master/divbug.d

But it need a more *automatic* way to use this package to be useful, i.e. minimal change on existing code base (all over the places), from:

----------------------------------------------------------
void default_div() {
  long[] a = [-5000L, 0L];
  long   c = sum(a) / a.length;
  writeln(c);  // 9223372036854773308
}
----------------------------------------------------------

to:

----------------------------------------------------------
alias Long = Checked!long;

void checked_div() {
  Long[] a = [checked(-5000L), checked(0L)];
  Long   c = sum(a) / a.length;  // compile error
  writeln(c);  // Checked!(long, Abort)(-2500)
}
----------------------------------------------------------


currently, there is a compile error
Error: template std.experimental.checkedint.Checked!(long, Abort).Checked.__ctor cannot deduce function from argument types !()(Checked!(ulong, Abort))

have to re-write it as:
```
  Long c = sum(a);
  c /= a.length;
```