Hi,

There seems to be a bug allocating large dynamic arrays in a 64-bit aware dmd (v2.062). Apparently, the size argument makes a trip through 32-bit ptrdiff_t land or something like that:

unittest
{
    immutable size_t size = 3 * 1024 * 1024 * 1024;
    auto data = new byte[size]; // compiler error:
    // file.d(line): Error: negative array index 18446744072635809792LU
}

unittest
{
    immutable size_t size = 4 * 1024 * 1024 * 1024;
    auto data = new byte[size]; // fails silently, zero length array
    assert(data.length != 0); // assert error
}

Have you seen this before? I can open a bug, but just checking.

In any case, I don't understand why the compiler doesn't complain about overflows at compile time:

unittest
{
    size_t s1 = uint.max + 1; // shouldn't it complain with -m32 flag? it does not.
    assert(s1 != 0); // fails for -m32, as expected
    uint s2 = 0xFFFFFFFF + 1; // shouldn't it complain? it does not.
}

Regards,
Luís

On 03/25/2013 07:23 PM, "Luís Marques" <luismarques@gmail.com>" wrote:
> Hi,
>
> There seems to be a bug allocating large dynamic arrays in a 64-bit
> aware dmd (v2.062). Apparently, the size argument makes a trip through
> 32-bit ptrdiff_t land or something like that:
>
> unittest
> {
> immutable size_t size = 3 * 1024 * 1024 * 1024;

On a tangent, despite appearances, the type of the right-hand side is int with the value of -1073741824. It is according to the arithmetic conversion rules:

  http://dlang.org/type.html

Ali

On Tuesday, 26 March 2013 at 05:38:41 UTC, Ali Çehreli wrote:
> On a tangent, despite appearances, the type of the right-hand side is int with the value of -1073741824. It is according to the arithmetic conversion rules:
>
>   http://dlang.org/type.html
>
> Ali

Ahh, right. If you do

    auto data = new byte[3 * 1024 * 1024 * 1024L];

with the L suffix, then it works, of course. But this is crazy! :-)

Really, something needs to be rethought here, do you really want your constant folding to to overflow at 32 bits by default?. Is this because of CTFE?

Luís

On Tuesday, 26 March 2013 at 13:56:26 UTC, Luís Marques wrote:
[cut]
> Ahh, right. If you do
>
>     auto data = new byte[3 * 1024 * 1024 * 1024L];
>
> with the L suffix, then it works, of course. But this is crazy! :-)
>
> Really, something needs to be rethought here, do you really want your constant folding to to overflow at 32 bits by default?. Is this because of CTFE?
>
> Luís

I don't know why there is this behaviour but I fully agree with you that this is a bug.
It should at least trigger a warning..

renoX

Luís Marques:

> Really, something needs to be rethought here, do you really want your constant folding to to overflow at 32 bits by default?

It's a unacceptable trap for a modern language:

http://d.puremagic.com/issues/show_bug.cgi?id=4835

Bye,
bearophile

On Tuesday, 26 March 2013 at 14:04:48 UTC, bearophile wrote:
> It's a unacceptable trap for a modern language:
>
> http://d.puremagic.com/issues/show_bug.cgi?id=4835

Thank you all for your feedback. I've added my vote to bug 4835.

On Tuesday, 26 March 2013 at 14:04:48 UTC, bearophile wrote:
> It's a unacceptable trap for a modern language:
>
> http://d.puremagic.com/issues/show_bug.cgi?id=4835

I agree, but apparently, for what its worth, Java doesn't complain either:

class Test
{
    public static void main(String[] args)
    {
        long a = 3 * 1024 * 1024 * 1024;
        long b = 3 * 1024 * 1024 * 1024L;

        assert(a < 0);
        assert(b > 0);
    }
}

$ javac test.java
$ java Test
$ (no error)

On the other hand, if this is fixed (err, improved?) then you have one more reason to say that D is better than Java ;-)

BTW, as far as I can see the overflow/underflow behavior never got specified by the language, in any case:

1) http://forum.dlang.org/thread/jo2c0a$31hh$1@digitalmars.com <-- no conclusion

2) Andrei's book doesn't seem to mention the topic.

If it is specified somewhere please do tell. Whatever the behavior should be (unspecified, modulus for unsigned integers, etc) there really should be an official stance.

On Tue, 26 Mar 2013 13:56:35 -0400, Luís Marques <luismarques@gmail.com> wrote:

> BTW, as far as I can see the overflow/underflow behavior never got specified by the language, in any case:
>
> 1) http://forum.dlang.org/thread/jo2c0a$31hh$1@digitalmars.com <-- no conclusion
>
> 2) Andrei's book doesn't seem to mention the topic.
>
> If it is specified somewhere please do tell. Whatever the behavior should be (unspecified, modulus for unsigned integers, etc) there really should be an official stance.

The official stance is, it's not an error.  If we treated it as an error, then it would be very costly to implement, every operation would have to check for overflow.  The CPU does not assist in this.

You can construct an "overflow-detecting" integer type that should be able to do what you want.

-Steve

On Tuesday, 26 March 2013 at 18:04:25 UTC, Steven Schveighoffer wrote:
> The official stance is, it's not an error.  If we treated it as an error, then it would be very costly to implement, every operation would have to check for overflow.  The CPU does not assist in this.

You say not an error as meaning the language definition does not guarantee checking for overflows/underflows and throwing an exception if one occurs.

But my point is even more simple: is there a stance on what the overflow/underflow semantics are? E.g., are they undefined (might wrap, might saturate, might have one's complement behavior, etc), defined only for unsigned integers (like C and C++), etc?