Hey,

[DMD 2.010, windows]

So.. was doing a bit of simple D coding and got some really odd behaviour, namely this code creates a 4gb (4,294,975,895 bytes) file on my disk!

My question is, is this a bug or am I doing something stupid?

The idea is to open a file and overwrite random blocks with '0' (the ascii number, not the decimal value - though I get the same behaviour with either).

When executed this code seems to hang after outputting "Close.." and at that point the file is growing, and growing... ctrl+c does nothing immediately but seems to work after the file reaches 4gb, at a guess when close actually returns.

Opening the file in textpad only shows 8232 characters, all 'a' - so none of the modifications have been applied and it seems an eof character has been inserted at that point in the 4gb file.

Adding a return after the file creation before the modification shows a file which is as expected 50000 characters long, all 'a'.  So that at least is working as expected.

import std.stdio;
import std.stream;
import std.file;
import std.random;

int main()
{
	// Create small file
	auto 	f = new BufferedFile("test", FileMode.OutNew);
	int	size = 50000;
	
	for(int i = 0; i < size; i++)
		f.write('a');
		
	f.close;

	// Open and modify	
	f = new BufferedFile("test", FileMode.In | FileMode.Out);
	ulong	total = getSize("test");
	long	block = total/1000;
	long 	offset;
	long	bytes;
	Random 	gen;
	
	writefln("Total: %d,%d", total, block);
	
	for(int i = 0; i < 2; i++)
	{
		offset = uniform!(long)(gen, 0L, block);
		bytes  = uniform!(long)(gen, 0L, block);
		
		writefln("Seek : %d,%d,%d", f.position, offset, bytes);
		
		f.seekCur(offset);
		
		writefln("Mod  : %d,%d", f.position, bytes);
		
		for(long j = 0; j < bytes; j++)
			f.write('0');
			
		writefln("Done : %d", f.position);
	}
	
	writefln("Close: %d", f.position);
	
	f.close;
	
	writefln("Done");
	
	return 0;
}

Regan Heath wrote:
> Hey,
> 
> [DMD 2.010, windows]
> 
> So.. was doing a bit of simple D coding and got some really odd behaviour, namely this code creates a 4gb (4,294,975,895 bytes) file on my disk!
> 
> My question is, is this a bug or am I doing something stupid?

Congratulations, you found another stupid integer promotion rule bug. Those have been a personal gripe for me. Hopefully we can one day fix this horrible design mistake from C.

The problem is that:

uint l = 1;
long c = -l;

yields c == 4294967295

wohoo... (Did I mention that I hate those?)

The error lies in BufferedStream.flush()

BufferedStream has a uint bufferSourcePos and does:

      if (bufferSourcePos != 0 && seekable) {
	// move actual file pointer to front of buffer
	streamPos = s.seek(-bufferSourcePos, SeekPos.Current);

and since seeks first argument is a long, it gets a value close to 4GB.

-- 
Oskar

On 06/03/2008, Oskar Linde <oskar.lindeREM@ovegmail.com> wrote:
>  The problem is that:
>
>  uint x = 1;
>  long y = -x;
>
>  yields y == 4294967295

(I changed the variable names because lowercase L looks like one to me).

Huh?

Why doesn't y equal minus one?

I wouldn't call that a bug in BufferedStream.flush(), I'd call it a
bug in D's arithmetic generally.

The way I see it, there are two possible fixes:

(1) Disallow unary minus completely for all unsigned types. Thus

    uint x = 1;
    long y = -x; /*ERROR*/

forcing the user to explicitly write

    long y = -cast(int)x;

or

    long y = -cast(long)x;

either of which should be OK, or

(2), when unary minus is applied to an unsigned type, promote ubyte to
short, ushort to int, uint to long, ulong to cent (...best make that
one illegal for now). That would mean

    uint x = 1;
    long y = -x; /* OK */

but

    uint x = 1;
    int y = -x; /* Error */

Either would work, but the status quo /can't/ be right!!!???

Janice Caron:
> I wouldn't call that a bug in BufferedStream.flush(), I'd call it a
> bug in D's arithmetic generally.

Another faster solution is to avoid unsigned integrals whenever you can, so you can avoid some bugs.


> (2), when unary minus is applied to an unsigned type, promote ubyte to
> short, ushort to int, uint to long, ulong to cent (...best make that
> one illegal for now). That would mean

Delphi has runtime cheeks for integer overflow too, they help.

Bye,
bearophile

Janice Caron wrote:
> On 06/03/2008, Oskar Linde <oskar.lindeREM@ovegmail.com> wrote:
>>  The problem is that:
>>
>>  uint x = 1;
>>  long y = -x;
>>
>>  yields y == 4294967295
> 
> (I changed the variable names because lowercase L looks like one to me).
> 
> Huh?
> 
> Why doesn't y equal minus one?
> 
> I wouldn't call that a bug in BufferedStream.flush(), I'd call it a
> bug in D's arithmetic generally.

It is hard to classify something as a bug when it behaves exactly as specified. See "Usual Arithmetic Conversion" under:

http://www.digitalmars.com/d/1.0/type.html

This is exactly how it works in C too.

> The way I see it, there are two possible fixes:
> 
> (1) Disallow unary minus completely for all unsigned types. Thus

Unary minus is not the only affected operator. All integer arithmetic is affected.

> (2), when unary minus is applied to an unsigned type, promote ubyte to
> short, ushort to int, uint to long, ulong to cent (...best make that
> one illegal for now). That would mean
> 
>     uint x = 1;
>     long y = -x; /* OK */
> 
> but
> 
>     uint x = 1;
>     int y = -x; /* Error */
> 
> Either would work, but the status quo /can't/ be right!!!???

There has been some previous discussions along those lines. For example:

http://www.digitalmars.com/d/archives/digitalmars/D/unsigned_policy_47929.html#N47954

Today, all arithmetic operations are promoted to int/uint, and by the magic of the 2's-complement representation, things actually work out most of the time.

for example:

int a = (rand() % 3) - 1;

works fine, while

double b = (rand() % 3) - 1;

doesn't.

-- 
Oskar

bearophile wrote:
> Delphi has runtime cheeks for integer overflow too, they help.

I'm sorry. I don't usually mind misspellings so much, but this one is really getting on my nerves. Maybe it's the fact you repeat it so often, combined with the fact the misspelling has an entirely different meaning; I don't know.

It's "checks". Not "cheeks".

Compare:
http://dictionary.reference.com/search?q=check
http://dictionary.reference.com/search?q=cheek

On 06/03/2008, Oskar Linde <oskar.lindeREM@ovegmail.com> wrote:
> There has been some previous discussions along those lines. For example:
>  http://www.digitalmars.com/d/archives/digitalmars/D/unsigned_policy_47929.html#N47954

Gosh - I've clearly missed out on that rather interesting line of conversation. Thanks for the link.

Frits van Bommel:

> I'm sorry. I don't usually mind misspellings so much, but this one is
> really getting on my nerves. Maybe it's the fact you repeat it so often,
> combined with the fact the misspelling has an entirely different
> meaning; I don't know.
> It's "checks". Not "cheeks".

This language is my third one, and it shows.
Don't be sorry, I'll try to remember it, checks!
If you find other problems in my language please feel free to tell me.
Thank you very much,
bearophile

"Janice Caron" <caron800@googlemail.com> wrote in message news:mailman.118.1204810684.2351.digitalmars-d@puremagic.com...
> On 06/03/2008, Oskar Linde <oskar.lindeREM@ovegmail.com> wrote:
>>  The problem is that:
>>
>>  uint x = 1;
>>  long y = -x;
>>
>>  yields y == 4294967295
>
> (I changed the variable names because lowercase L looks like one to me).
>
> Huh?
>
> Why doesn't y equal minus one?
>
> I wouldn't call that a bug in BufferedStream.flush(), I'd call it a
> bug in D's arithmetic generally.
>
> The way I see it, there are two possible fixes:
>
> (1) Disallow unary minus completely for all unsigned types. Thus
>
>    uint x = 1;
>    long y = -x; /*ERROR*/
>
> forcing the user to explicitly write
>
>    long y = -cast(int)x;
>
> or
>
>    long y = -cast(long)x;
>
> either of which should be OK, or
>
> (2), when unary minus is applied to an unsigned type, promote ubyte to
> short, ushort to int, uint to long, ulong to cent (...best make that
> one illegal for now). That would mean
>
>    uint x = 1;
>    long y = -x; /* OK */
>
> but
>
>    uint x = 1;
>    int y = -x; /* Error */
>
> Either would work, but the status quo /can't/ be right!!!???

Andrei Alexandrescu posted a .pdf a while back with a graph containing all types and directed edges for all allowed implicit promotions.. I think he has it on his site, but I can't seem to access it (erdani.org) from China. Has he been politcally active lately??  : )

L. 

Found it:

http://erdani.org/d-implicit-conversions.pdf