I have a very large file I need to read data from at certain positions, but I have run into this error
> std.conv.ConvOverflowException@std\conv.d(1328): Conversion positive overflow
when seeking to 6346890680. Seeking to smaller values such as 3580720 work with no problem. The file is well over 8GB, so it must be able to read data at positions that high.

On Sunday, 13 March 2016 at 10:32:41 UTC, stunaep wrote:
> I have a very large file I need to read data from at certain positions, but I have run into this error
>> std.conv.ConvOverflowException@std\conv.d(1328): Conversion positive overflow
> when seeking to 6346890680. Seeking to smaller values such as 3580720 work with no problem. The file is well over 8GB, so it must be able to read data at positions that high.

are you on a 32 or 64 bit system?
You could try 2 or more consecutive relative seeks in place of an absolute seek.
i.e.
File f = ... ;
f.seek(173445340 , SEEK_SET);
f.seek(173445340 , SEEK_REL);
also what does
f.seek(0,SEEK_END);
writeln(f.tell());
print?

> f.seek(173445340 , SEEK_SET);
> f.seek(173445340 , SEEK_REL);
oops that should be 3173445340.

On Sunday, 13 March 2016 at 12:21:11 UTC, Nicholas Wilson wrote:
> On Sunday, 13 March 2016 at 10:32:41 UTC, stunaep wrote:
>> I have a very large file I need to read data from at certain positions, but I have run into this error
>>> std.conv.ConvOverflowException@std\conv.d(1328): Conversion positive overflow
>> when seeking to 6346890680. Seeking to smaller values such as 3580720 work with no problem. The file is well over 8GB, so it must be able to read data at positions that high.
>
> are you on a 32 or 64 bit system?
> You could try 2 or more consecutive relative seeks in place of an absolute seek.
> i.e.
> File f = ... ;
> f.seek(173445340 , SEEK_SET);
> f.seek(173445340 , SEEK_REL);
> also what does
> f.seek(0,SEEK_END);
> writeln(f.tell());
> print?

I'm on 64 bit but it needs to work on both. It works for anything between 0 and 2147483647.

> f.seek(0,SEEK_END);
> writeln(f.tell());

that throws an error before reaching f.tell()

> std.exception.ErrnoException@std\stdio.d(920): Could not seek in file `./file.dat' (Invalid argument)
> ----------------
> 0x000000013FE67868 in @safe bool std.exception.errnoEnforce!(bool, "std\stdio.d", 920uL).errnoEnforce(bool, lazy immutable(char)[])
> 0x000000013FE4E7D3 in @trusted void std.stdio.File.seek(long, int)

Also, seeking relative to the current position throws the same error as in the original post if it's over max signed int.

> f.seek(2147483647, SEEK_SET);
> writeln(f.tell()); // prints 2147483647
> f.seek(4, SEEK_CUR); // throws error

On Monday, 14 March 2016 at 00:12:46 UTC, stunaep wrote:
> On Sunday, 13 March 2016 at 12:21:11 UTC, Nicholas Wilson wrote:
>> On Sunday, 13 March 2016 at 10:32:41 UTC, stunaep wrote:
>>> I have a very large file I need to read data from at certain positions, but I have run into this error
>>>> [...]
>>> when seeking to 6346890680. Seeking to smaller values such as 3580720 work with no problem. The file is well over 8GB, so it must be able to read data at positions that high.
>>
>> are you on a 32 or 64 bit system?
>> You could try 2 or more consecutive relative seeks in place of an absolute seek.
>> i.e.
>> File f = ... ;
>> f.seek(173445340 , SEEK_SET);
>> f.seek(173445340 , SEEK_REL);
>> also what does
>> f.seek(0,SEEK_END);
>> writeln(f.tell());
>> print?
>
> I'm on 64 bit but it needs to work on both. It works for anything between 0 and 2147483647.
>
>> f.seek(0,SEEK_END);
>> writeln(f.tell());
>
> that throws an error before reaching f.tell()
>
>> std.exception.ErrnoException@std\stdio.d(920): Could not seek in file `./file.dat' (Invalid argument)
>> ----------------
>> 0x000000013FE67868 in @safe bool std.exception.errnoEnforce!(bool, "std\stdio.d", 920uL).errnoEnforce(bool, lazy immutable(char)[])
>> 0x000000013FE4E7D3 in @trusted void std.stdio.File.seek(long, int)
>
> Also, seeking relative to the current position throws the same error as in the original post if it's over max signed int.
>
>> f.seek(2147483647, SEEK_SET);
>> writeln(f.tell()); // prints 2147483647
>> f.seek(4, SEEK_CUR); // throws error

Hmm. If you're getting an errno exception ( as opposed to a conv) I really don't
think that theres anything you can do about it, as its a problem with the C standard
lib. What OS/version are you running?

what does the equivalent in C give you.
i.e.

FILE* f = fopen(...,"r");
fseek(f,0,SEEK_END);
printf("%ld",ftell(f));
printf("%d",errno);

On Monday, 14 March 2016 at 03:07:05 UTC, Nicholas Wilson wrote:
> On Monday, 14 March 2016 at 00:12:46 UTC, stunaep wrote:
>> On Sunday, 13 March 2016 at 12:21:11 UTC, Nicholas Wilson wrote:
>>> [...]
>>
>> I'm on 64 bit but it needs to work on both. It works for anything between 0 and 2147483647.
>>
>>> [...]
>>
>> that throws an error before reaching f.tell()
>>
>>> [...]
>>
>> Also, seeking relative to the current position throws the same error as in the original post if it's over max signed int.
>>
>>> [...]
>
> Hmm. If you're getting an errno exception ( as opposed to a conv) I really don't
> think that theres anything you can do about it, as its a problem with the C standard
> lib. What OS/version are you running?
>
> what does the equivalent in C give you.
> i.e.
>
> FILE* f = fopen(...,"r");
> fseek(f,0,SEEK_END);
> printf("%ld",ftell(f));
> printf("%d",errno);

I'm currently on windows 7. The code you gave me prints 022. It's weird because it always tries to convert longs to ints and I think that is weird because the function uses a c_long.
> int fseek(FILE* stream, c_long offset, int whence)

It wont even compile if I try giving it a long
> Error: function core.stdc.stdio.fseek (shared(_iobuf)* stream, int offset, int whence) is not callable using argument types (shared(_iobuf)*, long, int)

I also tried giving it a c_long, but this line
>c_long test = 2147483649;
gives this error for some reason...
> Error: cannot implicitly convert expression (2147483649L) of type long to int

It's like c_longs are not longs at all...

It seems my lacking knowledge of C has gotten the best of me and longs in C only have a signed int range? It looks like in C, fseeko() is needed on linux and _fseeki64() is needed on windows, but I dont see either of these in stdc.stdio.

Just tested it on arch linux 64 bit and it works with no problem seeking to positions over 2^31-1

On Monday, 14 March 2016 at 05:24:48 UTC, stunaep wrote:
> On Monday, 14 March 2016 at 03:07:05 UTC, Nicholas Wilson wrote:
>> [...]
>
> I'm currently on windows 7. The code you gave me prints 022. It's weird because it always tries to convert longs to ints and I think that is weird because the function uses a c_long.
>> [...]
>
> It wont even compile if I try giving it a long
>> [...]
>
> I also tried giving it a c_long, but this line
>>[...]
> gives this error for some reason...
>> [...]
>
> It's like c_longs are not longs at all...

c_long maps to the target long which evidently is not 64-bit on win7
meaning you can't seek longer than int.max.
 if you cant't find _fseeki64 try looking in the windows section.
Glad you got it working on arch.

Nic

On Monday, 14 March 2016 at 07:15:01 UTC, Nicholas Wilson wrote:
> On Monday, 14 March 2016 at 05:24:48 UTC, stunaep wrote:
>> On Monday, 14 March 2016 at 03:07:05 UTC, Nicholas Wilson wrote:
>>> [...]
>>
>> I'm currently on windows 7. The code you gave me prints 022. It's weird because it always tries to convert longs to ints and I think that is weird because the function uses a c_long.
>>> [...]
>>
>> It wont even compile if I try giving it a long
>>> [...]
>>
>> I also tried giving it a c_long, but this line
>>>[...]
>> gives this error for some reason...
>>> [...]
>>
>> It's like c_longs are not longs at all...
>
> c_long maps to the target long which evidently is not 64-bit on win7
> meaning you can't seek longer than int.max.
>  if you cant't find _fseeki64 try looking in the windows section.
> Glad you got it working on arch.
>
> Nic

It looks like _fseeki64 is in the nightly build but not dmd 2.070.2; However, the nightly build says std.stdio and std.conv are deprecated and I cant use them.