View mode: basic / threaded / horizontal-split · Log in · Help
December 30, 2012
Re: Ranges longer than size_t.max
12/30/2012 10:23 PM, SomeDude пишет:
> On Saturday, 29 December 2012 at 06:12:52 UTC, Mehrdad wrote:
>> Uhm, 64-bit?
>>
>> What about large files (> 4 GiB) on 32-bit systems?
>>
>> (Say, if a range wants to provide random access to the bytes of an 8
>> GiB file?)
>>
>> And what about large bit vectors? A 600 MiB bit vector needs > 2^32
>> bits for length...
>
> AFAIK, no 32-bit OS supports files larger than 4Gb anyway.

False. It's solely a question of FS used.
NTFS supports files larger then 4 Gb regardless of version Windows 
(Win2K+ or even earlier). It doesn't matter what the bitness of OS in 
question is. I suspect 32bit linux also has > 4Gb files even with ext2 
no problem.

And e.g. FAT32 remarkably can't handle 4 Gb+ files with any OS.
-- 
Dmitry Olshansky
December 30, 2012
Re: Ranges longer than size_t.max
On Sunday, December 30, 2012 19:23:13 SomeDude wrote:
> On Saturday, 29 December 2012 at 06:12:52 UTC, Mehrdad wrote:
> > Uhm, 64-bit?
> > 
> > What about large files (> 4 GiB) on 32-bit systems?
> > 
> > (Say, if a range wants to provide random access to the bytes of
> > an 8 GiB file?)
> > 
> > And what about large bit vectors? A 600 MiB bit vector needs >
> > 2^32 bits for length...
> 
> AFAIK, no 32-bit OS supports files larger than 4Gb anyway.

Um. No. There's a reason that things like the macro _LARGE_FILE_API exist. 
While some file systems won't allow larger file sizes, that's generally an FS 
limitation, not an OS limitation. The OSes provide ways to operate on 64-bit 
files on 32-bit systems. If they didn't, it would be impossible to do something 
like read a Blu-ray on a 32-bit system.

http://en.wikipedia.org/wiki/Large_file_support

- Jonathan M Davis
December 31, 2012
Re: Ranges longer than size_t.max
On 29/12/2012 21:13, Peter Alexander wrote:
> On Saturday, 29 December 2012 at 20:59:56 UTC, Stewart Gordon wrote:
<snip>
>> If we define a length that may overflow, then sooner or later some
>> code will be called on it that calls hasLength on the range, finds it
>> to be true, and then relies on length and malfunctions because the
>> value returned doesn't match the actual length.
>
> Using this logic, chain shouldn't have length either (it can overflow,
> as I demonstrated in the original post), but it does, and it should.
> There are many ranges in Phobos that may overflow in length.

Yes, you have a point there.  Any facility for creating a range by 
combining ranges is liable to overflow if it defines a length property. 
 But I would expect that in practice 99% of finite ranges would be 
nowhere near 2^64 in size, so an overflow would be an exceptional case.

> The problem with permutations is that, for the kinds of permutations you
> are likely to use often, length will not overflow. It would be really
> useful to have length available when this is the case.

Maybe the solution is to define two permutation range classes: one with 
length that asserts that the number of elements <= 20, and one without 
length that places no bound.

If the set size is a template parameter rather than being set at run 
time, then it can define length conditionally.

Stewart.
December 31, 2012
Re: Ranges longer than size_t.max
Am Sun, 30 Dec 2012 19:23:13 +0100
schrieb "SomeDude" <lovelydear@mailmetrash.com>:

> On Saturday, 29 December 2012 at 06:12:52 UTC, Mehrdad wrote:
> AFAIK, no 32-bit OS supports files larger than 4Gb anyway.

Anachronism!!!
4.7 GB DVD burners: ~1997
64-bit personal computers: 2003
(Also backups of large partitions into disk images were made)

That said, my father owns a stand-alone, multi-media, USB
disk drive, that cannot store files > 4GB because of the used
FAT32.

-- 
Marco
December 31, 2012
Re: Ranges longer than size_t.max
On Saturday, 29 December 2012 at 01:18:37 UTC, Peter Alexander 
wrote:
> 1. Allow ranges to return BigInt for length.
> 2. Allow ranges like this but assert when .length overflows.
> 3. Allow ranges like this and just allow .length to overflow 
> dangerously.
> 4. Do not allow ranges like this.

 Option 5: Incorporate (emulation code?) for cent/ucent, then 
consider that the same as 1 but returning cent rather than BitInt.
December 31, 2012
Re: Ranges longer than size_t.max
On Monday, 31 December 2012 at 17:43:34 UTC, Era Scarecrow wrote:
> On Saturday, 29 December 2012 at 01:18:37 UTC, Peter Alexander 
> wrote:
>> 1. Allow ranges to return BigInt for length.
>> 2. Allow ranges like this but assert when .length overflows.
>> 3. Allow ranges like this and just allow .length to overflow 
>> dangerously.
>> 4. Do not allow ranges like this.
>
>  Option 5: Incorporate (emulation code?) for cent/ucent, then 
> consider that the same as 1 but returning cent rather than 
> BitInt.

Two problems with this:

1. Still complicates range code (have to accomodate for both 
size_t and ucent return value for .length).

2. ucent probably isn't enough either for these sorts of ranges.
December 31, 2012
Re: Ranges longer than size_t.max
On Sunday, 30 December 2012 at 19:11:51 UTC, Dmitry Olshansky 
wrote:
> False. It's solely a question of FS used.
> NTFS supports files larger then 4 Gb regardless of version 
> Windows (Win2K+ or even earlier). It doesn't matter what the 
> bitness of OS in question is. I suspect 32bit linux also has > 
> 4Gb files even with ext2 no problem.
>
> And e.g. FAT32 remarkably can't handle 4 Gb+ files with any OS.

 I was writing some code to go through the FAT12/16/32, and some 
interesting information was found (Although there was so much 
overhead I kinda stopped in the middle of it). FAT32 actually 
uses something like 28/29 bits for the sector id, the other 3-4 
bits were flags like bad sectors, last sector and other minor 
data. At least that's what I remember off hand.

 http://en.wikipedia.org/wiki/Fat32
December 31, 2012
Re: Ranges longer than size_t.max
On Monday, 31 December 2012 at 17:47:36 UTC, Peter Alexander 
wrote:
> On Monday, 31 December 2012 at 17:43:34 UTC, Era Scarecrow 
> wrote:
>> On Saturday, 29 December 2012 at 01:18:37 UTC, Peter Alexander 
>> wrote:
>>> 1. Allow ranges to return BigInt for length.
>>> 2. Allow ranges like this but assert when .length overflows.
>>> 3. Allow ranges like this and just allow .length to overflow 
>>> dangerously.
>>> 4. Do not allow ranges like this.
>>
>> Option 5: Incorporate (emulation code?) for cent/ucent, then 
>> consider that the same as 1 but returning cent rather than 
>> BitInt.
>
> Two problems with this:
>
> 1. Still complicates range code (have to accomodate for both 
> size_t and ucent return value for .length).
>
> 2. ucent probably isn't enough either for these sorts of ranges.

 Perhaps, but having cent/ucent available would make a few people 
happy at least, although except for a few cases with databases 
and encryption it may not be that important right now.

 I remember trying to learn C/C++ and trying to do a very very 
simple code to get the number 1 doubled something like 100 times. 
After 31 it went to garbage (hey I was 14 at the time, god that 
was a long time ago). It took me a while to understand the 
problem.

 I ended up re-writing the whole thing in assembly that took 101 
bytes (COM file) and could double the number as many times as I 
wanted.
December 31, 2012
Re: Ranges longer than size_t.max
On Monday, 31 December 2012 at 17:56:37 UTC, Era Scarecrow wrote:
>  Perhaps, but having cent/ucent available would make a few 
> people happy at least

Not to change the subject, but couldn't we just implement Cent 
and UCent as library types, such as Complex, or HalfFloat?

I'd be tempted to open an ER and see if walter is up for it...
December 31, 2012
Re: Ranges longer than size_t.max
On Monday, 31 December 2012 at 18:18:08 UTC, monarch_dodra wrote:
> On Monday, 31 December 2012 at 17:56:37 UTC, Era Scarecrow 
> wrote:
>> Perhaps, but having cent/ucent available would make a few 
>> people happy at least
>
> Not to change the subject, but couldn't we just implement Cent 
> and UCent as library types, such as Complex, or HalfFloat?
>
> I'd be tempted to open an ER and see if walter is up for it...

 Yes we could. Just that they are already defined built-in types 
according to the specs. *shrugs* But changing your code from Cent 
to cent after it's properly implemented seems like a minimal 
change.

 If walter says go ahead (and I'm sure he won't have a problem 
with it), then I guess it's up to whoever to make it.
1 2 3 4
Top | Discussion index | About this forum | D home