Thread overview
Handling large data structures.
May 03, 2006
rvw
May 03, 2006
Jan Knepper
May 03, 2006
rvw
May 03, 2006
Walter Bright
May 04, 2006
rvw
May 07, 2006
Walter Bright
May 09, 2006
rvw
May 03, 2006
I am writing programs that execute from the command line of Windows 2000.  I want to be able to do calculations with large data files and structures.  I have been compiling with the options:  -5 -o -A -r -v2 -w -mn.

I thought the -mn option would give me 4GB of data space.  However, if I create and write to an array of something over 256,000 unsigned longs (x4 bytes = 1MB), the program compiles but crashes.

How can I use the whole, or most of, the 4GB?

Thanks for you help.

rvw


May 03, 2006
How do you create that array?
How about posting an example?



rvw wrote:
> I am writing programs that execute from the command line of Windows 2000.  I
> want to be able to do calculations with large data files and structures.  I have
> been compiling with the options:  -5 -o -A -r -v2 -w -mn.
> 
> I thought the -mn option would give me 4GB of data space.  However, if I create
> and write to an array of something over 256,000 unsigned longs (x4 bytes = 1MB),
> the program compiles but crashes.
> 
> How can I use the whole, or most of, the 4GB?
> 
> Thanks for you help.
> 
> rvw
> 
> 


-- 
ManiaC++
Jan Knepper

But as for me and my household, we shall use Mozilla...
www.mozilla.org
May 03, 2006
The following is my test program.  It compiles and executes for up to 256000 or so, but compiles and fails at 260000.  The error message is "t.exe has generated errors and will be closed by Windows."

#include <stdio.h>
#include <stdlib.h>

unsigned long i;

int main()
{
unsigned long Long[256000];

for(i=0;i<=255999;i+=10000)
{
Long[i] = i;
printf("%u\n",Long[i]);
}

return EXIT_SUCCESS;
}


In article <e392o2$6l2$1@digitaldaemon.com>, Jan Knepper says...
>
>How do you create that array?
>How about posting an example?
>
>
>
>rvw wrote:
>> I am writing programs that execute from the command line of Windows 2000.  I want to be able to do calculations with large data files and structures.  I have been compiling with the options:  -5 -o -A -r -v2 -w -mn.
>> 
>> I thought the -mn option would give me 4GB of data space.  However, if I create and write to an array of something over 256,000 unsigned longs (x4 bytes = 1MB), the program compiles but crashes.
>> 
>> How can I use the whole, or most of, the 4GB?
>> 
>> Thanks for you help.
>> 
>> rvw
>> 
>> 
>
>
>-- 
>ManiaC++
>Jan Knepper
>
>But as for me and my household, we shall use Mozilla... www.mozilla.org


May 03, 2006
That's because Windows gives a limited amount of space to the stack, and you're allocating the array upon the stack. Try instead allocating the array with calloc().

rvw wrote:
> The following is my test program.  It compiles and executes for up to 256000 or
> so, but compiles and fails at 260000.  The error message is "t.exe has generated
> errors and will be closed by Windows."
> 
> #include <stdio.h>
> #include <stdlib.h>
> 
> unsigned long i;
> 
> int main()
> {
> unsigned long Long[256000];
> 
> for(i=0;i<=255999;i+=10000) {
> Long[i] = i;
> printf("%u\n",Long[i]);
> }
> 
> return EXIT_SUCCESS;
> }
> 
> 
> In article <e392o2$6l2$1@digitaldaemon.com>, Jan Knepper says...
>> How do you create that array?
>> How about posting an example?
>>
>>
>>
>> rvw wrote:
>>> I am writing programs that execute from the command line of Windows 2000.  I
>>> want to be able to do calculations with large data files and structures.  I have
>>> been compiling with the options:  -5 -o -A -r -v2 -w -mn.
>>>
>>> I thought the -mn option would give me 4GB of data space.  However, if I create
>>> and write to an array of something over 256,000 unsigned longs (x4 bytes = 1MB),
>>> the program compiles but crashes.
>>>
>>> How can I use the whole, or most of, the 4GB?
>>>
>>> Thanks for you help.
>>>
>>> rvw
>>>
>>>
>>
>> -- 
>> ManiaC++
>> Jan Knepper
>>
>> But as for me and my household, we shall use Mozilla...
>> www.mozilla.org
> 
> 
May 04, 2006
Thanks for your help Walter.  CALLOC does work through 400 million unsigned ints in the following program.  It fails at 500 million -- "Abnormal program termination."

My computer has 1GB of RAM. Is that the (first) limiting factor? (By the way,
there was a lot of hard drive activity during program execution -- Windows
finding working room?)

//t.c     This works at least through 400 million, as written.

#include <dos.h>
#include <stdio.h>
#include <stdlib.h>

unsigned long i;

int main()
{
unsigned long *A;

A = calloc (400000001, sizeof(unsigned long));
if (A == NULL)
{
fprintf (stderr, "Calloc failed");
abort();
}

for (i = 0;i<=399999999;i+=10000000)
{
A[i]=i;
printf("%-12u%-u   \n",i, A[i]);
}

return EXIT_SUCCESS;
}

_______________________________________________________________________ In article <e39ap8$jsg$1@digitaldaemon.com>, Walter Bright says...
>
>That's because Windows gives a limited amount of space to the stack, and you're allocating the array upon the stack. Try instead allocating the array with calloc().
>
>rvw wrote:
>> The following is my test program.  It compiles and executes for up to 256000 or so, but compiles and fails at 260000.  The error message is "t.exe has generated errors and will be closed by Windows."
>> 
>> #include <stdio.h>
>> #include <stdlib.h>
>> 
>> unsigned long i;
>> 
>> int main()
>> {
>> unsigned long Long[256000];
>> 
>> for(i=0;i<=255999;i+=10000)
>> {
>> Long[i] = i;
>> printf("%u\n",Long[i]);
>> }
>> 
>> return EXIT_SUCCESS;
>> }
>> 
>> 
>> In article <e392o2$6l2$1@digitaldaemon.com>, Jan Knepper says...
>>> How do you create that array?
>>> How about posting an example?
>>>
>>>
>>>
>>> rvw wrote:
>>>> I am writing programs that execute from the command line of Windows 2000.  I want to be able to do calculations with large data files and structures.  I have been compiling with the options:  -5 -o -A -r -v2 -w -mn.
>>>>
>>>> I thought the -mn option would give me 4GB of data space.  However, if I create and write to an array of something over 256,000 unsigned longs (x4 bytes = 1MB), the program compiles but crashes.
>>>>
>>>> How can I use the whole, or most of, the 4GB?
>>>>
>>>> Thanks for you help.
>>>>
>>>> rvw
>>>>
>>>>
>>>
>>> -- 
>>> ManiaC++
>>> Jan Knepper
>>>
>>> But as for me and my household, we shall use Mozilla... www.mozilla.org
>> 
>> 


May 07, 2006
500 million unsigned ints is 2Gb of memory.

rvw wrote:
> Thanks for your help Walter.  CALLOC does work through 400 million unsigned ints
> in the following program.  It fails at 500 million -- "Abnormal program
> termination." 
> 
> My computer has 1GB of RAM. Is that the (first) limiting factor? (By the way,
> there was a lot of hard drive activity during program execution -- Windows
> finding working room?)
May 09, 2006
I know.  That's why I was surprised that 400 million worked -- that's 1.6GB. But as I said, there was a lot of hard drive activity.  My guess is that Windows automatically used hard drive space.  Again, thanks for your help.

In article <e3je8h$1gmu$1@digitaldaemon.com>, Walter Bright says...
>
>500 million unsigned ints is 2Gb of memory.
>
>rvw wrote:
>> Thanks for your help Walter.  CALLOC does work through 400 million unsigned ints in the following program.  It fails at 500 million -- "Abnormal program termination."
>> 
>> My computer has 1GB of RAM. Is that the (first) limiting factor? (By the way,
>> there was a lot of hard drive activity during program execution -- Windows
>> finding working room?)