My program doesn't exit..

void main()
{

..
Data data[]

createData(data);
//This proces takes a minute or two.
//data now holds about 700000 Data elements

 writefln(`Creating buffer`);
 ubyte[] buffer;
 for (int i=0; i<data.length; i++)
 {
  buffer~=format(data[i].net,`,`,data[i].name,`,`,to!(int[])(data[i].output),`,`,to!(int[])(data[i].input),newline);
  //This takes like a minute and has a peak memory usage of 680MB
 }

 writefln(`Writing file`);
 std.file.write(outFilename, buffer);
writefln(`Writing file finished`);

//The file is readily made and delta memory usage shows a few a few times
around 20MB
//but the program doesn't stop, although 'Writing file finished' is shown.
//there is no static this or alike.
//and after 10 minutes the memory usage is about 370MB and slowly going up
//ctrl-c doesn't work anymore as well
}

On Tue, Jan 27, 2009 at 10:14 AM, Saaa <empty@needmail.com> wrote:
> My program doesn't exit..
>
> void main()
> {
>
> ..
> Data data[]
>
> createData(data);
> //This proces takes a minute or two.
> //data now holds about 700000 Data elements
>
>  writefln(`Creating buffer`);
>  ubyte[] buffer;
>  for (int i=0; i<data.length; i++)
>  {
>  buffer~=format(data[i].net,`,`,data[i].name,`,`,to!(int[])(data[i].output),`,`,to!(int[])(data[i].input),newline);
>  //This takes like a minute and has a peak memory usage of 680MB
>  }
>
>  writefln(`Writing file`);
>  std.file.write(outFilename, buffer);
> writefln(`Writing file finished`);
>
> //The file is readily made and delta memory usage shows a few a few times
> around 20MB
> //but the program doesn't stop, although 'Writing file finished' is shown.
> //there is no static this or alike.
> //and after 10 minutes the memory usage is about 370MB and slowly going up
> //ctrl-c doesn't work anymore as well
> }

Bug in the GC?

I suppose my question would be why you're generating that huge buffer in memory instead of just writing it all out to the file in the first place.  Use a std.stream.BufferedFile.

>
> Bug in the GC?
>
> I suppose my question would be why you're generating that huge buffer in memory instead of just writing it all out to the file in the first place.  Use a std.stream.BufferedFile.

Because it just needed to work and I have no experience with streams.. :/ And, writing it line by line was too slow (although I'm not sure anymore since it might be the same problem)

Maybe I can help...


Saaa Wrote:

> My program doesn't exit..
> 
> void main()
> {
> 
> ..
> Data data[]
> 
> createData(data);
> //This proces takes a minute or two.
> //data now holds about 700000 Data elements
> 
>  writefln(`Creating buffer`);
>  ubyte[] buffer;
>  for (int i=0; i<data.length; i++)
>  {
>   buffer~=format(data[i].net,`,`,data[i].name,`,`,to!(int[])(data[i].output),`,`,to!(int[])(data[i].input),newline);
>   //This takes like a minute and has a peak memory usage of 680MB
>  }
> 
>  writefln(`Writing file`);
>  std.file.write(outFilename, buffer);
> writefln(`Writing file finished`);
> 
> //The file is readily made and delta memory usage shows a few a few times
> around 20MB
> //but the program doesn't stop, although 'Writing file finished' is shown.
> //there is no static this or alike.
> //and after 10 minutes the memory usage is about 370MB and slowly going up
> //ctrl-c doesn't work anymore as well
> }
> 
> 

Alan Turing wrote:
> Maybe I can help...

At first glance, this might look like someone being silly and not at all helpful... but in truth, Mr. Turing has probably already fixed your source code with his incredible powers.  If you notice your firewall has melted into a pile of slag, and your PC seems to be running a lot faster, that'd be him.

Most people seem to think he's dead; this is only partially true. Whilst his physical body died, his mind lives on in the Internet, ever vigilant to ensure the safe delivery of our daily LOLCATs.

Some other random Alan Turing facts:

* The only reason Alan Turing hadn't solved the halting problem is that
it was too busy running from him in fear.
* Alan Turing could tell what a RAM chip contains simply by smelling it.
* Alan Turing programmed computers in raw machine code, using a
magnetised pin.
* Alan Turing didn't bother programming; he just stared at a computer
until it submits to his will.
* Alan Turing only designed the bombe because "he just stared at
ciphertexts for 10 seconds and wrote down the plaintext" wouldn't make
for an interesting documentary 70 years later.
* Steve Ballmer lives in perpetual fear of the day Alan Turing will
return to unmake Microsoft.
* When Alan Turing crashes a machine, all the other machines on the same
subnet dump core out of fear.
* Alan Turing can break RSA encryption up to 4096 bits; he doesn't
bother because it's not challenging enough.
* Alan Turing knows what you were browsing last summer.

  -- Daniel "insomnia is a terrible thing..." Keep

On Tue, Jan 27, 2009 at 11:04 AM, Alan Turing <at@bletchley.park.uk> wrote:
> Maybe I can help...
>

Looool.

Reply to Saaa,

> My program doesn't exit..
> 
> void main()
> {
> ..
> Data data[]
> createData(data);
> //This proces takes a minute or two.
> //data now holds about 700000 Data elements
> writefln(`Creating buffer`);
> ubyte[] buffer;
> for (int i=0; i<data.length; i++)
> {
> 
> buffer~=format(data[i].net,`,`,data[i].name,`,`,to!(int[])(data[i].out
> put),`,`,to!(int[])(data[i].input),newline);

~= in a short loop is a really bad thing. It will tend to need twice the ram you really needed and hammers the GC. Also if you start getting false pointers it gives them lots of memory segments to land in.

> //This takes like a minute and has a peak memory usage of 680MB
> }
> writefln(`Writing file`);
> std.file.write(outFilename, buffer);
> writefln(`Writing file finished`);
> //The file is readily made and delta memory usage shows a few a few
> times
> around 20MB
> //but the program doesn't stop, although 'Writing file finished' is
> shown.
> //there is no static this or alike.
> //and after 10 minutes the memory usage is about 370MB and slowly
> going up
> //ctrl-c doesn't work anymore as well
> }

"Saaa" wrote
> My program doesn't exit..

Have you tried a debugger, breaking into the program when it is hanging to find out where it is?

-Steve

No, normally writefln suffices.. :)
btw. Why do writef writes not always show up at the expected places?

>
> Have you tried a debugger, breaking into the program when it is hanging to find out where it is?
>
> -Steve
> 

On Tue, Jan 27, 2009 at 5:33 PM, Saaa <empty@needmail.com> wrote:
>
> No, normally writefln suffices.. :)
> btw. Why do writef writes not always show up at the expected places?

If you mean that sometimes you write something but it never shows up, it's because it buffers the output.  You have to use fflush(stdout) to see the output immediately.

Unless you mean something else.  <_<