Thread overview
Read a file to the end, new data is appended, no more is read?
Jun 20, 2013
Gary Willoughby
Jun 20, 2013
Gary Willoughby
Jun 20, 2013
Ali Çehreli
Jun 20, 2013
Gary Willoughby
June 20, 2013
I have a simple problem, I want to open a file and read data from it. When the file changes, i will read more data. The trouble is i only want to open the file once.

Here's the simplified code:

import core.thread;
import std.stdio;

void main(string[] args)
{
	auto file = File("file.txt", "r");

	string line;

	writeln("First Read:");

	while ((line = file.readln()) !is null)
	{
		write(line);
	}

	Thread.sleep(dur!("seconds")(5)); // <-- More data is appended to the file while waiting here.

	writeln("Second Read:");

	while ((line = file.readln()) !is null) // <-- Error! New data is not read!
	{
		write(line);
	}
}

I read the file, then when its paused, i add new lines to the opened file. When the program resumes it totally ignores the new lines. doh!

I guess this is the result of buffering somewhere so i've tried all sorts to try and escape this behaviour, including:

file.clearerr()
file.seek()
file.flush()

Nothing seems to work. What am i missing? It must be something simple. Any ideas?
June 20, 2013
	auto file = File("file.txt", "r");

	char[1024] buffer;
	char[] line;

	writeln("First Read:");

	while ((line = file.rawRead(buffer)) !is null)
	{
 		write(line);
	}

	Thread.sleep(dur!("seconds")(5));

	writeln("Second Read:");

	while ((line = file.rawRead(buffer)) !is null)
	{
	 	write(line);
	 }

Tried using the raw access commands still no joy. So i guess it's still the same issue.

Also tried C stdio too.

	char[1024] buffer;
	ulong offset;

	FILE* file = fopen("text.txt", "r");

	printf("First Read:\n");

	while(fgets(buffer.ptr, buffer.length, file) !is null)
	{
		printf("%s", buffer.ptr);
	}

	offset = ftell(file);
	clearerr(file);

	Thread.sleep(dur!("seconds")(5));

	fseek(file, offset, SEEK_SET);

	printf("Second Read:\n");

	while(fgets(buffer.ptr, buffer.length, file) !is null)
	{
		printf("%s", buffer.ptr);
	}

	fclose(file);

Any ideas?
June 20, 2013
On 06/20/2013 07:10 AM, Gary Willoughby wrote:
> I have a simple problem, I want to open a file and read data from it.
> When the file changes, i will read more data. The trouble is i only want
> to open the file once.
>
> Here's the simplified code:
>
> import core.thread;
> import std.stdio;
>
> void main(string[] args)
> {
>      auto file = File("file.txt", "r");
>
>      string line;
>
>      writeln("First Read:");
>
>      while ((line = file.readln()) !is null)
>      {
>          write(line);
>      }
>
>      Thread.sleep(dur!("seconds")(5)); // <-- More data is appended to
> the file while waiting here.
>
>      writeln("Second Read:");
>
>      while ((line = file.readln()) !is null) // <-- Error! New data is
> not read!
>      {
>          write(line);
>      }
> }
>
> I read the file, then when its paused, i add new lines to the opened
> file. When the program resumes it totally ignores the new lines. doh!
>
> I guess this is the result of buffering somewhere so i've tried all
> sorts to try and escape this behaviour, including:
>
> file.clearerr()
> file.seek()
> file.flush()
>
> Nothing seems to work. What am i missing? It must be something simple.
> Any ideas?

This must be platform-dependent. Your program works as expected under my Scientific Linux distribution. (Same as Red Hat.)

I have tried four kinds of modifications to the file:

1) Append a new line: works as expected

    echo new line >> file.txt

2) Overwrite the same file: works as expected

    echo new content > file.txt

3) Delete the file and recreate: works as expected

    rm file.txt; echo new content > file.txt

4) Rename the file: works better than expected :)

    mv file.txt file2.txt; echo new line >> file2.txt

In any case, there must be lots to learn from the 'tail' utility. Its '-f' command line switch achieves what you need, and '-F' achieves more than what you need:

  http://git.savannah.gnu.org/cgit/coreutils.git/tree/src/tail.c

Ali

June 20, 2013
> This must be platform-dependent. Your program works as expected under my Scientific Linux distribution. (Same as Red Hat.)
> Ali

You're right, it seems to be a Mac OS limitation. It works fine here on Ubuntu.