https://dlang.org/library/std/mmfile/mm_file.html doesn't seem to specify anything similar to lines() or byLine() or byLineCopy() etc.

On Sat, Mar 14, 2020 at 10:37:37PM +0000, Adnan via Digitalmars-d-learn wrote:
> https://dlang.org/library/std/mmfile/mm_file.html doesn't seem to
> specify anything similar to lines() or byLine() or byLineCopy() etc.

That's because a memory-mapped file appears directly in your program's memory address space as if it was an array of bytes (ubyte[]).  No interpretation is imposed upon the contents.  If you want lines out of it, try casting the memory to const(char)[] and using std.algorithm.splitter to get a range of lines. For example:

	auto mmfile = new MmFile("myfile.txt");
	auto data = cast(const(char)[]) mmfile[];
	auto lines = data.splitter("\n");
	foreach (line; lines) {
		...
	}


T

-- 
"You are a very disagreeable person." "NO."

On Sunday, 15 March 2020 at 00:37:35 UTC, H. S. Teoh wrote:
> On Sat, Mar 14, 2020 at 10:37:37PM +0000, Adnan via Digitalmars-d-learn wrote:
>> https://dlang.org/library/std/mmfile/mm_file.html doesn't seem to
>> specify anything similar to lines() or byLine() or byLineCopy() etc.
>
> That's because a memory-mapped file appears directly in your program's memory address space as if it was an array of bytes (ubyte[]).  No interpretation is imposed upon the contents.  If you want lines out of it, try casting the memory to const(char)[] and using std.algorithm.splitter to get a range of lines. For example:
>
> 	auto mmfile = new MmFile("myfile.txt");
> 	auto data = cast(const(char)[]) mmfile[];
> 	auto lines = data.splitter("\n");
> 	foreach (line; lines) {
> 		...
> 	}
>
>
> T

Would it be wasteful to cast the entire content into a const string? Can a memory mapped file be read with a buffer?

On Monday, 16 March 2020 at 13:09:08 UTC, Adnan wrote:
> On Sunday, 15 March 2020 at 00:37:35 UTC, H. S. Teoh wrote:
>> On Sat, Mar 14, 2020 at 10:37:37PM +0000, Adnan via Digitalmars-d-learn wrote:
>>> https://dlang.org/library/std/mmfile/mm_file.html doesn't seem to
>>> specify anything similar to lines() or byLine() or byLineCopy() etc.
>>
>> That's because a memory-mapped file appears directly in your program's memory address space as if it was an array of bytes (ubyte[]).  No interpretation is imposed upon the contents.  If you want lines out of it, try casting the memory to const(char)[] and using std.algorithm.splitter to get a range of lines. For example:
>>
>> 	auto mmfile = new MmFile("myfile.txt");
>> 	auto data = cast(const(char)[]) mmfile[];
>> 	auto lines = data.splitter("\n");
>> 	foreach (line; lines) {
>> 		...
>> 	}
>>
>>
>> T
>
> Would it be wasteful to cast the entire content into a const string? Can a memory mapped file be read with a buffer?

for more context here's the program

string lexHash(scope const ref string arg) {
	auto repeated = arg ~ arg;
	string result = arg;
	for (auto i = 1; i < arg.length; ++i) {
		const slice = repeated[i .. i + arg.length];
		if (slice < result)
			result = slice;
	}
	return result;
}

unittest {
	assert("cba".smallestRepr() == "acb");
}

void main(const string[] args) {
	import std.stdio : writeln, lines, File;
	import std.algorithm : splitter;
	import std.mmfile : MmFile;

	string[][const string] wordTable;
	scope auto mmfile = new MmFile(args[1]);
	auto data = cast(const string) mmfile[];

	foreach (string word; data.splitter()) {
		const string key = word.lexHash();
		wordTable.require(key, []) ~= word;
		if (wordTable[key].length == 4) {
			writeln(wordTable[key]);
			break;
		}
	}
}

On Monday, 16 March 2020 at 13:09:08 UTC, Adnan wrote:
> On Sunday, 15 March 2020 at 00:37:35 UTC, H. S. Teoh wrote:
>> On Sat, Mar 14, 2020 at 10:37:37PM +0000, Adnan via Digitalmars-d-learn wrote:
>>> [...]
>>
>> That's because a memory-mapped file appears directly in your program's memory address space as if it was an array of bytes (ubyte[]).  No interpretation is imposed upon the contents.  If you want lines out of it, try casting the memory to const(char)[] and using std.algorithm.splitter to get a range of lines. For example:
>>
>> 	auto mmfile = new MmFile("myfile.txt");
>> 	auto data = cast(const(char)[]) mmfile[];
>> 	auto lines = data.splitter("\n");
>> 	foreach (line; lines) {
>> 		...
>> 	}
>>
>>
>> T
>
> Would it be wasteful to cast the entire content into a const string? Can a memory mapped file be read with a buffer?

a string is the same thing as immutable(char)[] . It would make no difference with the example above.

On Mon, Mar 16, 2020 at 01:09:08PM +0000, Adnan via Digitalmars-d-learn wrote: [...]
> Would it be wasteful to cast the entire content into a const string?

Why would it be?  It's just reinterpreting a pointer.


> Can a memory mapped file be read with a buffer?

That totally defeats the purpose of memory-mapping.  The whole idea of memory-mapping is that the OS takes care of buffering the file data for you.  As far as your program is concerned, the entire file is actually "in memory" (it isn't really, but since the OS takes care of paging parts of it in and out as necessary, and this is completely transparent to your program, you might as well treat it as a huge consecutive array that resides, for all intents and purposes, "in memory" -- hence the term "virtual memory").


T

-- 
There are three kinds of people in the world: those who can count, and those who can't.