While working on a lot of enhancements for emesx'es tga library (almost a rewrite), I was thinking on implementing a way for handling memory locations as files, which would make handling archives mush easier, since I wouldn't have to write a separate overload (or more) for loading files from memory every time I need access to a new type.

I'm thinking on making a library for it. Let me know if you want to add any additional functions. I'll making it under the Boost license, and also planning to add better binary writing and reading capabilities (e.g. direct reading and writing of simple types and certain structs).

On Thursday, 6 December 2018 at 02:48:59 UTC, solidstate1991 wrote:
> While working on a lot of enhancements for emesx'es tga library (almost a rewrite), I was thinking on implementing a way for handling memory locations as files, which would make handling archives mush easier, since I wouldn't have to write a separate overload (or more) for loading files from memory every time I need access to a new type.
>
> I'm thinking on making a library for it. Let me know if you want to add any additional functions. I'll making it under the Boost license, and also planning to add better binary writing and reading capabilities (e.g. direct reading and writing of simple types and certain structs).

Didn't you think in the opposite direction - a memory mapped file (std.mmfile), accessing a file like it were a memory location ?

On Thu, Dec 06, 2018 at 02:48:59AM +0000, solidstate1991 via Digitalmars-d wrote:
> While working on a lot of enhancements for emesx'es tga library (almost a rewrite), I was thinking on implementing a way for handling memory locations as files, which would make handling archives mush easier, since I wouldn't have to write a separate overload (or more) for loading files from memory every time I need access to a new type.
[...]

There's no need for a separate overload.  All you need to do is:

	import std.stdio;
	auto myfunc(File = std.stdio.File, Args...)(File fp, Args args)
	{
		...
		fp.rawRead(...);
		...
		fp.rawWrite(...);
		... // etc
	}

	// uses std.stdio.File
	myfunc(File("realfile.txt", "r"), ...);

	struct InMemoryFile {
		...
		void[] rawRead(void[]) { ... }
		void rawWrite(void[]) { ... }
		...
	}

	// uses InMemoryFile
	myfunc(InMemoryFile(...), ...);

As long as the type you pass in has the same API as std.stdio.File (or a subset thereof, as long as that subset is the only thing you use in the called function), the code will work as-is with no additional work.

Writing it this way also makes it easy to test your code without having your unittests pollute the filesystem with temporaries (and needing to clean up the mess afterwards).

If you're concerned about template bloat, you could factor out the common API into an interface, then write a wrapper for std.stdio.File.


T

-- 
"A man's wife has more power over him than the state has." -- Ralph Emerson

On Thursday, 6 December 2018 at 16:11:55 UTC, H. S. Teoh wrote:
> There's no need for a separate overload.  All you need to do is:
>
> 	import std.stdio;
> 	auto myfunc(File = std.stdio.File, Args...)(File fp, Args args)
> 	{
> 		...
> 		fp.rawRead(...);
> 		...
> 		fp.rawWrite(...);
> 		... // etc
> 	}
>
> 	// uses std.stdio.File
> 	myfunc(File("realfile.txt", "r"), ...);
>
> 	struct InMemoryFile {
> 		...
> 		void[] rawRead(void[]) { ... }
> 		void rawWrite(void[]) { ... }
> 		...
> 	}
>
> 	// uses InMemoryFile
> 	myfunc(InMemoryFile(...), ...);
>
> As long as the type you pass in has the same API as std.stdio.File (or a subset thereof, as long as that subset is the only thing you use in the called function), the code will work as-is with no additional work.
>
> Writing it this way also makes it easy to test your code without having your unittests pollute the filesystem with temporaries (and needing to clean up the mess afterwards).
>
> If you're concerned about template bloat, you could factor out the common API into an interface, then write a wrapper for std.stdio.File.
>
>
> T

Thanks! It's mostly done except for some testing, which I'll do tomorrow.

It has a few extra capabilities (such as reading and writing single elements with heap allocation), which I might try to put into std.stdio.File too through a pull request.