We're long overdue for a "D is awesome" post

May 30, 2023

H. S. Teoh

May 30, 2023

Walter Bright

May 31, 2023

May 31, 2023

Jun 01, 2023

May 31, 2023

May 30, 2023

We're long overdue for a "D is awesome" post

Posted by H. S. Teoh

Permalink

H. S. Teoh

Permalink

So here's one.

Today, I was writing some code that iterates over a data structure and writes output to bunch of different files. It looks something like this:

void genSplitHtml(Data data, ...) {
	auto outputTemplate = File("template.html", "r");
	foreach (...) {
		auto filename = generateFilename(...);
		auto sink = File(filename, "w").lockingTextWriter;
		...
		while (line; outputTemplate.byLine) {
			if (line.canFind("MAGIC_TOKEN")) {
				generateOutput(...);
			} else {
				sink.put(line);
			}
		}
	}
}

Since the whole point of this function was to write output to different files (with automatically determined names), I wanted to write a unittest that tests whether it creates the correct files with the correct contents.  But I didn't want unittests to touch the actual filesystem either -- didn't want to have to clean up the mess during development where the code might sometimes break and leave detritus behind in a temporary directory, or interact badly with other unittests running in parallel, etc..

And I didn't want to do a massive code refactoring to make genSplitHtml more unittestable. (Which would have more chances of screwing up and having bugs creep in, which defeats the purpose of this exercise.)

So I came up with this solution:

1) Rewrite the function to:

	void genSplitHtml(File = std.stdio.File)(Data data, ...) { ... }

   The default parameter binds to the real filesystem by default, so
   other code that uses this function don't have to change to deal with
   a new API.  Then for the unittest code:

2) Create a fake virtual filesystem in my unittest block:

	static struct MockFile {
		static string[string] files;

		string fname;
		this(string _fname, string mode) {
			// ignore `mode` for this test
			fname = _fname;
		}

		// Mock replacement for std.stdio.File.lockingTextWriter
		auto lockingTextWriter() {
			return (const(char)[] data) {
				// simulate writing to a file
				files[fname] ~= data.dup;
			};
		}
		void rewind() {} // dummy
		void close() {} // dummy
		auto byLine() {
			// We're mostly writing to files, and only
			// reading from a specific one. So just fake its
			// contents here.
			if (fname != "template.html") return [];
			else return [
				"<html>",
				"MAGIC_TOKEN",
				"</html>"
			];
		}
	}

Then instead of calling genSplitHtml(...), the unittest calls it as genSplitHtml!MockFile(...). This replaces std.stdio.File with MockFile, and thanks to D templates and the range API, the rest of the code just adapted itself automatically to the MockFile fake filesystem. After the function is done, the unittest just has to inspect the contents of MockFile.files to verify that the correct files are there, and that their contents are correct.

Took me like 10 minutes to write MockFile and a unittest that checks for correct behaviour using the fake filesystem.

MockFile can be expanded in the future to simulate, e.g. a full filesystem, a filesystem that occasionally (or always) fails, or corrupts data, etc.: test cases that would be impractical to test with a real filesystem.  Best of all, I get all of this "for free": no existing code has to change except for that single new template parameter to the target function.

D not only r0x0rs, D boulders!!


T

-- 
Heads I win, tails you lose.

Excellent! The technique you're exhibiting is called "mocking". It works especially well when a function's inputs and outputs are properly parameterized. When I fixed the D lexer.d to not access any globals, I was able to simplify the unittests for it by mocking up the inputs and outputs. For example, instead of having to use the global "gagging" switch to suppress error messages, I used a mock error handler that just did nothing. On 5/30/2023 2:42 PM, H. S. Teoh wrote: > So here's one.

void genSplitHtml(Data data, ...) { auto outputTemplate = File("template.html", "r"); foreach (...) { auto filename = generateFilename(...); auto sink = File(filename, "w").lockingTextWriter; ... while (line; outputTemplate.byLine) { if (line.canFind("MAGIC_TOKEN")) { generateOutput(...); } else { sink.put(line); } } } }

static struct MockFile { static string[string] files; string fname; this(string _fname, string mode) { // ignore `mode` for this test fname = _fname; } // Mock replacement for std.stdio.File.lockingTextWriter auto lockingTextWriter() { return (const(char)[] data) { // simulate writing to a file files[fname] ~= data.dup; }; } void rewind() {} // dummy void close() {} // dummy auto byLine() { // We're mostly writing to files, and only // reading from a specific one. So just fake its // contents here. if (fname != "template.html") return []; else return [ "<html>", "MAGIC_TOKEN", "</html>" ]; } }

// returns an iopipe with the translated data auto genSplitHtml(Input)(Input inputPipe, Data data, ...) { ... } // and then you can call it like: File("template.html", "r").genSplitHtml(data).writeTo(File(filename, "w")).process(); // in unittest auto result = "contents of template".genSplitHtml(data); result.process(); // oof, would be nice if I could do this in one line assert(result.window == expectedData);

On 5/30/23 14:42, H. S. Teoh wrote: > So here's one. Thank you for reminding us. ;) > void genSplitHtml(File = std.stdio.File)(Data data, ...) { ... } Smart! In the past, I used runtime polymorhism to achieve the same. Oops! :) > MockFile can be expanded in the future to simulate, e.g. a full > filesystem, Yep. Mine was using associative arrays keyed by strings (file names) and data as dynamic arrays. Ali

Forums