Thread overview
Is there a File-like object for unit tests?
Jan 04, 2022
Amit
Jan 04, 2022
Paul Backus
Jan 04, 2022
Ali Çehreli
Jan 04, 2022
H. S. Teoh
Jan 04, 2022
Amit
January 04, 2022

Hi!

I wrote a text parser that takes a File argument and parses that file's contents. Now I would like to write a unit-test for that parser. I need a File (or a general IO interface) that reads from an in-memory buffer, similar to python's StringIO or go's strings.Reader.

How can I achieve that?

Example of what I have in mind:

unittest {
    string fakeContents = "foo\nbar\nbaz";
    File f = createFileFromString(fakeContents);
    assert(myParse(f) == MyParsedObject(...));
}
January 04, 2022

On Tuesday, 4 January 2022 at 17:01:41 UTC, Amit wrote:

>

Hi!

I wrote a text parser that takes a File argument and parses that file's contents. Now I would like to write a unit-test for that parser. I need a File (or a general IO interface) that reads from an in-memory buffer, similar to python's StringIO or go's strings.Reader.

How can I achieve that?

Probably the easiest way to do it is to have your parser take a generic range as its argument instead of a File. Then you can pass in whatever source of data you like--a file, an in-memory buffer, a class that generates data on-the-fly--and it will all Just Work.

For example, here's a function that parses an integer from an input range:

import std.range;

int parseInteger(Input)(Input input)
    if (isInputRange!Input && is(typeof(input.front - '0') : int))
{
    import std.ascii: isDigit;
    import std.exception: enforce;

    int result = 0;
    bool success = false;

    while (!input.empty && input.front.isDigit)
    {
        success = true;
        result *= 10;
        result += input.front - '0';
        input.popFront;
    }

    enforce(success, "input did not contain an integer");
    return result;
}

unittest
{
    import std.ascii: isDigit;
    import std.algorithm: filter;
    import std.exception: assertThrown;

    // can use strings
    assert(parseInteger("123") == 123);
    assert(parseInteger("123 hello") == 123);

    // can use arbitrary range types
    assert(parseInteger("321".retro) == 123);
    assert(parseInteger("a1b2c3".filter!isDigit) == 123);

    // failure cases
    assertThrown!Exception(parseInteger("hello"));
    assertThrown!Exception(parseInteger(""));
}
January 04, 2022
On 1/4/22 9:48 AM, Paul Backus wrote:
> On Tuesday, 4 January 2022 at 17:01:41 UTC, Amit wrote:

>> I need a File (or a general IO interface) that reads from an
>> in-memory buffer, similar to python's `StringIO` or go's
>> `strings.Reader`.
>>
>> How can I achieve that?

I don't think it exists in the standard library. So, I had to write this for work manually. Instead of using File on the interfaces, I created a Storage interface that implemented everything I did with a File: open, close, writeln, seek, etc.

interface Storage {
  // ...
}

And I had two versions of it:

class FileStorage : Storage {
  // ...
}

class InMemoryStorage : Storage {
  ubyte[] buffer;
  // ...
}

Worked like a charm after fixing a number of bugs. (I wish it were open source.)

> Probably the easiest way to do it is to have your parser take a generic
> [range][1] as its argument instead of a `File`.

Makes sense but in my case File was everywhere so it felt better to abstract it away.

> For example, here's a function that parses an integer from an input range:

In my case, it would have to be a RandomAccessRange because the file format had self-references through offsets.

Ali

January 04, 2022
On Tue, Jan 04, 2022 at 05:01:41PM +0000, Amit via Digitalmars-d-learn wrote:
> Hi!
> 
> I wrote a text parser that takes a File argument and parses that file's contents. Now I would like to write a unit-test for that parser. I need a File (or a general IO interface) that reads from an in-memory buffer, similar to python's `StringIO` or go's `strings.Reader`.
> 
> How can I achieve that?
> 
> Example of what I have in mind:
> 
> ```d
> unittest {
>     string fakeContents = "foo\nbar\nbaz";
>     File f = createFileFromString(fakeContents);
>     assert(myParse(f) == MyParsedObject(...));
> }
> ```

It's very easy, just have your text parser take File as a template argument, defaulted to std.file.File, that your unittest(s) can override with a custom type containing the needed methods. For example:

	auto myParser(File = std.stdio.File)(File input) {
		... // read input as usual
	}

	unittest {
		struct FakeFile {
			string contents = "...";
			void[] rawRead(void[] buf) {
				... // simulate a file read here
			}
			... // add whatever other File methods you might need here
		}
		FakeFile f;
		auto output = myParser(f);
		...
	}

	void main() {
		auto input = File("...", "r");
		auto output = myParser(input); // `File` defaults to std.stdio.File
		...
	}


T

-- 
"A man's wife has more power over him than the state has." -- Ralph Emerson
January 04, 2022

Wow, several different approaches!
Thanks everyone, I find this discussion enriching.

I find H. S. Teoh's template solution to be the closest to what I need. It adds minimal complexity to the existing implementation.