Thread overview
LLVM IR testsuite
Nov 10, 2015
Johan Engelen
Nov 10, 2015
Johan Engelen
Dec 19, 2015
Johan Engelen
November 10, 2015
Hi all,
  Recently, we got a new testsuite for testing the generated LLVM IR of a piece of code.
You can find the tests in /tests/ir/. I think this will be very useful for regression testing of the generated code, especially for optimizations or things that are hard to check/diagnose by executing a test program.

Using LLVM's LIT, all ".d" files in /tests/ir are automatically tested. Unfortunately, I have not found nice documentation on Lit that tells you how it interprets the tests files. But simple use, it is simple enough: it looks at lines starting with "// RUN: " and executes what comes after (shell command line). Often you will want to use this as a first line in a D source file:

// RUN: %ldc -c -output-ll -of=%t.ll %s && FileCheck %s < %t.ll

%s is replaced with the current filename, %t is replaced with a temporary file, %ldc is replaced with the LDC binary to be tested.
So this RUN line will compile the current file to LLVM IR, store it in %t, and pass that to FileCheck.

What is FileCheck?
FileCheck is an LLVM tool that checks if the file passed through stdin contains the test strings in the file passed as cmdline argument. "FileCheck %s < %t" tests if %t contains strings specified in %s. If the testfile contains this:

int main() {
// CHECK:  main
    return 42;
// CHECK:  ret i32 42
}

It will check if stdin contains the strings "main" and "ret i32 42", /in order/. FileCheck has some interesting other checks (independent of order, exactly one newline between strings, forbidden string, etc.). Have a look at its documentation:
http://llvm.org/docs/CommandGuide/FileCheck.html

A complete testfile would look like this:
// RUN: %ldc -c -output-ll -of=%t.ll %s && FileCheck %s < %t.ll
int main() {
// CHECK:  main
    return 42;
// CHECK:  ret i32 42
}

Simple!

cheers,
  Johan

(I imagine people attaching such testfiles in this format to bugreports... :)

November 10, 2015
Forgot to mention: instead of searching for Lit documentation, I recommend simply reading its (Python) sourcecode. We use the version here: https://pypi.python.org/pypi/lit
December 19, 2015
I have fixed some remaining issues on Windows, and now it is possible to link and execute in the IR tests.

For example, to compile and run a test case:
// RUN: %ldc -run %s