diff options
author | Daniel Dunbar <daniel@zuster.org> | 2012-05-08 16:50:35 +0000 |
---|---|---|
committer | Daniel Dunbar <daniel@zuster.org> | 2012-05-08 16:50:35 +0000 |
commit | 3b709d5f5532bacb6c351def8c9bc7efef36fa33 (patch) | |
tree | b102b42b9ae1e20ad6553249b348619b34f11669 /docs | |
parent | f647c1b7776f34125441f6edeb449be63b470dd7 (diff) |
[docs] Add ReST version of all the man pages.
- The POD versions are slated for execution, but are still around until
llvm.org machinery is in place.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@156384 91177308-0d34-0410-b5e6-96231b3b80d8
Diffstat (limited to 'docs')
-rw-r--r-- | docs/CommandGuide/FileCheck.rst | 284 | ||||
-rw-r--r-- | docs/CommandGuide/bugpoint.rst | 247 | ||||
-rw-r--r-- | docs/CommandGuide/index.rst | 53 | ||||
-rw-r--r-- | docs/CommandGuide/lit.rst | 474 | ||||
-rw-r--r-- | docs/CommandGuide/llc.rst | 251 | ||||
-rw-r--r-- | docs/CommandGuide/lli.rst | 300 | ||||
-rw-r--r-- | docs/CommandGuide/llvm-ar.rst | 482 | ||||
-rw-r--r-- | docs/CommandGuide/llvm-as.rst | 89 | ||||
-rw-r--r-- | docs/CommandGuide/llvm-bcanalyzer.rst | 424 | ||||
-rw-r--r-- | docs/CommandGuide/llvm-build.rst | 102 | ||||
-rw-r--r-- | docs/CommandGuide/llvm-config.rst | 176 | ||||
-rw-r--r-- | docs/CommandGuide/llvm-cov.rst | 51 | ||||
-rw-r--r-- | docs/CommandGuide/llvm-diff.rst | 56 | ||||
-rw-r--r-- | docs/CommandGuide/llvm-dis.rst | 69 | ||||
-rw-r--r-- | docs/CommandGuide/llvm-extract.rst | 104 | ||||
-rw-r--r-- | docs/CommandGuide/llvm-link.rst | 96 | ||||
-rw-r--r-- | docs/CommandGuide/llvm-nm.rst | 154 | ||||
-rw-r--r-- | docs/CommandGuide/llvm-prof.rst | 63 | ||||
-rw-r--r-- | docs/CommandGuide/llvm-ranlib.rst | 61 | ||||
-rw-r--r-- | docs/CommandGuide/llvm-stress.rst | 48 | ||||
-rw-r--r-- | docs/CommandGuide/opt.rst | 183 | ||||
-rw-r--r-- | docs/CommandGuide/tblgen.rst | 186 |
22 files changed, 3953 insertions, 0 deletions
diff --git a/docs/CommandGuide/FileCheck.rst b/docs/CommandGuide/FileCheck.rst new file mode 100644 index 0000000000..e7076c2f30 --- /dev/null +++ b/docs/CommandGuide/FileCheck.rst @@ -0,0 +1,284 @@ +FileCheck - Flexible pattern matching file verifier +=================================================== + + +SYNOPSIS +-------- + + +**FileCheck** *match-filename* [*--check-prefix=XXX*] [*--strict-whitespace*] + + +DESCRIPTION +----------- + + +**FileCheck** reads two files (one from standard input, and one specified on the +command line) and uses one to verify the other. This behavior is particularly +useful for the testsuite, which wants to verify that the output of some tool +(e.g. llc) contains the expected information (for example, a movsd from esp or +whatever is interesting). This is similar to using grep, but it is optimized +for matching multiple different inputs in one file in a specific order. + +The *match-filename* file specifies the file that contains the patterns to +match. The file to verify is always read from standard input. + + +OPTIONS +------- + + + +**-help** + + Print a summary of command line options. + + + +**--check-prefix** *prefix* + + FileCheck searches the contents of *match-filename* for patterns to match. By + default, these patterns are prefixed with "CHECK:". If you'd like to use a + different prefix (e.g. because the same input file is checking multiple + different tool or options), the **--check-prefix** argument allows you to specify + a specific prefix to match. + + + +**--strict-whitespace** + + By default, FileCheck canonicalizes input horizontal whitespace (spaces and + tabs) which causes it to ignore these differences (a space will match a tab). + The --strict-whitespace argument disables this behavior. + + + +**-version** + + Show the version number of this program. + + + + +EXIT STATUS +----------- + + +If **FileCheck** verifies that the file matches the expected contents, it exits +with 0. Otherwise, if not, or if an error occurs, it will exit with a non-zero +value. + + +TUTORIAL +-------- + + +FileCheck is typically used from LLVM regression tests, being invoked on the RUN +line of the test. A simple example of using FileCheck from a RUN line looks +like this: + + +.. code-block:: perl + + ; RUN: llvm-as < %s | llc -march=x86-64 | FileCheck %s + + +This syntax says to pipe the current file ("%s") into llvm-as, pipe that into +llc, then pipe the output of llc into FileCheck. This means that FileCheck will +be verifying its standard input (the llc output) against the filename argument +specified (the original .ll file specified by "%s"). To see how this works, +let's look at the rest of the .ll file (after the RUN line): + + +.. code-block:: perl + + define void @sub1(i32* %p, i32 %v) { + entry: + ; CHECK: sub1: + ; CHECK: subl + %0 = tail call i32 @llvm.atomic.load.sub.i32.p0i32(i32* %p, i32 %v) + ret void + } + + define void @inc4(i64* %p) { + entry: + ; CHECK: inc4: + ; CHECK: incq + %0 = tail call i64 @llvm.atomic.load.add.i64.p0i64(i64* %p, i64 1) + ret void + } + + +Here you can see some "CHECK:" lines specified in comments. Now you can see +how the file is piped into llvm-as, then llc, and the machine code output is +what we are verifying. FileCheck checks the machine code output to verify that +it matches what the "CHECK:" lines specify. + +The syntax of the CHECK: lines is very simple: they are fixed strings that +must occur in order. FileCheck defaults to ignoring horizontal whitespace +differences (e.g. a space is allowed to match a tab) but otherwise, the contents +of the CHECK: line is required to match some thing in the test file exactly. + +One nice thing about FileCheck (compared to grep) is that it allows merging +test cases together into logical groups. For example, because the test above +is checking for the "sub1:" and "inc4:" labels, it will not match unless there +is a "subl" in between those labels. If it existed somewhere else in the file, +that would not count: "grep subl" matches if subl exists anywhere in the +file. + +The FileCheck -check-prefix option +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + + +The FileCheck -check-prefix option allows multiple test configurations to be +driven from one .ll file. This is useful in many circumstances, for example, +testing different architectural variants with llc. Here's a simple example: + + +.. code-block:: perl + + ; RUN: llvm-as < %s | llc -mtriple=i686-apple-darwin9 -mattr=sse41 \ + ; RUN: | FileCheck %s -check-prefix=X32> + ; RUN: llvm-as < %s | llc -mtriple=x86_64-apple-darwin9 -mattr=sse41 \ + ; RUN: | FileCheck %s -check-prefix=X64> + + define <4 x i32> @pinsrd_1(i32 %s, <4 x i32> %tmp) nounwind { + %tmp1 = insertelement <4 x i32>; %tmp, i32 %s, i32 1 + ret <4 x i32> %tmp1 + ; X32: pinsrd_1: + ; X32: pinsrd $1, 4(%esp), %xmm0 + + ; X64: pinsrd_1: + ; X64: pinsrd $1, %edi, %xmm0 + } + + +In this case, we're testing that we get the expected code generation with +both 32-bit and 64-bit code generation. + + +The "CHECK-NEXT:" directive +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + + +Sometimes you want to match lines and would like to verify that matches +happen on exactly consecutive lines with no other lines in between them. In +this case, you can use CHECK: and CHECK-NEXT: directives to specify this. If +you specified a custom check prefix, just use "<PREFIX>-NEXT:". For +example, something like this works as you'd expect: + + +.. code-block:: perl + + define void @t2(<2 x double>* %r, <2 x double>* %A, double %B) { + %tmp3 = load <2 x double>* %A, align 16 + %tmp7 = insertelement <2 x double> undef, double %B, i32 0 + %tmp9 = shufflevector <2 x double> %tmp3, + <2 x double> %tmp7, + <2 x i32> < i32 0, i32 2 > + store <2 x double> %tmp9, <2 x double>* %r, align 16 + ret void + + ; CHECK: t2: + ; CHECK: movl 8(%esp), %eax + ; CHECK-NEXT: movapd (%eax), %xmm0 + ; CHECK-NEXT: movhpd 12(%esp), %xmm0 + ; CHECK-NEXT: movl 4(%esp), %eax + ; CHECK-NEXT: movapd %xmm0, (%eax) + ; CHECK-NEXT: ret + } + + +CHECK-NEXT: directives reject the input unless there is exactly one newline +between it an the previous directive. A CHECK-NEXT cannot be the first +directive in a file. + + +The "CHECK-NOT:" directive +~~~~~~~~~~~~~~~~~~~~~~~~~~ + + +The CHECK-NOT: directive is used to verify that a string doesn't occur +between two matches (or before the first match, or after the last match). For +example, to verify that a load is removed by a transformation, a test like this +can be used: + + +.. code-block:: perl + + define i8 @coerce_offset0(i32 %V, i32* %P) { + store i32 %V, i32* %P + + %P2 = bitcast i32* %P to i8* + %P3 = getelementptr i8* %P2, i32 2 + + %A = load i8* %P3 + ret i8 %A + ; CHECK: @coerce_offset0 + ; CHECK-NOT: load + ; CHECK: ret i8 + } + + + +FileCheck Pattern Matching Syntax +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + + +The CHECK: and CHECK-NOT: directives both take a pattern to match. For most +uses of FileCheck, fixed string matching is perfectly sufficient. For some +things, a more flexible form of matching is desired. To support this, FileCheck +allows you to specify regular expressions in matching strings, surrounded by +double braces: **{{yourregex}}**. Because we want to use fixed string +matching for a majority of what we do, FileCheck has been designed to support +mixing and matching fixed string matching with regular expressions. This allows +you to write things like this: + + +.. code-block:: perl + + ; CHECK: movhpd {{[0-9]+}}(%esp), {{%xmm[0-7]}} + + +In this case, any offset from the ESP register will be allowed, and any xmm +register will be allowed. + +Because regular expressions are enclosed with double braces, they are +visually distinct, and you don't need to use escape characters within the double +braces like you would in C. In the rare case that you want to match double +braces explicitly from the input, you can use something ugly like +**{{[{][{]}}** as your pattern. + + +FileCheck Variables +~~~~~~~~~~~~~~~~~~~ + + +It is often useful to match a pattern and then verify that it occurs again +later in the file. For codegen tests, this can be useful to allow any register, +but verify that that register is used consistently later. To do this, FileCheck +allows named variables to be defined and substituted into patterns. Here is a +simple example: + + +.. code-block:: perl + + ; CHECK: test5: + ; CHECK: notw [[REGISTER:%[a-z]+]] + ; CHECK: andw {{.*}}[REGISTER]] + + +The first check line matches a regex (**%[a-z]+**) and captures it into +the variable "REGISTER". The second line verifies that whatever is in REGISTER +occurs later in the file after an "andw". FileCheck variable references are +always contained in **[[ ]]** pairs, are named, and their names can be +name, then it is a definition of the variable, if not, it is a use. + +FileCheck variables can be defined multiple times, and uses always get the +latest value. Note that variables are all read at the start of a "CHECK" line +and are all defined at the end. This means that if you have something like +"**CHECK: [[XYZ:.\\*]]x[[XYZ]]**", the check line will read the previous +value of the XYZ variable and define a new one after the match is performed. If +you need to do something like this you can probably take advantage of the fact +that FileCheck is not actually line-oriented when it matches, this allows you to +define two separate CHECK lines that match on the same line. diff --git a/docs/CommandGuide/bugpoint.rst b/docs/CommandGuide/bugpoint.rst new file mode 100644 index 0000000000..2c66d10496 --- /dev/null +++ b/docs/CommandGuide/bugpoint.rst @@ -0,0 +1,247 @@ +bugpoint - automatic test case reduction tool +============================================= + + +SYNOPSIS +-------- + + +**bugpoint** [*options*] [*input LLVM ll/bc files*] [*LLVM passes*] **--args** +*program arguments* + + +DESCRIPTION +----------- + + +**bugpoint** narrows down the source of problems in LLVM tools and passes. It +can be used to debug three types of failures: optimizer crashes, miscompilations +by optimizers, or bad native code generation (including problems in the static +and JIT compilers). It aims to reduce large test cases to small, useful ones. +For more information on the design and inner workings of **bugpoint**, as well as +advice for using bugpoint, see *llvm/docs/Bugpoint.html* in the LLVM +distribution. + + +OPTIONS +------- + + + +**--additional-so** *library* + + Load the dynamic shared object *library* into the test program whenever it is + run. This is useful if you are debugging programs which depend on non-LLVM + libraries (such as the X or curses libraries) to run. + + + +**--append-exit-code**\ =\ *{true,false}* + + Append the test programs exit code to the output file so that a change in exit + code is considered a test failure. Defaults to false. + + + +**--args** *program args* + + Pass all arguments specified after -args to the test program whenever it runs. + Note that if any of the *program args* start with a '-', you should use: + + + .. code-block:: perl + + bugpoint [bugpoint args] --args -- [program args] + + + The "--" right after the **--args** option tells **bugpoint** to consider any + options starting with ``-`` to be part of the **--args** option, not as options to + **bugpoint** itself. + + + +**--tool-args** *tool args* + + Pass all arguments specified after --tool-args to the LLVM tool under test + (**llc**, **lli**, etc.) whenever it runs. You should use this option in the + following way: + + + .. code-block:: perl + + bugpoint [bugpoint args] --tool-args -- [tool args] + + + The "--" right after the **--tool-args** option tells **bugpoint** to consider any + options starting with ``-`` to be part of the **--tool-args** option, not as + options to **bugpoint** itself. (See **--args**, above.) + + + +**--safe-tool-args** *tool args* + + Pass all arguments specified after **--safe-tool-args** to the "safe" execution + tool. + + + +**--gcc-tool-args** *gcc tool args* + + Pass all arguments specified after **--gcc-tool-args** to the invocation of + **gcc**. + + + +**--opt-args** *opt args* + + Pass all arguments specified after **--opt-args** to the invocation of **opt**. + + + +**--disable-{dce,simplifycfg}** + + Do not run the specified passes to clean up and reduce the size of the test + program. By default, **bugpoint** uses these passes internally when attempting to + reduce test programs. If you're trying to find a bug in one of these passes, + **bugpoint** may crash. + + + +**--enable-valgrind** + + Use valgrind to find faults in the optimization phase. This will allow + bugpoint to find otherwise asymptomatic problems caused by memory + mis-management. + + + +**-find-bugs** + + Continually randomize the specified passes and run them on the test program + until a bug is found or the user kills **bugpoint**. + + + +**-help** + + Print a summary of command line options. + + + +**--input** *filename* + + Open *filename* and redirect the standard input of the test program, whenever + it runs, to come from that file. + + + +**--load** *plugin* + + Load the dynamic object *plugin* into **bugpoint** itself. This object should + register new optimization passes. Once loaded, the object will add new command + line options to enable various optimizations. To see the new complete list of + optimizations, use the **-help** and **--load** options together; for example: + + + .. code-block:: perl + + bugpoint --load myNewPass.so -help + + + + +**--mlimit** *megabytes* + + Specifies an upper limit on memory usage of the optimization and codegen. Set + to zero to disable the limit. + + + +**--output** *filename* + + Whenever the test program produces output on its standard output stream, it + should match the contents of *filename* (the "reference output"). If you + do not use this option, **bugpoint** will attempt to generate a reference output + by compiling the program with the "safe" backend and running it. + + + +**--profile-info-file** *filename* + + Profile file loaded by **--profile-loader**. + + + +**--run-{int,jit,llc,custom}** + + Whenever the test program is compiled, **bugpoint** should generate code for it + using the specified code generator. These options allow you to choose the + interpreter, the JIT compiler, the static native code compiler, or a + custom command (see **--exec-command**) respectively. + + + +**--safe-{llc,custom}** + + When debugging a code generator, **bugpoint** should use the specified code + generator as the "safe" code generator. This is a known-good code generator + used to generate the "reference output" if it has not been provided, and to + compile portions of the program that as they are excluded from the testcase. + These options allow you to choose the + static native code compiler, or a custom command, (see **--exec-command**) + respectively. The interpreter and the JIT backends cannot currently + be used as the "safe" backends. + + + +**--exec-command** *command* + + This option defines the command to use with the **--run-custom** and + **--safe-custom** options to execute the bitcode testcase. This can + be useful for cross-compilation. + + + +**--compile-command** *command* + + This option defines the command to use with the **--compile-custom** + option to compile the bitcode testcase. This can be useful for + testing compiler output without running any link or execute stages. To + generate a reduced unit test, you may add CHECK directives to the + testcase and pass the name of an executable compile-command script in this form: + + + .. code-block:: perl + + #!/bin/sh + llc "$@" + not FileCheck [bugpoint input file].ll < bugpoint-test-program.s + + + This script will "fail" as long as FileCheck passes. So the result + will be the minimum bitcode that passes FileCheck. + + + +**--safe-path** *path* + + This option defines the path to the command to execute with the + **--safe-{int,jit,llc,custom}** + option. + + + + +EXIT STATUS +----------- + + +If **bugpoint** succeeds in finding a problem, it will exit with 0. Otherwise, +if an error occurs, it will exit with a non-zero value. + + +SEE ALSO +-------- + + +opt|opt diff --git a/docs/CommandGuide/index.rst b/docs/CommandGuide/index.rst new file mode 100644 index 0000000000..73a4835dd7 --- /dev/null +++ b/docs/CommandGuide/index.rst @@ -0,0 +1,53 @@ +.. _commands: + +LLVM Command Guide +------------------ + +The following documents are command descriptions for all of the LLVM tools. +These pages describe how to use the LLVM commands and what their options are. +Note that these pages do not describe all of the options available for all +tools. To get a complete listing, pass the ``--help`` (general options) or +``--help-hidden`` (general and debugging options) arguments to the tool you are +interested in. + +Basic Commands +~~~~~~~~~~~~~~ + +.. toctree:: + :maxdepth: 1 + + llvm-as + llvm-dis + opt + llc + lli + llvm-link + llvm-ar + llvm-ranlib + llvm-nm + llvm-prof + llvm-config + llvm-diff + llvm-cov + llvm-stress + +Debugging Tools +~~~~~~~~~~~~~~~ + +.. toctree:: + :maxdepth: 1 + + bugpoint + llvm-extract + llvm-bcanalyzer + +Developer Tools +~~~~~~~~~~~~~~~ + +.. toctree:: + :maxdepth: 1 + + FileCheck + tblgen + lit + llvm-build diff --git a/docs/CommandGuide/lit.rst b/docs/CommandGuide/lit.rst new file mode 100644 index 0000000000..0073ebe2b3 --- /dev/null +++ b/docs/CommandGuide/lit.rst @@ -0,0 +1,474 @@ +lit - LLVM Integrated Tester +============================ + + +SYNOPSIS +-------- + + +**lit** [*options*] [*tests*] + + +DESCRIPTION +----------- + + +**lit** is a portable tool for executing LLVM and Clang style test suites, +summarizing their results, and providing indication of failures. **lit** is +designed to be a lightweight testing tool with as simple a user interface as +possible. + +**lit** should be run with one or more *tests* to run specified on the command +line. Tests can be either individual test files or directories to search for +tests (see "TEST DISCOVERY"). + +Each specified test will be executed (potentially in parallel) and once all +tests have been run **lit** will print summary information on the number of tests +which passed or failed (see "TEST STATUS RESULTS"). The **lit** program will +execute with a non-zero exit code if any tests fail. + +By default **lit** will use a succinct progress display and will only print +summary information for test failures. See "OUTPUT OPTIONS" for options +controlling the **lit** progress display and output. + +**lit** also includes a number of options for controlling how tests are executed +(specific features may depend on the particular test format). See "EXECUTION +OPTIONS" for more information. + +Finally, **lit** also supports additional options for only running a subset of +the options specified on the command line, see "SELECTION OPTIONS" for +more information. + +Users interested in the **lit** architecture or designing a **lit** testing +implementation should see "LIT INFRASTRUCTURE" + + +GENERAL OPTIONS +--------------- + + + +**-h**, **--help** + + Show the **lit** help message. + + + +**-j** *N*, **--threads**\ =\ *N* + + Run *N* tests in parallel. By default, this is automatically chosen to match + the number of detected available CPUs. + + + +**--config-prefix**\ =\ *NAME* + + Search for *NAME.cfg* and *NAME.site.cfg* when searching for test suites, + instead of *lit.cfg* and *lit.site.cfg*. + + + +**--param** *NAME*, **--param** *NAME*\ =\ *VALUE* + + Add a user defined parameter *NAME* with the given *VALUE* (or the empty + string if not given). The meaning and use of these parameters is test suite + dependent. + + + + +OUTPUT OPTIONS +-------------- + + + +**-q**, **--quiet** + + Suppress any output except for test failures. + + + +**-s**, **--succinct** + + Show less output, for example don't show information on tests that pass. + + + +**-v**, **--verbose** + + Show more information on test failures, for example the entire test output + instead of just the test result. + + + +**--no-progress-bar** + + Do not use curses based progress bar. + + + + +EXECUTION OPTIONS +----------------- + + + +**--path**\ =\ *PATH* + + Specify an addition *PATH* to use when searching for executables in tests. + + + +**--vg** + + Run individual tests under valgrind (using the memcheck tool). The + *--error-exitcode* argument for valgrind is used so that valgrind failures will + cause the program to exit with a non-zero status. + + + +**--vg-arg**\ =\ *ARG* + + When *--vg* is used, specify an additional argument to pass to valgrind itself. + + + +**--time-tests** + + Track the wall time individual tests take to execute and includes the results in + the summary output. This is useful for determining which tests in a test suite + take the most time to execute. Note that this option is most useful with *-j + 1*. + + + + +SELECTION OPTIONS +----------------- + + + +**--max-tests**\ =\ *N* + + Run at most *N* tests and then terminate. + + + +**--max-time**\ =\ *N* + + Spend at most *N* seconds (approximately) running tests and then terminate. + + + +**--shuffle** + + Run the tests in a random order. + + + + +ADDITIONAL OPTIONS +------------------ + + + +**--debug** + + Run **lit** in debug mode, for debugging configuration issues and **lit** itself. + + + +**--show-suites** + + List the discovered test suites as part of the standard output. + + + +**--no-tcl-as-sh** + + Run Tcl scripts internally (instead of converting to shell scripts). + + + +**--repeat**\ =\ *N* + + Run each test *N* times. Currently this is primarily useful for timing tests, + other results are not collated in any reasonable fashion. + + + + +EXIT STATUS +----------- + + +**lit** will exit with an exit code of 1 if there are any FAIL or XPASS +results. Otherwise, it will exit with the status 0. Other exit codes are used +for non-test related failures (for example a user error or an internal program +error). + + +TEST DISCOVERY +-------------- + + +The inputs passed to **lit** can be either individual tests, or entire +directories or hierarchies of tests to run. When **lit** starts up, the first +thing it does is convert the inputs into a complete list of tests to run as part +of *test discovery*. + +In the **lit** model, every test must exist inside some *test suite*. **lit** +resolves the inputs specified on the command line to test suites by searching +upwards from the input path until it finds a *lit.cfg* or *lit.site.cfg* +file. These files serve as both a marker of test suites and as configuration +files which **lit** loads in order to understand how to find and run the tests +inside the test suite. + +Once **lit** has mapped the inputs into test suites it traverses the list of +inputs adding tests for individual files and recursively searching for tests in +directories. + +This behavior makes it easy to specify a subset of tests to run, while still +allowing the test suite configuration to control exactly how tests are +interpreted. In addition, **lit** always identifies tests by the test suite they +are in, and their relative path inside the test suite. For appropriately +configured projects, this allows **lit** to provide convenient and flexible +support for out-of-tree builds. + + +TEST STATUS RESULTS +------------------- + + +Each test ultimately produces one of the following six results: + + +**PASS** + + The test succeeded. + + + +**XFAIL** + + The test failed, but that is expected. This is used for test formats which allow + specifying that a test does not currently work, but wish to leave it in the test + suite. + + + +**XPASS** + + The test succeeded, but it was expected to fail. This is used for tests which + were specified as expected to fail, but are now succeeding (generally because + the feature they test was broken and has been fixed). + + + +**FAIL** + + The test failed. + + + +**UNRESOLVED** + + The test result could not be determined. For example, this occurs when the test + could not be run, the test itself is invalid, or the test was interrupted. + + + +**UNSUPPORTED** + + The test is not supported in this environment. This is used by test formats + which can report unsupported tests. + + + +Depending on the test format tests may produce additional information about +their status (generally only for failures). See the Output|"OUTPUT OPTIONS" +section for more information. + + +LIT INFRASTRUCTURE +------------------ + + +This section describes the **lit** testing architecture for users interested in +creating a new **lit** testing implementation, or extending an existing one. + +**lit** proper is primarily an infrastructure for discovering and running +arbitrary tests, and to expose a single convenient interface to these +tests. **lit** itself doesn't know how to run tests, rather this logic is +defined by *test suites*. + +TEST SUITES +~~~~~~~~~~~ + + +As described in "TEST DISCOVERY", tests are always located inside a *test +suite*. Test suites serve to define the format of the tests they contain, the +logic for finding those tests, and any additional information to run the tests. + +**lit** identifies test suites as directories containing *lit.cfg* or +*lit.site.cfg* files (see also **--config-prefix**). Test suites are initially +discovered by recursively searching up the directory hierarchy for all the input +files passed on the command line. You can use **--show-suites** to display the +discovered test suites at startup. + +Once a test suite is discovered, its config file is loaded. Config files +themselves are Python modules which will be executed. When the config file is +executed, two important global variables are predefined: + + +**lit** + + The global **lit** configuration object (a *LitConfig* instance), which defines + the builtin test formats, global configuration parameters, and other helper + routines for implementing test configurations. + + + +**config** + + This is the config object (a *TestingConfig* instance) for the test suite, + which the config file is expected to populate. The following variables are also + available on the *config* object, some of which must be set by the config and + others are optional or predefined: + + **name** *[required]* The name of the test suite, for use in reports and + diagnostics. + + **test_format** *[required]* The test format object which will be used to + discover and run tests in the test suite. Generally this will be a builtin test + format available from the *lit.formats* module. + + **test_src_root** The filesystem path to the test suite root. For out-of-dir + builds this is the directory that will be scanned for tests. + + **test_exec_root** For out-of-dir builds, the path to the test suite root inside + the object directory. This is where tests will be run and temporary |