aboutsummaryrefslogtreecommitdiff
path: root/tools/testing
AgeCommit message (Collapse)Author
2012-02-27Merge tag 'ktest-fix-make-min-failed-build-for-real' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-ktest While demoing ktest at ELC in 2012, it was embarrassing that the make_min_config test failed to work because the snowball board I was testing it against had a config that would not build. But the make_min_config only tested the testing part and ignored build failures. The end result was a config file that would not boot. This time, for real. * tag 'ktest-fix-make-min-failed-build-for-real' of git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-ktest: ktest: Fix make_min_config test when build fails
2012-02-27ktest: Fix make_min_config test when build failsSteven Rostedt
The make_min_config does not take into account when the build fails, resulting in a invalid MIN_CONFIG .config file. When the build fails, it is ignored and the boot test is executed, using the previous built kernel. The configs that should be tested are not tested and they may be added or removed depending on the result of the last kernel that succeeded to be built. If the build fails, mark the current config as a failure and the configs that were disabled may still be needed. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2012-01-16Merge branch 'for-linus' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-ktest * 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-ktest: ktest: Add INGORE_ERRORS to ignore warnings in boot up ktest: Still do reboot even for REBOOT_TYPE = script ktest: Fix compare script to test if options are not documented ktest: Detect typos in option names ktest: Have all values be set by defaults ktest: Change initialization of defaults hash to perl format ktest: Add options SWITCH_TO_GOOD and SWITCH_TO_TEST ktest: Allow overriding bisect test results ktest: Evaluate options before processing them ktest: Evaluate $KERNEL_VERSION in both install and post install ktest: Only ask options needed for install ktest: When creating a new config, ask for BUILD_OPTIONS ktest: Do not ask for some options if the only test is build ktest: Ask for type of test when creating a new config ktest: Allow bisect test to restart where it left off ktest: When creating new config, allow the use of ${THIS_DIR} ktest: Add default for ssh-user, build-target and target-image ktest: Allow success logs to be stored ktest: Save test output
2012-01-12selftests: new x86 breakpoints selftestFrederic Weisbecker
Bring a first selftest in the relevant directory. This tests several combinations of breakpoints and watchpoints in x86, as well as icebp traps and int3 traps. Given the amount of breakpoint regressions we raised after we merged the generic breakpoint infrastructure, such selftest became necessary and can still serve today as a basis for new patches that touch the do_debug() path. Signed-off-by: Frederic Weisbecker <fweisbec@gmail.com> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Ingo Molnar <mingo@elte.hu> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Jason Wessel <jason.wessel@windriver.com> Cc: Will Deacon <will.deacon@arm.com> Cc: Michal Marek <mmarek@suse.cz> Cc: Sam Ravnborg <sam@ravnborg.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2012-01-12selftests: new very basic kernel selftests directoryFrederic Weisbecker
Bring a new kernel selftests directory in tools/testing/selftests. To add a new selftest, create a subdirectory with the sources and a makefile that creates a target named "run_test" then add the subdirectory name to the TARGET var in tools/testing/selftests/Makefile and tools/testing/selftests/run_tests script. This can help centralizing and maintaining any useful selftest that developers usually tend to let rust in peace on some random server. Suggested-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Frederic Weisbecker <fweisbec@gmail.com> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Ingo Molnar <mingo@elte.hu> Cc: "H. Peter Anvin" <hpa@zytor.com> Cc: Jason Wessel <jason.wessel@windriver.com> Cc: Will Deacon <will.deacon@arm.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Michal Marek <mmarek@suse.cz> Cc: Sam Ravnborg <sam@ravnborg.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2012-01-08Merge branch 'for-linus' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/jikos/trivial * 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/jikos/trivial: (53 commits) Kconfig: acpi: Fix typo in comment. misc latin1 to utf8 conversions devres: Fix a typo in devm_kfree comment btrfs: free-space-cache.c: remove extra semicolon. fat: Spelling s/obsolate/obsolete/g SCSI, pmcraid: Fix spelling error in a pmcraid_err() call tools/power turbostat: update fields in manpage mac80211: drop spelling fix types.h: fix comment spelling for 'architectures' typo fixes: aera -> area, exntension -> extension devices.txt: Fix typo of 'VMware'. sis900: Fix enum typo 'sis900_rx_bufer_status' decompress_bunzip2: remove invalid vi modeline treewide: Fix comment and string typo 'bufer' hyper-v: Update MAINTAINERS treewide: Fix typos in various parts of the kernel, and fix some comments. clockevents: drop unknown Kconfig symbol GENERIC_CLOCKEVENTS_MIGR gpio: Kconfig: drop unknown symbol 'CS5535_GPIO' leds: Kconfig: Fix typo 'D2NET_V2' sound: Kconfig: drop unknown symbol ARCH_CLPS7500 ... Fix up trivial conflicts in arch/powerpc/platforms/40x/Kconfig (some new kconfig additions, close to removed commented-out old ones)
2012-01-04ktest: Add INGORE_ERRORS to ignore warnings in boot upSteven Rostedt
When testing a kernel that has warnings, ktest.pl will fail the test when it sees the warning. If you need to test the the kernel and want to ignore the errors that are produced, the option IGNORE_ERRORS has been added. When IGNORE_ERRORS is set to something other than 0, it will ignore call traces due to WARN_ON(). Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2012-01-04ktest: Still do reboot even for REBOOT_TYPE = scriptSteven Rostedt
The REBOOT_TYPE may be either grub or script, if it is script it is expected that a REBOOT_SCRIPT is defined. With the SWITCH_TO_TEST which is the complement of SWITCH_TO_GOOD, which does basically the same thing as REBOOT_SCRIPT and but for both grub and script, the REBOOT_SCRIPT does not need to be mandatory anymore. Do not require the REBOOT_SCRIPT and always run the reboot code for both grub and script. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2012-01-02misc latin1 to utf8 conversionsAl Viro
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk> Signed-off-by: Jiri Kosina <jkosina@suse.cz>
2011-12-22ktest: Fix compare script to test if options are not documentedSteven Rostedt
The compare script compare-ktest-sample.pl checks for options that are defined in ktest.pl and not documented in samples.conf, as well as samples in samples.conf that are not used in ktest.pl. With the switch to the hash format to initialize the ktest variables the compare script needs to be updated to handle the change. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2011-12-22ktest: Detect typos in option namesSteven Rostedt
It becomes quite annoying when you go to run a test and then realize that you typed an option name wrong, and the test starts doing the default action and not what you expected it to do. It is even more annoying when you wake up the next day after running the test over night when you discover this. By testing if all options specified in a config file are used by either ktest or were used in one of the option's values we can see if there are any dangling options that were not used. In such a case, show the user the options that were not used and ask them if they want to continue or not. The option IGNORE_UNUSED was also added to allow the user to override this feature. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2011-12-22ktest: Have all values be set by defaultsSteven Rostedt
Currently the patchcheck, bisect, and config_bisect variables are only able to be set per test. You can not set a default value for them. By letting default values be set, it makes some config files a bit easier, and also makes it easier to find typos in the option names. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2011-12-22ktest: Change initialization of defaults hash to perl formatSteven Rostedt
Initializing each default value by specifying the hash name is ugly. This is one of the rare cases that the "perl way" is actually much cleaner and easier to read. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2011-12-22ktest: Add options SWITCH_TO_GOOD and SWITCH_TO_TESTSteven Rostedt
For machines that do no use grub, it may be needed to update an external image (tftp) before doing a reboot into either the test image or the known good image. The option SWITCH_TO_GOOD is added, where if it is defined, the command that is specified as its value will be executed before doing a reboot into a known good image. The option SWITCH_TO_TEST is added, where if it is defined, the command that is specified as its value will be executed before doing a reboot into the test image. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2011-12-22ktest: Allow overriding bisect test resultsSteven Rostedt
When running the ktest git bisect test, if the BISECT_TYPE is "test", the bisect is determined to be good or bad based off of the error code of the test that is run. Currently, if the test returns 0, it is considered a pass (good), a non-zero is considered a fail (bad). But it has been requested to add more options, and also change the meanings of the error codes of the test. For example, one may want the test to detect if the commit is not good or bad, (maybe the bisect came to a point where the code in question does not exist). The test could report an error code that should tell ktest to skip the commit. Also, a test could detect that something is horribly wrong and the biscet should just be aborted. The new options: BISECT_RET_GOOD BISECT_RET_BAD BISECT_RET_SKIP BISECT_RET_ABORT BISECT_RET_DEFAULT have been added. The first 4 take an integer value that will represent if the test should be considered a pass, fail, neither good nor bad, or abort respectively. The BISECT_RET_DEFAULT will bo whatever is not defined by the above codes. If only BISECT_RET_DEFAULT is defined, then all tests will do the default. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2011-12-22ktest: Evaluate options before processing themSteven Rostedt
All options can take variables "${var}". Before doing any processing or decision making on the content of an option, evaluate it incase there are variables that may change the outcome. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2011-12-22ktest: Evaluate $KERNEL_VERSION in both install and post installSteven Rostedt
The install process may also need to know what the kernel version is, to add it to the name. Evaluate it for both install and post install. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2011-12-22ktest: Only ask options needed for installSteven Rostedt
If all the tests are only for build or install, do not ask for options not needed to do the install, if the options do not exist. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2011-12-22ktest: When creating a new config, ask for BUILD_OPTIONSSteven Rostedt
When creating a new config, ask for the BUILD_OPTIONS variable that lets users add things like -j20 to the make. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2011-12-22ktest: Do not ask for some options if the only test is buildSteven Rostedt
When creating a ktest config or if te config only has build only tests, some of the manditory config options are not needed. Do not ask for them if all tests in the config file are just build tests. Suggested-by: Darren Hart <dvhart@linux.intel.com> Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2011-12-22ktest: Ask for type of test when creating a new configSteven Rostedt
When no argument is supplied to ktest, or the config applied does not exist and a new config is being created, instead of just using the default test type, give the user an option to pick the test type of either 'build, install, or boot'. Other options may be added later but then those would require more questions as they require more fields. But that's for another release of ktest to add that feature. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2011-12-22ktest: Allow bisect test to restart where it left offSteven Rostedt
If a bisect is killed for some reason, have ktest detect that a bisect is in progress and if so, allow the user to start the bisect where it left off. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2011-11-23ktest: When creating new config, allow the use of ${THIS_DIR}Steven Rostedt
Typing in a full path when you know that the path exists within the directory your are running is tedious and unnecessary. Allow the user to use ${PWD} if they want a dynamic path name which will be the path that ktest.pl is executed from or use ${THIS_DIR} which is a variable assigned `pwd` and the the variable will exist within the config, allowing the user to change it and affect all other paths using this variable as well Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2011-11-21ktest: Add default for ssh-user, build-target and target-imageSteven Rostedt
When a user runs ktest without an argument, or the argument given is not a config file that exists, ktest will ask the user a few questions to create a simple ktest config file. A few of the questions should have a default value set, that if anything it will make it easier for the user to know what is suppose to be in that value. These new values are: SSH_USER, BUILD_TARGET and TARGET_IMAGE Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2011-11-18ktest: Allow success logs to be storedRabin Vincent
Add a STORE_SUCCESSES option, to allow success logs to be stored, for example to double-check or otherwise post-process the test logs. Link: http://lkml.kernel.org/r/1321616131-21352-3-git-send-email-rabin@rab.in Signed-off-by: Rabin Vincent <rabin@rab.in> Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2011-11-18ktest: Save test outputRabin Vincent
The test output may contain useful information; save it along with the already-saved buildlog, dmesg, and .config. Link: http://lkml.kernel.org/r/1321616131-21352-1-git-send-email-rabin@rab.in Signed-off-by: Rabin Vincent <rabin@rab.in> Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2011-11-18ktest: Check parent options for iterated testsRabin Vincent
Let's say we have "OUTPUT_DIR = build/${TEST_NAME}", and we're iterating a test. In the second iteration of a test, the TEST_NAME of the test we're repeating is not used. Instead, ${TEST_NAME} appears literally: touch /home/rabin/kernel/test/build/${TEST_NAME}/.config ... SUCCESS Fix this by making __eval_option() check the parent test options for a repeated test. Link: http://lkml.kernel.org/r/1321616131-21352-2-git-send-email-rabin@rab.in Signed-off-by: Rabin Vincent <rabin@rab.in> Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2011-10-28ktest: Evaluate variables entered on the command lineSteven Rostedt
When ktest.pl is called without any arguments, or if the config file does not exist, ktest.pl will ask the user for some information. Some of these questions are code paths. Allowing the user to type ${PWD} for the current directory greatly simplifies these entries. Add variable processing to the entered values. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2011-10-22ktest: Add variable ${PWD}Steven Rostedt
Adding the variable ${PWD} that equals `pwd` makes the config files much simpler. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2011-10-22ktest: Add another monitor flush before installing kernelSteven Rostedt
On some tests that do multiple boots (patchcheck, bisect, etc), the build of the next kernel to run may finish before the stable kernel has finished booting. Then the install of the new kernel will fail when it tries to connect as the machine has not finished the boot process. Do one more monitor flush to make sure the machine is up and running before trying to connect to it again. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2011-10-20ktest: Do not opencode reboot in grub settingSteven Rostedt
When setting the next kernel to boot to with grub, do not opencode the reboot operation. The normal reboot operation can be modified by config options (namely POWERCYCLE_AFTER_REBOOT). This needs to affect all reboots. Remove the opencoded reboot to make sure that any changes to the reboot code also affect all reboots. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2011-10-17ktest: Add processing of complex conditionalsSteven Rostedt
The IF statements for DEFAULTS and TEST_START sections now handle complex statements (&&,||) Example: TEST_START IF (DEFINED ALL_TESTS || ${MYTEST} == boottest) && ${MACHINE} == gandalf Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2011-10-17ktest: Fix parsing of config section linesSteven Rostedt
The order for some of the keywords on a section line (TEST_START or DEFAULTS) does not really matter. Simply need to remove the keyword from the line as we process it and evaluate the next keyword in the line. By removing the keywords as we find them, we do not need to keep track of where on the line they were found. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2011-10-17ktest: Sort make_min_config configs by dependeciesSteven Rostedt
The make_min_config test will turn off one config at a time and check if the config boots or not, and if it does, it will remove that config plus any config that depended on that config. ktest already looks if a config has a dependency and will try the dependency config first. But by sorting the configs and trying the config with the most configs dependent on it, we can shrink the minconfig faster. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2011-10-17ktest: Add DEFINED keyword for IF statementsSteven Rostedt
Have IF statements process if a config variable or option has been defined or not. Can use NOT DEFINED in the case for telling if a variable or option has not been defined. DEFAULTS IF NOT DEFINED SSH_USER SSH_USER = root Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2011-10-17ktest: Add OVERRIDE keyword to DEFAULTS sectionSteven Rostedt
The OVERRIDE keyword will allow options defined in the given DEFAULTS section to override options defined in previous DEFAULT sections. Normally, options will error if they were previous defined. The OVERRIDE keyword allows options that have been previously defined to be changed in the given section. Note, the same option can not be defined in the same DEFAULT section even if that section is marked as OVERRIDE. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2011-10-17ktest: Consolidate TEST_TYPE and DEFAULT codeSteven Rostedt
The code that handles parsing the TEST_TYPE and DEFAULT code share a lot of common functionality. Combine the two and add a if statement that does what is different between them. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2011-10-17ktest: Add INCLUDE keyword to include other config filesSteven Rostedt
Have the reading of the config file allow reading of other config files using the INCLUDE keyword. This allows multiple config files to share config options. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2011-10-17ktest: Let IF keyword take comparisonsSteven Rostedt
Allow ==, !=, <=, >=, <, and > to be used in IF statements to compare if a section should be processed or not. For example: BITS := 32 DEFAULTS IF ${BITS} == 32 MIN_CONFIG = ${CONFIG_DIR}/config-32 ELSE MIN_CONFIG = ${CONFIG_DIR}/config-64 Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2011-10-17ktest: Add IF and ELSE to config sectionsSteven Rostedt
Add IF keyword to sections within the config. Also added an ELSE keyword that allows different config options to be set for a given section. For example: TYPE := 1 STATUS := 0 DEFAULTS IF ${TYPE} [...] ELSE IF ${STATUS} [...] ELSE [...] The above will process the first section as $TYPE is true. If it was false, it would process the last section as $STATUS is false. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2011-10-17ktest: Do not reboot on config or build issuesSteven Rostedt
Even if REBOOT_ON_ERROR is set, it becomes annoying that the target machine is rebooted when a config option is incorrect or a build fails. There's no reason to reboot the target for host only issues. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2011-10-17ktest: Add option REBOOT_SUCCESS_LINE to stop waiting after a rebootSteven Rostedt
When ktest.pl reboots, it will usuall wait SLEEP_TIME seconds of idle console before starting the next test. By setting the REBOOT_SUCCESS_LINE, ktest will not wait SLEEP_TIME when it detects the line while rebooting to a new kernel. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2011-10-17ktest: Add NO_INSTALL option to not install for a testSteven Rostedt
There's cases where running the same kernel over and over again is useful, and being able to not install the same kernel can save time between tests. Add a NO_INSTALL option that tells ktest.pl to not install the new kernel. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2011-10-17ktest: Fail when grub menu not foundSteven Rostedt
Currently if the grub menu that is supplied is not found, it will just boot into the last grub menu in menu.lst. Fail instead of confusing the user why their kernel is not booting. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2011-10-17ktest: Include monitor in reboot codeAndrew Jones
Several places that call reboot do the same thing with respect to the monitor. By adding this code into the reboot code, redundant code is removed and it paves the way for the the reset time patch. Signed-off-by: Andrew Jones <drjones@redhat.com> Link: http://lkml.kernel.org/r/1313155932-20092-4-git-send-email-drjones@redhat.com Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2011-10-17ktest: Only need to save .config when doing mrproperAndrew Jones
Only save the .config file if we're doing mrproper Signed-off-by: Andrew Jones <drjones@redhat.com> Link: http://lkml.kernel.org/r/1313155932-20092-3-git-send-email-drjones@redhat.com Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2011-10-17ktest: Create outputdir if it does not existAndrew Jones
Signed-off-by: Andrew Jones <drjones@redhat.com> Link: http://lkml.kernel.org/r/1313155932-20092-2-git-send-email-drjones@redhat.com Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2011-10-17ktest: Add TEST_TYPE install optionSteven Rostedt
In testing one of my boxes, I found that I only wanted to build and install the kernel. I wanted to manually reboot the box and test it. Adding a TEST_TYPE option "install" allows this to happen. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2011-07-15ktest: Fix bug when ADD_CONFIG is set but MIN_CONFIG is notSteven Rostedt
The MIN_CONFIG is a single config that is considered to have all the configs that are required to boot the box. ADD_CONFIG is a list of configs that we add that may contain configs known to be broken (set off) or just configs that we want every box to have and this can include shared configs. If a config has no MIN_CONFIG defined, but has multiple files defined for the ADD_CONFIG, the test will die, because the MIN_CONFIG will default to ADD_CONFIG. The problem is the code to open MIN_CONFIG expects a string of one file, not multiple, and the open will fail. Since the real minconfig that is used is a concatination of MIN_CONFIG and ADD_CONFIG files, we change the code to open that instead of whatever MIN_CONFIG defaults to. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2011-07-15ktest: Keep fonud configs separate from default configsSteven Rostedt
The IGNORE_CONFIG file holds the configs that we don't want to change (with their proper settings). But on start up, the make noconfig is executed, and the configs that are on are also put into the ignore config category. But these are configs that were forced on by the kconfig scripts and not something that we found must be enabled to boot our machine. By keeping the configs that are forced on by default, separate from the configs we found that are required to boot the box, we can get a much more interesting IGNORE_CONFIG. In fact, the IGNORE_CONFIG can usually end up being the must have configs to boot, and only have 6 or 7 configs set. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>