summaryrefslogtreecommitdiffstats
path: root/tools/testing
Commit message (Collapse)AuthorAgeFilesLines
* ktest: Ignore unset values of the minconfig in config_bisectSteven Rostedt2011-06-021-1/+1
| | | | | | | | | | By ignoring the unset values of the minconfig in deciding what to test in the config_bisect can cause the problem config from being tested too. Just do not test the configs that are set in the minconfig. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
* ktest: Fix result of rebooting the kernelSteven Rostedt2011-06-021-1/+1
| | | | | | | | | | | The command that is called that reboots the kernel may fail but the return code is not passed back to the ktest.pl script. This is because a ';' is used between the two commands and if the second command fails, only the first command's return code is returned. Using a '&&' between the two commands fixes this. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
* ktest: Fix off-by-one in config bisect resultSteven Rostedt2011-06-021-2/+2
| | | | | | | | | | Because in perl the array size returned by $#arr, is the last index and not the actually size of the array, we end the config bisect early, thinking there is only one config left when there are in fact two. Thus the result has a 50% chance of picking the correct config that caused the problem. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
* ktest: Allow options to be used by other optionsSteven Rostedt2011-05-202-1/+97
| | | | | | | | | | | | | | | | | | | | There are cases where one ktest option may be used within another ktest option. Allow them to be reused just like config variables but there are evaluated at time of test not config processing time. Thus having something like: MAKE_CMD = make ARCH=${ARCH} TEST_START ARCH = powerpc TEST_START ARCH = arm Will have the arch defined for each test iteration. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
* ktest: Create variables for the ktest config filesSteven Rostedt2011-05-202-0/+121
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | I found that I constantly reuse information for each test case. It would be nice to just define a variable to reuse. For example I may have: TEST_START [...] TEST = ssh root@mybox /path/to/my/script TEST_START [...] TEST = ssh root@mybox /path/to/my/script [etc] The issue is, I may wont to change that script or one of the other fields. Then I need to update each line individually. With the addition of config variables (variables only used during parsing the config) we can simplify the config files. These variables can also be defined multiple times and each time the new value will overwrite the old value. The convention to use a config variable over a ktest option is to use := instead of =. Now we could do: USER := root TARGET := mybox TEST_SCRIPT := /path/to/my/script TEST_CASE := ${USER}@${TARGET} ${TEST_SCRIPT} TEST_START [...] TEST = ${TEST_CASE} TEST_START [...] TEST = ${TEST_CASE} [etc] Now we just need to update the variables at the top. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
* ktest: Reboot after each patchcheck runSteven Rostedt2011-05-202-0/+17
| | | | | | | | | | | | The patches being checked may not leave the kernel in a state that the next run will allow the new kernel to be copied to the machine. Reboot to a known good kernel before continuing to the next kernel to test. Added option PATCHCHECK_SLEEP_TIME for the max time to sleep between patchcheck reboots. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
* ktest: Reboot to good kernel after every bisect runSteven Rostedt2011-05-201-5/+5
| | | | | | | | | Reboot after each bisect run regardless if the bisect passed or failed. The test may just be to boot the kernel and that kernel may not have a way to copy the next kerne to it. Reboot to a known good kernel after each bisect run. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
* ktest: If test failed due to timeout, print thatSteven Rostedt2011-05-201-0/+1
| | | | | | | If the test failed due to timeout for boot, print a message saying so. Otherwise the user will be confused to why their test just failed. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
* ktest: Fix post install commandSteven Rostedt2011-05-201-1/+1
| | | | | | | | The command to run post install (for those that want initrds) was broken. Instead of doing a substitution for the $KERNEL_VERSION variable. It was replacing the entire command with nothing. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
* Merge branch 'for-linus' of ↵Linus Torvalds2011-03-212-20/+161
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-2.6-ktest * 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-2.6-ktest: ktest: Add STOP_TEST_AFTER to stop the test after a period of time ktest: Monitor kernel while running of user tests ktest: Fix bug where the test would not end after failure ktest: Add BISECT_FILES to run git bisect on paths ktest: Add BISECT_SKIP ktest: Add manual bisect ktest: Handle kernels before make oldnoconfig ktest: Start failure timeout on panic too ktest: Print logfile name on failure
| * ktest: Add STOP_TEST_AFTER to stop the test after a period of timeSteven Rostedt2011-03-082-1/+20
| | | | | | | | | | | | | | | | | | | | Currently, if a test causes constant output but never reaches a boot prompt, or crashes, the test will never stop. Add STOP_TEST_AFTER to create a variable that will stop (and fail) the test after it has run for this amount of time. The default is 10 minutes. Setting this variable to -1 will disable it. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
| * ktest: Monitor kernel while running of user testsSteven Rostedt2011-03-081-0/+14
| | | | | | | | | | | | | | | | | | | | Record the console of tests to both the console and the log. Also, record the bug reports afte the test has completed. Currently, if a kernel bug happens while running the userland test, the test stops and will not record the kernel bug. This makes it difficult to solve what happened. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
| * ktest: Fix bug where the test would not end after failureSteven Rostedt2011-03-081-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | The config STOP_AFTER_FAILURE is the number of seconds to continue the test when a failure is detected. This lets the monitor record more data to the logs and console that may be helpful in solving the bug that was found. But the test had a bug. If the failure caused multiple "Call Trace" stack dumps, the start time to compare the STOP_AFTER_FAILURE would constantly be reset. Only update the start time at the first "Call Trace" instance. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
| * ktest: Add BISECT_FILES to run git bisect on pathsSteven Rostedt2011-03-082-1/+17
| | | | | | | | | | | | | | Add the config option BISECT_FILES that allows the user to specify what path in the kernel to run the git bisect on. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
| * ktest: Add BISECT_SKIPSteven Rostedt2011-03-082-8/+37
| | | | | | | | | | | | | | | | | | | | If a during a git bisect, ktest fails on something other than what it is testing (if BISECT_TYPE is test but it fails on build), if BISECT_SKIP is set, then it will do a "git bisect skip" instead of just failing the bisect and letting the user find a good commit to test. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
| * ktest: Add manual bisectSteven Rostedt2011-03-082-2/+47
| | | | | | | | | | | | | | | | | | | | For both git bisect and config bisect, if BISECT_MANUAL is set to 1, then bisect will stop between iterations and ask the user for the result. The actual result is ignored. This makes it possible to use ktest.pl for bisecting configs and git and let the user examine the results themselves and enter their own results. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
| * ktest: Handle kernels before make oldnoconfigSteven Rostedt2011-03-081-7/+20
| | | | | | | | | | | | | | | | | | | | | | | | When bisecting, one may come across a kernel that does not have make oldnoconfig. In this case, we need to run the command "yes" into a make oldconfig. This will select defaults instead of 'n' into each command, but it works as a work around. Note, "yes n" will not work because a config may have a value that "n" is not acceptable for. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
| * ktest: Start failure timeout on panic tooSteven Rostedt2011-03-081-0/+1
| | | | | | | | | | | | | | | | | | Currently we just look for a Call Trace to start the time out when to reboot the box. But if the kernel panics and does not show a Call Trace, the test will not reboot the box after the specified timeout. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
| * ktest: Print logfile name on failureSteven Rostedt2011-03-081-0/+4
| | | | | | | | | | | | | | | | If the test fails and a logfile was specified. Print the name to let the user know where to look for more information on the failure. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
* | trivial: Fix Steven's Copyright typosUwe Kleine-König2011-02-081-1/+1
|/ | | | | | | | | OK, the copyright allows you to write a copy, still I think the lawyers prefer the correct spelling. Signed-off-by: Uwe Kleine-König <u.kleine-koenig@pengutronix.de> LKML-Reference: <1295899921-11333-1-git-send-email-u.kleine-koenig@pengutronix.de> Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
* ktest: Ask for the manditory config options instead of just failingSteven Rostedt2010-11-181-14/+206
| | | | | | | | | In keeping with the notion that all tools should be simple for all to use. I've changed ktest.pl to ask for mandatory options instead of just failing. It will append (or create) the options the user types in, onto the config file. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
* ktest: Copy the last good and bad configs in config_bisectSteven Rostedt2010-11-181-0/+4
| | | | | | | | | | | During the config_bisect, in case of failure, it is nice to have the last good and bad .configs that were used. This would let us restart the config_bisect from those configs. Copy the last good config into the output dir as config_good, and the last bad config as config_bad. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
* ktest: For grub reboot, use run_ssh instead of run_commandSteven Rostedt2010-11-181-1/+1
| | | | | | | The run_ssh handles the ssh variable $SSH_COMMAND, which was not being used by the run_command in reboot_to function. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
* ktest: Added force stop after success and failureSteven Rostedt2010-11-182-1/+44
| | | | | | | | | | Added the options STOP_AFTER_SUCCESS and STOP_AFTER_FAILURE to allow the user to give a time (in seconds) to stop the monitor after a stack trace or login has been detected. Sometimes the kernel constantly prints out to the console and this may cause the test to run indefinitely. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
* ktest: Parse off the directory name in useconfig for failuresSteven Rostedt2010-11-181-1/+6
| | | | | | | | | | When we store failures, we create a directory that has the build_type in it. For useconfig, it also contains the name path of the config file it uses. This unfortunately gets its own directory on failure. Parse off the directory name when creating the directory to store the failures. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
* ktest: Use different temp config name for minconfigSteven Rostedt2010-11-181-2/+2
| | | | | | | | | By using the "use_config" for minconfig and addconfig we risk trying to copy itself to itself, which will cause an unexpected failure. Use a different name instead. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
* ktest: Updated the sample.conf for the latest optionsSteven Rostedt2010-11-181-1/+89
| | | | | | | Added documentation for SSH_EXEC, SCP_TO_TARGET, REBOOT, and CONFIG_BISECT and friends. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
* ktest: Added compare script to test ktest.pl to sample.confSteven Rostedt2010-11-181-0/+30
| | | | | | | | Add a compare script that makes sure that all the options in sample.conf are used in ktest.pl, and all the options in ktest.pl are described in sample.conf. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
* ktest: Added config_bisect test typeSteven Rostedt2010-11-181-21/+388
| | | | | | | | | | | | | | | | | | | Added the ability to do a config_bisect. It starts with a bad config and does the following loop. Enable half the configs. if none of the configs to check are not enabled (caused by missing dependencies) enable the other half. Run the test if the test passes, remove the configs from the check but enabled them for further tests (to satisfy dependencies). else Remove any config that was not enabled, as we have found a new config that can cause a failure. loop till we have only one config left. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
* ktest/cleanups: Added version 0.2, ssh as optionsSteven Rostedt2010-11-181-18/+63
| | | | | | | | | | | Updated to version 0.2. Now have SSH_EXEC options. Also added some cleanups for keeping track of success and reading the config file. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
* ktest: Output something easy to parse for failure or successSteven Rostedt2010-11-181-2/+2
| | | | | | | | | Have a easy way to parse the log file for success or failure. KTEST RESULT: ... Suggested-by: Tim Bird <tim.bird@am.sony.com> Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
* ktest: Allow a test case to undefine a default valueSteven Rostedt2010-11-181-0/+5
| | | | | | | | | | Allow a test case in the config file to undefine a default value by specifying the option and equal sign but not assigning it a value: OPTION = Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
* ktest: Use $output_config instead of typing $outputdir/.configSteven Rostedt2010-11-181-8/+10
| | | | | | | To help prevent typos, use $output_config as the reference to "$outputdir/.config". Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
* ktest: Write to stdout if no log file is givenSteven Rostedt2010-11-181-2/+10
| | | | | | | If no LOG_FILE option is set, then write what would be logged to that file to standard output. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
* ktest: Use oldnoconfig instead of yes commandSteven Rostedt2010-11-181-3/+2
| | | | | | | | | | Running the command "yes ''" through the make oldconfig may enable things we do not want enabled. If something is default enabled, the yes command with '' as an argument will enable it. Use oldnoconfig, which runs everything as if 'no' was used. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
* ktest: Update the sample config file with more documentationSteven Rostedt2010-11-181-91/+103
| | | | Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
* ktest: New TEST_START instead of using [], and use real SHA1sSteven Rostedt2010-11-182-118/+433
| | | | | | | | | | | | | | | | | | | | | | | Change the config to use TEST_START where the options after a TEST_START automatically get the [] as it is read and they do not need to exist in the config file; TEST_START MIN_CONFIG = myconfig is the same as MIN_CONFIG[1] = myconfig The benefit is that you no longer need to keep track of test numbers with tests. Also process the commit ids that are passed to the options to get the actually SHA1 so it is no longer relative to the branch. Ie, saying HEAD will get the current SHA1 and then that will be used, and will work even if another branch is checked out. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
* ktest: Add poweroff after halt and powercycle after rebootSteven Rostedt2010-11-182-15/+66
| | | | | | | | | | Added the options POWEROFF_AFTER_HALT to handle boxes that do not really shut off after a halt is called. Added POWERCYCLE_AFTER_REBOOT to force a power cycle for boxes that don't reboot but get stuck during the reboot. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
* ktest: Add POST_INSTALL to allow initrds to be createdSteven Rostedt2010-11-182-0/+16
| | | | | | | | | | | Add a POST_INSTALL option that runs after the build and install but before rebooting to the test kernel. This alls the user to run a script that will install an initrd (or anything else that may be special) before booting. An environment variable KERNEL_VERSION is set. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
* ktest: Added sample.conf, new %default option formatSteven Rostedt2010-11-182-142/+649
| | | | | | | | | Added sample.conf as a nice document to show new users. Use a %default hash to separate out the options that are default and allow us to complain about options being set twice. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
* ktest: Add open and close console and start stop monitorSteven Rostedt2010-11-181-87/+170
| | | | | | | | | | It is much better to keep the monitor running throughout a test than to constantly start and stop it. Some console readers will show everything that has happened before when opening the console, and by opening it several times, causes the old content to be read multiple times in a single test. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
* ktest: Added continuing on success, clear log and timeoutSteven Rostedt2010-11-181-38/+78
| | | | | | | | | | Add option to continue after a test fails. Add option to reset the log at start of running ktest. Update default timeout to 2 minutes. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
* ktest: Add reverse bisect, better logging, copyrightSteven Rostedt2010-11-181-6/+45
| | | | | | | | | | Added the ability to do a reverse bisect. Better logging of running commands. Added the copyright statement. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
* ktest: Added patchcheckSteven Rostedt2010-11-181-2/+140
| | | | | | | Added patchcheck functionality. It will checkout a given SHA1 and test that commit and all commits to another given SHA1. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
* ktest: Added reboot on successSteven Rostedt2010-11-181-1/+6
| | | | Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
* ktest: Added better console, add test buildSteven Rostedt2010-11-181-48/+159
| | | | | | | | Better reading of the console. Added running a script to do testing after build succeeds. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
* ktest: Bisecting, install modules, add loggingSteven Rostedt2010-11-181-94/+227
| | | | | | | Added bisecting, modules, logging of the output. Banners that show success. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
* ktest: New features reboot on error, make optionsSteven Rostedt2010-11-181-16/+46
| | | | | | | | | | REBOOT_ON_ERROR to reboot the box on error BUILD_OPTIONS to add options to the make build (like -j40) Added "useconfig:<config>". Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
* ktest: New features: noclean, dodie, poweroff on error and successSteven Rostedt2010-11-181-21/+94
| | | | | | | | | | | | Added dodie function to have a bit more control over die calls. BUILD_NOCLEAN to not run make mrproper or remove .config. POWEROFF_ON_{SUCCESS,ERROR} to turn off the power after tests. Skip backtrace calls that were done by the backtrace tests. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
* Initial start of ktest.plSteven Rostedt2010-11-181-0/+334
Originally named autotest.pl, but renamed to ktest.pl now because the autotest name is used by other projects. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
OpenPOWER on IntegriCloud