summaryrefslogtreecommitdiffstats
path: root/clang/utils/analyzer/SATestBuild.py
Commit message (Collapse)AuthorAgeFilesLines
* [analyzer] SATestBuild.py: Use driver for analyzing single-file tests.Artem Dergachev2019-05-291-10/+5
| | | | | | | | Don't bother coming up with a -cc1 run-line ourselves. This, in particular, gets rid of a macOS-specific code path. llvm-svn: 362009
* Portable Python script across Python versionSerge Guelton2018-12-181-2/+1
| | | | | | | | | | | In Python3, dict.items, dict.keys, dict.values, zip, map and filter no longer return lists, they create generator instead. The portability patch consists in forcing an extra `list` call if the result is actually used as a list. `map` are replaced by list comprehension and `filter` by filtered list comprehension. Differential Revision: https://reviews.llvm.org/D55197 llvm-svn: 349501
* Portable Python script across Python versionSerge Guelton2018-12-181-7/+7
| | | | | | | | Using from __future__ import print_function it is possible to have a compatible behavior of `print(...)` across Python version. Differential Revision: https://reviews.llvm.org/D55213 llvm-svn: 349454
* Portable Python script across Python versionSerge Guelton2018-12-131-2/+5
| | | | | | | | Queue module as been renamed into queue in Python3 Differential Revision: https://reviews.llvm.org/D55202 llvm-svn: 349009
* Portable Python script across Python versionSerge Guelton2018-12-031-1/+1
| | | | | | | | | | | | | | Python2 supports the two following equivalent construct raise ExceptionType, exception_value and raise ExceptionType(exception_value) Only the later is supported by Python3. Differential Revision: https://reviews.llvm.org/D55195 llvm-svn: 348126
* [analyzer] [tests] Hotfix: missing spaceGeorge Karpenkov2018-10-021-1/+1
| | | | llvm-svn: 343643
* [analyzer] [tests] Allow specifying entire -analyze-config on the command ↵George Karpenkov2018-10-021-14/+16
| | | | | | | | line, make sure it's always propagated Differential Revision: https://reviews.llvm.org/D52801 llvm-svn: 343636
* [analyzer] [testing] Pass through an extra argument for specifying extra ↵George Karpenkov2018-09-271-16/+22
| | | | | | | | analyzer options Differential Revision: https://reviews.llvm.org/D52585 llvm-svn: 343158
* [analyzer] [tests] Style fixes for testing harness.George Karpenkov2018-07-301-9/+9
| | | | llvm-svn: 338322
* [analyzer] [tests] Pass clang executable path to prefix-less executor scripts.George Karpenkov2018-07-021-0/+1
| | | | llvm-svn: 336124
* [analyzer] [tests] Allow the tested project to specify it's own analyzer wrapperGeorge Karpenkov2018-06-291-0/+14
| | | | llvm-svn: 336023
* [analyzer] [tests] Fix 80 column violation in SATestBuild.pyGeorge Karpenkov2018-06-291-5/+5
| | | | llvm-svn: 336022
* [analyzer] Fix string not being formatted with extra argumentsMikhail R. Gadelha2018-06-271-1/+2
| | | | | Signed-off-by: Mikhail Ramalho <mikhail.ramalho@gmail.com> llvm-svn: 335739
* [analyzer] [tests] Include statistics in tests.George Karpenkov2018-06-261-1/+7
| | | | llvm-svn: 335685
* [analyzer] [testing] Be less verbose by default in integration testing.George Karpenkov2018-03-291-1/+1
| | | | llvm-svn: 328752
* [analyzer] [tests] Update CmpRuns to write to stdout correctly in ↵George Karpenkov2018-02-131-1/+3
| | | | | | multithreaded environment llvm-svn: 325070
* [analyzer] [tests] Fix a typo in analyzer testing script.George Karpenkov2018-02-121-1/+1
| | | | | | Incorrect option instance construction. llvm-svn: 324946
* [analyzer] [tests] [NFC] Remove a fragile tightly-coupled component ↵George Karpenkov2018-02-091-1/+2
| | | | | | | | | | emulating parser output ...when we can just use the real parser instead. Differential Revision: https://reviews.llvm.org/D43098 llvm-svn: 324759
* [analyzer] [tests] Test different projects concurrentlyGeorge Karpenkov2018-02-081-65/+175
| | | | | | Differential Revision: https://reviews.llvm.org/D43031 llvm-svn: 324652
* [analyzer] Use stable filenames in analyzer testing infrastructureGeorge Karpenkov2018-01-291-0/+1
| | | | | | | | Makes finding the right file in test results easier. Differential Revision: https://reviews.llvm.org/D42445 llvm-svn: 323697
* [analyzer] [tests] Remove empty folders in reference results, do not store ↵George Karpenkov2017-10-301-4/+12
| | | | | | | | | | | diffs.txt Storing diffs.txt is now redundant, as we simply dump the CmpRuns output to stdout (it is saved in CI and tends to be small). Not generating those files enables us to remove empty folders, which confuse git, as it would not add them with reference results. llvm-svn: 316948
* [Analyzer] [Tests] Fixing typo from the previous commit.George Karpenkov2017-10-271-1/+1
| | | | | | Can not open a non-existent file with r+. llvm-svn: 316808
* [Analyzer] [Tests] Dump the output of scan-build to stdout on failure.George Karpenkov2017-10-271-5/+6
| | | | | | Eliminates extra lookup step during debugging. llvm-svn: 316806
* [Analyzer] [Tests] Write analyzers crashes to stdout, and not to a separate fileGeorge Karpenkov2017-10-261-20/+12
| | | | | | | With this change it would be sufficient to look at CI console to see the failure. llvm-svn: 316687
* [Analyzer] [Tests] Consistently use exit codes. Use code=42 to signify ↵George Karpenkov2017-10-261-8/+11
| | | | | | different results llvm-svn: 316632
* [Analyzer] [Tests] Do not discard output from CmpRuns.py when running ↵George Karpenkov2017-10-251-7/+1
| | | | | | | | | | integration tests Contrary to the deleted comment, in most cases CmpRuns.py produces a fairly small amount of output, which is useful to see straight away to see what has changed when executing the integration tests. llvm-svn: 316618
* [Analyzer] [Tests] Minor refactor of testing infrastructure:George Karpenkov2017-10-241-115/+14
| | | | | | | Move utilities functions into a separate file to make comprehension easier. llvm-svn: 316535
* [Analyzer] [Tests] Remove temporary fields from generated reference results.George Karpenkov2017-10-241-0/+8
| | | | | | | Pointer to HTML diagnostics is removed (as it is not stored) as well as the version (as it would be available from the commit message). llvm-svn: 316534
* [Analyzer] Fix bug in testing scripts, which always marked result as failure.George Karpenkov2017-10-241-0/+1
| | | | llvm-svn: 316522
* [Analyzer] Remove dead code from testing scriptsGeorge Karpenkov2017-10-111-53/+2
| | | | | | Differential Revision: https://reviews.llvm.org/D38488 llvm-svn: 315489
* [Analyzer Tests] Run static analyzer integration tests until the end,George Karpenkov2017-10-051-8/+20
| | | | | | | | Do not stop at the first failure. Differential Revision: https://reviews.llvm.org/D38589 llvm-svn: 314992
* [Analyzer] Make testing scripts flake8 compliantGeorge Karpenkov2017-10-021-162/+230
| | | | | | Differential Review: https://reviews.llvm.org/D38213 llvm-svn: 314692
* [Analyzer] Add nullability to the list of tested checkers in SATestBuildGeorge Karpenkov2017-09-301-1/+12
| | | | | | Differential Revision: https://reviews.llvm.org/D38162 llvm-svn: 314572
* [Analyzer] Fix minor errors in python scripts.George Karpenkov2017-09-221-1/+0
| | | | llvm-svn: 314042
* [Analyzer] Refactor analyzer testing scripts.George Karpenkov2017-09-221-86/+74
| | | | | | | | | | | | | - Exporting needed function for future reuse. - Idiomatic python: using with `file as f` instead of `try/finally`. - Fixing some indentation issues. - No need to reinvent python `multiprocessing.getCPUCount()` - Removing a function parameter which is always the same under all invocations. - Adding some docstrings. Differential Revision: https://reviews.llvm.org/D38156 llvm-svn: 313949
* [Analyzer] Use CC environment variable to select analyzer path in SATestBuild.George Karpenkov2017-09-211-1/+4
| | | | | | | | | This change is required to easily test the given checkout of the analyzer, rather than the one bundled with a system compiler. Differential Revision: https://reviews.llvm.org/D38155 llvm-svn: 313927
* [analyzer] SATestBuild.py: Treat '#' as comment in projectMap.csvDevin Coughlin2016-09-191-0/+8
| | | | | | | | | | Treat lines in projectMap.csv that start with '#' as comments. This enables a workflow where projects can be temporarily disabled with a comment describing when they should be turned back on. Differential Revision: https://reviews.llvm.org/D24709 llvm-svn: 281880
* [analyzer] SATestBuild.py: Remove html and log when producing reference results.Devin Coughlin2016-01-231-7/+29
| | | | | | | | | | | The html reports are huge -- every issue in a given file results in a separate copy of the source code, in HTML form, for the file. This gets very large quickly and it doesn't make sense to check this into a reference repository. Also remove the log when generating reference results because it can leak absolute path names. We still keep both the html and the log around when producing non-reference results. llvm-svn: 258594
* [analyzer] SATestBuild.py: Handle spaces in path passed to --use-analyzer.Devin Coughlin2016-01-221-2/+2
| | | | | | I missed this one in r258493. llvm-svn: 258517
* [analyzer] Update SATestBuild.py to handle spaces in paths.Devin Coughlin2016-01-221-10/+10
| | | | | | | The Jenkins workspace on the new Green Dragon builder for the static analyzer has spaces in its path. llvm-svn: 258493
* [analyzer] Update SATestBuild.py to enable a 'download and patch' model for ↵Devin Coughlin2015-11-071-12/+87
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | projects. Currently the SATestBuild.py and SATestAdd.py buildbot scripts expect project sources to be checked into the project repository. This commit changes these scripts to additionally support a model where project sources are downloaded rather than checked into the repository. Sometimes projects may need to be modified (for example, to support a newer versions of clang), so the updated scripts also allow for an optional patch file that will be applied to the downloaded project source before analysis. To support this workflow, this commit changes the expected layout of a project in the repository. The project-specific helper scripts will stay in the root of each project directory, but the benchmark source itself (if checked into the repo) should now be stored in a subdirectory named 'CachedSource': project_name/ cleanup_run_static_analyzer.sh [optional] run_static_analyzer.cmd [required] download_project.sh [optional] CachedSource/ [optional] changes_for_analyzer.patch [optional] If the 'CachedSource' source directory is not present, the download script will be executed. This script should download the project source into 'CachedSource'. Then, if 'changes_for_analyzer.patch' is present its changes will be applied to a copy of 'CachedSource' before analysis. Differential Revision: http://reviews.llvm.org/D14345 llvm-svn: 252410
* [analyzer] SATestBuild.py: Move additional checkers logic so SATestAdd.py ↵Devin Coughlin2015-09-161-4/+6
| | | | | | | | | | | | | can use it as well. Move the logic looking for additional checkers in the SA_ADDITIONAL_CHECKERS environmental variable from SATestBuild's main() to runScanBuild(). This allows SATestAdd.py to use the variable as well. Without it, we won't include additional checkers when building reference results for the build bot. Differential Revision: http://reviews.llvm.org/D12891 llvm-svn: 247767
* [analyzer] Update SATestBuild.py to set -isysroot for preprocessed filesDevin Coughlin2015-09-141-2/+19
| | | | | | | | | Update the static analyzer buildbot script to set -isysroot to the OS X SDK path when analyzing preprocessed files on OS X. Differential Revision: http://reviews.llvm.org/D12769 llvm-svn: 247617
* [analyzer] Apply whitespace cleanups by Honggyu Kim.Ted Kremenek2015-09-081-113/+113
| | | | llvm-svn: 246978
* [Static Analyzer] Made it easier to test new checkers using the test suite.Gabor Horvath2015-08-201-0/+7
| | | | llvm-svn: 245637
* [Static Analyzer] Make the paths relative from the project root when ↵Gabor Horvath2015-07-081-1/+14
| | | | | | generating reference results in the test suite. llvm-svn: 241706
* [Static Analyzer] No longer include the full path of the source file in the ↵Gabor Horvath2015-07-021-1/+1
| | | | | | single file tests' bug identifiers. llvm-svn: 241292
* [Static Analyzer] Minor improvements to SATest.Gabor Horvath2015-06-301-19/+40
| | | | | | Differential Revision: http://reviews.llvm.org/D10812 llvm-svn: 241073
* [analyzer] Rename NewDeleteLeaks checker in the test script.Anna Zaks2014-10-311-1/+1
| | | | | | Fixup to r220289. llvm-svn: 220976
* Correct hyphenations in comments and assert messagesAlp Toker2013-12-051-2/+2
| | | | | | | This patch tries to avoid unrelated changes other than fixing a few hyphen-related ambiguities in nearby lines. llvm-svn: 196466
OpenPOWER on IntegriCloud