summaryrefslogtreecommitdiffstats
path: root/compiler-rt/test/scudo
Commit message (Collapse)AuthorAgeFilesLines
* [Scudo] [GWP-ASan] Add GWP-ASan to Scudo Standalone.Mitch Phillips2019-12-131-0/+4
| | | | | | | | | | | | | | | | Summary: Adds GWP-ASan to Scudo standalone. Default parameters are pulled across from the GWP-ASan build. No backtrace support as of yet. Reviewers: cryptoad, eugenis, pcc Reviewed By: cryptoad Subscribers: merge_guards_bot, mgorny, #sanitizers, llvm-commits, cferris, vlad.tsyrklevich, pcc Tags: #sanitizers, #llvm Differential Revision: https://reviews.llvm.org/D71229
* compiler-rt: Remove .cc from all lit config filesNico Weber2019-08-051-1/+1
| | | | | | All cc files have been renamed to cpp now. llvm-svn: 367911
* [compiler-rt] Rename lit.*.cfg.* -> lit.*.cfg.py.*Reid Kleckner2019-06-275-7/+7
| | | | | | | | | | | | | These lit configuration files are really Python source code. Using the .py file extension helps editors and tools use the correct language mode. LLVM and Clang already use this convention for lit configuration, this change simply applies it to all of compiler-rt. Reviewers: vitalybuka, dberris Differential Revision: https://reviews.llvm.org/D63658 llvm-svn: 364591
* [GWP-ASan] Integration with Scudo [5].Mitch Phillips2019-06-171-0/+4
| | | | | | | | | | | | | | | | | | | | | | Summary: See D60593 for further information. This patch adds GWP-ASan support to the Scudo hardened allocator. It also implements end-to-end integration tests using Scudo as the backing allocator. The tests include crash handling for buffer over/underflow as well as use-after-free detection. Reviewers: vlad.tsyrklevich, cryptoad Reviewed By: vlad.tsyrklevich, cryptoad Subscribers: kubamracek, mgorny, #sanitizers, llvm-commits, morehouse Tags: #sanitizers, #llvm Differential Revision: https://reviews.llvm.org/D62929 llvm-svn: 363584
* [sanitizer] Remove "-android" from test_arch.Evgeniy Stepanov2019-02-251-9/+1
| | | | | | | | | | | | | | | | | | Summary: ASan and Scudo tests are adding "-android" to test arch. There are no tests that depend on it as far as I can see. If necessary, do this instead: REQUIRES: aarch64-target-arch && android Reviewers: pcc, vitalybuka Subscribers: srhines, kubamracek, mgorny, javed.absar, kristof.beyls, cryptoad, #sanitizers, llvm-commits Tags: #sanitizers, #llvm Differential Revision: https://reviews.llvm.org/D58532 llvm-svn: 354829
* [scudo] Initial standalone skeleton check-inKostya Kortchinsky2019-02-043-0/+29
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: This is the initial check-in for the Standalone version of Scudo. The project is initially going to live in scudo/standalone then will replace scudo. See http://lists.llvm.org/pipermail/llvm-dev/2019-January/129113.html for details. This initial CL is meant to lay out the project structure, of both code & tests, providing a minimal amount of functionalities, namely various definitions, some atomic helpers and an intrusive list. (empty.cc is just here to have a compilation unit, but will go away in the upcoming CLs). Initial support is restricted to Linux i386 & x86_64 in make files and will be extended once things land & work. We will grow organically from here, adding functionalities in limited amounts. Reviewers: morehouse, eugenis, vitalybuka, kcc, mcgrathr, flowerhack Reviewed By: morehouse, vitalybuka Subscribers: srhines, mgorny, krytarowski, delcypher, jfb, #sanitizers, llvm-commits Tags: #llvm, #sanitizers Differential Revision: https://reviews.llvm.org/D57412 llvm-svn: 353055
* [scudo] Delay allocations in the RSS check testKostya Kortchinsky2019-01-251-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | Summary: D57116 fails on the armv7 bots, which is I assume due to the timing of the RSS check on the platform. While I don't have a platform to test that change on, I assume this would do. The test could be made more reliable by either delaying more the allocations, or allocating more large-chunks, but both those options have a somewhat non negligible impact (more memory used, longer test). Hence me trying to keep the additional sleeping/allocating to a minimum. Reviewers: eugenis, yroux Reviewed By: yroux Subscribers: javed.absar, kristof.beyls, delcypher, #sanitizers, llvm-commits Differential Revision: https://reviews.llvm.org/D57241 llvm-svn: 352220
* [scudo] Tuning changes based on feedback from current useKostya Kortchinsky2019-01-241-12/+12
| | | | | | | | | | | | | | | | | | | | | | Summary: This tunes several of the default parameters used within the allocator: - disable the deallocation type mismatch on Android by default; this was causing too many issues with third party libraries; - change the default `SizeClassMap` to `Dense`, it caches less entries and is way more memory efficient overall; - relax the timing of the RSS checks, 10 times per second was too much, lower it to 4 times (every 250ms), and update the test so that it passes with the new default. Reviewers: eugenis Reviewed By: eugenis Subscribers: srhines, delcypher, #sanitizers, llvm-commits Differential Revision: https://reviews.llvm.org/D57116 llvm-svn: 352057
* [scudo] Fix race condition in deallocation path when Quarantine is bypassedKostya Kortchinsky2018-08-141-0/+69
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: There is a race window in the deallocation path when the Quarantine is bypassed. Initially we would just erase the header of a chunk if we were not to use the Quarantine, as opposed to using a compare-exchange primitive, to make things faster. It turned out to be a poor decision, as 2 threads (or more) could simultaneously deallocate the same pointer, and if the checks were to done before the header got erased, this would result in the pointer being added twice (or more) to distinct thread caches, and eventually be reused. Winning the race is not trivial but can happen with enough control over the allocation primitives. The repro added attempts to trigger the bug, with a moderate success rate, but it should be enough to notice if the bug ever make its way back into the code. Since I am changing things in this file, there are 2 smaller changes tagging along, marking a variable `const`, and improving the Quarantine bypass test at runtime. Reviewers: alekseyshl, eugenis, kcc, vitalybuka Reviewed By: eugenis, vitalybuka Subscribers: delcypher, #sanitizers, llvm-commits Differential Revision: https://reviews.llvm.org/D50655 llvm-svn: 339705
* [scudo] Use mkdir -p when creating directories for a testFilipe Cabecinhas2018-07-101-1/+1
| | | | llvm-svn: 336673
* [compiler-rt] Get rid of "%T" expansionsFilipe Cabecinhas2018-07-101-8/+8
| | | | | | | | | | | | | | | | | | | Summary: Original patch by Kuba Mracek The %T lit expansion expands to a common directory shared between all the tests in the same directory, which is unexpected and unintuitive, and more importantly, it's been a source of subtle race conditions and flaky tests. In https://reviews.llvm.org/D35396, it was agreed that it would be best to simply ban %T and only keep %t, which is unique to each test. When a test needs a temporary directory, it can just create one using mkdir %t. This patch removes %T in compiler-rt. Differential Revision: https://reviews.llvm.org/D48618 llvm-svn: 336661
* [scudo] Add some runtime tests for the minimal runtimeKostya Kortchinsky2018-06-294-4/+43
| | | | | | | | | | | | | | | | Summary: As well as some tests to ensure that various combinations of the clang command line flags work (shared/static/minimal). Reviewers: eugenis, alekseyshl, vitalybuka Reviewed By: vitalybuka Subscribers: srhines, delcypher, #sanitizers, llvm-commits Differential Revision: https://reviews.llvm.org/D48553 llvm-svn: 335981
* Support for multiarch runtimes layoutPetr Hosek2018-06-281-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | This change adds a support for multiarch style runtimes layout, so in addition to the existing layout where runtimes get installed to: lib/clang/$version/lib/$os Clang now allows runtimes to be installed to: lib/clang/$version/$target/lib This also includes libc++, libc++abi and libunwind; today those are assumed to be in Clang library directory built for host, with the new layout it is possible to install libc++, libc++abi and libunwind into the runtime directory built for different targets. The use of new layout is enabled by setting the LLVM_ENABLE_RUNTIME_TARGET_DIR CMake variable and is supported by both projects and runtimes layouts. The runtimes CMake build has been further modified to use the new layout when building runtimes for multiple targets. Differential Revision: https://reviews.llvm.org/D45604 llvm-svn: 335809
* [scudo] Add verbose failures in place of CHECK(0)Kostya Kortchinsky2018-06-154-10/+16
| | | | | | | | | | | | | | | | | | | | | | | | Summary: The current `FailureHandler` mechanism was fairly opaque with regard to the failure reason due to using `CHECK(0)`. Scudo is a bit different from the other Sanitizers as it prefers to avoid spurious processing in its failure path. So we just `dieWithMessage` using a somewhat explicit string. Adapted the tests for the new strings. While this takes care of the `OnBadRequest` & `OnOOM` failures, the next step is probably to migrate the other Scudo failures in the same failes (header corruption, invalid state and so on). Reviewers: alekseyshl Reviewed By: alekseyshl Subscribers: filcab, mgorny, delcypher, #sanitizers, llvm-commits Differential Revision: https://reviews.llvm.org/D48199 llvm-svn: 334843
* [scudo] Add C++17 aligned new/delete operators supportKostya Kortchinsky2018-06-121-0/+84
| | | | | | | | | | | | | | | | | | | Summary: This CL adds support for aligned new/delete operators (C++17). Currently we do not support alignment inconsistency detection on deallocation, as this requires a header change, but the APIs are introduced and are functional. Add a smoke test for the aligned version of the operators. Reviewers: alekseyshl Reviewed By: alekseyshl Subscribers: delcypher, #sanitizers, llvm-commits Differential Revision: https://reviews.llvm.org/D48031 llvm-svn: 334505
* [scudo] Adding an interface function to print allocator statsKostya Kortchinsky2018-04-251-0/+21
| | | | | | | | | | | | | | | | Summary: This adds `__scudo_print_stats` as an interface function to display the Primary and Secondary allocator statistics for Scudo. Reviewers: alekseyshl, flowerhack Reviewed By: alekseyshl Subscribers: delcypher, llvm-commits, #sanitizers Differential Revision: https://reviews.llvm.org/D46016 llvm-svn: 330857
* [scudo] Make logging more consistentKostya Kortchinsky2018-03-072-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: A few changes related to logging: - prepend `Scudo` to the error messages so that users can identify that we reported an error; - replace a couple of `Report` calls in the RSS check code with `dieWithMessage`/`Print`, mark a condition as `UNLIKELY` in the process; - change some messages so that they all look more or less the same. This includes the `CHECK` message; - adapt a couple of tests with the new strings. A couple of side notes: this results in a few 1-line-blocks, for which I left brackets. There doesn't seem to be any style guide for that, I can remove them if need be. I didn't use `SanitizerToolName` in the strings, but directly `Scudo` because we are the only users, I could change that too. Reviewers: alekseyshl, flowerhack Reviewed By: alekseyshl Subscribers: mgorny, delcypher, llvm-commits, #sanitizers Differential Revision: https://reviews.llvm.org/D44171 llvm-svn: 326901
* [scudo] Make some tests less Linux-yKostya Kortchinsky2018-02-265-44/+43
| | | | | | | | | | | | | | | | | | | | | Summary: Start making the Scudo tests less Linux-y: - `malloc_usable_size` doesn't exist everywhere, so replace them with `__sanitizer_get_allocated_size` which we provide; - move all the `memalign` related tests into `memalign.c` since it's also not available everywhere. I also noticed that the `memalign.c` was missing a line in one of the loops. Reviewers: alekseyshl Reviewed By: alekseyshl Subscribers: delcypher, #sanitizers, llvm-commits Differential Revision: https://reviews.llvm.org/D43393 llvm-svn: 326100
* [scudo] Use -fsanitize=scudo rather than --whole-archive in testsKostya Kortchinsky2018-01-181-8/+5
| | | | | | | | | | | | | | | | | | | | | | | | Summary: Tests were being run by whole-linking the static library with our test binaries. But since `-fsanitize=scudo` landed with rL317337, we might as well change how the tests are compiled to use it. The only difference will be on Android, where the clang flag links in the dynamic library instead, but the bots are already pushing `libclang_rt.*-android.so` to the device there is no additional change needed. Tested locally, including with a standalone build, and an Android one on a O device, and it all passes. Reviewers: alekseyshl Reviewed By: alekseyshl Subscribers: #sanitizers, llvm-commits Differential Revision: https://reviews.llvm.org/D42243 llvm-svn: 322882
* [scudo] Attempt to re-enable the valloc test on armhfKostya Kortchinsky2018-01-041-1/+1
| | | | | | | | | | | | | | | | Summary: It used to fail on the bots, but I could not repro it locally. So turn it back on to try and see if it still fails and maybe get to the heart of it. Reviewers: alekseyshl, flowerhack Reviewed By: alekseyshl Subscribers: aemerson, srhines, kristof.beyls, llvm-commits, #sanitizers Differential Revision: https://reviews.llvm.org/D41707 llvm-svn: 321812
* [scudo] Touch memory to count as RSSJonas Hahnfeld2018-01-011-5/+10
| | | | | | | | This should fix the test from https://reviews.llvm.org/D41128. Differential Revision: https://reviews.llvm.org/D41649 llvm-svn: 321627
* [scudo] Disabling the interface test on armhfKostya Kortchinsky2017-12-141-0/+1
| | | | | | | | | | | | | | | | | | Summary: I will investigate the breakage tomorrow, disable on armhf to turn the bots green over the night. http://lab.llvm.org:8011/builders/clang-cmake-thumbv7-a15-full-sh/builds/2952/steps/ninja%20check%202/logs/FAIL%3A%20Scudo-armhf%3A%3A%20interface.cpp This is post https://reviews.llvm.org/D41128. Reviewers: alekseyshl Subscribers: aemerson, kristof.beyls, llvm-commits, #sanitizers Differential Revision: https://reviews.llvm.org/D41218 llvm-svn: 320665
* [scudo] Adding a public Scudo interfaceKostya Kortchinsky2017-12-131-3/+42
| | | | | | | | | | | | | | | | Summary: The first and only function to start with allows to set the soft or hard RSS limit at runtime. Add associated tests. Reviewers: alekseyshl Reviewed By: alekseyshl Subscribers: mgorny, #sanitizers, llvm-commits Differential Revision: https://reviews.llvm.org/D41128 llvm-svn: 320611
* [ubsan] Re-commit: lit changes for lld testing, future lto testing.Roman Lebedev2017-12-011-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: As discussed in https://github.com/google/oss-fuzz/issues/933, it would be really awesome to be able to use ThinLTO for fuzzing. However, as @kcc has pointed out, it is currently undefined (untested) whether the sanitizers actually function properly with LLD and/or LTO. This patch is inspired by the cfi test, which already do test with LTO (and/or LLD), since LTO is required for CFI to function. I started with UBSan, because it's cmakelists / lit.* files appeared to be the cleanest. This patch adds the infrastructure to easily add LLD and/or LTO sub-variants of the existing lit test configurations. Also, this patch adds the LLD flavor, that explicitly does use LLD to link. The check-ubsan does pass on my machine. And to minimize the [initial] potential buildbot breakage i have put some restrictions on this flavour. Please review carefully, i have not worked with lit/sanitizer tests before. The original attempt, r319525 was reverted in r319526 due to the failures in compiler-rt standalone builds. Reviewers: eugenis, vitalybuka Reviewed By: eugenis Subscribers: #sanitizers, pcc, kubamracek, mgorny, llvm-commits, mehdi_amini, inglorion, kcc Differential Revision: https://reviews.llvm.org/D39508 llvm-svn: 319575
* Revert "[ubsan] lit changes for lld testing, future lto testing."Roman Lebedev2017-12-011-1/+1
| | | | | | | | | | | | | | | | | | | | This reverts commit r319525. This change has introduced a problem with the Lit tests build for compiler-rt using Gold: http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux/builds/6047/steps/test%20standalone%20compiler-rt/logs/stdio llvm-lit: /b/sanitizer-x86_64-linux/build/llvm/utils/lit/lit/TestingConfig.py:101: fatal: unable to parse config file '/b/sanitizer-x86_64-linux/build/llvm/projects/compiler-rt/test/profile/Linux/lit.local.cfg', traceback: Traceback (most recent call last): File "/b/sanitizer-x86_64-linux/build/llvm/utils/lit/lit/TestingConfig.py", line 88, in load_from_path exec(compile(data, path, 'exec'), cfg_globals, None) File "/b/sanitizer-x86_64-linux/build/llvm/projects/compiler-rt/test/profile/Linux/lit.local.cfg", line 37, in <module> if root.host_os not in ['Linux'] or not is_gold_linker_available(): File "/b/sanitizer-x86_64-linux/build/llvm/projects/compiler-rt/test/profile/Linux/lit.local.cfg", line 27, in is_gold_linker_available stderr = subprocess.PIPE) File "/usr/lib/python2.7/subprocess.py", line 390, in __init__ errread, errwrite) File "/usr/lib/python2.7/subprocess.py", line 1024, in _execute_child raise child_exception OSError: [Errno 2] No such file or directory llvm-svn: 319529
* [ubsan] lit changes for lld testing, future lto testing.Roman Lebedev2017-12-011-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: As discussed in https://github.com/google/oss-fuzz/issues/933, it would be really awesome to be able to use ThinLTO for fuzzing. However, as @kcc has pointed out, it is currently undefined (untested) whether the sanitizers actually function properly with LLD and/or LTO. This patch is inspired by the cfi test, which already do test with LTO (and/or LLD), since LTO is required for CFI to function. I started with UBSan, because it's cmakelists / lit.* files appeared to be the cleanest. This patch adds the infrastructure to easily add LLD and/or LTO sub-variants of the existing lit test configurations. Also, this patch adds the LLD flavor, that explicitly does use LLD to link. The check-ubsan does pass on my machine. And to minimize the [initial] potential buildbot breakage i have put some restrictions on this flavour. Please review carefully, i have not worked with lit/sanitizer tests before. Reviewers: eugenis, vitalybuka Reviewed By: eugenis Subscribers: #sanitizers, pcc, kubamracek, mgorny, llvm-commits, mehdi_amini, inglorion, kcc Differential Revision: https://reviews.llvm.org/D39508 llvm-svn: 319525
* [scudo] Soft and hard RSS limit checksKostya Kortchinsky2017-11-151-0/+56
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: This implements an opportunistic check for the RSS limit. For ASan, this was implemented thanks to a background thread checking the current RSS vs the set limit every 100ms. This was deemed problematic for Scudo due to potential Android concerns (Zygote as pointed out by Aleksey) as well as the general inconvenience of having a permanent background thread. If a limit (soft or hard) is specified, we will attempt to update the RSS limit status (exceeded or not) every 100ms. This is done in an opportunistic way: if we can update it, we do it, if not we return the current status, mostly because we don't need it to be fully consistent (it's done every 100ms anyway). If the limit is exceeded `allocate` will act as if OOM for a soft limit, or just die for a hard limit. We use the `common_flags()`'s `hard_rss_limit_mb` & `soft_rss_limit_mb` for configuration of the limits. Reviewers: alekseyshl Reviewed By: alekseyshl Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D40038 llvm-svn: 318301
* [scudo] Fix standlone build -lrt requirementKostya Kortchinsky2017-11-011-1/+2
| | | | | | | | | | | | | | | | Summary: The split in D39461 introduced separate C++ flags, but `cxx_flags` needs `-lrt` as well for the standalone build. Reviewers: alekseyshl Reviewed By: alekseyshl Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D39497 llvm-svn: 317103
* [scudo] Implement stricter separation of C vs C++Kostya Kortchinsky2017-11-0120-26/+30
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: Initially, Scudo had a monolithic design where both C and C++ functions were living in the same library. This was not necessarily ideal, and with the work on -fsanitize=scudo, it became more apparent that this needed to change. We are splitting the new/delete interceptor in their own C++ library. This allows more flexibility, notably with regard to std::bad_alloc when the work is done. This also allows us to not link new & delete when using pure C. Additionally, we add the UBSan runtimes with Scudo, in order to be able to have a -fsanitize=scudo,undefined in Clang (see work in D39334). The changes in this patch: - split the cxx specific code in the scudo cmake file into a new library; (remove the spurious foreach loop, that was not necessary) - add the UBSan runtimes (both C and C++); - change the test cmake file to allow for specific C & C++ tests; - make C tests pure C, rename their extension accordingly. Reviewers: alekseyshl Reviewed By: alekseyshl Subscribers: srhines, mgorny, llvm-commits Differential Revision: https://reviews.llvm.org/D39461 llvm-svn: 317097
* [sanitizer] Random shuffling of chunks for the 32-bit Primary AllocatorKostya Kortchinsky2017-10-251-2/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: The 64-bit primary has had random shuffling of chunks for a while, this implements it for the 32-bit primary. Scudo is currently the only user of `kRandomShuffleChunks`. This change consists of a few modifications: - move the random shuffling functions out of the 64-bit primary to `sanitizer_common.h`. Alternatively I could move them to `sanitizer_allocator.h` as they are only used in the allocator, I don't feel strongly either way; - small change in the 64-bit primary to make the `rand_state` initialization `UNLIKELY`; - addition of a `rand_state` in the 32-bit primary's `SizeClassInfo` and shuffling of chunks when populating the free list. - enabling the `random_shuffle.cpp` test on platforms using the 32-bit primary for Scudo. Some comments on why the shuffling is done that way. Initially I just implemented a `Shuffle` function in the `TransferBatch` which was simpler but I came to realize this wasn't good enough: for chunks of 10000 bytes for example, with a `CompactSizeClassMap`, a batch holds only 1 chunk, meaning shuffling the batch has no effect, while a region is usually 1MB, eg: 104 chunks of that size. So I decided to "stage" the newly gathered chunks in a temporary array that would be shuffled prior to placing the chunks in batches. The result is looping twice through n_chunks even if shuffling is not enabled, but I didn't notice any significant significant performance impact. Reviewers: alekseyshl Reviewed By: alekseyshl Subscribers: srhines, llvm-commits, kubamracek Differential Revision: https://reviews.llvm.org/D39244 llvm-svn: 316596
* [scudo] Add a shared runtimeKostya Kortchinsky2017-10-232-6/+27
| | | | | | | | | | | | | | | | | Summary: Up to now, the Scudo cmake target only provided a static library that had to be linked to an executable to benefit from the hardened allocator. This introduces a shared library as well, that can be LD_PRELOAD'ed. Reviewers: alekseyshl Reviewed By: alekseyshl Subscribers: srhines, mgorny, llvm-commits Differential Revision: https://reviews.llvm.org/D38980 llvm-svn: 316342
* [scudo] Temporary disabling the valloc test on armhfKostya Kortchinsky2017-09-271-1/+1
| | | | | | | | | | | | | | | | | | | | | Summary: Weird failure where `errno != ENOMEM` on valloc failure. The returned pointer is null since it passes the previous assert, so this shouldn't happen. http://lab.llvm.org:8011/builders/clang-cmake-armv7-a15-full/builds/10931 http://lab.llvm.org:8011/builders/clang-cmake-thumbv7-a15-full-sh/builds/2469 Disabling until we figure out what's going on. Reviewers: alekseyshl Reviewed By: alekseyshl Subscribers: aemerson, srhines, llvm-commits, kristof.beyls Differential Revision: https://reviews.llvm.org/D38324 llvm-svn: 314344
* [scudo] Additional modifications for Android tests supportKostya Kortchinsky2017-09-183-2/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: With the recent move of `android_commands` to `sanitizer_common`, some things have to be updated with regard to Scudo on Android. Notably: - `config.android` is dealt with in the common code - `config.compile_wrapper` can be prepended to allow for the use of the android commands - `SCUDO_OPTIONS` must be passed with the environment when running a test - `preinit.cpp` fails with some API levels, not sure why, I will have to dig into this later. Note that `check-scudo` is not enabled yet in the bots. It's all local testing for now until everything looks good. Reviewers: alekseyshl, vitalybuka Reviewed By: vitalybuka Subscribers: srhines, kubamracek, llvm-commits Differential Revision: https://reviews.llvm.org/D37990 llvm-svn: 313561
* [scudo] Android build supportKostya Kortchinsky2017-09-1813-46/+72
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: Mark Android as supported in the cmake configuration for Scudo. Scudo is not added yet in the Android build bots, but code builds and tests pass locally. It is for a later CL. I also checked that Scudo builds as part of the Android toolchain. A few modifications had to be made: - Android defaults to `abort_on_error=1`, which doesn't work well with the current tests. So change the default way to pass `SCUDO_OPTIONS` to the tests to account for this, setting it to 0 by default; - Disable the `valloc.cpp` & `random_shuffle.cpp` tests on Android; - There is a bit of gymnatic to be done with the `SCUDO_TEST_TARGET_ARCH` string, due to android using the `-android` suffix, and `i686` instead of `i386`; - Android doesn't need `-lrt`. Reviewers: alekseyshl, eugenis Reviewed By: alekseyshl Subscribers: srhines, mgorny, llvm-commits Differential Revision: https://reviews.llvm.org/D37907 llvm-svn: 313538
* [scudo] Fix bad request handling when allocator has not been initializedKostya Kortchinsky2017-09-142-4/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: In a few functions (`scudoMemalign` and the like), we would call `ScudoAllocator::FailureHandler::OnBadRequest` if the parameters didn't check out. The issue is that if the allocator had not been initialized (eg: if this is the first heap related function called), we would use variables like `allocator_may_return_null` and `exitcode` that still had their default value (as opposed to the one set by the user or the initialization path). To solve this, we introduce `handleBadRequest` that will call `initThreadMaybe`, allowing the options to be correctly initialized. Unfortunately, the tests were passing because `exitcode` was still 0, so the results looked like success. Change those tests to do what they were supposed to. Reviewers: alekseyshl Reviewed By: alekseyshl Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D37853 llvm-svn: 313294
* [scudo] Fix improper TSD init after TLS destructors are calledKostya Kortchinsky2017-09-111-0/+42
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: Some of glibc's own thread local data is destroyed after a user's thread local destructors are called, via __libc_thread_freeres. This might involve calling free, as is the case for strerror_thread_freeres. If there is no prior heap operation in the thread, this free would end up initializing some thread specific data that would never be destroyed properly (as user's pthread destructors have already been called), while still being deallocated when the TLS goes away. As a result, a program could SEGV, usually in __sanitizer::AllocatorGlobalStats::Unregister, where one of the doubly linked list links would refer to a now unmapped memory area. To prevent this from happening, we will not do a full initialization from the deallocation path. This means that the fallback cache & quarantine will be used if no other heap operation has been called, and we effectively prevent the TSD being initialized and never destroyed. The TSD will be fully initialized for all other paths. In the event of a thread doing only frees and nothing else, a TSD would never be initialized for that thread, but this situation is unlikely and we can live with that. Reviewers: alekseyshl Reviewed By: alekseyshl Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D37697 llvm-svn: 312939
* Reland r311842 - [cmake] Remove i686 target that is duplicate to i386Michal Gorny2017-08-281-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | Remove the explicit i686 target that is completely duplicate to the i386 target, with the latter being used more commonly. 1. The runtime built for i686 will be identical to the one built for i386. 2. Supporting both -i386 and -i686 suffixes causes unnecessary confusion on the clang end which has to expect either of them. 3. The checks are based on wrong assumption that __i686__ is defined for all newer x86 CPUs. In fact, it is only declared when -march=i686 is explicitly used. It is not available when a more specific (or newer) -march is used. Curious enough, if CFLAGS contain -march=i686, the runtime will be built both for i386 and i686. For any other value, only i386 variant will be built. Differential Revision: https://reviews.llvm.org/D26764 llvm-svn: 311924
* Revert r311842 - [cmake] Remove i686 target that is duplicate to i386Michal Gorny2017-08-271-1/+1
| | | | | | | The required change in clang is being reverted because of the Android build bot failure. llvm-svn: 311859
* [cmake] Remove i686 target that is duplicate to i386Michal Gorny2017-08-271-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | Remove the explicit i686 target that is completely duplicate to the i386 target, with the latter being used more commonly. 1. The runtime built for i686 will be identical to the one built for i386. 2. Supporting both -i386 and -i686 suffixes causes unnecessary confusion on the clang end which has to expect either of them. 3. The checks are based on wrong assumption that __i686__ is defined for all newer x86 CPUs. In fact, it is only declared when -march=i686 is explicitly used. It is not available when a more specific (or newer) -march is used. Curious enough, if CFLAGS contain -march=i686, the runtime will be built both for i386 and i686. For any other value, only i386 variant will be built. Differential Revision: https://reviews.llvm.org/D26764 llvm-svn: 311842
* [scudo] Application & platform compatibility changesKostya Kortchinsky2017-08-163-48/+79
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: This patch changes a few (small) things around for compatibility purposes for the current Android & Fuchsia work: - `realloc`'ing some memory that was not allocated with `malloc`, `calloc` or `realloc`, while UB according to http://pubs.opengroup.org/onlinepubs/009695399/functions/realloc.html is more common that one would think. We now only check this if `DeallocationTypeMismatch` is set; change the "mismatch" error messages to be more homogeneous; - some sketchily written but widely used libraries expect a call to `realloc` to copy the usable size of the old chunk to the new one instead of the requested size. We have to begrundingly abide by this de-facto standard. This doesn't seem to impact security either way, unless someone comes up with something we didn't think about; - the CRC32 intrinsics for 64-bit take a 64-bit first argument. This is misleading as the upper 32 bits end up being ignored. This was also raising `-Wconversion` errors. Change things to take a `u32` as first argument. This also means we were (and are) only using 32 bits of the Cookie - not a big thing, but worth mentioning. - Includes-wise: prefer `stddef.h` to `cstddef`, move `scudo_flags.h` where it is actually needed. - Add tests for the memalign-realloc case, and the realloc-usable-size one. (Edited typos) Reviewers: alekseyshl Reviewed By: alekseyshl Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D36754 llvm-svn: 311018
* [scudo] Check for pvalloc overflowKostya Kortchinsky2017-07-252-7/+70
| | | | | | | | | | | | | | | | | | | | | | | | Summary: Previously we were rounding up the size passed to `pvalloc` to the next multiple of page size no matter what. There is an overflow possibility that wasn't accounted for. So now, return null in the event of an overflow. The man page doesn't seem to indicate the errno to set in this particular situation, but the glibc unit tests go for ENOMEM (https://code.woboq.org/userspace/glibc/malloc/tst-pvalloc.c.html#54) so we'll do the same. Update the aligned allocation funtions tests to check for properly aligned returned pointers, and the `pvalloc` corner cases. @alekseyshl: do you want me to do the same in the other Sanitizers? Reviewers: alekseyshl Reviewed By: alekseyshl Subscribers: kubamracek, alekseyshl, llvm-commits Differential Revision: https://reviews.llvm.org/D35818 llvm-svn: 309033
* [scudo] Fix QuarantineChunksUpToSize failing test on AArch64Kostya Kortchinsky2017-07-241-0/+6
| | | | | | | | | | | | | | | | Summary: Warm-up the other 2 sizes used by the tests, which should get rid of a failure on AArch64. Reviewers: alekseyshl Reviewed By: alekseyshl Subscribers: aemerson, rengolin, llvm-commits, kristof.beyls Differential Revision: https://reviews.llvm.org/D35806 llvm-svn: 308907
* [scudo] Quarantine overhaulKostya Kortchinsky2017-07-242-9/+70
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: First, some context. The main feedback we get about the quarantine is that it's too memory hungry. A single MB of quarantine will have an impact of 3 to 4MB of PSS/RSS, and things quickly get out of hand in terms of memory usage, and the quarantine ends up disabled. The main objective of the quarantine is to protect from use-after-free exploitation by making it harder for an attacker to reallocate a controlled chunk in place of the targeted freed chunk. This is achieved by not making it available to the backend right away for reuse, but holding it a little while. Historically, what has usually been the target of such attacks was objects, where vtable pointers or other function pointers could constitute a valuable targeti to replace. Those are usually on the smaller side. There is barely any advantage in putting the quarantine several megabytes of RGB data or the like. Now for the patch. This patch introduces a new way the Quarantine behaves in Scudo. First of all, the size of the Quarantine will be defined in KB instead of MB, then we introduce a new option: the size up to which (lower than or equal to) a chunk will be quarantined. This way, we only quarantine smaller chunks, and the size of the quarantine remains manageable. It also prevents someone from triggering a recycle by allocating something huge. We default to 512 bytes on 32-bit and 2048 bytes on 64-bit platforms. In details, the patches includes the following: - introduce `QuarantineSizeKb`, but honor `QuarantineSizeMb` if set to fall back to the old behavior (meaning no threshold in that case); `QuarantineSizeMb` is described as deprecated in the options descriptios; documentation update will follow; - introduce `QuarantineChunksUpToSize`, the new threshold value; - update the `quarantine.cpp` test, and other tests using `QuarantineSizeMb`; - remove `AllocatorOptions::copyTo`, it wasn't used; - slightly change the logic around `quarantineOrDeallocateChunk` to accomodate for the new logic; rename a couple of variables there as well; Rewriting the tests, I found a somewhat annoying bug where non-default aligned chunks would account for more than needed when placed in the quarantine due to `<< MinAlignment` instead of `<< MinAlignmentLog`. This is fixed and tested for now. Reviewers: alekseyshl, kcc Reviewed By: alekseyshl Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D35694 llvm-svn: 308884
* [Sanitizers] Scudo allocator set errno on failure.Alex Shlyapnikov2017-07-141-4/+4
| | | | | | | | | | | | | | | Summary: Set proper errno code on alloction failure and change pvalloc and posix_memalign implementation to satisfy their man-specified requirements. Reviewers: cryptoad Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D35429 llvm-svn: 308053
* [scudo] Change aligned alloc functions to be more compliant & perf changesKostya Kortchinsky2017-06-291-5/+19
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: We were not following the `man` documented behaviors for invalid arguments to `memalign` and associated functions. Using `CHECK` for those was a bit extreme, so we relax the behavior to return null pointers as expected when this happens. Adapt the associated test. I am using this change also to change a few more minor performance improvements: - mark as `UNLIKELY` a bunch of unlikely conditions; - the current `CHECK` in `__sanitizer::RoundUpTo` is redundant for us in *all* calls. So I am introducing our own version without said `CHECK`. - change our combined allocator `GetActuallyAllocatedSize`. We already know if the pointer is from the Primary or Secondary, so the `PointerIsMine` check is redundant as well, and costly for the 32-bit Primary. So we get the size by directly using the available Primary functions. Finally, change a `int` to `uptr` to avoid a warning/error when compiling on Android. Reviewers: alekseyshl Reviewed By: alekseyshl Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D34782 llvm-svn: 306698
* [Sanitizers] Operator new() interceptors always die on allocation errorAlex Shlyapnikov2017-06-281-13/+32
| | | | | | | | | | | | | | | | | | | | | Summary: Operator new interceptors behavior is now controlled by their nothrow property as well as by allocator_may_return_null flag value: - allocator_may_return_null=* + new() - die on allocation error - allocator_may_return_null=0 + new(nothrow) - die on allocation error - allocator_may_return_null=1 + new(nothrow) - return null Ideally new() should throw std::bad_alloc exception, but that is not trivial to achieve, hence TODO. Reviewers: eugenis Subscribers: kubamracek, llvm-commits Differential Revision: https://reviews.llvm.org/D34731 llvm-svn: 306604
* [scudo] Enabling MIPS support for ScudoSagar Thakur2017-06-191-1/+1
| | | | | | | | | Adding MIPS 32-bit and 64-bit support for Scudo. Reviewed by cryptoad, sdardis. Differential: D31803 llvm-svn: 305682
* Revert [scudo] Enabling MIPS support for ScudoSagar Thakur2017-04-241-1/+1
| | | | | | This patch broke the buildbot clang-cmake-mips. Investigating the issue. llvm-svn: 301173
* [scudo] Enabling MIPS support for ScudoSagar Thakur2017-04-241-1/+1
| | | | | | | | | Adding MIPS 32-bit and 64-bit support for Scudo. Reviewed by cryptoad Differential: D31803 llvm-svn: 301158
* [scudo] Add test exercising pthreadsKostya Kortchinsky2017-03-231-0/+65
| | | | | | | | | | | | | | Summary: Scudo didn't have any test using multiple threads. Add one, borrowed from lsan. Reviewers: kcc, alekseyshl Reviewed By: alekseyshl Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D31297 llvm-svn: 298636
OpenPOWER on IntegriCloud