summaryrefslogtreecommitdiffstats
path: root/compiler-rt/lib/scudo
Commit message (Collapse)AuthorAgeFilesLines
...
* [scudo] Add a minimal runtime for -fsanitize-minimal-runtime compatibilityKostya Kortchinsky2018-06-211-8/+34
| | | | | | | | | | | | | | | | | | | Summary: This patch follows D48373. The point is to be able to use Scudo with `-fsanitize-minimal-runtime`. For that we need a runtime that doesn't embed the UBSan one. This results in binaries that can be compiled with `-fsanitize=scudo,integer -fsanitize-minimal-runtime`. Reviewers: eugenis Reviewed By: eugenis Subscribers: mgorny, delcypher, llvm-commits, #sanitizers Differential Revision: https://reviews.llvm.org/D48377 llvm-svn: 335296
* [scudo] Move noinline functions definitions out of lineKostya Kortchinsky2018-06-191-63/+67
| | | | | | | | | | | | | | | | Summary: Mark `isRssLimitExceeded` as `NOINLINE`, and move it's definition as well as the one of `performSanityChecks` out of the class definition, as requested. Reviewers: filcab, alekseyshl Reviewed By: alekseyshl Subscribers: delcypher, #sanitizers, llvm-commits Differential Revision: https://reviews.llvm.org/D48228 llvm-svn: 335054
* [scudo] Add verbose failures in place of CHECK(0)Kostya Kortchinsky2018-06-156-25/+158
| | | | | | | | | | | | | | | | | | | | | | | | Summary: The current `FailureHandler` mechanism was fairly opaque with regard to the failure reason due to using `CHECK(0)`. Scudo is a bit different from the other Sanitizers as it prefers to avoid spurious processing in its failure path. So we just `dieWithMessage` using a somewhat explicit string. Adapted the tests for the new strings. While this takes care of the `OnBadRequest` & `OnOOM` failures, the next step is probably to migrate the other Scudo failures in the same failes (header corruption, invalid state and so on). Reviewers: alekseyshl Reviewed By: alekseyshl Subscribers: filcab, mgorny, delcypher, #sanitizers, llvm-commits Differential Revision: https://reviews.llvm.org/D48199 llvm-svn: 334843
* [scudo] Make Secondary linker-initialized compliantKostya Kortchinsky2018-06-141-6/+2
| | | | | | | | | | | | | | | | Summary: As a follow up to D48142 for Scudo, switch the `SpinMutex` to its static counterpart, and ensure zero-initialization by memset'ing the whole class. Reviewers: alekseyshl Reviewed By: alekseyshl Subscribers: delcypher, #sanitizers, llvm-commits Differential Revision: https://reviews.llvm.org/D48148 llvm-svn: 334716
* [scudo] Add C++17 aligned new/delete operators supportKostya Kortchinsky2018-06-124-62/+87
| | | | | | | | | | | | | | | | | | | Summary: This CL adds support for aligned new/delete operators (C++17). Currently we do not support alignment inconsistency detection on deallocation, as this requires a header change, but the APIs are introduced and are functional. Add a smoke test for the aligned version of the operators. Reviewers: alekseyshl Reviewed By: alekseyshl Subscribers: delcypher, #sanitizers, llvm-commits Differential Revision: https://reviews.llvm.org/D48031 llvm-svn: 334505
* [scudo] Improve the scalability of the shared TSD modelKostya Kortchinsky2018-06-116-51/+83
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: The shared TSD model in its current form doesn't scale. Here is an example of rpc2-benchmark (with default parameters, which is threading heavy) on a 72-core machines (defaulting to a `CompactSizeClassMap` and no Quarantine): - with tcmalloc: 337K reqs/sec, peak RSS of 338MB; - with scudo (exclusive): 321K reqs/sec, peak RSS of 637MB; - with scudo (shared): 241K reqs/sec, peak RSS of 324MB. This isn't great, since the exclusive model uses a lot of memory, while the shared model doesn't even come close to be competitive. This is mostly due to the fact that we are consistently scanning the TSD pool starting at index 0 for an available TSD, which can result in a lot of failed lock attempts, and touching some memory that needs not be touched. This CL attempts to make things better in most situations: - first, use a thread local variable on Linux (intead of pthread APIs) to store the current TSD in the shared model; - move the locking boolean out of the TSD: this allows the compiler to use a register and potentially optimize out a branch instead of reading it from the TSD everytime (we also save a tiny bit of memory per TSD); - 64-bit atomic operations on 32-bit ARM platforms happen to be expensive: so store the `Precedence` in a `uptr` instead of a `u64`. We lose some nanoseconds of precision and we'll wrap around at some point, but the benefit is worth it; - change a `CHECK` to a `DCHECK`: this should never happen, but if something is ever terribly wrong, we'll crash on a near null AV if the TSD happens to be null; - based on an idea by dvyukov@, we are implementing a bound random scan for an available TSD. This requires computing the coprimes for the number of TSDs, and attempting to lock up to 4 TSDs in an random order before falling back to the current one. This is obviously slightly more expansive when we have just 2 TSDs (barely noticeable) but is otherwise beneficial. The `Precedence` still basically corresponds to the moment of the first contention on a TSD. To seed on random choice, we use the precedence of the current TSD since it is very likely to be non-zero (since we are in the slow path after a failed `tryLock`) With those modifications, the benchmark yields to: - with scudo (shared): 330K reqs/sec, peak RSS of 327MB. So the shared model for this specific situation not only becomes competitive but outperforms the exclusive model. I experimented with some values greater than 4 for the number of TSDs to attempt to lock and it yielded a decrease in QPS. Just sticking with the current TSD is also a tad slower. Numbers on platforms with less cores (eg: Android) remain similar. Reviewers: alekseyshl, dvyukov, javed.absar Reviewed By: alekseyshl, dvyukov Subscribers: srhines, kristof.beyls, delcypher, llvm-commits, #sanitizers Differential Revision: https://reviews.llvm.org/D47289 llvm-svn: 334410
* [scudo] Quarantine optimizationKostya Kortchinsky2018-05-161-1/+2
| | | | | | | | | | | | | | | | | | | | Summary: It turns out that the previous code construct was not optimizing the allocation and deallocation of batches. The class id was read as a class member (even though a precomputed one) and nothing else was optimized. By changing the construct this way, the compiler actually optimizes most of the allocation and deallocation away to only work with a single class id, which not only saves some CPU but also some code footprint. Reviewers: alekseyshl, dvyukov Reviewed By: dvyukov Subscribers: dvyukov, delcypher, llvm-commits, #sanitizers Differential Revision: https://reviews.llvm.org/D46961 llvm-svn: 332502
* [scudo] Adding an interface function to print allocator statsKostya Kortchinsky2018-04-253-0/+17
| | | | | | | | | | | | | | | | Summary: This adds `__scudo_print_stats` as an interface function to display the Primary and Secondary allocator statistics for Scudo. Reviewers: alekseyshl, flowerhack Reviewed By: alekseyshl Subscribers: delcypher, llvm-commits, #sanitizers Differential Revision: https://reviews.llvm.org/D46016 llvm-svn: 330857
* [scudo] Read ARM feature bits using Fuchsia APIs.Kostya Kortchinsky2018-04-231-1/+12
| | | | | | | | | | | | | | | | | Summary: Fuchsia uses zx_system_get_features in lieu of getauxval. Use this call when checking for CRC32 support. Reviewers: cryptoad Reviewed By: cryptoad Subscribers: delcypher, llvm-commits, #sanitizers, kristof.beyls, chrib Differential Revision: https://reviews.llvm.org/D45896 llvm-svn: 330598
* [sanitizer] Split Symbolizer/StackTraces from core RTSanitizerCommonKostya Kortchinsky2018-04-161-3/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: Host symbolizer & stacktraces related code in their own RT: `RTSanitizerCommonSymbolizer`, which is "libcdep" by nature. Symbolizer & stacktraces specific code that used to live in common files is moved to a new file `sanitizer_symbolizer_report.cc` as is. The purpose of this is the enforce a separation between code that relies on symbolization and code that doesn't. This saves the inclusion of spurious code due to the interface functions with default visibility, and the extra data associated. The following sanitizers makefiles were modified & tested locally: - dfsan: doesn't require the new symbolizer RT - esan: requires it - hwasan: requires it - lsan: requires it - msan: requires it - safestack: doesn't require it - xray: doesn't require it - tsan: requires it - ubsan: requires it - ubsan_minimal: doesn't require it - scudo: requires it (but not for Fuchsia that has a minimal runtime) This was tested locally on Linux, Android, Fuchsia. Reviewers: alekseyshl, eugenis, dberris, kubamracek, vitalybuka, dvyukov, mcgrathr Reviewed By: alekseyshl, vitalybuka Subscribers: srhines, kubamracek, mgorny, krytarowski, delcypher, llvm-commits, #sanitizers Differential Revision: https://reviews.llvm.org/D45457 llvm-svn: 330131
* [sanitizer] Allow for the allocator "names" to be set by the toolsKostya Kortchinsky2018-04-132-1/+4
| | | | | | | | | | | | | | | | | | | Summary: In the same spirit of SanitizerToolName, allow the Primary & Secondary allocators to have names that can be set by the tools via PrimaryAllocatorName and SecondaryAllocatorName. Additionally, set a non-default name for Scudo. Reviewers: alekseyshl, vitalybuka Reviewed By: alekseyshl, vitalybuka Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits Differential Revision: https://reviews.llvm.org/D45600 llvm-svn: 330055
* [scudo] Remove duplicate cmake flagsKostya Kortchinsky2018-04-121-1/+1
| | | | | | | | | | | | | | | | Summary: Now that common options are propagated again for runtimes build with D45507, the -f{data,function}-sections flags are now duplicates, remove them. Reviewers: alekseyshl Reviewed By: alekseyshl Subscribers: mgorny, delcypher, #sanitizers, llvm-commits Differential Revision: https://reviews.llvm.org/D45575 llvm-svn: 329925
* [scudo] Fuchsia minimal shared runtimeKostya Kortchinsky2018-03-271-21/+24
| | | | | | | | | | | | | | | | | | | Summary: Fuchsia requires its Scudo shared runtime to not be C++ dependant. Since they don't use UBSan in conjunction with Scudo, we can just remove the runtime, and add the extra `nostdinc++` and `nostdlib++` flags. No need for Coverage either. This allows to keep things going while working on additional splits of sanitizer_commong and a more minimal runtime. Reviewers: phosek, flowerhack, alekseyshl Reviewed By: phosek, alekseyshl Subscribers: mgorny, delcypher, #sanitizers, llvm-commits Differential Revision: https://reviews.llvm.org/D44791 llvm-svn: 328625
* [sanitizer] Split coverage into separate RT in sanitizer_commonKostya Kortchinsky2018-03-221-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: `sanitizer_common`'s coverage support is fairly well separated, and libcdep by default. Several sanitizers don't make use of coverage, and as far as I can tell do no benefit from the extra dependencies pulled in by the coverage public interface functions. The following sanitizers call `InitializeCoverage` explicitely: MSan, ASan, LSan, HWAsan, UBSan. On top of this, any sanitizer bundling RTUBSan should add the coverage RT as well: ASan, Scudo, UBSan, CFI (diag), TSan, MSan, HWAsan. So in the end the following have no need: DFSan, ESan, CFI, SafeStack (nolibc anyway), XRay, and the upcoming Scudo minimal runtime. I tested this with all the sanitizers check-* with gcc & clang, and in standalone on Linux & Android, and there was no issue. I couldn't test this on Mac, Fuchsia, BSDs, & Windows for lack of an environment, so adding a bunch of people for additional scrunity. I couldn't test HWAsan either. Reviewers: eugenis, vitalybuka, alekseyshl, flowerhack, kubamracek, dberris, rnk, krytarowski Reviewed By: vitalybuka, alekseyshl, flowerhack, dberris Subscribers: mgorny, delcypher, #sanitizers, llvm-commits Differential Revision: https://reviews.llvm.org/D44701 llvm-svn: 328204
* [scudo] Add Chunk::getSize, rework Chunk::getUsableSizeKostya Kortchinsky2018-03-142-25/+28
| | | | | | | | | | | | | | | | | | | | | | | | Summary: Using `getActuallyAllocatedSize` from the Combined resulting in mediocre compiled code, as the `ClassId != 0` predicament was not propagated there, resulting in additional branches and dead code. Move the logic in the frontend, which results in better compiled code. Also I think it makes it slightly easier to distinguish between the size the user requested, and the size that was actually allocated by the allocator. `const` a couple of things as well. This has no functional impact. Reviewers: alekseyshl Reviewed By: alekseyshl Subscribers: delcypher, #sanitizers, llvm-commits Differential Revision: https://reviews.llvm.org/D44444 llvm-svn: 327525
* [scudo] Secondary allocator overhaul to support WindowsKostya Kortchinsky2018-03-122-56/+113
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: The need for this change stems from the fact that Windows doesn't support partial unmapping (`MEM_RELEASE` implies the entire allocated region). So we now have to keep track of the reserved region and the committed region, so that we can function without the trimming we did when dealing with larger alignments. Instead of just having a `ReservedAddressRange` per chunk, we introduce a `LargeChunkHeader` (and `LargeChunk` namespace) that additionally holds the committed size and the usable size. The former is needed for stats purposes, the latter is used by the frontend. Requiring both is debatable, we could only work with the usable size but then be off by up to a page per chunk when dealing with stats. Additionally, we introduce more stats since they turned out to be useful for experiments, and a `PrintStats` function that will be used by the combined allocator in later patch. Reviewers: alekseyshl, flowerhack Reviewed By: alekseyshl Subscribers: delcypher, #sanitizers, llvm-commits Differential Revision: https://reviews.llvm.org/D43949 llvm-svn: 327321
* [scudo] Make logging more consistentKostya Kortchinsky2018-03-073-63/+43
| | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: A few changes related to logging: - prepend `Scudo` to the error messages so that users can identify that we reported an error; - replace a couple of `Report` calls in the RSS check code with `dieWithMessage`/`Print`, mark a condition as `UNLIKELY` in the process; - change some messages so that they all look more or less the same. This includes the `CHECK` message; - adapt a couple of tests with the new strings. A couple of side notes: this results in a few 1-line-blocks, for which I left brackets. There doesn't seem to be any style guide for that, I can remove them if need be. I didn't use `SanitizerToolName` in the strings, but directly `Scudo` because we are the only users, I could change that too. Reviewers: alekseyshl, flowerhack Reviewed By: alekseyshl Subscribers: mgorny, delcypher, llvm-commits, #sanitizers Differential Revision: https://reviews.llvm.org/D44171 llvm-svn: 326901
* [scudo] Use gc-sections by defaultKostya Kortchinsky2018-03-061-0/+4
| | | | | | | | | | | | | | | | | | | | | | | Summary: If not using `-Wl,--gc-sections`, a whole lot of unused `sanitizer_common` code and related static variables are pulled into the shared library. Keep the binary size smaller, and its memory footprint smaller as well, by using the compiler flags `-ffunction-section` & `-fdata-sections` by default, as well as the linker flags `-Wl,--gc-sections`. Current experiments show a large discrepency between binary sizes generated by gcc (big) and clang (small). I am not sure yet how I can make a test that would encompass both, so it's an outstanding work item. Reviewers: alekseyshl, flowerhack Reviewed By: alekseyshl Subscribers: mgorny, delcypher, llvm-commits, #sanitizers Differential Revision: https://reviews.llvm.org/D44121 llvm-svn: 326833
* [scudo] Introduce Chunk::getHeaderSizeKostya Kortchinsky2018-02-273-25/+27
| | | | | | | | | | | | | | | | | | | | | Summary: Instead of using `AlignedChunkHeaderSize`, introduce a `constexpr` function `getHeaderSize` in the `Chunk` namespace. Switch `RoundUpTo` to a `constexpr` as well (so we can use it in `constexpr` declarations). Mark a few variables in the areas touched as `const`. Overall this has no functional change, and is mostly to make things a bit more consistent. Reviewers: alekseyshl Reviewed By: alekseyshl Subscribers: delcypher, #sanitizers, llvm-commits Differential Revision: https://reviews.llvm.org/D43772 llvm-svn: 326206
* [scudo] Allow options to be defined at compile timeKostya Kortchinsky2018-02-081-0/+11
| | | | | | | | | | | | | | | | Summary: Allow for options to be defined at compile time, like is already the case for other sanitizers, via `SCUDO_DEFAULT_OPTIONS`. Reviewers: alekseyshl, dberris Reviewed By: alekseyshl, dberris Subscribers: kubamracek, delcypher, llvm-commits, #sanitizers Differential Revision: https://reviews.llvm.org/D42980 llvm-svn: 324620
* [scudo] Minor Secondary changesKostya Kortchinsky2018-02-011-8/+8
| | | | | | | | | | | | | | | | | | | Summary: Few changes to the secondary: - mark `const` variables as such; - change some `CHECK` to `DCHECK`: I don't feel we need to be as conservative as we were with out checks, as they are the results of our own computation. - mark a condition as `UNLIKELY`. Reviewers: alekseyshl Reviewed By: alekseyshl Subscribers: delcypher, #sanitizers, llvm-commits Differential Revision: https://reviews.llvm.org/D42696 llvm-svn: 323997
* [scudo] Add default implementations for weak functionsKostya Kortchinsky2018-01-303-2/+22
| | | | | | | | | | | | | | | | Summary: This is in preparation for platforms where `SANITIZER_SUPPORTS_WEAK_HOOKS` is 0. They require a default implementation. Reviewers: alekseyshl Reviewed By: alekseyshl Subscribers: delcypher, llvm-commits, #sanitizers Differential Revision: https://reviews.llvm.org/D42557 llvm-svn: 323795
* [scudo] Overhaul malloc related interceptorsKostya Kortchinsky2018-01-253-72/+85
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: This is a follow-up to D42506. There are a few of things that bothered me about `scudo_interceptors.cpp`: - the filename is a misnomer: it intercepts some functions, but the rest (C++) is actually in `scudo_new_delete.cpp`. I feel like `scudo_malloc.cpp` is more appropriate (ASan uses the same naming scheme); - we do not need "full" interceptors, since we are never accessing the unsanitized version of the functions, we just need the `extern "C" INTERCEPTOR_ATTRIBUTE` part of it to just call our functions; - a couple of functions where duplicated while they could just be `ALIAS`'d; - use the `SANITIZER_INTERCEPT_*` defines to hide the unneeded interceptors; - use `SIZE_T` instead of `uptr`: while it's the same behind the curtain, the former is meant for this use case. In the end there is no functional change on the currently supported platforms (Linux, Android). Reviewers: alekseyshl Reviewed By: alekseyshl Subscribers: mgorny, hintonda, delcypher, #sanitizers, llvm-commits Differential Revision: https://reviews.llvm.org/D42546 llvm-svn: 323464
* [scudo] Remove SANITIZER_LINUX requirement for the malloc interceptorsKostya Kortchinsky2018-01-241-6/+1
| | | | | | | | | | | | | | | | | | | | Summary: Currently all platforms are using the `scudo_interceptors.cpp` interceptors. We might to come up with platform specific interceptors when/if we get Apple & Windows, but as of now, that allows for Fuchsia to use them. `scudo_new_delete.cpp` didn't have the `#if SANITIZER_LINUX` so it's good to go. Reviewers: alekseyshl, flowerhack Reviewed By: flowerhack Subscribers: delcypher, #sanitizers, llvm-commits Differential Revision: https://reviews.llvm.org/D42506 llvm-svn: 323386
* [scudo] Allow for weak hooks, gated by a defineKostya Kortchinsky2018-01-232-2/+10
| | | | | | | | | | | | | | | | | | | | | | Summary: Hooks in the allocation & deallocation paths can be a security risk (see for an example https://scarybeastsecurity.blogspot.com/2016/11/0day-exploit-advancing-exploitation.html which used the glibc's __free_hook to complete exploitation). But some users have expressed a need for them, even if only for tests and memory benchmarks. So allow for `__sanitizer_malloc_hook` & `__sanitizer_free_hook` to be called if defined, and gate them behind a global define `SCUDO_CAN_USE_HOOKS` defaulting to 0. Reviewers: alekseyshl Reviewed By: alekseyshl Subscribers: #sanitizers, llvm-commits Differential Revision: https://reviews.llvm.org/D42430 llvm-svn: 323278
* [scudo] Pass SANITIZER_COMMON_LINK_FLAGS to the shared library LINK_FLAGSKostya Kortchinsky2018-01-191-0/+3
| | | | | | | | | | | | | | | | | | Summary: We somehow never did it, and it raised no issue until now, when trying to enable Fuchsia as a supported Scudo platform in the cmake config. So propagate SANITIZER_COMMON_LINK_FLAGS for now. Reviewers: alekseyshl, flowerhack Reviewed By: flowerhack Subscribers: mgorny, #sanitizers, llvm-commits Differential Revision: https://reviews.llvm.org/D42314 llvm-svn: 322999
* [scudo] Fix for the Scudo interface function scopeKostya Kortchinsky2018-01-171-2/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: A forgotten include in `scudo_allocator.cpp` made the symbol only local :/ Before: ``` nm ./lib/clang/7.0.0/lib/linux/libclang_rt.scudo-i686-android.so | grep rss 00024730 t __scudo_set_rss_limit ``` After: ``` nm ./lib/clang/7.0.0/lib/linux/libclang_rt.scudo-i686-android.so | grep rs 00024760 T __scudo_set_rss_limit ``` And we want `T`! This include also means that we can get rid of the `extern "C"` in the C++ file, the compiler does fine without it (note that this was already the case for all the `__sanitizer_*` interface functions. Reviewers: alekseyshl, eugenis Reviewed By: eugenis Subscribers: #sanitizers, llvm-commits Differential Revision: https://reviews.llvm.org/D42199 llvm-svn: 322782
* [scudo] Limit by default the TSD pool to 2 on AndroidKostya Kortchinsky2018-01-171-1/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: jemalloc on Android currently uses 2 arenas (https://android.googlesource.com/platform/external/jemalloc/+/master/Android.bp#64). Since the Android toolchain absorbs compiler-rt and compiles it as is, we have to enforce the same limit to somehow stay competitive in terms of memory usage. The changes could either go in: - `scudo_platform.h` with a default for Android of 2 (this is the solution implemented here); - in `CMakeLists.txt` adding -DSCUDO_SHARED_TSD_POOL_SIZE=2 for Android. - something else? I don't have a strong opinion on how to do it, but it has to be done upstream anyway. Reviewers: alekseyshl, eugenis Reviewed By: alekseyshl, eugenis Subscribers: srhines, #sanitizers, llvm-commits Differential Revision: https://reviews.llvm.org/D42194 llvm-svn: 322764
* [scudo] Add SANITIZER_CXX_ABI_LIBRARY to SCUDO_DYNAMIC_LIBSKostya Kortchinsky2018-01-121-1/+2
| | | | | | | | | | | | | | | | | | Summary: This is needed for the shared runtime since we are pulling RTUbsan in. Otherwise some builds might fail with errors such as: `error: undefined reference to '__dynamic_cast'` Reviewers: alekseyshl, srhines Reviewed By: srhines Subscribers: kongyi, pirama, chh, mgorny, llvm-commits, #sanitizers Differential Revision: https://reviews.llvm.org/D41995 llvm-svn: 322389
* [scudo] s/unsigned long/size_t/ for __scudo_set_rss_limitKostya Kortchinsky2018-01-042-4/+9
| | | | | | | | | | | | | | | | | | | | Summary: `__scudo_set_rss_limit`'s `LimitMb` should really be a `size_t`. Update accordingly the prototype. To avoid the `NOLINT` and conform with the other Sanitizers, use the sanitizers types for the internal definition. This should have no functional change. Additionally, capitalize a variable name to follow the LLVM coding standards. Reviewers: alekseyshl, flowerhack Reviewed By: alekseyshl Subscribers: #sanitizers, llvm-commits Differential Revision: https://reviews.llvm.org/D41704 llvm-svn: 321803
* [scudo] Refactor ScudoChunkKostya Kortchinsky2017-12-141-119/+117
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: The initial implementation used an ASan like Chunk class that was deriving from a Header class. Due to potential races, we ended up working with local copies of the Header and never using the parent class fields. ScudoChunk was never constructed but cast, and we were using `this` as the pointer needed for our computations. This was meh. So we refactored ScudoChunk to be now a series of static functions within the namespace `__scudo::Chunk` that take a "user" pointer as first parameter (former `this`). A compiled binary doesn't really change, but the code is more sensible. Clang tends to inline all those small function (in -O2), but GCC left a few not inlined, so we add the `INLINE` keyword to all. Since we don't have `ScudoChunk` pointers anymore, a few variables were renamed here and there to introduce a clearer distinction between a user pointer (usually `Ptr`) and a backend pointer (`BackendPtr`). Reviewers: alekseyshl, flowerhack Reviewed By: alekseyshl Subscribers: #sanitizers, llvm-commits Differential Revision: https://reviews.llvm.org/D41200 llvm-svn: 320745
* [scudo] Adding a public Scudo interfaceKostya Kortchinsky2017-12-133-0/+46
| | | | | | | | | | | | | | | | Summary: The first and only function to start with allows to set the soft or hard RSS limit at runtime. Add associated tests. Reviewers: alekseyshl Reviewed By: alekseyshl Subscribers: mgorny, #sanitizers, llvm-commits Differential Revision: https://reviews.llvm.org/D41128 llvm-svn: 320611
* [sanitizer] Introduce a vDSO aware timing functionKostya Kortchinsky2017-12-132-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | Summary: See D40657 & D40679 for previous versions of this patch & description. A couple of things were fixed here to have it not break some bots. Weak symbols can't be used with `SANITIZER_GO` so the previous version was breakin TsanGo. I set up some additional local tests and those pass now. I changed the workaround for the glibc vDSO issue: `__progname` is initialized after the vDSO and is actually public and of known type, unlike `__vdso_clock_gettime`. This works better, and with all compilers. The rest is the same. Reviewers: alekseyshl Reviewed By: alekseyshl Subscribers: srhines, kubamracek, krytarowski, llvm-commits, #sanitizers Differential Revision: https://reviews.llvm.org/D41121 llvm-svn: 320594
* [scudo] Inline getScudoChunk function.Kostya Kortchinsky2017-12-131-1/+1
| | | | | | | | | | | | | | | Summary: getScudoChunk function is implicitly inlined for optimized builds on clang, but not on gcc. It's a small enough function that it seems sensible enough to just inline it by default. Reviewers: cryptoad, alekseyshl Reviewed By: cryptoad Differential Revision: https://reviews.llvm.org/D41138 llvm-svn: 320592
* [sanitizer] Revert rL320409Kostya Kortchinsky2017-12-112-3/+3
| | | | | | | | | | | | | | Summary: D40679 broke a couple of builds, reverting while investigating. Reviewers: alekseyshl Reviewed By: alekseyshl Subscribers: srhines, kubamracek, krytarowski, llvm-commits, #sanitizers Differential Revision: https://reviews.llvm.org/D41088 llvm-svn: 320417
* [sanitizer] Introduce a vDSO aware time function, and use it in the ↵Kostya Kortchinsky2017-12-112-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | allocator [redo] Summary: Redo of D40657, which had the initial discussion. The initial code had to move into a libcdep file, and things had to be shuffled accordingly. `NanoTime` is a time sink when checking whether or not to release memory to the OS. While reducing the amount of calls to said function is in the works, another solution that was found to be beneficial was to use a timing function that can leverage the vDSO. We hit a couple of snags along the way, like the fact that the glibc crashes when clock_gettime is called from a preinit_array, or the fact that `__vdso_clock_gettime` is mangled (for security purposes) and can't be used directly, and also that clock_gettime can be intercepted. The proposed solution takes care of all this as far as I can tell, and significantly improve performances and some Scudo load tests with memory reclaiming enabled. @mcgrathr: please feel free to follow up on https://reviews.llvm.org/D40657#940857 here. I posted a reply at https://reviews.llvm.org/D40657#940974. Reviewers: alekseyshl, krytarowski, flowerhack, mcgrathr, kubamracek Reviewed By: alekseyshl, krytarowski Subscribers: #sanitizers, mcgrathr, srhines, llvm-commits, kubamracek Differential Revision: https://reviews.llvm.org/D40679 llvm-svn: 320409
* [scudo] Minor code generation improvementKostya Kortchinsky2017-12-082-11/+9
| | | | | | | | | | | | | | | | | | | | | | | | | Summary: It looks like clang was generating somewhat weird assembly with the current code. `FromPrimary`, even though `const`, was replaced every time with the code generated for `size <= SizeClassMap::kMaxSize` instead of using a variable or register, and `FromPrimary` didn't induce `ClassId != 0` for the compiler, so a dead branch was generated for `getActuallyAllocatedSize(Ptr, ClassId)` since it's never called for `ClassId = 0` (Secondary backed allocations) [this one was more wishful thinking on my side than anything else]. I rearranged the code bit so that the generated assembly is less clunky. Also changed 2 whitespace inconsistencies that were bothering me. Reviewers: alekseyshl, flowerhack Reviewed By: flowerhack Subscribers: llvm-commits, #sanitizers Differential Revision: https://reviews.llvm.org/D40976 llvm-svn: 320160
* [scudo] Correct performance regression in SecondaryKostya Kortchinsky2017-12-061-8/+10
| | | | | | | | | | | | | | | | | | | | | | Summary: This wasn't noticed: `RoundUpTo` doesn't produce a constant expression, so the sizes were not constant either. Enforce them to be static const, replace `RoundUpTo` by its expression. The compiler can now optimize the associated computations accordingly. Also looking at the produced assembly, `PageSize` was fetched multiple times during `Allocate`, so keep a local value of it. As a result it's fetched once and kept in a register. Reviewers: alekseyshl, flowerhack Reviewed By: alekseyshl Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D40862 llvm-svn: 319903
* [scudo] Get rid of the thread local PRNG & header saltKostya Kortchinsky2017-12-055-115/+63
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: It was deemed that the salt in the chunk header didn't improve security significantly (and could actually decrease it). The initial idea was that the same chunk would different headers on different allocations, allowing for less predictability. The issue is that gathering the same chunk header with different salts can give information about the other "secrets" (cookie, pointer), and that if an attacker leaks a header, they can reuse it anyway for that same chunk anyway since we don't enforce the salt value. So we get rid of the salt in the header. This means we also get rid of the thread local Prng, and that we don't need a global Prng anymore as well. This makes everything faster. We reuse those 8 bits to store the `ClassId` of a chunk now (0 for a secondary based allocation). This way, we get some additional speed gains: - `ClassId` is computed outside of the locked block; - `getActuallyAllocatedSize` doesn't need the `GetSizeClass` call; - same for `deallocatePrimary`; We add a sanity check at init for this new field (all sanity checks are moved in their own function, `init` was getting crowded). Reviewers: alekseyshl, flowerhack Reviewed By: alekseyshl Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D40796 llvm-svn: 319791
* [scudo] Allow for compile-time choice of the SizeClassMapKostya Kortchinsky2017-11-291-11/+12
| | | | | | | | | | | | | | | | | | | | | | | | Summary: With this change, we allow someone to chose the `SizeClassMap` they want to use at compile time via a define. I feel somewhat unimaginative with the name of the defines, so if someone has a better idea, let me know. I have been alternating between those and `SCUDO_USE_xxx_SIZECLASSMAP` which is clearer but also longer. The issue with those is that it wouldn't be consistent with `SCUDO_TSD_EXCLUSIVE` that should probably become `SCUDO_USE_EXCLUSIVE_TSD` maybe? Anyway, naming is hard, and I am not sure what makes more sense! Reviewers: alekseyshl, flowerhack Reviewed By: alekseyshl Subscribers: llvm-commits, srhines Differential Revision: https://reviews.llvm.org/D40521 llvm-svn: 319350
* [scudo] Workaround for uninitialized Bionic globalsKostya Kortchinsky2017-11-271-1/+11
| | | | | | | | | | | | | | | | | | | Summary: Bionic doesn't initialize its globals early enough. This causes issues when trying to access them from a preinit_array (b/25751302) or from another constructor called before the libc one (b/68046352). __progname is initialized after the other globals, so we can check its value to know if calling `getauxval` is safe. Reviewers: alekseyshl Reviewed By: alekseyshl Subscribers: srhines, llvm-commits Differential Revision: https://reviews.llvm.org/D40504 llvm-svn: 319099
* [scudo] Overhaul hardware CRC32 feature detectionKostya Kortchinsky2017-11-223-101/+49
| | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: This patch aims at condensing the hardware CRC32 feature detection and making it slightly more effective on Android. The following changes are included: - remove the `CPUFeature` enum, and get rid of one level of nesting of functions: we only used CRC32, so we just implement and use `hasHardwareCRC32`; - allow for a weak `getauxval`: the Android toolchain is compiled at API level 14 for Android ARM, meaning no `getauxval` at compile time, yet we will run on API level 27+ devices. The `/proc/self/auxv` fallback can work but is worthless for a process like `init` where the proc filesystem doesn't exist yet. If a weak `getauxval` doesn't exist, then fallback. - couple of extra corrections. Reviewers: alekseyshl Reviewed By: alekseyshl Subscribers: kubamracek, aemerson, srhines, kristof.beyls, llvm-commits Differential Revision: https://reviews.llvm.org/D40322 llvm-svn: 318859
* [scudo] Make getNumberOfCPUs Fuchsia compliant v2Kostya Kortchinsky2017-11-211-8/+1
| | | | | | | | | | | | | | | | | | | Summary: This change allows Fuchsia to boot properly using the Scudo allocator. A first version of this commit was reverted by rL317834 because it broke Android builds for toolchains generated with older NDKs. This commit introduces a fall back to solve that issue. Reviewers: cryptoad, krytarowski, rnk, alekseyshl Reviewed By: cryptoad, krytarowski, alekseyshl Subscribers: llvm-commits, srhines, kubamracek, krytarowski Differential Revision: https://reviews.llvm.org/D40121 llvm-svn: 318802
* [scudo] Soft and hard RSS limit checksKostya Kortchinsky2017-11-151-0/+50
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: This implements an opportunistic check for the RSS limit. For ASan, this was implemented thanks to a background thread checking the current RSS vs the set limit every 100ms. This was deemed problematic for Scudo due to potential Android concerns (Zygote as pointed out by Aleksey) as well as the general inconvenience of having a permanent background thread. If a limit (soft or hard) is specified, we will attempt to update the RSS limit status (exceeded or not) every 100ms. This is done in an opportunistic way: if we can update it, we do it, if not we return the current status, mostly because we don't need it to be fully consistent (it's done every 100ms anyway). If the limit is exceeded `allocate` will act as if OOM for a soft limit, or just die for a hard limit. We use the `common_flags()`'s `hard_rss_limit_mb` & `soft_rss_limit_mb` for configuration of the limits. Reviewers: alekseyshl Reviewed By: alekseyshl Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D40038 llvm-svn: 318301
* [scudo] Simplify initialization and flagsKostya Kortchinsky2017-11-142-66/+24
| | | | | | | | | | | | | | | | | | | | | | | Summary: This is mostly some cleanup and shouldn't affect functionalities. Reviewing some code for a future addition, I realized that the complexity of the initialization path was unnecessary, and so was maintaining a structure for the allocator options throughout the initialization. So we get rid of that structure, of an extraneous level of nesting for the `init` function, and correct a couple of related code inaccuracies in the flags cpp. Reviewers: alekseyshl Reviewed By: alekseyshl Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D39974 llvm-svn: 318157
* [sanitizer] Update scudo to use new APIKostya Kortchinsky2017-11-131-32/+33
| | | | | | | | | | | | | | | | | | | Summary: The ScudoAllocator uses a SecondaryHeader to keep track of the size and base address of each mmap'd chunk. This aligns well with what the ReservedAddressRange is trying to do. This changeset converts the scudo allocator from using the MmapNoAccess/MmapFixed APIs to the ReservedAddressRange::Init and ::Map APIs. In doing so, it replaces the SecondayHeader struct with the ReservedAddressRange object. This is part 3 of a 4 part changeset; part 1 https://reviews.llvm.org/D39072 and part 2 https://reviews.llvm.org/D38592 Reviewers: alekseyshl, mcgrathr, cryptoad, phosek Reviewed By: cryptoad Subscribers: llvm-commits, cryptoad, kubamracek Differential Revision: https://reviews.llvm.org/D38593 llvm-svn: 318080
* [scudo] Bump the Android API level requirement to 21 for getauxvalKostya Kortchinsky2017-11-101-3/+3
| | | | | | | | | | | | | | | | | | | | Summary: `getauxval` was introduced in 18 & 21 depending on the architecture. Bump the requirement to 21. It also turns out that the NDK is finicky: NDK r13b doesn't include sys/auxv.h when creating a standalone toolchain at API level 19 for ARM. So 18 didn't work well with older NDKs. Reviewers: alekseyshl Reviewed By: alekseyshl Subscribers: aemerson, srhines, llvm-commits, kristof.beyls Differential Revision: https://reviews.llvm.org/D39905 llvm-svn: 317907
* [sanitizer] Revert rL317822Kostya Kortchinsky2017-11-091-1/+8
| | | | | | | | | | | | | | | | | | | Summary: This reverts D39490. For toolchains generated with older NDKs (<=r13b as far as we tested), `cpu_set_t` doesn't exist in `sched.h`. We have to figure out another way to get the number of CPUs without this. Reviewers: rnk Reviewed By: rnk Subscribers: kubamracek, llvm-commits, krytarowski Differential Revision: https://reviews.llvm.org/D39867 llvm-svn: 317834
* [scudo] Make getNumberOfCPUs Fuchsia compliantKostya Kortchinsky2017-11-091-8/+1
| | | | | | | | | | | | | | Summary: This change allows Fuchsia to boot properly using the Scudo allocator. Reviewers: cryptoad, alekseyshl, krytarowski Reviewed By: cryptoad, krytarowski Subscribers: rnk, krytarowski, kubamracek, llvm-commits Differential Revision: https://reviews.llvm.org/D39490 llvm-svn: 317822
* [sanitizer] Add Scudo to the sanitizer lint checks.Kostya Kortchinsky2017-11-083-6/+3
| | | | | | | | | | | | | | | | | | | | Summary: Scudo abides by the coding style enforced by the sanitizer_common linter, but as of right now, it's not linter-enforced. Add Scudo to the list of directories checked by check_lint.sh. Also: fixes some linter errors found after getting this running. Reviewers: cryptoad Reviewed By: cryptoad Subscribers: llvm-commits, kubamracek Differential Revision: https://reviews.llvm.org/D39757 llvm-svn: 317699
OpenPOWER on IntegriCloud