summaryrefslogtreecommitdiffstats
path: root/compiler-rt/lib/scudo
Commit message (Collapse)AuthorAgeFilesLines
...
* [scudo][standalone] Introduce the C & C++ wrappers [fixed]Kostya Kortchinsky2019-06-2713-37/+921
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: This is a redo of D63612. Two problems came up on some bots: - `__builtin_umull_overflow` was not declared. This is likely due to an older clang or gcc, so add a guard with `__has_builtin` and fallback to a division in the event the builtin doesn't exist; - contradicting definition for `malloc`, etc. This is AFAIU due to the fact that we ended up transitively including `stdlib.h` in the `.inc` due to it being the flags parser header: so move the include to the cc instead. This should fix the issues, but since those didn't come up in my local tests it's mostly guesswork. Rest is the same! Reviewers: morehouse, hctim, eugenis, vitalybuka, dyung, hans Reviewed By: morehouse, dyung, hans Subscribers: srhines, mgorny, delcypher, jfb, #sanitizers, llvm-commits Tags: #llvm, #sanitizers Differential Revision: https://reviews.llvm.org/D63831 llvm-svn: 364547
* Revert r364332 "[scudo][standalone] Introduce the C & C++ wrappers"Hans Wennborg2019-06-2611-909/+36
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Makes the build fail with e.g. llvm/projects/compiler-rt/lib/scudo/standalone/wrappers_c.inc:20:68: error: declaration of 'void* calloc(size_t, size_t)' has a different exception specifier INTERFACE WEAK void *SCUDO_PREFIX(calloc)(size_t nmemb, size_t size) { ^ See llvm-commits thread. > Summary: > This CL adds C & C++ wrappers and associated tests. Those use default > configurations for a Scudo combined allocator that will likely be > tweaked in the future. > > This is the final CL required to have a functional C & C++ allocator > based on Scudo. > > The structure I have chosen is to define the core C allocation > primitives in an `.inc` file that can be customized through defines. > This allows to easily have 2 (or more) sets of wrappers backed by > different combined allocators, as demonstrated by the `Bionic` > wrappers: one set for the "default" allocator, one set for the "svelte" > allocator. > > Currently all the tests added have been gtests, but I am planning to > add some more lit tests as well. > > Reviewers: morehouse, eugenis, vitalybuka, hctim, rengolin > > Reviewed By: morehouse > > Subscribers: srhines, mgorny, delcypher, jfb, #sanitizers, llvm-commits > > Tags: #llvm, #sanitizers > > Differential Revision: https://reviews.llvm.org/D63612 llvm-svn: 364400
* [scudo] Correct a behavior on the shared TSD registryKostya Kortchinsky2019-06-252-6/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: There is an error in the shared TSD registry logic when looking for a TSD in the slow path. There is an unlikely event when a TSD's precedence was 0 after attempting a `tryLock` which indicated that it was grabbed by another thread in between. We dealt with that case by continuing to the next iteration, but that meant that the `Index` was not increased and we ended up trying to lock the same TSD. This would manifest in heavy contention, and in the end we would still lock a TSD, but that was a wasted iteration. So, do not `continue`, just skip the TSD as a potential candidate. This is in both the standalone & non-standalone versions. Reviewers: morehouse, eugenis, vitalybuka, hctim Reviewed By: morehouse Subscribers: delcypher, #sanitizers, llvm-commits Tags: #llvm, #sanitizers Differential Revision: https://reviews.llvm.org/D63783 llvm-svn: 364345
* [scudo][standalone] Introduce the C & C++ wrappersKostya Kortchinsky2019-06-2511-36/+909
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: This CL adds C & C++ wrappers and associated tests. Those use default configurations for a Scudo combined allocator that will likely be tweaked in the future. This is the final CL required to have a functional C & C++ allocator based on Scudo. The structure I have chosen is to define the core C allocation primitives in an `.inc` file that can be customized through defines. This allows to easily have 2 (or more) sets of wrappers backed by different combined allocators, as demonstrated by the `Bionic` wrappers: one set for the "default" allocator, one set for the "svelte" allocator. Currently all the tests added have been gtests, but I am planning to add some more lit tests as well. Reviewers: morehouse, eugenis, vitalybuka, hctim, rengolin Reviewed By: morehouse Subscribers: srhines, mgorny, delcypher, jfb, #sanitizers, llvm-commits Tags: #llvm, #sanitizers Differential Revision: https://reviews.llvm.org/D63612 llvm-svn: 364332
* [scudo][standalone] Fuchsia related changesKostya Kortchinsky2019-06-183-38/+37
| | | | | | | | | | | | | | | | | | | | | | Summary: Fuchsia wants to use mutexes with PI in the Scudo code, as opposed to our own implementation. This required making `lock` & `unlock` platform specific (as opposed to `wait` & `wake`) [code courtesy of John Grossman]. There is an additional flag required now for mappings as well: `ZX_VM_ALLOW_FAULTS`. Reviewers: morehouse, mcgrathr, eugenis, vitalybuka, hctim Reviewed By: morehouse Subscribers: delcypher, jfb, #sanitizers, llvm-commits Tags: #llvm, #sanitizers Differential Revision: https://reviews.llvm.org/D63435 llvm-svn: 363705
* [GWP-ASan] Integration with Scudo [5].Mitch Phillips2019-06-172-0/+56
| | | | | | | | | | | | | | | | | | | | | | Summary: See D60593 for further information. This patch adds GWP-ASan support to the Scudo hardened allocator. It also implements end-to-end integration tests using Scudo as the backing allocator. The tests include crash handling for buffer over/underflow as well as use-after-free detection. Reviewers: vlad.tsyrklevich, cryptoad Reviewed By: vlad.tsyrklevich, cryptoad Subscribers: kubamracek, mgorny, #sanitizers, llvm-commits, morehouse Tags: #sanitizers, #llvm Differential Revision: https://reviews.llvm.org/D62929 llvm-svn: 363584
* [scudo][standalone] Introduce the combined allocatorKostya Kortchinsky2019-06-175-0/+870
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: The Combined allocator hold together all the other components, and provides a memory allocator interface based on various template parameters. This will be in turn used by "wrappers" that will provide the standard C and C++ memory allocation functions, but can be used as is as well. This doesn't depart significantly from the current Scudo implementation except for a few details: - Quarantine batches are now protected by a header a well; - an Allocator instance has its own TSD registry, as opposed to a static one for everybody; - a function to iterate over busy chunks has been added, for Android purposes; This also adds the associated tests, and a few default configurations for several platforms, that will likely be further tuned later on. Reviewers: morehouse, hctim, eugenis, vitalybuka Reviewed By: morehouse Subscribers: srhines, mgorny, delcypher, jfb, #sanitizers, llvm-commits Tags: #llvm, #sanitizers Differential Revision: https://reviews.llvm.org/D63231 llvm-svn: 363569
* Improve error message when '=' is missing in {ASAN,...}_OPTIONS.Vitaly Buka2019-06-151-1/+1
| | | | | | | | | | | | | | | | | | Summary: It's handling isses as described here: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=89832 Patch by Martin Liška. Reviewers: kcc, vitalybuka Reviewed By: vitalybuka Subscribers: cryptoad, kubamracek Differential Revision: https://reviews.llvm.org/D59876 llvm-svn: 363480
* [scudo][standalone] Unmap memory in testsKostya Kortchinsky2019-06-118-16/+74
| | | | | | | | | | | | | | | | | | | | | Summary: The more tests are added, the more we are limited by the size of the address space on 32-bit. Implement `unmapTestOnly` all around (like it is in sanitzer_common) to be able to free up some memory. This is not intended to be a proper "destructor" for an allocator, but allows us to not fail due to having no memory left. Reviewers: morehouse, vitalybuka, eugenis, hctim Reviewed By: morehouse Subscribers: delcypher, jfb, #sanitizers, llvm-commits Tags: #llvm, #sanitizers Differential Revision: https://reviews.llvm.org/D63146 llvm-svn: 363095
* [scudo][standalone] Introduce the thread specific data structuresKostya Kortchinsky2019-06-1010-11/+512
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: This CL adds the structures dealing with thread specific data for the allocator. This includes the thread specific data structure itself and two registries for said structures: an exclusive one, where each thread will have its own TSD struct, and a shared one, where a pool of TSD structs will be shared by all threads, with dynamic reassignment at runtime based on contention. This departs from the current Scudo implementation: we intend to make the Registry a template parameter of the allocator (as opposed to a single global entity), allowing various allocators to coexist with different TSD registry models. As a result, TSD registry and Allocator are tightly coupled. This also corrects a couple of things in other files that I noticed while adding this. Reviewers: eugenis, vitalybuka, morehouse, hctim Reviewed By: morehouse Subscribers: srhines, mgorny, delcypher, jfb, #sanitizers, llvm-commits Tags: #llvm, #sanitizers Differential Revision: https://reviews.llvm.org/D62258 llvm-svn: 362962
* [scudo][standalone] Introduce the Primary(s) and LocalCacheKostya Kortchinsky2019-05-207-9/+1128
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: This CL introduces the 32 & 64-bit primary allocators, and associated Local Cache. While the general idea is mostly similar to what exists in sanitizer_common, it departs from the original code somewhat significantly: - the 64-bit primary no longer uses a free array at the end of a region but uses batches of free blocks in region 0, allowing for a convergence with the 32-bit primary behavior; - as a result, there is only one (templated) local cache type for both primary allocators, and memory reclaiming can be implemented similarly for the 32-bit & 64-bit platforms; - 64-bit primary regions are handled a bit differently: we do not reserve 4TB of memory that we split, but reserve `NumClasses * 2^RegionSizeLog`, each region being offseted by a random number of pages from its computed base. A side effect of this is that the 64-bit primary works on 32-bit platform (I don't think we want to encourage it but it's an interesting side effect); Reviewers: vitalybuka, eugenis, morehouse, hctim Reviewed By: morehouse Subscribers: srhines, mgorny, delcypher, jfb, #sanitizers, llvm-commits Tags: #llvm, #sanitizers Differential Revision: https://reviews.llvm.org/D61745 llvm-svn: 361159
* [scudo][standalone] Introduce the chunk headerKostya Kortchinsky2019-05-086-3/+249
| | | | | | | | | | | | | | | | | | | | | | Summary: ... and its related functions. The structure and its functionalities are identical to existing ones. The header stores information on a `scudo::Chunk` to be able to detect inconsitencies or potential corruption attempts. It is checksummed for that purpose. Reviewers: morehouse, eugenis, vitalybuka, hctim Reviewed By: vitalybuka Subscribers: mgorny, delcypher, jfb, #sanitizers, llvm-commits Tags: #llvm, #sanitizers Differential Revision: https://reviews.llvm.org/D61654 llvm-svn: 360290
* [scudo][standalone] Introduce the QuarantineKostya Kortchinsky2019-05-075-0/+534
| | | | | | | | | | | | | | | | | | | | | Summary: The Quarantine is used to hold chunks for a little while prior to actually releasing them for potential reuse. The code is pretty much the same as the sanitizer_common one, with additional shuffling of the quarantine batches to decrease predictability of allocation patterns when it is enabled. Reviewers: vitalybuka, eugenis, hctim, morehouse Reviewed By: morehouse Subscribers: mgorny, delcypher, jfb, #sanitizers, llvm-commits Tags: #llvm, #sanitizers Differential Revision: https://reviews.llvm.org/D61385 llvm-svn: 360163
* [compiler-rt] Set the ZX_VMO_RESIZABLE option for zx_vmo_createPetr Hosek2019-05-021-1/+1
| | | | | | | | | | | | | Currently VMO in Zircon create using the zx_vmo_create is resizable by default, but we'll be changing this in the future, requiring an explicit flag to make the VMO resizable. Prepare for this change by passing ZX_VMO_RESIZABLE option to all zx_vmo_create calls that need resizable VMO. Differential Revision: https://reviews.llvm.org/D61450 llvm-svn: 359803
* [compiler-rt][tests] Propagate COMPILER_RT_UNITTEST_LINK_FLAGSHubert Tong2019-05-011-1/+1
| | | | | | | | | | | | | | | | | | | | `COMPILER_RT_UNITTEST_LINK_FLAGS` is dropped in many places, unlike `COMPILER_RT_UNITTEST_CFLAGS`. This patch attempts to remove that inconsistency. Previously reviewed as part of D58951. Reviewers: sfertile, peter.smith, pzheng, phosek, Hahnfeld, nemanjai, jasonliu Reviewed By: sfertile Subscribers: jsji, kubamracek, dberris, mgorny, delcypher, jdoerfert, #sanitizers, llvm-commits Tags: #llvm, #sanitizers Differential Revision: https://reviews.llvm.org/D60143 llvm-svn: 359733
* [scudo][NFC] Remove unneeded template from scudo::CombinedAllocatorVitaly Buka2019-05-012-4/+7
| | | | | | | | | | | | | | Reviewers: cryptoad, eugenis Reviewed By: cryptoad Subscribers: #sanitizers, llvm-commits Tags: #sanitizers, #llvm Differential Revision: https://reviews.llvm.org/D61162 llvm-svn: 359718
* [sanitizer][NFC] Get type of AllocatorCache from CombinedAllocatorVitaly Buka2019-05-012-4/+4
| | | | | | | | | | | | | | Reviewers: eugenis, cryptoad, kcc Reviewed By: kcc Subscribers: kcc, kubamracek, #sanitizers, llvm-commits Tags: #sanitizers, #llvm Differential Revision: https://reviews.llvm.org/D61155 llvm-svn: 359715
* [scudo][standalone] Add the memory reclaiming mechanismKostya Kortchinsky2019-04-306-3/+527
| | | | | | | | | | | | | | | | | | | | | | Summary: This CL implements the memory reclaiming function `releaseFreeMemoryToOS` and its associated classes. Most of this code was originally written by Aleksey for the Primary64 in sanitizer_common, and I made some changes to be able to implement 32-bit reclaiming as well. The code has be restructured a bit to accomodate for freelist of batches instead of the freearray used in the current sanitizer_common code. Reviewers: eugenis, vitalybuka, morehouse, hctim Reviewed By: vitalybuka Subscribers: srhines, mgorny, delcypher, #sanitizers, llvm-commits Tags: #llvm, #sanitizers Differential Revision: https://reviews.llvm.org/D61214 llvm-svn: 359567
* [sanitizer] Calculate SizeClassAllocator32::ByteMap type from ↵Vitaly Buka2019-04-271-7/+0
| | | | | | | | | | | | | | Params::kSpaceSize and Params::kRegionSizeLog Reviewers: eugenis Subscribers: kubamracek, cryptoad, #sanitizers, llvm-commits Tags: #sanitizers, #llvm Differential Revision: https://reviews.llvm.org/D61206 llvm-svn: 359374
* [scudo][standalone] Introduce the SizeClassMapKostya Kortchinsky2019-04-255-1/+190
| | | | | | | | | | | | | | | | | | | | | | Summary: As with the sanitizer_common allocator, the SCM allows for efficient mapping between sizes and size-classes, table-free. It doesn't depart significantly from the original, except that we allow the use of size-class 0 for other purposes (as opposed to chunks of size 0). The Primary will use it to hold TransferBatches. Reviewers: vitalybuka, eugenis, hctim, morehouse Reviewed By: vitalybuka Subscribers: srhines, mgorny, delcypher, #sanitizers, llvm-commits Tags: #llvm, #sanitizers Differential Revision: https://reviews.llvm.org/D61088 llvm-svn: 359199
* [scudo][standalone] Introduce the Secondary allocatorKostya Kortchinsky2019-04-2414-88/+476
| | | | | | | | | | | | | | | | | | | | | | | | Summary: The Secondary allocator wraps the platform allocation primitives. It is meant to be used for larger sizes that the Primary can't fullfill, as it will be slower, and sizes are multiple of the system page size. This also changes some of the existing code, notably the opaque platform data being passed to the platform specific functions: we can shave a couple of syscalls on Fuchsia by storing additional data (this addresses a TODO). Reviewers: eugenis, vitalybuka, hctim, morehouse Reviewed By: morehouse Subscribers: mgorny, delcypher, jfb, #sanitizers, llvm-commits Tags: #llvm, #sanitizers Differential Revision: https://reviews.llvm.org/D60787 llvm-svn: 359097
* [scudo][standalone] Add flags & related parsersKostya Kortchinsky2019-04-099-0/+510
| | | | | | | | | | | | | | | | | | | | | | | Summary: As with other Sanitizers, and the current version of Scudo, we can provide flags in differents way: at compile time, through a weak function, through an environment variable. This change adds support for the configuration flags, and the string parsers. Those are fairly similar to the sanitizer_common way of doing things. Reviewers: morehouse, hctim, vitalybuka Reviewed By: morehouse, vitalybuka Subscribers: mgorny, delcypher, jdoerfert, #sanitizers, llvm-commits Tags: #llvm, #sanitizers Differential Revision: https://reviews.llvm.org/D59597 llvm-svn: 358011
* [scudo][standalone] Add error reportsKostya Kortchinsky2019-03-206-37/+339
| | | | | | | | | | | | | | | | | | | | | Summary: This change adds fatal error messages for various error conditions that will be added later in the code. This also addresses a `TODO` now that we have `reportCheckFailed` (which lead me to notice a few variables that were not cased properly so I changed that as well). Reviewers: morehouse, hctim, vitalybuka Reviewed By: morehouse, hctim, vitalybuka Subscribers: mgorny, delcypher, jfb, jdoerfert, #sanitizers, llvm-commits Tags: #llvm, #sanitizers Differential Revision: https://reviews.llvm.org/D59551 llvm-svn: 356556
* [scudo][standalone] Add string utility functionsKostya Kortchinsky2019-03-197-13/+385
| | | | | | | | | | | | | | | | | | | | | | | | | Summary: Add some string utility functions, notably to format strings, get lengths, convert a string to a number. Those functions will be used in reports and flags (coming up next). They were mostly borrowed from sanitizer_common. Make use of the string length function in a couple places in the platform code that was checked in with inlined version of it. Add some tests. Reviewers: morehouse, eugenis, vitalybuka, hctim Reviewed By: morehouse, vitalybuka Subscribers: mgorny, delcypher, jdoerfert, #sanitizers, llvm-commits Tags: #llvm, #sanitizers Differential Revision: https://reviews.llvm.org/D59262 llvm-svn: 356457
* [scudo][standalone] Implement checksumming functionsKostya Kortchinsky2019-03-126-0/+216
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: This CL implements the checksumming functions. This departs from the current Scudo code in one aspect: the software version is no longer CRC32 but a BSD checksum. This is because the software CRC32 was too impactful in terms of performances, the BSD checksum has no array lookup which is better (and saves 1KB of data). As with the current version, we only flip the CRC compiler flag for a single compilation unit by default, to allow for a library compiled with HW CRC32 to work on a system without HW CRC32. There is some platform & hardware specific code in those files, but since departs from a mere platform split, it felt right to me to have it that way. Reviewers: morehouse, eugenis, vitalybuka, hctim, mcgrathr Reviewed By: morehouse Subscribers: mgorny, delcypher, jfb, jdoerfert, #sanitizers, llvm-commits Tags: #llvm, #sanitizers Differential Revision: https://reviews.llvm.org/D59116 llvm-svn: 355923
* [scudo][standalone] Adding a stats classKostya Kortchinsky2019-03-074-0/+152
| | | | | | | | | | | | | | | | | | | | Summary: This adds simple local & global stats classes to be used by the Primary and Secondary, and associated test. Note that we don't need the strict atomicity of the addition & subtraction (as is in sanitizer_common) so we just use load & store. Reviewers: morehouse, vitalybuka, eugenis, flowerhack, dmmoore415 Reviewed By: morehouse, vitalybuka Subscribers: mgorny, delcypher, jfb, #sanitizers, llvm-commits Tags: #llvm, #sanitizers Differential Revision: https://reviews.llvm.org/D59031 llvm-svn: 355643
* [scudo][standalone] Add bytemap classesKostya Kortchinsky2019-03-054-0/+178
| | | | | | | | | | | | | | | | | | | | Summary: The bytemap classes will be used by the primary32 allocator to associate classes with memory regions. It's similar to the sanitizer_common one except for the fact that the base (level1) maps are mapped instead of being static to reduce the memory footprint of an uninitialized allocator. Reviewers: vitalybuka, eugenis, morehouse, flowerhack, dmmoore415, mcgrathr Reviewed By: vitalybuka, morehouse Subscribers: mgorny, delcypher, jfb, #sanitizers, llvm-commits Tags: #llvm, #sanitizers Differential Revision: https://reviews.llvm.org/D58723 llvm-svn: 355416
* [scudo][standalone] Fix tests makefileKostya Kortchinsky2019-03-011-2/+2
| | | | | | | | | | | | | | | | | | | | | Summary: A missing `STATIC` entailed some annoying to debug failures wrt 32 vs 64 binaries. Additionally I noticed I was using the wrong variable (the Scudo one as opposed to the Scudo Standalone one). See https://reviews.llvm.org/D58184#1412417 and below for discussion. Reviewers: vitalybuka, eugenis, brzycki Reviewed By: vitalybuka, brzycki Subscribers: mgorny, delcypher, #sanitizers, llvm-commits Tags: #llvm, #sanitizers Differential Revision: https://reviews.llvm.org/D58794 llvm-svn: 355203
* [scudo][standalone] Add a standalone vector classKostya Kortchinsky2019-02-275-4/+167
| | | | | | | | | | | | | | | | | | | | | | | | | Summary: This CL adds a standalone vector class that will be used by the scoped strings when they land. We reimplement our own vector class because we can't use the std library one. It's mostly borrowed from the current sanitizer_common one, with LLVM code style changes. Additionnally a casing change in a function name that slipped through the previous review (the function isn't used yet). Reviewers: vitalybuka, eugenis, flowerhack, dmmoore415, mcgrathr, morehouse Reviewed By: vitalybuka Subscribers: mgorny, delcypher, #sanitizers, llvm-commits Tags: #llvm, #sanitizers Differential Revision: https://reviews.llvm.org/D58689 llvm-svn: 354999
* [scudo][standalone] Introduce platform specific code & mutexesKostya Kortchinsky2019-02-2615-52/+986
| | | | | | | | | | | | | | | | | | | | | | | | | Summary: This CL adds the platform specific code for Fuchsia, Linux & Android, as well as some tests related to those (more tests to come later). While some of it is pretty much a straight port of the existing scudo & sanitizer_common code, the memory mapping functions have been reworked a bit to fit the limited usage scenario that Scudo has for them. For Fuchsia, I can now track the Vmar/Vmo pair for memory mappings if there is an intent to grow or decommit some mapping (that will be useful for the Primary). Reviewers: eugenis, vitalybuka, mcgrathr, phosek, flowerhack, morehouse, dmmoore415 Reviewed By: vitalybuka, morehouse Subscribers: kcc, dvyukov, srhines, mgorny, delcypher, jfb, jdoerfert, #sanitizers, llvm-commits Tags: #llvm, #sanitizers Differential Revision: https://reviews.llvm.org/D58184 llvm-svn: 354895
* [compiler-rt] Cleanup usage of C++ ABI libraryJonas Hahnfeld2019-02-162-2/+6
| | | | | | | | | | | | | Add missed value "libcxxabi" and introduce SANITIZER_TEST_CXX for linking unit tests. This needs to be a full C++ library and cannot be libcxxabi. Recommit r354132 which I reverted in r354153 because it broke a sanitizer bot. This was because of the "fixes" for pthread linking, so I've removed these changes. Differential Revision: https://reviews.llvm.org/D58012 llvm-svn: 354198
* Revert "[compiler-rt] Cleanup usage of C++ ABI library"Jonas Hahnfeld2019-02-152-5/+2
| | | | | | | This reverts r354132 because it breaks sanitizer-x86_64-linux: http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux/builds/19915 llvm-svn: 354153
* [compiler-rt] Cleanup usage of C++ ABI libraryJonas Hahnfeld2019-02-152-2/+5
| | | | | | | | | Add missed value "libcxxabi" and introduce SANITIZER_TEST_CXX for linking unit tests. This needs to be a full C++ library and cannot be libcxxabi. Differential Revision: https://reviews.llvm.org/D58012 llvm-svn: 354132
* [scudo][standalone] Do not error out on spurious C(XX) flagsKostya Kortchinsky2019-02-071-1/+3
| | | | | | | | | | | | | | | | | | | | | | Summary: The standalone Scudo version is being built with `-Werror` which can be tripped by extraneous command line arguments. We have little control over those as they can be passed down to us by `CMAKE_C(XX)_FLAGS`, the reported scenario involving `-stdlib=libc++` (see https://reviews.llvm.org/D57412#1384504). To work around this, disable `-Wunused-command-line-argument`. Reviewers: eugenis, vitalybuka, Eugene.Zelenko Reviewed By: eugenis Subscribers: mgorny, delcypher, #sanitizers, llvm-commits Tags: #llvm, #sanitizers Differential Revision: https://reviews.llvm.org/D57757 llvm-svn: 353418
* [scudo] Initial standalone skeleton check-inKostya Kortchinsky2019-02-0410-0/+917
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: This is the initial check-in for the Standalone version of Scudo. The project is initially going to live in scudo/standalone then will replace scudo. See http://lists.llvm.org/pipermail/llvm-dev/2019-January/129113.html for details. This initial CL is meant to lay out the project structure, of both code & tests, providing a minimal amount of functionalities, namely various definitions, some atomic helpers and an intrusive list. (empty.cc is just here to have a compilation unit, but will go away in the upcoming CLs). Initial support is restricted to Linux i386 & x86_64 in make files and will be extended once things land & work. We will grow organically from here, adding functionalities in limited amounts. Reviewers: morehouse, eugenis, vitalybuka, kcc, mcgrathr, flowerhack Reviewed By: morehouse, vitalybuka Subscribers: srhines, mgorny, krytarowski, delcypher, jfb, #sanitizers, llvm-commits Tags: #llvm, #sanitizers Differential Revision: https://reviews.llvm.org/D57412 llvm-svn: 353055
* [scudo] Tuning changes based on feedback from current useKostya Kortchinsky2019-01-243-4/+6
| | | | | | | | | | | | | | | | | | | | | | Summary: This tunes several of the default parameters used within the allocator: - disable the deallocation type mismatch on Android by default; this was causing too many issues with third party libraries; - change the default `SizeClassMap` to `Dense`, it caches less entries and is way more memory efficient overall; - relax the timing of the RSS checks, 10 times per second was too much, lower it to 4 times (every 250ms), and update the test so that it passes with the new default. Reviewers: eugenis Reviewed By: eugenis Subscribers: srhines, delcypher, #sanitizers, llvm-commits Differential Revision: https://reviews.llvm.org/D57116 llvm-svn: 352057
* Update the file headers across all of the LLVM projects in the monorepoChandler Carruth2019-01-1923-92/+69
| | | | | | | | | | | | | | | | | to reflect the new license. We understand that people may be surprised that we're moving the header entirely to discuss the new license. We checked this carefully with the Foundation's lawyer and we believe this is the correct approach. Essentially, all code in the project is now made available by the LLVM project under our new license, so you will see that the license headers include that license only. Some of our contributors have contributed code under our old license, and accordingly, we have retained a copy of our old license notice in the top-level files in each project and repository. llvm-svn: 351636
* Introduce `AddressSpaceView` template parameter to `SizeClassAllocator64`.Dan Liew2018-12-211-0/+1
| | | | | | | | | | | | | | | | | | | | | | Summary: This is a follow up patch to r349138. This patch makes a `AddressSpaceView` a type declaration in the allocator parameters used by `SizeClassAllocator64`. For ASan, LSan, and the unit tests the AP64 declarations have been made templated so that `AddressSpaceView` can be changed at compile time. For the other sanitizers we just hard-code `LocalAddressSpaceView` because we have no plans to use these allocators in an out-of-process manner. rdar://problem/45284065 Reviewers: kcc, dvyukov, vitalybuka, cryptoad, eugenis, kubamracek, george.karpenkov Subscribers: #sanitizers, llvm-commits Differential Revision: https://reviews.llvm.org/D55764 llvm-svn: 349954
* Introduce `AddressSpaceView` template parameter to `SizeClassAllocator32`, ↵Dan Liew2018-12-141-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | `FlatByteMap`, and `TwoLevelByteMap`. Summary: This is a follow up patch to r346956 for the `SizeClassAllocator32` allocator. This patch makes `AddressSpaceView` a template parameter both to the `ByteMap` implementations (but makes `LocalAddressSpaceView` the default), some `AP32` implementations and is used in `SizeClassAllocator32`. The actual changes to `ByteMap` implementations and `SizeClassAllocator32` are very simple. However the patch is large because it requires changing all the `AP32` definitions, and users of those definitions. For ASan and LSan we make `AP32` and `ByteMap` templateds type that take a single `AddressSpaceView` argument. This has been done because we will instantiate the allocator with a type that isn't `LocalAddressSpaceView` in the future patches. For the allocators used in the other sanitizers (i.e. HWAsan, MSan, Scudo, and TSan) use of `LocalAddressSpaceView` is hard coded because we do not intend to instantiate the allocators with any other type. In the cases where untemplated types have become templated on a single `AddressSpaceView` parameter (e.g. `PrimaryAllocator`) their name has been changed to have a `ASVT` suffix (Address Space View Type) to indicate they are templated. The only exception to this are the `AP32` types due to the desire to keep the type name as short as possible. In order to check that template is instantiated in the correct a way a `static_assert(...)` has been added that checks that the `AddressSpaceView` type used by `Params::ByteMap::AddressSpaceView` matches the `Params::AddressSpaceView`. This uses the new `sanitizer_type_traits.h` header. rdar://problem/45284065 Reviewers: kcc, dvyukov, vitalybuka, cryptoad, eugenis, kubamracek, george.karpenkov Subscribers: mgorny, llvm-commits, #sanitizers Differential Revision: https://reviews.llvm.org/D54904 llvm-svn: 349138
* [sanitizer] Fix mallopt interceptor.Evgeniy Stepanov2018-10-251-1/+1
| | | | | | On error, mallopt is supposed to return 0, not -1. llvm-svn: 345323
* [sanitizer] Use -Wl,-z,global on Android for sanitizers except UBsanStephen Hines2018-10-021-0/+8
| | | | | | | | | | | | | | | | | | Summary: This essentially reverts r337010 since it breaks UBSan, which is used for a few platform libraries. The "-z global" flag is now added for Scudo as well. The only other sanitizer shared libraries are for asan and hwasan, which have also been reinstated to use the global flag. Reviewers: cryptoad, eugenis Reviewed By: cryptoad Subscribers: kubamracek, mgorny, delcypher, #sanitizers, nickdesaulniers, chh, kongyi, pirama, llvm-commits Differential Revision: https://reviews.llvm.org/D52770 llvm-svn: 343599
* [sanitizer] Revert D50940Kostya Kortchinsky2018-08-291-1/+1
| | | | | | | | | | | | | | | | | Summary: The previous version of the patch makes some code unable to distinguish failure to map address 0 and error. Revert to turn the bots back to green while figuring out a new approach. Reviewers: eugenis Reviewed By: eugenis Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits Differential Revision: https://reviews.llvm.org/D51451 llvm-svn: 340957
* [scudo] Replace eraseHeader with compareExchangeHeader for Quarantined chunksKostya Kortchinsky2018-08-241-10/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: The reason for the existence of `eraseHeader` was that it was deemed faster to null-out a chunk header, effectively making it invalid, rather than marking it as available, which incurred a checksum computation and a cmpxchg. A previous use of `eraseHeader` was removed with D50655 due to a race. Now we remove the second use of it in the Quarantine deallocation path and replace is with a `compareExchangeHeader`. The reason for this is that greatly helps debugging some heap bugs as the chunk header is now valid and the chunk marked available, as opposed to the header being invalid. Eg: we get an invalid state error, instead of an invalid header error, which reduces the possibilities. The computational penalty is negligible. Reviewers: alekseyshl, flowerhack, eugenis Reviewed By: eugenis Subscribers: delcypher, jfb, #sanitizers, llvm-commits Differential Revision: https://reviews.llvm.org/D51224 llvm-svn: 340633
* [sanitizer] Change Mmap*NoAccess to return nullptr on errorKostya Kortchinsky2018-08-231-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: `MmapNoAccess` & `MmapFixedNoAccess` return directly the result of `internal_mmap`, as opposed to other Mmap functions that return nullptr. This inconsistency leads to some confusion for the callers, as some check for `~(uptr)0` (`MAP_FAILED`) for failure (while it can fail with `-ENOMEM` for example). Two potential solutions: change the callers, or make the functions return `nullptr` on failure to follow the precedent set by the other functions. The second option looked more appropriate to me. Correct the callers that were wrongly checking for `~(uptr)0` or `MAP_FAILED`. TODO for follow up CLs: - There are a couple of `internal_mmap` calls in XRay that check for MMAP_FAILED as a result as well (cc: @dberris); they should use `internal_iserror`; Reviewers: eugenis, alekseyshl, dberris, kubamracek Reviewed By: alekseyshl Subscribers: kristina, kubamracek, delcypher, #sanitizers, dberris, llvm-commits Differential Revision: https://reviews.llvm.org/D50940 llvm-svn: 340576
* [scudo] Fix race condition in deallocation path when Quarantine is bypassedKostya Kortchinsky2018-08-141-5/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: There is a race window in the deallocation path when the Quarantine is bypassed. Initially we would just erase the header of a chunk if we were not to use the Quarantine, as opposed to using a compare-exchange primitive, to make things faster. It turned out to be a poor decision, as 2 threads (or more) could simultaneously deallocate the same pointer, and if the checks were to done before the header got erased, this would result in the pointer being added twice (or more) to distinct thread caches, and eventually be reused. Winning the race is not trivial but can happen with enough control over the allocation primitives. The repro added attempts to trigger the bug, with a moderate success rate, but it should be enough to notice if the bug ever make its way back into the code. Since I am changing things in this file, there are 2 smaller changes tagging along, marking a variable `const`, and improving the Quarantine bypass test at runtime. Reviewers: alekseyshl, eugenis, kcc, vitalybuka Reviewed By: eugenis, vitalybuka Subscribers: delcypher, #sanitizers, llvm-commits Differential Revision: https://reviews.llvm.org/D50655 llvm-svn: 339705
* [scudo] Simplify internal names (NFC)Kostya Kortchinsky2018-07-205-56/+51
| | | | | | | | | | | | | | | | | | | | | | | | | Summary: There is currently too much redundancy in the class/variable/* names in Scudo: - we are in the namespace `__scudo`, so there is no point in having something named `ScudoX` to end up with a final name of `__scudo::ScudoX`; - there are a lot of types/* that have `Allocator` in the name, given that Scudo is an allocator I figure this doubles up as well. So change a bunch of the Scudo names to make them shorter, less redundant, and overall simpler. They should still be pretty self explaining (or at least it looks so to me). The TSD part will be done in another CL (eg `__scudo::ScudoTSD`). Reviewers: alekseyshl, eugenis Reviewed By: alekseyshl Subscribers: delcypher, #sanitizers, llvm-commits Differential Revision: https://reviews.llvm.org/D49505 llvm-svn: 337557
* [CMake] Add compiler-rt header files to the list of sources for targetsDan Liew2018-07-101-0/+21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | when building with an IDE so that header files show up in the UI. This massively improves the development workflow in IDEs. To implement this a new function `compiler_rt_process_sources(...)` has been added that adds header files to the list of sources when the generator is an IDE. For non-IDE generators (e.g. Ninja/Makefile) no changes are made to the list of source files. The function can be passed a list of headers via the `ADDITIONAL_HEADERS` argument. For each runtime library a list of explicit header files has been added and passed via `ADDITIONAL_HEADERS`. For `tsan` and `sanitizer_common` a list of headers was already present but it was stale and has been updated to reflect the current state of the source tree. The original version of this patch used file globbing (`*.{h,inc,def}`) to find the headers but the approach was changed due to this being a CMake anti-pattern (if the list of headers changes CMake won't automatically re-generate if globbing is used). The LLVM repo contains a similar function named `llvm_process_sources()` but we don't use it here for several reasons: * It depends on the `LLVM_ENABLE_OPTION` cache variable which is not set in standalone compiler-rt builds. * We would have to `include(LLVMProcessSources)` which I'd like to avoid because it would include a bunch of stuff we don't need. Differential Revision: https://reviews.llvm.org/D48422 llvm-svn: 336663
* [scudo] Add some logs for AndroidKostya Kortchinsky2018-07-061-0/+3
| | | | | | | | | | | | | | | | Summary: Namely, set the abort message, and allow to write the message to syslog if the option is enabled. Reviewers: alekseyshl Reviewed By: alekseyshl Subscribers: delcypher, #sanitizers, llvm-commits Differential Revision: https://reviews.llvm.org/D48902 llvm-svn: 336445
* [scudo] Get rid of builtin-declaration-mismatch warningsKostya Kortchinsky2018-07-031-10/+12
| | | | | | | | | | | | | | | | | | | | | | | | Summary: The C interceptors were using `SIZE_T` defined in the interception library as a `__sanitizer::uptr`. On some 32-bit platforms, this lead to the following warning: ``` warning: declaration of ‘void* malloc(SIZE_T)’ conflicts with built-in declaration ‘void* malloc(unsigned int)’ [-Wbuiltin-declaration-mismatch] INTERCEPTOR_ATTRIBUTE void *malloc(SIZE_T size) { ``` `__sanitizer::uptr` is indeed defined as an `unsigned long` on those. So just include `stddef.h` and use `size_t` instead. Reviewers: alekseyshl Reviewed By: alekseyshl Subscribers: delcypher, llvm-commits, #sanitizers Differential Revision: https://reviews.llvm.org/D48885 llvm-svn: 336221
* [scudo] Enable Scudo memory hooks for Fuchsia.Kostya Kortchinsky2018-07-021-2/+6
| | | | | | | | | | | | | | | | | Summary: It would be useful for Flutter apps, especially, to be able to use malloc hooks to debug memory leaks on Fuchsia. They're not able to do this right now, so it'd be a nice bonus to throw in with the Scudo switchover. Reviewers: cryptoad, alekseyshl Reviewed By: cryptoad Differential Revision: https://reviews.llvm.org/D48618 llvm-svn: 336139
OpenPOWER on IntegriCloud