summaryrefslogtreecommitdiffstats
path: root/compiler-rt/lib/scudo/standalone/mutex.h
Commit message (Collapse)AuthorAgeFilesLines
* [scudo][standalone] Add more stats to mallinfoKostya Kortchinsky2019-08-141-4/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: Android requires additional stats in mallinfo. While we can provide right away the number of bytes mapped (Primary+Secondary), there was no way to get the number of free bytes (only makes sense for the Primary since the Secondary unmaps everything on deallocation). An approximation could be `StatMapped - StatAllocated`, but since we are mapping in `1<<17` increments for the 64-bit Primary, it's fairly inaccurate. So we introduce `StatFree` (note it's `Free`, not `Freed`!), which keeps track of the amount of Primary blocks currently unallocated. Reviewers: cferris, eugenis, vitalybuka, hctim, morehouse Reviewed By: morehouse Subscribers: delcypher, #sanitizers, llvm-commits Tags: #llvm, #sanitizers Differential Revision: https://reviews.llvm.org/D66112 llvm-svn: 368866
* [scudo][standalone] Optimization passKostya Kortchinsky2019-07-241-7/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: This introduces a bunch of small optimizations with the purpose of making the fastpath tighter: - tag more conditions as `LIKELY`/`UNLIKELY`: as a rule of thumb we consider that every operation related to the secondary is unlikely - attempt to reduce the number of potentially extraneous instructions - reorganize the `Chunk` header to not straddle a word boundary and use more appropriate types Note that some `LIKELY`/`UNLIKELY` impact might be less obvious as they are in slow paths (for example in `secondary.cc`), but at this point I am throwing a pretty wide net, and it's consistant and doesn't hurt. This was mosly done for the benfit of Android, but other platforms benefit from it too. An aarch64 Android benchmark gives: - before: ``` BM_youtube/min_time:15.000/repeats:4/manual_time_mean 445244 us 659385 us 4 BM_youtube/min_time:15.000/repeats:4/manual_time_median 445007 us 658970 us 4 BM_youtube/min_time:15.000/repeats:4/manual_time_stddev 885 us 1332 us 4 ``` - after: ``` BM_youtube/min_time:15.000/repeats:4/manual_time_mean 415697 us 621925 us 4 BM_youtube/min_time:15.000/repeats:4/manual_time_median 415913 us 622061 us 4 BM_youtube/min_time:15.000/repeats:4/manual_time_stddev 990 us 1163 us 4 ``` Additional since `-Werror=conversion` is enabled on some platforms we are built on, enable it upstream to catch things early: a few sign conversions had slept through and needed additional casting. Reviewers: hctim, morehouse, eugenis, vitalybuka Reviewed By: vitalybuka Subscribers: srhines, mgorny, javed.absar, kristof.beyls, delcypher, #sanitizers, llvm-commits Tags: #llvm, #sanitizers Differential Revision: https://reviews.llvm.org/D64664 llvm-svn: 366918
* [scudo][standalone] Merge Spin & Blocking mutex into a Hybrid oneKostya Kortchinsky2019-07-111-56/+36
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: We ran into a problem on Fuchsia where yielding threads would never be deboosted, ultimately resulting in several threads spinning on the same TSD, and no possibility for another thread to be scheduled, dead-locking the process. While this was fixed in Zircon, this lead to discussions about if spinning without a break condition was a good decision, and settled on a new hybrid model that would spin for a while then block. Currently we are using a number of iterations for spinning that is mostly arbitrary (based on sanitizer_common values), but this can be tuned in the future. Since we are touching `common.h`, we also use this change as a vehicle for an Android optimization (the page size is fixed in Bionic, so use a fixed value too). Reviewers: morehouse, hctim, eugenis, dvyukov, vitalybuka Reviewed By: hctim Subscribers: srhines, delcypher, jfb, #sanitizers, llvm-commits Tags: #llvm, #sanitizers Differential Revision: https://reviews.llvm.org/D64358 llvm-svn: 365790
* [scudo][standalone] Fuchsia related changesKostya Kortchinsky2019-06-181-19/+4
| | | | | | | | | | | | | | | | | | | | | | Summary: Fuchsia wants to use mutexes with PI in the Scudo code, as opposed to our own implementation. This required making `lock` & `unlock` platform specific (as opposed to `wait` & `wake`) [code courtesy of John Grossman]. There is an additional flag required now for mappings as well: `ZX_VM_ALLOW_FAULTS`. Reviewers: morehouse, mcgrathr, eugenis, vitalybuka, hctim Reviewed By: morehouse Subscribers: delcypher, jfb, #sanitizers, llvm-commits Tags: #llvm, #sanitizers Differential Revision: https://reviews.llvm.org/D63435 llvm-svn: 363705
* [scudo][standalone] Introduce the thread specific data structuresKostya Kortchinsky2019-06-101-1/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: This CL adds the structures dealing with thread specific data for the allocator. This includes the thread specific data structure itself and two registries for said structures: an exclusive one, where each thread will have its own TSD struct, and a shared one, where a pool of TSD structs will be shared by all threads, with dynamic reassignment at runtime based on contention. This departs from the current Scudo implementation: we intend to make the Registry a template parameter of the allocator (as opposed to a single global entity), allowing various allocators to coexist with different TSD registry models. As a result, TSD registry and Allocator are tightly coupled. This also corrects a couple of things in other files that I noticed while adding this. Reviewers: eugenis, vitalybuka, morehouse, hctim Reviewed By: morehouse Subscribers: srhines, mgorny, delcypher, jfb, #sanitizers, llvm-commits Tags: #llvm, #sanitizers Differential Revision: https://reviews.llvm.org/D62258 llvm-svn: 362962
* [scudo][standalone] Introduce the QuarantineKostya Kortchinsky2019-05-071-0/+1
| | | | | | | | | | | | | | | | | | | | | Summary: The Quarantine is used to hold chunks for a little while prior to actually releasing them for potential reuse. The code is pretty much the same as the sanitizer_common one, with additional shuffling of the quarantine batches to decrease predictability of allocation patterns when it is enabled. Reviewers: vitalybuka, eugenis, hctim, morehouse Reviewed By: morehouse Subscribers: mgorny, delcypher, jfb, #sanitizers, llvm-commits Tags: #llvm, #sanitizers Differential Revision: https://reviews.llvm.org/D61385 llvm-svn: 360163
* [scudo][standalone] Introduce platform specific code & mutexesKostya Kortchinsky2019-02-261-0/+108
Summary: This CL adds the platform specific code for Fuchsia, Linux & Android, as well as some tests related to those (more tests to come later). While some of it is pretty much a straight port of the existing scudo & sanitizer_common code, the memory mapping functions have been reworked a bit to fit the limited usage scenario that Scudo has for them. For Fuchsia, I can now track the Vmar/Vmo pair for memory mappings if there is an intent to grow or decommit some mapping (that will be useful for the Primary). Reviewers: eugenis, vitalybuka, mcgrathr, phosek, flowerhack, morehouse, dmmoore415 Reviewed By: vitalybuka, morehouse Subscribers: kcc, dvyukov, srhines, mgorny, delcypher, jfb, jdoerfert, #sanitizers, llvm-commits Tags: #llvm, #sanitizers Differential Revision: https://reviews.llvm.org/D58184 llvm-svn: 354895
OpenPOWER on IntegriCloud