summaryrefslogtreecommitdiffstats
path: root/compiler-rt/lib/tsan/rtl/tsan_defs.h
Commit message (Collapse)AuthorAgeFilesLines
* Update the file headers across all of the LLVM projects in the monorepoChandler Carruth2019-01-191-4/+3
| | | | | | | | | | | | | | | | | to reflect the new license. We understand that people may be surprised that we're moving the header entirely to discuss the new license. We checked this carefully with the Foundation's lawyer and we believe this is the correct approach. Essentially, all code in the project is now made available by the LLVM project under our new license, so you will see that the license headers include that license only. Some of our contributors have contributed code under our old license, and accordingly, we have retained a copy of our old license notice in the top-level files in each project and repository. llvm-svn: 351636
* tsan: optimize sync clock memory consumptionDmitry Vyukov2017-07-141-4/+29
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This change implements 2 optimizations of sync clocks that reduce memory consumption: Use previously unused first level block space to store clock elements. Currently a clock for 100 threads consumes 3 512-byte blocks: 2 64-bit second level blocks to store clock elements +1 32-bit first level block to store indices to second level blocks Only 8 bytes of the first level block are actually used. With this change such clock consumes only 2 blocks. Share similar clocks differing only by a single clock entry for the current thread. When a thread does several release operations on fresh sync objects without intervening acquire operations in between (e.g. initialization of several fields in ctor), the resulting clocks differ only by a single entry for the current thread. This change reuses a single clock for such release operations. The current thread time (which is different for different clocks) is stored in dirty entries. We are experiencing issues with a large program that eats all 64M clock blocks (32GB of non-flushable memory) and crashes with dense allocator overflow. Max number of threads in the program is ~170 which is currently quite unfortunate (consume 4 blocks per clock). Currently it crashes after consuming 60+ GB of memory. The first optimization brings clock block consumption down to ~40M and allows the program to work. The second optimization further reduces block consumption to "modest" 16M blocks (~8GB of RAM) and reduces overall RAM consumption to ~30GB. Measurements on another real world C++ RPC benchmark show RSS reduction from 3.491G to 3.186G and a modest speedup of ~5%. Go parallel client/server HTTP benchmark: https://github.com/golang/benchmarks/blob/master/http/http.go shows RSS reduction from 320MB to 240MB and a few percent speedup. Reviewed in https://reviews.llvm.org/D35323 llvm-svn: 308018
* [tsan] Detect races on modifying accesses in Swift codeKuba Mracek2017-05-031-0/+9
| | | | | | | | This patch allows the Swift compiler to emit calls to `__tsan_external_write` before starting any modifying access, which will cause TSan to detect races on arrays, dictionaries and other classes defined in non-instrumented modules. Races on collections from the Swift standard library and user-defined structs and a frequent cause of subtle bugs and it's important that TSan detects those on top of existing LLVM IR instrumentation, which already detects races in direct memory accesses. Differential Revision: https://reviews.llvm.org/D31630 llvm-svn: 302050
* [tsan] Provide API for libraries for race detection on custom objectsKuba Mracek2017-02-021-1/+2
| | | | | | | | This patch allows a non-instrumented library to call into TSan runtime, and tell us about "readonly" and "modifying" accesses to an arbitrary "object" and provide the caller and tag (type of object). This allows TSan to detect violations of API threading contracts where "read-only" methods can be called simulatenously from multiple threads, while modifying methods must be exclusive. Differential Revision: https://reviews.llvm.org/D28836 llvm-svn: 293885
* tsan: always define SANITIZER_GODmitry Vyukov2016-10-281-12/+2
| | | | | | | | | | | | | | | | | | | | | | Currently we either define SANITIZER_GO for Go or don't define it at all for C++. This works fine with preprocessor (ifdef/ifndef/defined), but does not work for C++ if statements (e.g. if (SANITIZER_GO) {...}). Also this is different from majority of SANITIZER_FOO macros which are always defined to either 0 or 1. Always define SANITIZER_GO to either 0 or 1. This allows to use SANITIZER_GO in expressions and in flag default values. Also remove kGoMode and kCppMode, which were meant to be used in expressions, but they are not defined in sanitizer_common code, so SANITIZER_GO become prevalent. Also convert some preprocessor checks to C++ if's or ternary expressions. Majority of this change is done mechanically with: sed "s#ifdef SANITIZER_GO#if SANITIZER_GO#g" sed "s#ifndef SANITIZER_GO#if \!SANITIZER_GO#g" sed "s#defined(SANITIZER_GO)#SANITIZER_GO#g" llvm-svn: 285443
* tsan: split thread into logical and physical stateDmitry Vyukov2016-04-271-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | This is reincarnation of http://reviews.llvm.org/D17648 with the bug fix pointed out by Adhemerval (zatrazz). Currently ThreadState holds both logical state (required for race-detection algorithm, user-visible) and physical state (various caches, most notably malloc cache). Move physical state in a new Process entity. Besides just being the right thing from abstraction point of view, this solves several problems: Cache everything on P level in Go. Currently we cache on a mix of goroutine and OS thread levels. This unnecessary increases memory consumption. Properly handle free operations in Go. Frees are issue by GC which don't have goroutine context. As the result we could not do anything more than just clearing shadow. For example, we leaked sync objects and heap block descriptors. This will allow to get rid of libc malloc in Go (now we have Processor context for internal allocator cache). This in turn will allow to get rid of dependency on libc entirely. Potentially we can make Processor per-CPU in C++ mode instead of per-thread, which will reduce resource consumption. The distinction between Thread and Processor is currently used only by Go, C++ creates Processor per OS thread, which is equivalent to the current scheme. llvm-svn: 267678
* tsan: revert r262037Dmitry Vyukov2016-02-261-1/+0
| | | | | | Broke aarch64 and darwin bots. llvm-svn: 262046
* tsan: split thread into logical and physical stateDmitry Vyukov2016-02-261-0/+1
| | | | | | | | | | | | | | | | | | | | Currently ThreadState holds both logical state (required for race-detection algorithm, user-visible) and physical state (various caches, most notably malloc cache). Move physical state in a new Process entity. Besides just being the right thing from abstraction point of view, this solves several problems: 1. Cache everything on P level in Go. Currently we cache on a mix of goroutine and OS thread levels. This unnecessary increases memory consumption. 2. Properly handle free operations in Go. Frees are issue by GC which don't have goroutine context. As the result we could not do anything more than just clearing shadow. For example, we leaked sync objects and heap block descriptors. 3. This will allow to get rid of libc malloc in Go (now we have Processor context for internal allocator cache). This in turn will allow to get rid of dependency on libc entirely. 4. Potentially we can make Processor per-CPU in C++ mode instead of per-thread, which will reduce resource consumption. The distinction between Thread and Processor is currently used only by Go, C++ creates Processor per OS thread, which is equivalent to the current scheme. llvm-svn: 262037
* Fix another -Wexpansion-to-defined warning in compiler-rt.Nico Weber2016-01-191-1/+5
| | | | llvm-svn: 258202
* [tsan] Fix weakly imported functions on OS XKuba Brecka2015-11-301-3/+0
| | | | | | | | | | On OS X, for weak function (that user can override by providing their own implementation in the main binary), we need extern `"C" SANITIZER_INTERFACE_ATTRIBUTE SANITIZER_WEAK_ATTRIBUTE NOINLINE`. Fixes a broken test case on OS X, java_symbolization.cc, which uses a weak function __tsan_symbolize_external. Differential Revision: http://reviews.llvm.org/D14907 llvm-svn: 254298
* [tsan] Handle libdispatch worker threads on OS XKuba Brecka2015-11-041-0/+2
| | | | | | | | On OS X, GCD worker threads are created without a call to pthread_create. We need to properly register these threads with ThreadCreate and ThreadStart. This patch uses a libpthread API (`pthread_introspection_hook_install`) to get notifications about new threads and about threads that are about to be destroyed. Differential Revision: http://reviews.llvm.org/D14328 llvm-svn: 252049
* Allow UBSan+MSan and UBSan+TSan combinations (Clang part).Alexey Samsonov2015-04-281-0/+5
| | | | | | | | Embed UBSan runtime into TSan and MSan runtimes in the same as we do in ASan. Extend UBSan test suite to also run tests for these combinations. llvm-svn: 235954
* [TSan] Provide default values for compile definitions.Alexey Samsonov2015-02-171-1/+10
| | | | | | | | | Provide defaults for TSAN_COLLECT_STATS and TSAN_NO_HISTORY. Replace #ifdef directives with #if. This fixes a bug introduced in r229112, where building TSan runtime with -DTSAN_COLLECT_STATS=0 would still enable stats collection and reporting. llvm-svn: 229581
* tsan: reduce size of vector clock in Go modeDmitry Vyukov2015-02-131-0/+4
| | | | | | | | | Go does not have freed memory. Reduces per-goroutine overhead from 455K to 356K. https://code.google.com/p/thread-sanitizer/issues/detail?id=89 llvm-svn: 229113
* tsan: remove stats from ThreadState ifndef TSAN_COLLECT_STATSDmitry Vyukov2015-02-131-6/+0
| | | | | | | Issue 89: Uses a lot of memory for each goroutine https://code.google.com/p/thread-sanitizer/issues/detail?id=89 llvm-svn: 229112
* tsan: remove TSAN_SHADOW_COUNTDmitry Vyukov2015-01-191-29/+0
| | | | | | | | | TSAN_SHADOW_COUNT is defined to 4 in all environments. Other values of TSAN_SHADOW_COUNT were never tested and were broken by recent changes to shadow mapping. Remove it as there is no reason to fix nor maintain it. llvm-svn: 226466
* Remove TSAN_DEBUG in favor of SANITIZER_DEBUG.Alexey Samsonov2015-01-031-6/+2
| | | | llvm-svn: 225111
* [tsan] remove TSAN_GO in favor of SANITIZER_GOKostya Serebryany2014-12-091-1/+1
| | | | llvm-svn: 223732
* [TSan] Use StackTrace from sanitizer_common where applicableAlexey Samsonov2014-11-031-2/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | Summary: This change removes `__tsan::StackTrace` class. There are now three alternatives: # Lightweight `__sanitizer::StackTrace`, which doesn't own a buffer of PCs. It is used in functions that need stack traces in read-only mode, and helps to prevent unnecessary allocations/copies (e.g. for StackTraces fetched from StackDepot). # `__sanitizer::BufferedStackTrace`, which stores buffer of PCs in a constant array. It is used in TraceHeader (non-Go version) # `__tsan::VarSizeStackTrace`, which owns buffer of PCs, dynamically allocated via TSan internal allocator. Test Plan: compiler-rt test suite Reviewers: dvyukov, kcc Reviewed By: kcc Subscribers: llvm-commits, kcc Differential Revision: http://reviews.llvm.org/D6004 llvm-svn: 221194
* tsan: optimize memory access functionsDmitry Vyukov2014-05-301-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The optimization is two-fold: First, the algorithm now uses SSE instructions to handle all 4 shadow slots at once. This makes processing faster. Second, if shadow contains the same access, we do not store the event into trace. This increases effective trace size, that is, tsan can remember up to 10x more previous memory accesses. Perofrmance impact: Before: [ OK ] DISABLED_BENCH.Mop8Read (2461 ms) [ OK ] DISABLED_BENCH.Mop8Write (1836 ms) After: [ OK ] DISABLED_BENCH.Mop8Read (1204 ms) [ OK ] DISABLED_BENCH.Mop8Write (976 ms) But this measures only fast-path. On large real applications the speedup is ~20%. Trace size impact: On app1: Memory accesses : 1163265870 Including same : 791312905 (68%) on app2: Memory accesses : 166875345 Including same : 150449689 (90%) 90% of filtered events means that trace size is effectively 10x larger. llvm-svn: 209897
* tsan: refactor storage of meta information for heap blocks and sync objectsDmitry Vyukov2014-05-291-1/+16
| | | | | | | | | | | | | | | The new storage (MetaMap) is based on direct shadow (instead of a hashmap + per-block lists). This solves a number of problems: - eliminates quadratic behaviour in SyncTab::GetAndLock (https://code.google.com/p/thread-sanitizer/issues/detail?id=26) - eliminates contention in SyncTab - eliminates contention in internal allocator during allocation of sync objects - removes a bunch of ad-hoc code in java interface - reduces java shadow from 2x to 1/2x - allows to memorize heap block meta info for Java and Go - allows to cleanup sync object meta info for Go - which in turn enabled deadlock detector for Go llvm-svn: 209810
* tsan: allow to disable history collectionDmitry Vyukov2014-05-151-0/+6
| | | | | | | The mode is enabled with -DTSAN_NO_HISTORY=1 flag. Intended mostly for research purposes (how fast can it go w/o history). llvm-svn: 208878
* tsan: fix vector clocksDmitry Vyukov2014-04-111-0/+1
| | | | | | | the new optimizations break when thread ids gets reused (clocks go backwards) add the necessary tests as well llvm-svn: 206035
* tsan: better diagnostics if thread finishes with ignores enabledDmitry Vyukov2013-11-271-0/+1
| | | | | | | print thread creation stack and stacks where ignores were enabled. llvm-svn: 195836
* tsan: move shadow stack from thread descriptors to fixed addressesDmitry Vyukov2013-10-161-4/+2
| | | | | | | | This allows to increase max shadow stack size to 64K, and reliably catch shadow stack overflows instead of silently corrupting memory. llvm-svn: 192797
* [tsan] Move some suppressions-related code to common.Sergey Matveev2013-06-261-1/+0
| | | | | | Factor out code to be reused in LSan. Also switch from linked list to vector. llvm-svn: 184957
* tsan: print matched suppressions if print_suppressions=1 flag is providedDmitry Vyukov2013-03-271-0/+1
| | | | llvm-svn: 178159
* [TSan] Switch TSan runtime to use ThreadRegistry class from sanitizer_commonAlexey Samsonov2013-03-151-1/+0
| | | | llvm-svn: 177154
* tsan: detect races between plain and atomic memory accessesDmitry Vyukov2013-02-011-1/+1
| | | | llvm-svn: 174163
* tsan: fix CPP_WEAK definition (it must be the other way around)Dmitry Vyukov2013-01-301-3/+3
| | | | llvm-svn: 173932
* tsan: introduce a helped macro CPP_WEAK (Go linker does not support weak ↵Dmitry Vyukov2013-01-301-0/+3
| | | | | | symbols) llvm-svn: 173917
* tsan: add mutexsets to reportsDmitry Vyukov2012-12-061-0/+6
| | | | | | With this change reports say what mutexes the threads hold around the racy memory accesses. llvm-svn: 169493
* tsan: fix trace handling when trace is reused between threadsDmitry Vyukov2012-12-041-1/+7
| | | | llvm-svn: 169259
* tsan: address several review commentsDmitry Vyukov2012-11-281-0/+4
| | | | llvm-svn: 168789
* tsan: switch to 4 shadow cells by default (since that's what we use ↵Dmitry Vyukov2012-11-151-1/+1
| | | | | | everywhere now) llvm-svn: 168059
* tsan: use GORACE env for options for GoDmitry Vyukov2012-11-081-0/+6
| | | | llvm-svn: 167575
* tsan: lazily allocate shadow for GoDmitry Vyukov2012-11-061-0/+3
| | | | llvm-svn: 167464
* tsan: fix constant typesDmitry Vyukov2012-11-061-4/+4
| | | | llvm-svn: 167453
* tsan: fix code styleDmitry Vyukov2012-09-061-1/+1
| | | | llvm-svn: 163326
* tsan: increase max shadow stack size + reduce memory consumption at the same ↵Dmitry Vyukov2012-09-061-1/+2
| | | | | | time (by not memorizing full stacks in traces) llvm-svn: 163322
* tsan: switch to new allocatorDmitry Vyukov2012-08-151-0/+1
| | | | llvm-svn: 161953
* tsan: use dynamic shadow stack for GoDmitry Vyukov2012-07-161-3/+1
| | | | llvm-svn: 160288
* tsan: Go language supportDmitry Vyukov2012-07-051-3/+5
| | | | llvm-svn: 159754
* tsan/asan: unify atomics (move atomics from tsan to sanitizer_common)Dmitry Vyukov2012-06-291-18/+0
| | | | llvm-svn: 159437
* tsan: remove own memset/memcpy/memcmp (too messy)Dmitry Vyukov2012-06-291-25/+0
| | | | llvm-svn: 159430
* tsan: prevent insertion of unwanted memset/memcpy/memcmp into runtimeDmitry Vyukov2012-06-271-5/+24
| | | | llvm-svn: 159294
* tsan: do not call malloc/free in memory access handling routine.Dmitry Vyukov2012-06-221-0/+1
| | | | | | This improves signal-/fork-safety of instrumented programs. llvm-svn: 158988
* [Sanitizer] Use DEFINE_REAL macro in TSan runtime to call libc ↵Alexey Samsonov2012-06-151-7/+6
| | | | | | implementations of functions. Move strchr to sanitizer_libc. llvm-svn: 158517
* [Sanitizer] Move internal_memcmp to common sanitizer libcAlexey Samsonov2012-06-141-1/+0
| | | | llvm-svn: 158450
* [TSan] use efficient real_memcpy inside runtimeAlexey Samsonov2012-06-091-0/+1
| | | | llvm-svn: 158260
OpenPOWER on IntegriCloud