summaryrefslogtreecommitdiffstats
path: root/llvm/lib/Transforms/Instrumentation
Commit message (Collapse)AuthorAgeFilesLines
* [hwasan] Inline instrumentation & fixed shadow.Evgeniy Stepanov2017-12-131-3/+48
| | | | | | | | | | | | Summary: This brings CPU overhead on bzip2 down from 5.5x to 2x. Reviewers: kcc, alekseyshl Subscribers: kubamracek, hiraditya, llvm-commits Differential Revision: https://reviews.llvm.org/D41137 llvm-svn: 320538
* Split IndirectBr critical edges before PGO gen/use passes.Hiroshi Yamauchi2017-12-121-0/+6
| | | | | | | | | | | | | | | | | | | | | | Summary: The PGO gen/use passes currently fail with an assert failure if there's a critical edge whose source is an IndirectBr instruction and that edge needs to be instrumented. To avoid this in certain cases, split IndirectBr critical edges in the PGO gen/use passes. This works for blocks with single indirectbr predecessors, but not for those with multiple indirectbr predecessors (splitting an IndirectBr critical edge isn't always possible.) Reviewers: davidxl, xur Reviewed By: davidxl Subscribers: efriedma, llvm-commits, mehdi_amini Differential Revision: https://reviews.llvm.org/D40699 llvm-svn: 320511
* ASAN: Provide reliable debug info for local variables at -O0.Adrian Prantl2017-12-111-2/+12
| | | | | | | | | | | | | | | | | | | | | | The function stack poisioner conditionally stores local variables either in an alloca or in malloc'ated memory, which has the unfortunate side-effect, that the actual address of the variable is only materialized when the variable is accessed, which means that those variables are mostly invisible to the debugger even when compiling without optimizations. This patch stores the address of the local stack base into an alloca, which can be referred to by the debug info and is available throughout the function. This adds one extra pointer-sized alloca to each stack frame (but mem2reg can optimize it away again when optimizations are enabled, yielding roughly the same debug info quality as before in optimized code). rdar://problem/30433661 Differential Revision: https://reviews.llvm.org/D41034 llvm-svn: 320415
* [MSan] Hotfix compilationAlexander Potapenko2017-12-111-2/+2
| | | | | | | For some reason the override directives got removed in r320373. I suspect this to be an unwanted effect of clang-format. llvm-svn: 320381
* [MSan] introduce getShadowOriginPtr(). NFC.Alexander Potapenko2017-12-111-129/+191
| | | | | | | | | | | | | | This patch introduces getShadowOriginPtr(), a method that obtains both the shadow and origin pointers for an address as a Value pair. The existing callers of getShadowPtr() and getOriginPtr() are updated to use getShadowOriginPtr(). The rationale for this change is to simplify KMSAN instrumentation implementation. In KMSAN origins tracking is always enabled, and there's no direct mapping between the app memory and the shadow/origin pages. Both the shadow and the origin pointer for a given address are obtained by calling a single runtime hook from the instrumentation, therefore it's easier to work with those pointers together. Reviewed at https://reviews.llvm.org/D40835. llvm-svn: 320373
* [PGO] change arg type to uint64_t to match member field typeXinliang David Li2017-12-101-2/+2
| | | | llvm-svn: 320285
* Register NetBSD/x86_64 in MemorySanitizer.cppKamil Rytarowski2017-12-091-0/+22
| | | | | | | | | | | | | | | | | | | Summary: Reuse the Linux new mapping as it is. Sponsored by <The NetBSD Foundation> Reviewers: joerg, eugenis, vitalybuka Reviewed By: vitalybuka Subscribers: llvm-commits, #sanitizers Tags: #sanitizers Differential Revision: https://reviews.llvm.org/D41022 llvm-svn: 320219
* Hardware-assisted AddressSanitizer (llvm part).Evgeniy Stepanov2017-12-093-0/+284
| | | | | | | | | | | | | | | | | | | | | Summary: This is LLVM instrumentation for the new HWASan tool. It is basically a stripped down copy of ASan at this point, w/o stack or global support. Instrumenation adds a global constructor + runtime callbacks for every load and store. HWASan comes with its own IR attribute. A brief design document can be found in clang/docs/HardwareAssistedAddressSanitizerDesign.rst (submitted earlier). Reviewers: kcc, pcc, alekseyshl Subscribers: srhines, mehdi_amini, mgorny, javed.absar, eraman, llvm-commits, hiraditya Differential Revision: https://reviews.llvm.org/D40932 llvm-svn: 320217
* Generalize llvm::replaceDbgDeclare and actually support the use-case thatAdrian Prantl2017-12-081-1/+2
| | | | | | is mentioned in the documentation (inserting a deref before the plus_uconst). llvm-svn: 320203
* Revert r320104: infinite loop profiling bug fixXinliang David Li2017-12-082-32/+38
| | | | | | | | | | | Causes unexpected memory issue with New PM this time. The new PM invalidates BPI but not BFI, leaving the reference to BPI from BFI invalid. Abandon this patch. There is a more general solution which also handles runtime infinite loop (but not statically). llvm-svn: 320180
* [PowerPC][asan] Update asan to handle changed memory layouts in newer kernelsBill Seurer2017-12-071-1/+1
| | | | | | | | | | | | | | | In more recent Linux kernels with 47 bit VMAs the layout of virtual memory for powerpc64 changed causing the address sanitizer to not work properly. This patch adds support for 47 bit VMA kernels for powerpc64 and fixes up test cases. https://reviews.llvm.org/D40907 There is an associated patch for compiler-rt. Tested on several 4.x and 3.x kernel releases. llvm-svn: 320109
* [PGO] detect infinite loop and form MST properlyXinliang David Li2017-12-072-38/+32
| | | | | | Differential Revision: http://reviews.llvm.org/D40873 llvm-svn: 320104
* [PGO] Make indirect call promotion a utilityMatthew Simpson2017-12-061-313/+10
| | | | | | | | | | | | | | | | This patch factors out the main code transformation utilities in the pgo-driven indirect call promotion pass and places them in Transforms/Utils. The change is intended to be a non-functional change, letting non-pgo-driven passes share a common implementation with the existing pgo-driven pass. The common utilities are used to conditionally promote indirect call sites to direct call sites. They perform the underlying transformation, and do not consider profile information. The pgo-specific details (e.g., the computation of branch weight metadata) have been left in the indirect call promotion pass. Differential Revision: https://reviews.llvm.org/D40658 llvm-svn: 319963
* Revert r319794: [PGO] detect infinite loop and form MST properly: memory ↵Xinliang David Li2017-12-052-57/+15
| | | | | | leak problem llvm-svn: 319841
* [PGO] detect infinite loop and form MST properlyXinliang David Li2017-12-052-15/+57
| | | | | | Differential Revision: http://reviews.llvm.org/D40702 llvm-svn: 319794
* [msan] Add a fixme note for a minor deficiency.Evgeniy Stepanov2017-12-041-0/+2
| | | | llvm-svn: 319708
* [PGO] Skip counter promotion for infinite loopsXinliang David Li2017-11-301-0/+3
| | | | | | Differential Revision: http://reviews.llvm.org/D40662 llvm-svn: 319462
* MSan: remove an unnecessary cast. NFC for userspace instrumenetation.Alexander Potapenko2017-11-231-3/+3
| | | | llvm-svn: 318923
* [MSan] Move the access address check before the shadow access for that addressAlexander Potapenko2017-11-231-2/+1
| | | | | | | | | | | | MSan used to insert the shadow check of the store pointer operand _after_ the shadow of the value operand has been written. This happens to work in the userspace, as the whole shadow range is always mapped. However in the kernel the shadow page may not exist, so the bug may cause a crash. This patch moves the address check in front of the shadow access. llvm-svn: 318901
* [msan] Don't sanitize "nosanitize" instructionsVitaly Buka2017-11-201-2/+14
| | | | | | | | | | Reviewers: eugenis Subscribers: hiraditya, llvm-commits Differential Revision: https://reviews.llvm.org/D40205 llvm-svn: 318708
* Add heuristics for irreducible loop metadata under PGOHiroshi Yamauchi2017-11-201-1/+12
| | | | | | | | | | | | | | | | | | | | | | | | | Summary: Add the following heuristics for irreducible loop metadata: - When an irreducible loop header is missing the loop header weight metadata, give it the minimum weight seen among other headers. - Annotate indirectbr targets with the loop header weight metadata (as they are likely to become irreducible loop headers after indirectbr tail duplication.) These greatly improve the accuracy of the block frequency info of the Python interpreter loop (eg. from ~3-16x off down to ~40-55% off) and the Python performance (eg. unpack_sequence from ~50% slower to ~8% faster than GCC) due to better register allocation under PGO. Reviewers: davidxl Reviewed By: davidxl Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D39980 llvm-svn: 318693
* [asan] Use dynamic shadow on 32-bit Android, try 2.Evgeniy Stepanov2017-11-201-11/+63
| | | | | | | | | | | | | | | | Summary: This change reverts r318575 and changes FindDynamicShadowStart() to keep the memory range it found mapped PROT_NONE to make sure it is not reused. We also skip MemoryRangeIsAvailable() check, because it is (a) unnecessary, and (b) would fail anyway. Reviewers: pcc, vitalybuka, kcc Subscribers: srhines, kubamracek, mgorny, llvm-commits, hiraditya Differential Revision: https://reviews.llvm.org/D40203 llvm-svn: 318666
* Revert "[asan] Use dynamic shadow on 32-bit Android" and 3 more.Evgeniy Stepanov2017-11-181-63/+11
| | | | | | | | | | | | | | Revert the following commits: r318369 [asan] Fallback to non-ifunc dynamic shadow on android<22. r318235 [asan] Prevent rematerialization of &__asan_shadow. r317948 [sanitizer] Remove unnecessary attribute hidden. r317943 [asan] Use dynamic shadow on 32-bit Android. MemoryRangeIsAvailable() reads /proc/$PID/maps into an mmap-ed buffer that may overlap with the address range that we plan to use for the dynamic shadow mapping. This is causing random startup crashes. llvm-svn: 318575
* [asan] Fix small X86_64 ShadowOffset for non-default shadow scaleWalter Lee2017-11-161-7/+9
| | | | | | | | | | The requirement is that shadow memory must be aligned to page boundaries (4k in this case). Use a closed form equation that always satisfies this requirement. Differential Revision: https://reviews.llvm.org/D39471 llvm-svn: 318421
* [asan] Fix size/alignment issues with non-default shadow scaleWalter Lee2017-11-161-2/+5
| | | | | | | | | | | | | | | Fix a couple places where the minimum alignment/size should be a function of the shadow granularity: - alignment of AllGlobals - the minimum left redzone size on the stack Added a test to verify that the metadata_array is properly aligned for shadow scale of 5, to be enabled when we add build support for testing shadow scale of 5. Differential Revision: https://reviews.llvm.org/D39470 llvm-svn: 318395
* [asan] Fallback to non-ifunc dynamic shadow on android<22.Evgeniy Stepanov2017-11-161-1/+3
| | | | | | | | | | | | Summary: Android < 22 does not support ifunc. Reviewers: pcc Subscribers: srhines, kubamracek, hiraditya, llvm-commits Differential Revision: https://reviews.llvm.org/D40116 llvm-svn: 318369
* [asan] Prevent rematerialization of &__asan_shadow.Evgeniy Stepanov2017-11-151-12/+30
| | | | | | | | | | | | | | | | | | | | Summary: In the mode when ASan shadow base is computed as the address of an external global (__asan_shadow, currently on android/arm32 only), regalloc prefers to rematerialize this value to save register spills. Even in -Os. On arm32 it is rather expensive (2 loads + 1 constant pool entry). This changes adds an inline asm in the function prologue to suppress this behavior. It reduces AsanTest binary size by 7%. Reviewers: pcc, vitalybuka Subscribers: aemerson, kristof.beyls, hiraditya, llvm-commits Differential Revision: https://reviews.llvm.org/D40048 llvm-svn: 318235
* [PM] Port BoundsChecking to the new PM.Chandler Carruth2017-11-142-35/+40
| | | | | | | | | | | Registers it and everything, updates all the references, etc. Next patch will add support to Clang's `-fexperimental-new-pass-manager` path to actually enable BoundsChecking correctly. Differential Revision: https://reviews.llvm.org/D39084 llvm-svn: 318128
* [PM] Refactor BoundsChecking further to prepare it to be exposed both asChandler Carruth2017-11-141-65/+65
| | | | | | | | | | | | a legacy and new PM pass. This essentially moves the class state to parameters and re-shuffles the code to make that reasonable. It also does some minor cleanups along the way and leaves some comments. Differential Revision: https://reviews.llvm.org/D39081 llvm-svn: 318124
* Update some code.google.com linksHans Wennborg2017-11-132-5/+5
| | | | llvm-svn: 318115
* [PowerPC][msan] Update msan to handle changed memory layouts in newer kernelsBill Seurer2017-11-131-1/+1
| | | | | | | | | | | | | In more recent Linux kernels (including those with 47 bit VMAs) the layout of virtual memory for powerpc64 changed causing the memory sanitizer to not work properly. This patch adjusts a bit mask in the memory sanitizer to work on the newer kernels while continuing to work on the older ones as well. This is the non-runtime part of the patch and finishes it. ref: r317802 Tested on several 4.x and 3.x kernel releases. llvm-svn: 318045
* [asan] Use dynamic shadow on 32-bit Android.Evgeniy Stepanov2017-11-101-7/+39
| | | | | | | | | | | | | | | | | | | Summary: The following kernel change has moved ET_DYN base to 0x4000000 on arm32: https://marc.info/?l=linux-kernel&m=149825162606848&w=2 Switch to dynamic shadow base to avoid such conflicts in the future. Reserve shadow memory in an ifunc resolver, but don't use it in the instrumentation until PR35221 is fixed. This will eventually let use save one load per function. Reviewers: kcc Subscribers: aemerson, srhines, kubamracek, kristof.beyls, hiraditya, llvm-commits Differential Revision: https://reviews.llvm.org/D39393 llvm-svn: 317943
* Irreducible loop metadata for more accurate block frequency under PGO.Hiroshi Yamauchi2017-11-021-2/+26
| | | | | | | | | | | | | | | | | | | | | | | Summary: Currently the block frequency analysis is an approximation for irreducible loops. The new irreducible loop metadata is used to annotate the irreducible loop headers with their header weights based on the PGO profile (currently this is approximated to be evenly weighted) and to help improve the accuracy of the block frequency analysis for irreducible loops. This patch is a basic support for this. Reviewers: davidxl Reviewed By: davidxl Subscribers: mehdi_amini, llvm-commits, eraman Differential Revision: https://reviews.llvm.org/D39028 llvm-svn: 317278
* [asan] Upgrade private linkage globals to internal linkage on COFFReid Kleckner2017-10-311-2/+7
| | | | | | | COFF comdats require symbol table entries, which means the comdat leader cannot have private linkage. llvm-svn: 317009
* [Transforms] Fix some Clang-tidy modernize and Include What You Use ↵Eugene Zelenko2017-10-213-121/+215
| | | | | | warnings; other minor fixes (NFC). llvm-svn: 316253
* [Transforms] Fix some Clang-tidy modernize and Include What You Use ↵Eugene Zelenko2017-10-193-302/+449
| | | | | | warnings; other minor fixes (NFC). llvm-svn: 316190
* [PM] Refactor the bounds checking pass to remove a method only called inChandler Carruth2017-10-181-28/+25
| | | | | | one place. llvm-svn: 316135
* Disable gcov instrumentation of functions using funclet-based exception handlingMarco Castelluccio2017-10-131-0/+13
| | | | | | | | | | | | | | Summary: This patch fixes the crash from https://bugs.llvm.org/show_bug.cgi?id=34659 and https://bugs.llvm.org/show_bug.cgi?id=34833. Reviewers: rnk, majnemer Reviewed By: rnk, majnemer Subscribers: majnemer, llvm-commits Differential Revision: https://reviews.llvm.org/D38223 llvm-svn: 315677
* [NFC] Convert OptimizationRemarkEmitter old emit() calls to new closureVivek Pandya2017-10-113-20/+34
| | | | | | | | | | | | | | parameterized emit() calls Summary: This is not functional change to adopt new emit() API added in r313691. Reviewed By: anemet Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D38285 llvm-svn: 315476
* Rename OptimizationDiagnosticInfo.* to OptimizationRemarkEmitter.*Adam Nemet2017-10-093-3/+3
| | | | | | | Sync it up with the name of the class actually defined here. This has been bothering me for a while... llvm-svn: 315249
* Directly return promoted direct call instead of rely on stripPointerCast.Dehao Chen2017-10-061-8/+13
| | | | | | | | | | | | | | Summary: stripPointerCast is not reliably returning the value that's being type-casted. Instead it may look further at function attributes to further propagate the value. Instead of relying on stripPOintercast, the more reliable solution is to directly use the pointer to the promoted direct call. Reviewers: tejohnson, davidxl Reviewed By: tejohnson Subscribers: llvm-commits, sanjoy Differential Revision: https://reviews.llvm.org/D38603 llvm-svn: 315077
* Don't move llvm.localescape outside the entry block in the GCOV profiling passSylvestre Ledru2017-09-261-1/+11
| | | | | | | | | | | | | | | | | Summary: This fixes https://bugs.llvm.org/show_bug.cgi?id=34714. Patch by Marco Castelluccio Reviewers: rnk Reviewed By: rnk Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D38224 llvm-svn: 314201
* Add section headers to SpecialCaseListsVlad Tsyrklevich2017-09-251-5/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: Sanitizer blacklist entries currently apply to all sanitizers--there is no way to specify that an entry should only apply to a specific sanitizer. This is important for Control Flow Integrity since there are several different CFI modes that can be enabled at once. For maximum security, CFI blacklist entries should be scoped to only the specific CFI mode(s) that entry applies to. Adding section headers to SpecialCaseLists allows users to specify more information about list entries, like sanitizer names or other metadata, like so: [section1] fun:*fun1* [section2|section3] fun:*fun23* The section headers are regular expressions. For backwards compatbility, blacklist entries entered before a section header are put into the '[*]' section so that blacklists without sections retain the same behavior. SpecialCaseList has been modified to also accept a section name when matching against the blacklist. It has also been modified so the follow-up change to clang can define a derived class that allows matching sections by SectionMask instead of by string. Reviewers: pcc, kcc, eugenis, vsk Reviewed By: eugenis, vsk Subscribers: vitalybuka, llvm-commits Differential Revision: https://reviews.llvm.org/D37924 llvm-svn: 314170
* [MSan] Disable sanitization for __sanitizer_dtor_callback.Matt Morehouse2017-09-201-0/+1
| | | | | | | | | | | | | | | | Summary: Eliminate unnecessary instrumentation at __sanitizer_dtor_callback call sites. Fixes https://github.com/google/sanitizers/issues/861. Reviewers: eugenis, kcc Reviewed By: eugenis Subscribers: vitalybuka, llvm-commits, cfe-commits, hiraditya Differential Revision: https://reviews.llvm.org/D38063 llvm-svn: 313831
* [gcov] Emit errors when opening the notes file failsReid Kleckner2017-09-181-0/+6
| | | | | | | | No time to write a test case, on to the next bug. =P Discovered while investigating PR34659 llvm-svn: 313571
* Add options to dump PGO counts in text.Hiroshi Yamauchi2017-09-131-20/+38
| | | | | | | | | | | | | | | | | Summary: Added text options to -pgo-view-counts and -pgo-view-raw-counts that dump block frequency and branch probability info in text. This is useful when the graph is very large and complex (the dot command crashes, lines/edges too close to tell apart, hard to navigate without textual search) or simply when text is preferred. Reviewers: davidxl Reviewed By: davidxl Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D37776 llvm-svn: 313159
* [sanitizer-coverage] call appendToUsed once per module, not once per ↵Kostya Serebryany2017-09-091-8/+8
| | | | | | function (which is too slow) llvm-svn: 312855
* [SanitizeCoverage] Enable stack-depth coverage for -fsanitize=fuzzerMatt Morehouse2017-08-301-9/+21
| | | | | | | | | | | | | | | | | | Summary: - Don't sanitize __sancov_lowest_stack. - Don't instrument leaf functions. - Add CoverageStackDepth to Fuzzer and FuzzerNoLink. - Only enable on Linux. Reviewers: vitalybuka, kcc, george.karpenkov Reviewed By: kcc Subscribers: kubamracek, cfe-commits, llvm-commits, hiraditya Differential Revision: https://reviews.llvm.org/D37156 llvm-svn: 312185
* Revert "[SanitizeCoverage] Enable stack-depth coverage for -fsanitize=fuzzer"Matt Morehouse2017-08-291-21/+9
| | | | | | This reverts r312026 due to bot breakage. llvm-svn: 312047
* [SanitizeCoverage] Enable stack-depth coverage for -fsanitize=fuzzerMatt Morehouse2017-08-291-9/+21
| | | | | | | | | | | | | | | | | | Summary: - Don't sanitize __sancov_lowest_stack. - Don't instrument leaf functions. - Add CoverageStackDepth to Fuzzer and FuzzerNoLink. - Disable stack depth tracking on Mac. Reviewers: vitalybuka, kcc, george.karpenkov Reviewed By: kcc Subscribers: kubamracek, cfe-commits, llvm-commits, hiraditya Differential Revision: https://reviews.llvm.org/D37156 llvm-svn: 312026
OpenPOWER on IntegriCloud