summaryrefslogtreecommitdiffstats
path: root/llvm/lib/Transforms/IPO
Commit message (Collapse)AuthorAgeFilesLines
* [globalopt] Don't emit DWARF fragments for membersDavid Spickett2020-05-181-6/+10
| | | | | | | | | | | | | | | | of a struct that cover the whole struct This can happen when the rest of the members of are zero length. Following the same pattern applied to the SROA pass in: d7f6f1636d53c3e2faf55cdf20fbb44a1a149df1 Fixes: https://bugs.llvm.org/show_bug.cgi?id=45335 Differential Revision: https://reviews.llvm.org/D78720 (cherry picked from commit 3929429347d398773577b79f7fdb780d4f7ed887)
* [PassManagerBuilder] Remove global extension when a plugin is unloadedElia Geretto2020-01-291-8/+33
| | | | | | | | | | | | This commit fixes PR39321. GlobalExtensions is not guaranteed to be destroyed when optimizer plugins are unloaded. If it is indeed destroyed after a plugin is dlclose-d, the destructor of the corresponding ExtensionFn is not mapped anymore, causing a call to unmapped memory during destruction. This commit guarantees that extensions coming from external plugins are removed from GlobalExtensions when the plugin is unloaded if GlobalExtensions has not been destroyed yet. Differential Revision: https://reviews.llvm.org/D71959 (cherry picked from commit ab2300bc154f7bed43f85f74fd3fe31be71d90e0)
* [Attributor] AAValueConstantRange: Value range analysis using constant rangeHideto Ueno2020-01-151-7/+499
| | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: This patch introduces `AAValueConstantRange`, which answers a possible range for integer value in a specific program point. One of the motivations is propagating existing `range` metadata. (I think we need to change the situation that `range` metadata cannot be put to Argument). The state is a tuple of `ConstantRange` and it is initialized to (known, assumed) = ([-∞, +∞], empty). Currently, AAValueConstantRange is created in `getAssumedConstant` method when `AAValueSimplify` returns `nullptr`(worst state). Supported - BinaryOperator(add, sub, ...) - CmpInst(icmp eq, ...) - !range metadata `AAValueConstantRange` is not intended to extend to polyhedral range value analysis. Reviewers: jdoerfert, sstefan1 Reviewed By: jdoerfert Subscribers: phosek, davezarzycki, baziotis, hiraditya, javed.absar, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D71620
* [NewPM] Port MergeFunctions passNikita Popov2020-01-142-15/+34
| | | | | | | | | | This ports the MergeFunctions pass to the NewPM. This was rather straightforward, as no analyses are used. Additionally MergeFunctions needs to be conditionally enabled in the PassBuilder, but I left that part out of this patch. Differential Revision: https://reviews.llvm.org/D72537
* [ThinLTO/WPD] Remove an overly-aggressive assertTeresa Johnson2020-01-141-8/+3
| | | | | | | | | | | | | | | | | | | | Summary: An assert added to the index-based WPD was trying to verify that we only have multiple vtables for a given guid when they are all non-external linkage. This is too conservative because we may have multiple external vtable with the same guid when they are in comdat. Remove the assert, as we don't have comdat information in the index, the linker should issue an error in this case. See discussion on D71040 for more information. Reviewers: evgeny777, aganea Subscribers: mehdi_amini, inglorion, hiraditya, steven_wu, dexonsmith, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D72648
* Removed PointerUnion3 and PointerUnion4 aliases in favor of the variadic ↵Dmitri Gribenko2020-01-141-1/+1
| | | | template
* Revert "Recommit "[GlobalOpt] Pass DTU to removeUnreachableBlocks instead of ↵Florian Hahn2020-01-141-3/+7
| | | | | | | | | | | recomputing."" This reverts commit a03d7b0f24b65d69721dbbbc871df0629efcf774. As discussed in D68298, this causes a compile-time regression, in case the DTs requested are not used elsewhere in GlobalOpt. We should only get the DTs if they are available here, but this seems not possible with the legacy pass manager from a module pass.
* [ThinLTO/WPD] Fix index-based WPD for alias vtablesTeresa Johnson2020-01-131-1/+1
| | | | | | | | | | | | | | | | | Summary: A recent fix in D69452 fixed index based WPD in the presence of available_externally vtables. It added a cast of the vtable def summary to a GlobalVarSummary. However, in some cases one def may be an alias, in which case we need to get the base object before casting, otherwise we will crash. Reviewers: evgeny777, steven_wu, aganea Subscribers: mehdi_amini, inglorion, hiraditya, dexonsmith, arphaman, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D71040
* [Attributor][FIX] Carefully change invokes to calls (after manifest)Johannes Doerfert2020-01-081-84/+35
| | | | | | Before we manually inserted unreachable early but that could lead to broken PHI nodes. Now we use the existing late modification functionality.
* [Attributor][FIX] Avoid dangling value pointers during code modificationJohannes Doerfert2020-01-081-2/+3
| | | | | | | When we replace instructions with unreachable we delete instructions. We now avoid dangling pointers to those deleted instructions in the `ToBeChangedToUnreachableInsts` set. Other modification collections might need to be updated in the future as well.
* Add Triple::isX86()Fangrui Song2020-01-061-2/+1
| | | | | | Reviewed By: craig.topper, skan Differential Revision: https://reviews.llvm.org/D72247
* [NFC] Fix trivial typos in commentsJames Henderson2020-01-062-4/+4
| | | | | | | | Reviewed By: jhenderson Differential Revision: https://reviews.llvm.org/D72143 Patch by Kazuaki Ishizaki.
* [Transforms][GlobalSRA] huge array causes long compilation time and huge ↵Alexey Lapshin2020-01-041-65/+77
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | memory usage. Summary: For artificial cases (huge array, few usages), Global SRA optimization creates a lot of redundant data. It creates an instance of GlobalVariable for each array element. For huge array, that means huge compilation time and huge memory usage. Following example compiles for 10 minutes and requires 40GB of memory. namespace { char LargeBuffer[64 * 1024 * 1024]; } int main ( void ) { LargeBuffer[0] = 0; printf("\n "); return LargeBuffer[0] == 0; } The fix is to avoid Global SRA for large arrays. Reviewers: craig.topper, rnk, efriedma, fhahn Reviewed By: rnk Subscribers: xbolva00, lebedev.ri, lkail, merge_guards_bot, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D71993
* [Attributor][FIX] Allow dead users of rewritten functionJohannes Doerfert2020-01-031-13/+15
| | | | | | | If we replace a function with a new one because we rewrite the signature, dead users may still refer to the old version. With this patch we reuse the code that deals with dead functions, which the old versions are, to avoid problems.
* [Attributor][NFC] Unify the way we delete dead functionsJohannes Doerfert2020-01-031-13/+14
|
* [Attributor][FIX] Don't crash on ptr2int/int2ptr instructionsJohannes Doerfert2020-01-031-1/+2
| | | | | An integer isn't allowed in getAlignmentForValue so we need to stop at a ptr2int instruction during exploration.
* [Attributor][FIX] Do not derive nonnull and dereferenceable w/o accessJohannes Doerfert2020-01-031-14/+0
| | | | | | | An inbounds GEP results in poison if the value is not "inbounds", not in UB. We accidentally derived nonnull and dereferenceable from these inbounds GEPs even in the absence of accesses that would make the poison to UB.
* [Attributor][FIX] Return CHANGED once a pessimistic fixpoint is reached.Johannes Doerfert2020-01-031-1/+2
|
* Revert "[Attributor] AAValueConstantRange: Value range analysis using ↵Hideto Ueno2020-01-031-499/+7
| | | | | | constant range" This reverts commit e9963034314edf49a12ea5e29f694d8f9f52734a.
* [Attributor] AAValueConstantRange: Value range analysis using constant rangeHideto Ueno2020-01-011-7/+499
| | | | | | | | | | | | | | | | | | | | | This patch introduces `AAValueConstantRange`, which answers a possible range for integer value in a specific program point. One of the motivations is propagating existing `range` metadata. (I think we need to change the situation that `range` metadata cannot be put to Argument). The state is a tuple of `ConstantRange` and it is initialized to (known, assumed) = ([-∞, +∞], empty). Currently, AAValueConstantRange is created when AAValueSimplify cannot simplify the value. Supported - BinaryOperator(add, sub, ...) - CmpInst(icmp eq, ...) - !range metadata `AAValueConstantRange` is not intended to extend to polyhedral range value analysis. Reviewed By: jdoerfert Differential Revision: https://reviews.llvm.org/D71620
* [Attributor] Suppress unused warnings when assertions are disabled. NFCIlya Biryukov2019-12-311-0/+2
|
* [Attributor] Function signature rewrite infrastructureJohannes Doerfert2019-12-311-0/+259
| | | | | | | | | | | | | | | As part of the Attributor manifest we want to change the signature of functions. This patch introduces a fairly generic interface to do so. As a first, very simple, use case, we remove unused arguments. A second use case, pointer privatization, will be committed with this patch as well. A lot of the code and ideas are taken from argument promotion and we run all argument promotion tests through this framework as well. Reviewed By: uenoku Differential Revision: https://reviews.llvm.org/D68765
* [Attributor] Propagate known align from arguments to call sites argumentsJohannes Doerfert2019-12-311-0/+12
| | | | | | | Since the information is known we can simply use it at the call site. This is especially useful for callbacks but also helps regular calls. The test changes are mechanical.
* [Attributor] Use abstract call sites to determine associated argumentsJohannes Doerfert2019-12-311-2/+92
| | | | | | | | | | | | | | | | | | | | | | This is the second step after D67871 to make use of abstract call sites. In this patch the argument we associate with a abstract call site argument can be the one in the callback callee instead of the one in the callback broker. Caveat: We cannot allow no-alias arguments for problematic callbacks: As described in [1], adding no-alias (or restrict) to arguments could break synchronization as the synchronization effect, e.g., a barrier, does not "alias" with the pointer anymore. This disables no-alias annotation for potentially problematic arguments until we implement the fix described in [1]. Reviewed By: uenoku Differential Revision: https://reviews.llvm.org/D68008 [1] Compiler Optimizations for OpenMP, J. Doerfert and H. Finkel, International Workshop on OpenMP 2018, http://compilers.cs.uni-saarland.de/people/doerfert/par_opt18.pdf
* [Attributor] Annotate the memory behavior of call site argumentsJohannes Doerfert2019-12-311-2/+7
| | | | | | | | | Especially for callbacks, annotating the call site arguments is important. Doing so exposed a too strong dependence of AAMemoryBehavior on AANoCapture since we handle the case of potentially captured pointers explicitly. The changes to the tests are all mechanical.
* [Attributor] Use `changeUseAfterManifest` in AAValueSimplify manifestHideto Ueno2019-12-301-4/+2
| | | | | | | | | | | | | | Summary: This patch makes `AAValueSimplify` use `changeUsesAfterManifest` in `manifest`. This will invoke simple folding after the manifest. Reviewers: jdoerfert, sstefan1 Reviewed By: jdoerfert Subscribers: hiraditya, arphaman, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D71972
* [Attributor] AAUndefinedBehavior: Check for branches on undef value.Hideto Ueno2019-12-291-53/+139
| | | | | | | | | | | | | | | A branch is considered UB if it depends on an undefined / uninitialized value. At this point this handles simple UB branches in the form: `br i1 undef, ...` We query `AAValueSimplify` to get a value for the branch condition, so the branch can be more complicated than just: `br i1 undef, ...`. Patch By: Stefanos Baziotis (@baziotis) Reviewers: jdoerfert, sstefan1, uenoku Reviewed By: uenoku Differential Revision: https://reviews.llvm.org/D71799
* [Attributor] Add helper to change an instruction to `unreachable` instHideto Ueno2019-12-271-5/+4
| | | | | | | | | | | | | | Summary: Calling `changeToUnreachable` in `manifest` from different places might cause really unpredictable problems. As other deleting functions are doing, we need to change these instructions after all `manifest`. Reviewers: jdoerfert, sstefan1 Reviewed By: jdoerfert Subscribers: hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D71910
* [Attributor] Reach optimistic fixpoint in AAValueSimplify when the value is ↵Hideto Ueno2019-12-251-1/+9
| | | | | | | | | | | | | | | | | constant or undef Summary: As discussed in D71799, we have found that it is more useful to reach an optimistic fixpoint in AAValueSimpify when the value is constant or undef. Reviewers: jdoerfert, sstefan1 Reviewed By: jdoerfert Subscribers: baziotis, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D71852
* [Attributor] UB Attribute now handles all instructions that access memory ↵Johannes Doerfert2019-12-241-33/+45
| | | | | | | | | | | | | | | | | | | | | | | | | through a pointer Summary: Follow-up on: https://reviews.llvm.org/D71435 We basically use `checkForAllInstructions` to loop through all the instructions in a function that access memory through a pointer: load, store, atomicrmw, atomiccmpxchg Note that we can now use the `getPointerOperand()` that gets us the pointer operand for an instruction that belongs to the aforementioned set. Question: This function returns `nullptr` if the instruction is `volatile`. Why? Guess: Because if it is volatile, we don't want to do any transformation to it. Another subtle point is that I had to add AtomicRMW, AtomicCmpXchg to `initializeInformationCache()`. Following `checkAllInstructions()` path, that seemed the most reasonable place to add it and correct the fact that these instructions were ignored (they were not in `OpcodeInstMap` etc.). Is that ok? Reviewers: jdoerfert, sstefan1 Reviewed By: jdoerfert, sstefan1 Subscribers: hiraditya, jfb, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D71787
* [Attributor] Function level undefined behavior attributeJohannes Doerfert2019-12-241-0/+97
| | | | | | | | | | | | | | | _Eventually_, this attribute will be assigned to a function if it contains undefined behavior. As a first small step, I tried to make it loop through the load instructions in a function (eventually, the plan is to check if a load instructions causes undefined behavior, because e.g. dereferences a null pointer - Also eventually, this won't happen in initialize() but in updateImpl()). Patch By: Stefanos Baziotis (@baziotis) Reviewed By: jdoerfert Differential Revision: https://reviews.llvm.org/D71435
* [Transforms] Fixes -Wrange-loop-analysis warningsMark de Wever2019-12-221-1/+1
| | | | | | This avoids new warnings due to D68912 adds -Wrange-loop-analysis to -Wall. Differential Revision: https://reviews.llvm.org/D71810
* HotColdSplitting: Do not outline within noreturn functionsVedant Kumar2019-12-191-0/+5
| | | | | | | A function marked `noreturn` may contain unreachable terminators: these should not be considered cold, as the function may be a trampoline. rdar://58068594
* [Attributor] H2S fix.Stefan Stipanovic2019-12-171-2/+2
| | | | | | | | | | Summary: Fixing issues that were noticed in D71521 Reviewers: jdoerfert, lebedev.ri, uenoku Subscribers: Differential Revision: https://reviews.llvm.org/D71564
* [Attributor][NFC] Clang format the AttributorJohannes Doerfert2019-12-161-11/+11
| | | | | | | The Attributor is always kept formatted so diffs are cleaner. Sometime we get out of sync for various reasons so we need to format the file once in a while.
* [Attributor] Annotate call sites of declarations with a callbackJohannes Doerfert2019-12-131-1/+2
| | | | | Even if a declaration is called, if there is a callback we might need the information during CG-SCC traversal (D70767).
* [Attributor][NFC] Simplify debug printing for abstract attributesJohannes Doerfert2019-12-131-10/+37
| | | | This also fixes a type in the debug printing of AANoAlias.
* [Attributor] Only replace instruction operandsJohannes Doerfert2019-12-131-4/+18
| | | | | | | | This was part of D70767. When we replace the value of (call/invoke) instructions we do not want to disturb the old call graph so we will only replace instruction uses until we get rid of the old PM. Accepted as part of D70767.
* [profile] Fix a crash when -fprofile-remapping-file= triggers an errorFangrui Song2019-12-131-1/+2
| | | | | | Reviewed By: wmi Differential Revision: https://reviews.llvm.org/D71485
* [Attributor][FIX] Do treat byval arguments specialJohannes Doerfert2019-12-121-12/+60
| | | | | | | | | | | | | | When we reason about the pointer argument that is byval we actually reason about a local copy of the value passed at the call site. This was not the case before and we wrongly introduced attributes based on the surrounding function. AAMemoryBehaviorArgument, AAMemoryBehaviorCallSiteArgument and AANoCaptureCallSiteArgument are made aware of byval now. The code to skip "subsuming positions" for reasoning follows a common pattern and we should refactor it. A TODO was added. Discovered by @efriedma as part of D69748.
* [Matrix] Add first set of matrix intrinsics and initial lowering pass.Florian Hahn2019-12-121-0/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is the first patch adding an initial set of matrix intrinsics and a corresponding lowering pass. This has been discussed on llvm-dev: http://lists.llvm.org/pipermail/llvm-dev/2019-October/136240.html The first patch introduces four new intrinsics (transpose, multiply, columnwise load and store) and a LowerMatrixIntrinsics pass, that lowers those intrinsics to vector operations. Matrixes are embedded in a 'flat' vector (e.g. a 4 x 4 float matrix embedded in a <16 x float> vector) and the intrinsics take the dimension information as parameters. Those parameters need to be ConstantInt. For the memory layout, we initially assume column-major, but in the RFC we also described how to extend the intrinsics to support row-major as well. For the initial lowering, we split the input of the intrinsics into a set of column vectors, transform those column vectors and concatenate the result columns to a flat result vector. This allows us to lower the intrinsics without any shape propagation, as mentioned in the RFC. In follow-up patches, we plan to submit the following improvements: * Shape propagation to eliminate the embedding/splitting for each intrinsic. * Fused & tiled lowering of multiply and other operations. * Optimization remarks highlighting matrix expressions and costs. * Generate loops for operations on large matrixes. * More general block processing for operation on large vectors, exploiting shape information. We would like to add dedicated transpose, columnwise load and store intrinsics, even though they are not strictly necessary. For example, we could instead emit a large shufflevector instruction instead of the transpose. But we expect that to (1) become unwieldy for larger matrixes (even for 16x16 matrixes, the resulting shufflevector masks would be huge), (2) risk instcombine making small changes, causing us to fail to detect the transpose, preventing better lowerings For the load/store, we are additionally planning on exploiting the intrinsics for better alias analysis. Reviewers: anemet, Gerolf, reames, hfinkel, andrew.w.kaylor, efriedma, rengolin Reviewed By: anemet Differential Revision: https://reviews.llvm.org/D70456
* [Attributor][NFC] Fix comments and unnecessary commaHideto Ueno2019-12-121-6/+7
|
* [Attributor] [NFC] Use `checkForAllUses` helpr in ↵Hideto Ueno2019-12-121-35/+18
| | | | | | | | | | | | | | | | `AAHeapToStackImpl::updateImpl` Summary: Remove `Worklist` iteration and make use `checkForAllUses`. There is no test chage. Reviewers: sstefan1, jdoerfert Reviewed By: jdoerfert Subscribers: hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D71352
* [Attributor][NFC] Refactoring `AANoFreeArgument::updateImpl`Hideto Ueno2019-12-121-38/+15
| | | | | | | | | | | | | | Summary: Refactoring `AANoFreeArgument::updateImpl`. There is no test change. Reviewers: sstefan1, jdoerfert Reviewed By: sstefan1 Subscribers: hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D71349
* [AutoFDO] Statistic for context sensitive profile guided inliningWenlei He2019-12-111-3/+40
| | | | | | | | | | | | Summary: AutoFDO compilation has two places that do inlining - the sample profile loader that does inlining with context sensitive profile, and the regular inliner as CGSCC pass. Ideally we want most inlining to come from sample profile loader as that is driven by context sensitive profile and also retains context sensitivity after inlining. However the reality is most of the inlining actually happens during regular inliner. To track the number of inline instances from sample profile loader and help move more inlining to sample profile loader, I'm adding statistics and optimization remarks for sample profile loader's inlining. Reviewers: wmi, davidxl Subscribers: hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D70584
* [MergeFuncs] Remove incorrect attribute copyingNikita Popov2019-12-111-22/+4
| | | | | | | | | | | | | | | Fix for https://bugs.llvm.org/show_bug.cgi?id=44236. This code was originally introduced in rG36512330041201e10f5429361bbd79b1afac1ea1. However, the attribute copying was done in the wrong place (in general call replacement, not thunk generation) and a proper fix was implemented in D12581. Previously this code was just unnecessary but harmless (because FunctionComparator ensured that the attributes of the two functions are exactly the same), but since byval was changed to accept a type this copying is actively wrong and may result in malformed IR. Differential Revision: https://reviews.llvm.org/D71173
* [WPD] Remove unused parameter (NFC)Teresa Johnson2019-12-061-4/+3
| | | | Remove unused parameter.
* [AutoFDO] Inline replay for cold/small callees from sample profile loaderWenlei He2019-12-061-3/+35
| | | | | | | | | | | | | | | Summary: Sample profile loader of AutoFDO tries to replay previous inlining using context sensitive profile. The replay only repeats inlining if the call site block is hot. As a result it punts inlining of small functions, some of which can be beneficial for size, and will still be inlined by CSGCC inliner later. The oscillation between sample profile loader's inlining and regular CGSSC inlining cause unnecessary loss of context-sensitive profile. It doesn't have much impact for inline decision itself, but it negatively affects post-inline profile quality as CGSCC inliner have to scale counts which is not as accurate as the original context sensitive profile, and bad post-inline profile can misguide code layout. This change added regular Inline Cost calculation for sample profile loader, so we can inline small functions upfront under switch -sample-profile-inline-size. In addition -sample-profile-cold-inline-threshold is added so we can tune the separate size threshold - currently the default is chosen to be the same as regular inliner's cold call-site threshold. Reviewers: wmi, davidxl Subscribers: hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D70750
* [ThinLTO] Add option to disable readonly/writeonly attribute propagationTeresa Johnson2019-12-051-12/+1
| | | | | | | | | | | | | | | | | | | | | Summary: Add an option to allow the attribute propagation on the index to be disabled, to allow a workaround for issues (such as that fixed by D70977). Also move the setting of the WithAttributePropagation flag on the index into propagateAttributes(), and remove some old stale code that predated this flag and cleared the maybe read/write only bits when we need to disable the propagation (previously only when importing disabled, now also when the new option disables it). Reviewers: evgeny777, steven_wu Subscribers: mehdi_amini, inglorion, hiraditya, dexonsmith, arphaman, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D70984
* [AutoFDO] Top-down Inlining for specialization with context-sensitive profileWenlei He2019-12-051-9/+46
| | | | | | | | | | | | | | | | | Summary: AutoFDO's sample profile loader processes function in arbitrary source code order, so if I change the order of two functions in source code, the inline decision can change. This also prevented the use of context-sensitive profile to do specialization while inlining. This commit enforces SCC top-down order for sample profile loader. With this change, we can now do specialization, as illustrated by the added test case: Say if we have A->B->C and D->B->C call path, we want to inline C into B when root inliner is B, but not when root inliner is A or D, this is not possible without enforcing top-down order. E.g. Once C is inlined into B, A and D can only choose to inline (B->C) as a whole or nothing, but what we want is only inline B into A and D, not its recursive callee C. If we process functions in top-down order, this is no longer a problem, which is what this commit is doing. This change is guarded with a new switch "-sample-profile-top-down-load" for tuning, and it depends on D70653. Eventually, top-down can be the default order for sample profile loader. Reviewers: wmi, davidxl Subscribers: hiraditya, llvm-commits, tejohnson Tags: #llvm Differential Revision: https://reviews.llvm.org/D70655
OpenPOWER on IntegriCloud