summaryrefslogtreecommitdiffstats
path: root/llvm/lib/Transforms/IPO
Commit message (Collapse)AuthorAgeFilesLines
...
* LowerTypeTests: Generate simpler IR for br(llvm.type.test, then, else).Peter Collingbourne2017-06-071-2/+19
| | | | | | | | | | | | | This makes it so that the code quality for CFI checks when compiling with -O2 and linking with --lto-O0 is similar to that of the rest of the code. Reduces the size of a chrome binary built with -O2/--lto-O0 by about 750KB. Differential Revision: https://reviews.llvm.org/D33925 llvm-svn: 304921
* Sort the remaining #include lines in include/... and lib/....Chandler Carruth2017-06-0611-14/+13
| | | | | | | | | | | | | | | | | | | | | | | | | I did this a long time ago with a janky python script, but now clang-format has built-in support for this. I fed clang-format every line with a #include and let it re-sort things according to the precise LLVM rules for include ordering baked into clang-format these days. I've reverted a number of files where the results of sorting includes isn't healthy. Either places where we have legacy code relying on particular include ordering (where possible, I'll fix these separately) or where we have particular formatting around #include lines that I didn't want to disturb in this patch. This patch is *entirely* mechanical. If you get merge conflicts or anything, just ignore the changes in this patch and run clang-format over your #include lines in the files. Sorry for any noise here, but it is important to keep these things stable. I was seeing an increasing number of patches with irrelevant re-ordering of #include lines because clang-format was used. This patch at least isolates that churn, makes it easy to skip when resolving conflicts, and gets us to a clean baseline (again). llvm-svn: 304787
* Revert "[CFI] Remove LinkerSubsectionsViaSymbols."Evgeniy Stepanov2017-06-031-12/+23
| | | | | | This reverts commit r304582: breaks cfi-devirt :: anon-namespace.cpp on Darwin. llvm-svn: 304626
* Fix debug build test failureXinliang David Li2017-06-021-2/+3
| | | | llvm-svn: 304600
* [PartialInlining] Minor cost anaysis tuningXinliang David Li2017-06-021-9/+56
| | | | | | Also added a test option and 2 cost analysis related tests. llvm-svn: 304599
* FunctionAttrs: Skip it if the effective SCC (ignoring optnone functions) is ↵David Blaikie2017-06-021-0/+4
| | | | | | | | | empty Minor optimization but mostly simplifies my debugging so I'm not dealing with empty SCCNodeSets while investigating issues in this optimization. llvm-svn: 304597
* [CFI] Remove LinkerSubsectionsViaSymbols.Evgeniy Stepanov2017-06-021-23/+12
| | | | | | | | | | Since D17854 LinkerSubsectionsViaSymbols is unnecessary. It is interfering with ThinLTO implementation of CFI-ICall, where the aliases used on the !LinkerSubsectionsViaSymbols branch are needed to export jump tables to ThinLTO backends. llvm-svn: 304582
* Skip CFI for dead functions.Evgeniy Stepanov2017-06-021-2/+1
| | | | | | Differential Revision: https://reviews.llvm.org/D33805 llvm-svn: 304578
* [PM] GVNSink is off by default, fix an obvious typo.Davide Italiano2017-06-011-1/+1
| | | | llvm-svn: 304497
* (NFC) Track global summary liveness in GVFlags.Evgeniy Stepanov2017-06-011-60/+47
| | | | | | | | Replace GVFlags::LiveRoot with GVFlags::Live and use that instead of all the DeadSymbols sets. This is refactoring in order to make liveness information available in the RegularLTO pipeline. llvm-svn: 304466
* [ThinLTO] Migrate ThinLTOBitcodeWriter to the new PM.Tim Shen2017-06-011-7/+13
| | | | | | | | | | | | Summary: Also see D33429 for other ThinLTO + New PM related changes. Reviewers: davide, chandlerc, tejohnson Subscribers: mehdi_amini, Prazek, cfe-commits, inglorion, llvm-commits, eraman Differential Revision: https://reviews.llvm.org/D33525 llvm-svn: 304378
* [PartialInlining] Reduce outlining overhead by removing unneeded live-out(s)Xinliang David Li2017-06-011-0/+23
| | | | | | Differential Revision: http://reviews.llvm.org/D33694 llvm-svn: 304375
* [PartialInlining] Replace delete with unique_ptr in ↵Vitaly Buka2017-05-271-7/+3
| | | | | | | | | | | | | | computeCallsiteToProfCountMap Reviewers: davidxl Reviewed By: davidxl Subscribers: vsk, llvm-commits Differential Revision: https://reviews.llvm.org/D33220 llvm-svn: 304064
* PMB: Run the whole-program-devirt pass during LTO at --lto-O0.Peter Collingbourne2017-05-261-0/+6
| | | | | | | | | | The whole-program-devirt pass needs to run at -O0 because only it knows about the llvm.type.checked.load intrinsic: it needs to both lower the intrinsic itself and handle it in the summary. Differential Revision: https://reviews.llvm.org/D33571 llvm-svn: 304019
* [GVNSink] GVNSink passJames Molloy2017-05-251-0/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch provides an initial prototype for a pass that sinks instructions based on GVN information, similar to GVNHoist. It is not yet ready for commiting but I've uploaded it to gather some initial thoughts. This pass attempts to sink instructions into successors, reducing static instruction count and enabling if-conversion. We use a variant of global value numbering to decide what can be sunk. Consider: [ %a1 = add i32 %b, 1 ] [ %c1 = add i32 %d, 1 ] [ %a2 = xor i32 %a1, 1 ] [ %c2 = xor i32 %c1, 1 ] \ / [ %e = phi i32 %a2, %c2 ] [ add i32 %e, 4 ] GVN would number %a1 and %c1 differently because they compute different results - the VN of an instruction is a function of its opcode and the transitive closure of its operands. This is the key property for hoisting and CSE. What we want when sinking however is for a numbering that is a function of the *uses* of an instruction, which allows us to answer the question "if I replace %a1 with %c1, will it contribute in an equivalent way to all successive instructions?". The (new) PostValueTable class in GVN provides this mapping. This pass has some shown really impressive improvements especially for codesize already on internal benchmarks, so I have high hopes it can replace all the sinking logic in SimplifyCFG. Differential revision: https://reviews.llvm.org/D24805 llvm-svn: 303850
* [PartialInlining] Add internal options to enable partial inlining in pass ↵Xinliang David Li2017-05-221-2/+8
| | | | | | | | | | | pipeline (off by default) 1. Legacy: -mllvm -enable-partial-inlining 2. New: -mllvm -enable-npm-partial-inlining -fexperimental-new-pass-manager Differential Revision: http://reviews.llvm.org/D33382 llvm-svn: 303567
* Revert "ThinLTO: Verify bitcode before lauching the ThinLTOCodeGenerator."Adrian Prantl2017-05-191-4/+1
| | | | | | This reverts commit r303438 while deliberating buildbot breakage. llvm-svn: 303467
* ThinLTO: Verify bitcode before lauching the ThinLTOCodeGenerator.Adrian Prantl2017-05-191-1/+4
| | | | | | | | rdar://problem/31233625 Differential Revision: https://reviews.llvm.org/D33151 llvm-svn: 303438
* Fix memory leakXinliang David Li2017-05-151-0/+4
| | | | llvm-svn: 303126
* Fix test failure on windows -- do not return deleted funcXinliang David Li2017-05-141-2/+8
| | | | llvm-svn: 302999
* [PartialInlining] Profile based cost analysisXinliang David Li2017-05-121-45/+363
| | | | | | | | | | | | Implemented frequency based cost/saving analysis and related options. The pass is now in a state ready to be turne on in the pipeline (in follow up). Differential Revision: http://reviews.llvm.org/D32783 llvm-svn: 302967
* Remove spurious cast of nullptr. NFC.Serge Guelton2017-05-111-1/+1
| | | | | | Conversion rules allow automatic casting of nullptr to any pointer type. llvm-svn: 302780
* Ensure non-null ProfileSummaryInfo passed to ModuleSummaryIndex builderTeresa Johnson2017-05-101-1/+3
| | | | | | | | | | | | This fixes a ubsan bot failure after r302597, which made getProfileCount non-static, but ended up invoking it on a null ProfileSummaryInfo object in some cases from buildModuleSummaryIndex. Most testing passed because the non-static getProfileCount currently doesn't access any member variables, but I found this when testing a follow on patch (D32877) that adds a member variable access. llvm-svn: 302705
* [ProfileSummary] Make getProfileCount a non-static member function.Easwaran Raman2017-05-092-3/+3
| | | | | | | | | | This change is required because the notion of count is different for sample profiling and getProfileCount will need to determine the underlying profile type. Differential revision: https://reviews.llvm.org/D33012 llvm-svn: 302597
* FunctionImport: Simplify function llvm::thinLTOInternalizeModule. NFCI.Peter Collingbourne2017-05-091-10/+5
| | | | llvm-svn: 302595
* [PartialInlining] Capture by reference rather than by value.Davide Italiano2017-05-081-3/+3
| | | | llvm-svn: 302464
* Re-apply r302108, "IR: Use pointers instead of GUIDs to represent edges in ↵Peter Collingbourne2017-05-043-56/+45
| | | | | | | | the module summary. NFCI." with a fix for the clang backend. llvm-svn: 302176
* Revert "IR: Use pointers instead of GUIDs to represent edges in the module ↵Eric Liu2017-05-043-45/+56
| | | | | | | | | | summary. NFCI." This reverts commit r302108. This causes crash in clang bootstrap with LTO. Contacted the auther in the original commit. llvm-svn: 302140
* [ArgPromotion] Fix a truncated variableMartin Storsjo2017-05-041-1/+1
| | | | | | | | | | | | This fixes a regression since SVN rev 273808 (which was supposed to not change functionality). The regression caused miscompilations (noted in the wild when targeting AArch64) on platforms with 32 bit long. Differential Revision: https://reviews.llvm.org/D32850 llvm-svn: 302137
* IR: Use pointers instead of GUIDs to represent edges in the module summary. ↵Peter Collingbourne2017-05-043-56/+45
| | | | | | | | | | | | | | | | | | | | NFCI. When profiling a no-op incremental link of Chromium I found that the functions computeImportForFunction and computeDeadSymbols were consuming roughly 10% of the profile. The goal of this change is to improve the performance of those functions by changing the map lookups that they were previously doing into pointer dereferences. This is achieved by changing the ValueInfo data structure to be a pointer to an element of the global value map owned by ModuleSummaryIndex, and changing reference lists in the GlobalValueSummary to hold ValueInfos instead of GUIDs. This means that a ValueInfo will take a client directly to the summary list for a given GUID. Differential Revision: https://reviews.llvm.org/D32471 llvm-svn: 302108
* [IR] Abstract away ArgNo+1 attribute indexing as much as possibleReid Kleckner2017-05-032-7/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: Do three things to help with that: - Add AttributeList::FirstArgIndex, which is an enumerator currently set to 1. It allows us to change the indexing scheme with fewer changes. - Add addParamAttr/removeParamAttr. This just shortens addAttribute call sites that would otherwise need to spell out FirstArgIndex. - Remove some attribute-specific getters and setters from Function that take attribute list indices. Most of these were only used from BuildLibCalls, and doesNotAlias was only used to test or set if the return value is malloc-like. I'm happy to split the patch, but I think they are probably easier to review when taken together. This patch should be NFC, but it sets the stage to change the indexing scheme to this, which is more convenient when indexing into an array: 0: func attrs 1: retattrs 2...: arg attrs Reviewers: chandlerc, pete, javed.absar Subscribers: david2050, llvm-commits Differential Revision: https://reviews.llvm.org/D32811 llvm-svn: 302060
* [PartialInlining] Add more early filteringXinliang David Li2017-05-021-0/+10
| | | | | | | This is a follow up to the previous inline cost patch for quicker filtering. llvm-svn: 301959
* [PartialInlining] Hook up inline cost analysisXinliang David Li2017-05-021-11/+98
| | | | | | Differential Revision: http://reviews.llvm.org/D32666 llvm-svn: 301894
* IPO: Add missing build dep.Peter Collingbourne2017-05-011-1/+1
| | | | llvm-svn: 301835
* Object: Remove ModuleSummaryIndexObjectFile class.Peter Collingbourne2017-05-011-1/+1
| | | | | | Differential Revision: https://reviews.llvm.org/D32195 llvm-svn: 301832
* Rename WeakVH to WeakTrackingVH; NFCSanjoy Das2017-05-012-10/+12
| | | | | | This relands r301424. llvm-svn: 301812
* [IPO/MergeFunctions] This function is used only under DEBUG().Davide Italiano2017-04-281-0/+4
| | | | llvm-svn: 301672
* Use Argument::hasAttribute and AttributeList::ReturnIndex moreReid Kleckner2017-04-282-19/+15
| | | | | | | | | | | This eliminates many extra 'Idx' induction variables in loops over arguments in CodeGen/ and Target/. It also reduces the number of places where we assume that ReturnIndex is 0 and that we should add one to argument numbers to get the corresponding attribute list index. NFC llvm-svn: 301666
* [asan] Fix dead stripping of globals on Linux.Evgeniy Stepanov2017-04-271-33/+2
| | | | | | | | | | | | | | | | | | | | | | Use a combination of !associated, comdat, @llvm.compiler.used and custom sections to allow dead stripping of globals and their asan metadata. Sometimes. Currently this works on LLD, which supports SHF_LINK_ORDER with sh_link pointing to the associated section. This also works on BFD, which seems to treat comdats as all-or-nothing with respect to linker GC. There is a weird quirk where the "first" global in each link is never GC-ed because of the section symbols. At this moment it does not work on Gold (as in the globals are never stripped). This is a second re-land of r298158. This time, this feature is limited to -fdata-sections builds. llvm-svn: 301587
* [PM/LoopUnswitch] Introduce a new, simpler loop unswitch pass.Chandler Carruth2017-04-271-1/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently, this pass only focuses on *trivial* loop unswitching. At that reduced problem it remains significantly better than the current loop unswitch: - Old pass is worse than cubic complexity. New pass is (I think) linear. - New pass is much simpler in its design by focusing on full unswitching. (See below for details on this). - New pass doesn't carry state for thresholds between pass iterations. - New pass doesn't carry state for correctness (both miscompile and infloop) between pass iterations. - New pass produces substantially better code after unswitching. - New pass can handle more trivial unswitch cases. - New pass doesn't recompute the dominator tree for the entire function and instead incrementally updates it. I've ported all of the trivial unswitching test cases from the old pass to the new one to make sure that major functionality isn't lost in the process. For several of the test cases I've worked to improve the precision and rigor of the CHECKs, but for many I've just updated them to handle the new IR produced. My initial motivation was the fact that the old pass carried state in very unreliable ways between pass iterations, and these mechansims were incompatible with the new pass manager. However, I discovered many more improvements to make along the way. This pass makes two very significant assumptions that enable most of these improvements: 1) Focus on *full* unswitching -- that is, completely removing whatever control flow construct is being unswitched from the loop. In the case of trivial unswitching, this means removing the trivial (exiting) edge. In non-trivial unswitching, this means removing the branch or switch itself. This is in opposition to *partial* unswitching where some part of the unswitched control flow remains in the loop. Partial unswitching only really applies to switches and to folded branches. These are very similar to full unrolling and partial unrolling. The full form is an effective canonicalization, the partial form needs a complex cost model, cannot be iterated, isn't canonicalizing, and should be a separate pass that runs very late (much like unrolling). 2) Leverage LLVM's Loop machinery to the fullest. The original unswitch dates from a time when a great deal of LLVM's loop infrastructure was missing, ineffective, and/or unreliable. As a consequence, a lot of complexity was added which we no longer need. With these two overarching principles, I think we can build a fast and effective unswitcher that fits in well in the new PM and in the canonicalization pipeline. Some of the remaining functionality around partial unswitching may not be relevant today (not many test cases or benchmarks I can find) but if they are I'd like to add support for them as a separate layer that runs very late in the pipeline. Purely to make reviewing and introducing this code more manageable, I've split this into first a trivial-unswitch-only pass and in the next patch I'll add support for full non-trivial unswitching against a *fixed* threshold, exactly like full unrolling. I even plan to re-use the unrolling thresholds, as these are incredibly similar cost tradeoffs: we're cloning a loop body in order to end up with simplified control flow. We should only do that when the total growth is reasonably small. One of the biggest changes with this pass compared to the previous one is that previously, each individual trivial exiting edge from a switch was unswitched separately as a branch. Now, we unswitch the entire switch at once, with cases going to the various destinations. This lets us unswitch multiple exiting edges in a single operation and also avoids numerous extremely bad behaviors, where we would introduce 1000s of branches to test for thousands of possible values, all of which would take the exact same exit path bypassing the loop. Now we will use a switch with 1000s of cases that can be efficiently lowered into a jumptable. This avoids relying on somehow forming a switch out of the branches or getting horrible code if that fails for any reason. Another significant change is that this pass actively updates the CFG based on unswitching. For trivial unswitching, this is actually very easy because of the definition of loop simplified form. Doing this makes the code coming out of loop unswitch dramatically more friendly. We still should run loop-simplifycfg (at the least) after this to clean up, but it will have to do a lot less work. Finally, this pass makes much fewer attempts to simplify instructions based on the unswitch. Something like loop-instsimplify, instcombine, or GVN can be used to do increasingly powerful simplifications based on the now dominating predicate. The old simplifications are things that something like loop-instsimplify should get today or a very, very basic loop-instcombine could get. Keeping that logic separate is a big simplifying technique. Most of the code in this pass that isn't in the old one has to do with achieving specific goals: - Updating the dominator tree as we go - Unswitching all cases in a switch in a single step. I think it is still shorter than just the trivial unswitching code in the old pass despite having this functionality. Differential Revision: https://reviews.llvm.org/D32409 llvm-svn: 301576
* [GlobalOpt] Correctly update metadata when localizing a global.Eli Friedman2017-04-271-1/+3
| | | | | | | | | Just calling dropAllReferences leaves pointers to the ConstantExpr behind, so we would eventually crash with a null pointer dereference. Differential Revision: https://reviews.llvm.org/D32551 llvm-svn: 301575
* [PartialInlining]: Improve partial inlining to handle complex conditionsXinliang David Li2017-04-271-41/+243
| | | | | | Differential Revision: http://reviews.llvm.org/D32249 llvm-svn: 301561
* Disable GVN Hoist due to still more bugs being found in it. There isChandler Carruth2017-04-271-2/+2
| | | | | | | | | | also a discussion about exactly what we should do prior to re-enabling it. The current bug is http://llvm.org/PR32821 and the discussion about this is in the review thread for r300200. llvm-svn: 301505
* Reverts commit r301424, r301425 and r301426Sanjoy Das2017-04-262-12/+10
| | | | | | | | | | | | Commits were: "Use WeakVH instead of WeakTrackingVH in AliasSetTracker's UnkownInsts" "Add a new WeakVH value handle; NFC" "Rename WeakVH to WeakTrackingVH; NFC" The changes assumed pointers are 8 byte aligned on all architectures. llvm-svn: 301429
* Rename WeakVH to WeakTrackingVH; NFCSanjoy Das2017-04-262-10/+12
| | | | | | | | | | | | | | | | Summary: I plan to use WeakVH to mean "nulls itself out on deletion, but does not track RAUW" in a subsequent commit. Reviewers: dblaikie, davide Reviewed By: davide Subscribers: arsenm, mehdi_amini, mcrosier, mzolotukhin, jfb, llvm-commits, nhaehnle Differential Revision: https://reviews.llvm.org/D32266 llvm-svn: 301424
* Simplify the CFG after loop pass cleanup.Filipe Cabecinhas2017-04-261-0/+5
| | | | | | | | | | | | | | | | Summary: Otherwise we might end up with some empty basic blocks or single-entry-single-exit basic blocks. This fixes PR32085 Reviewers: chandlerc, danielcdh Subscribers: mehdi_amini, RKSimon, llvm-commits Differential Revision: https://reviews.llvm.org/D30468 llvm-svn: 301395
* [PM] Run IndirectCallPromotion only when PGO is enabled.Davide Italiano2017-04-251-9/+9
| | | | | | Differential Revision: https://reviews.llvm.org/D32465 llvm-svn: 301327
* [PartialInine]: add triaging optionsXinliang David Li2017-04-231-0/+24
| | | | | | | There are more bugs (runtime failures) triggered when partial inlining is turned on. Add options to help triaging problems. llvm-svn: 301148
* [PartialInlining] Add optimization remark supportXinliang David Li2017-04-231-4/+20
| | | | | | Differential Revision: http://reviews.llvm.org/D32387 llvm-svn: 301143
* [ThinLTO/Summary] Rename anonymous globals as last action ...Davide Italiano2017-04-231-3/+6
| | | | | | | | | | | | | | ... in the per-TU -O0 pipeline. The problem is that there could be passes registered using `addExtensionsToPM()` introducing unnamed globals. Asan is an example, but there may be others. Building cppcheck with `-flto=thin` and `-fsanitize=address` triggers an assertion while we're reading bitcode (in lib/LTO), as the BitcodeReader assumes there are no unnamed globals (because the namer has run). Unfortunately I wasn't able to find an easy way to test this. I added a comment in the hope nobody moves this again. llvm-svn: 301102
OpenPOWER on IntegriCloud