summaryrefslogtreecommitdiffstats
path: root/llvm/lib/CodeGen/CodeGenPrepare.cpp
Commit message (Collapse)AuthorAgeFilesLines
* [NFC] Change MemIntrinsicInst::setAlignment() to take an unsigned instead of ↵Daniel Neilson2018-01-121-1/+1
| | | | | | | | | | | a Constant Summary: In preparation for https://reviews.llvm.org/D41675 this NFC changes this prototype of MemIntrinsicInst::setAlignment() to accept an unsigned instead of a Constant. llvm-svn: 322403
* [CGP] Re-enable Select in complex addressing modeSerguei Katkov2018-01-121-1/+1
| | | | | | | | Re-enable Select after a couple of fixes. Differential Revision: https://reviews.llvm.org/D40634 llvm-svn: 322358
* Tidy some grammar in some commentsEric Christopher2018-01-091-3/+3
| | | | llvm-svn: 322133
* [CGP] Fix Complex addressing mode for offsetSerguei Katkov2018-01-091-1/+6
| | | | | | | | | | | | | | If the offset is differ in two addressing mode we can continue only if ScaleReg is not set due to we will use it as merge of different offsets. It should fix PR35799 and PR35805. Reviewers: john.brawn, reames Reviewed By: reames Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D41227 llvm-svn: 322056
* Use phi ranges to simplify code. No functionality change intended.Benjamin Kramer2017-12-301-37/+21
| | | | llvm-svn: 321585
* [PGO] Function section hotness prefix should look at all blocksTeresa Johnson2017-12-201-11/+4
| | | | | | | | | | | | | | | | | | | | Summary: The function section prefix for PGO based layout (e.g. hot/unlikely) should look at the hotness of all blocks not just the entry BB. A function with a cold entry but a very hot loop should be placed in the hot section, for example, so that it is located close to other hot functions it may call. For SamplePGO it was already looking at the branch weights on calls, and I made that code conditional on whether this is SamplePGO since it was essentially a noop for instrumentation PGO anyway. Reviewers: davidxl Subscribers: eraman, llvm-commits Differential Revision: https://reviews.llvm.org/D41395 llvm-svn: 321197
* [CGP] Format. NFCHaicheng Wu2017-12-191-1/+1
| | | | | | Clang-format. llvm-svn: 321107
* [CGP] Fix the handling select inst in complex addressing modeSerguei Katkov2017-12-181-2/+2
| | | | | | | | | | | | | | | When we put the value in select placeholder we must pass the value through simplification tracker due to the value might be already simplified and erased. This is a fix for PR35658. Reviewers: john.brawn, uabelho Reviewed By: john.brawn Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D41251 llvm-svn: 320956
* Revert "[CGP] Enable select in complex addr mode"Serguei Katkov2017-12-131-1/+1
| | | | | | | | Causes: Assertion `ScaledReg == nullptr' failed. This actually a revert of rL320551. llvm-svn: 320553
* [CGP] Enable select in complex addr modeSerguei Katkov2017-12-131-1/+1
| | | | | | | | | | | Enable select instruction handling in complex addr modes. Reviewers: john.brawn, reames, aaboud Reviewed By: reames Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D40634 llvm-svn: 320551
* Move splitIndirectCriticalEdges() to BasicBlockUtils.h.Hiroshi Yamauchi2017-12-041-159/+1
| | | | | | | | | | | | | | | | Summary: Move splitIndirectCriticalEdges() from CodeGenPrepare to BasicBlockUtils.h so that it can be called from other places. Reviewers: davidxl Reviewed By: davidxl Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D40750 llvm-svn: 319689
* [CGP] Enable complex addr modeSerguei Katkov2017-11-291-1/+1
| | | | | | Enable complex addr modes after two critical fixes: rL319109 and rL319292 llvm-svn: 319302
* [CGP] Fix common type handling in optimizeMemoryInstSerguei Katkov2017-11-291-6/+10
| | | | | | | | | | | | | | | | | If common type is different we should bail out due to we will not be able to create a select or Phi of these values. Basically it is done in ExtAddrMode::compare however it does not work if we handle the null first and then two values of different types. so add a check in initializeMap as well. The check in ExtAddrMode::compare is used as earlier bail out. Reviewers: reames, john.brawn Reviewed By: john.brawn Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D40479 llvm-svn: 319292
* [CGP] Fix handling of null pointer values in optimizeMemoryInstJohn Brawn2017-11-271-9/+7
| | | | | | | | | | | The current way that trivial addressing modes are detected incorrectly thinks that null pointers are non-trivial, leading to an infinite loop where we keep duplicating the same select. Fix this by aware of null when deciding if an addressing mode is trivial. Differential Revision: https://reviews.llvm.org/D40447 llvm-svn: 319019
* [CodeGenPrepare] Check that erased sunken address are not reusedSimon Dardis2017-11-241-5/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | CodeGenPrepare sinks address computations from one basic block to another and attempts to reuse address computations that have already been sunk. If the same address computation appears twice with the first instance as an operand of a load whose result is an operand to a simplifable select, CodeGenPrepare simplifies the select and recursively erases the now dead instructions. CodeGenPrepare then attempts to use the erased address computation for the second load. Fix this by erasing the cached address value if it has zero uses before looking for the address value in the sunken address map. This partially resolves PR35209. Thanks to Alexander Richardson for reporting the issue! This fixed version relands r318032 which was reverted in r318049 due to sanitizer buildbot failures. Reviewers: john.brawn Differential Revision: https://reviews.llvm.org/D39841 llvm-svn: 318956
* [CGP] Make optimizeMemoryInst able to combine more kinds of ExtAddrMode fieldsJohn Brawn2017-11-241-12/+94
| | | | | | | | | | | | | This patch extends the recent work in optimizeMemoryInst to make it able to combine more ExtAddrMode fields than just the BaseReg. This fixes some benchmark regressions introduced by r309397, where GVN PRE is hoisting a getelementptr such that it can no longer be combined into the addressing mode of the load or store that uses it. Differential Revision: https://reviews.llvm.org/D38133 llvm-svn: 318949
* Revert "[CGP] Enable complex addr mode (2nd attempt)"Serguei Katkov2017-11-211-1/+1
| | | | | | Revert the patch rl318728 causing buildbot hangs-ups. llvm-svn: 318731
* [CGP] Enable complex addr mode (2nd attempt)Serguei Katkov2017-11-211-1/+1
| | | | | | | 2nd attempt to enable complex addr modes after fix of the crash by rL318638. llvm-svn: 318728
* [CGP] Fix the crash caused by enable of complex addr modeSerguei Katkov2017-11-201-9/+5
| | | | | | | | | | | | | We must collect all AddModes even if they are the same. This is due to Original value is different but we need all original values collected as they are used as anchors in common phi finding. Reviewers: john.brawn, reames Reviewed By: john.brawn Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D40166 llvm-svn: 318638
* Fix a bunch more layering of CodeGen headers that are in TargetDavid Blaikie2017-11-171-2/+2
| | | | | | | | All these headers already depend on CodeGen headers so moving them into CodeGen fixes the layering (since CodeGen depends on Target, not the other way around). llvm-svn: 318490
* [CodeGenPrepare] Disable div bypass when working set size is huge.Easwaran Raman2017-11-141-3/+4
| | | | | | | | | | | | | | | | | | | Summary: Bypass of slow divs based on operand values is currently disabled for -Os. Do the same when profile summary is available and the working set size of the application is huge. This is similar to how loop peeling is guarded by hasHugeWorkingSetSize. In the div bypass case, the generated extra code (and the extra branch) tendss to outweigh the benefits of the bypass. This results in noticeable performance improvement on an internal application. Reviewers: davidxl Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D39992 llvm-svn: 318179
* Revert "[CodeGenPrepare] Check that erased sunken address are not reused"Simon Dardis2017-11-131-6/+1
| | | | | | This reverts commit r318032. The test broke some sanitizer bots. llvm-svn: 318049
* [CodeGenPrepare] Check that erased sunken address are not reusedSimon Dardis2017-11-131-1/+6
| | | | | | | | | | | | | | | | | | | | | | | CodeGenPrepare sinks address computations from one basic block to another and attempts to reuse address computations that have already been sunk. If the same address computation appears twice with the first instance as an operand of a load whose result is an operand to a simplifable select, CodeGenPrepare simplifies the select and recursively erases the now dead instructions. CodeGenPrepare then attempts to use the erased address computation for the second load. Fix this by erasing the cached address value if it has zero uses before looking for the address value in the sunken address map. This partially resolves PR35209. Thanks to Alexander Richardson for reporting the issue! Reviewers: john.brawn Differential Revision: https://reviews.llvm.org/D39841 llvm-svn: 318032
* Revert "[CGP] Enable extending scope of optimizeMemoryInst"Serguei Katkov2017-11-081-1/+1
| | | | | | Revert the patch r317665 causing buildbot failures. llvm-svn: 317667
* [CGP] Enable extending scope of optimizeMemoryInstSerguei Katkov2017-11-081-1/+1
| | | | | | | | | | | This patch enables the folding of address computation in memory instruction in case adress is represented by Phi node. The inputs of Phi node might be different in base register. Differential Revision: https://reviews.llvm.org/D36073 llvm-svn: 317665
* [CodeGenPrepare] Fix typo in comment. NFCCraig Topper2017-11-071-1/+1
| | | | llvm-svn: 317614
* [CGP] Disable Select instruction handling in optimizeMemoryInst. NFCSerguei Katkov2017-11-071-1/+1
| | | | | | | | | | | | | | | | | | This patch disables the handling of selects in optimization extensing scope of optimizeMemoryInst. The optimization itself is disable by default. The idea here is just to switch optimiztion level step by step. Specifically, first optimization will be enabled only for Phi nodes, then select instructions will be added. In case someone will complain about perfromance it will be easier to detect what part of optimizations is responsible for that. Differential Revision: https://reviews.llvm.org/D36073 llvm-svn: 317555
* [CGP] Fix the bug found by asan.Serguei Katkov2017-11-051-2/+2
| | | | | | Try to fix the asan failure introduced by r317429. llvm-svn: 317431
* [CGP] Extends the scope of optimizeMemoryInst optimizationSerguei Katkov2017-11-051-5/+438
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is an implementation of PR26223. Currently optimizeMemoryInst optimization tries to fold address computation if all possible way to get compute the address are of the form baseGV + base + scale * Index + offset where scale and offset are constants and baseGV, base and Index are exactly the same instructions if defined. The patch extends this optimization to allow different bases. In this case it tries to find/build a Phi node merging all possible bases and use this Phi node as a base for sunk address computation. Also it supports Select instruction on the way. The main motivation for this scope extension is GCRelocateInst. If there is a relocation of derived pointer it will be represented as relocation of base + offset. Also there will be a Phi node merging address computation for relocated derived pointer and derived pointer itself. If we have a Phi node merging original base and relocated base and can fold the address computation of derived pointer then we can potentially reduce the code size and Phi node for derived pointer. The later can have a positive impact to register allocator. Reviewers: efriedma, dberlin, mkazantsev, reames, john.brawn Reviewed By: john.brawn Subscribers: javed.absar, john.brawn, dneilson, llvm-commits Differential Revision: https://reviews.llvm.org/D36073 llvm-svn: 317429
* Invoke salvageDebugInfo from CodeGenPrepare's SinkCast()Adrian Prantl2017-11-031-0/+1
| | | | | | | | | | This preserves the debug info for the cast operation in the original location. rdar://problem/33460652 Reapplied r317340 with the test moved into an ARM-specific directory. llvm-svn: 317375
* Revert "Invoke salvageDebugInfo from CodeGenPrepare's SinkCast()"Adrian Prantl2017-11-031-1/+0
| | | | | | This reverts commit 317342 while investigating bot breakage. llvm-svn: 317345
* Invoke salvageDebugInfo from CodeGenPrepare's SinkCast()Adrian Prantl2017-11-031-0/+1
| | | | | | | | This preserves the debug info for the cast operation in the original location. rdar://problem/33460652 llvm-svn: 317340
* re-land [ExpandMemCmp] Split ExpandMemCmp from CodeGen into its own pass."Clement Courbet2017-11-031-710/+0
| | | | | | Fix undefined references: ExpandMemCmp belongs to CodeGen/, not Scalar/. llvm-svn: 317318
* Revert "[ExpandMemCmp] Split ExpandMemCmp from CodeGen into its own pass."Clement Courbet2017-11-021-0/+710
| | | | | | | | | undefined reference to `llvm::TargetPassConfig::ID' on clang-ppc64le-linux-multistage This reverts commit eea333c33fa73ad225ef28607795984829f65688. llvm-svn: 317213
* [ExpandMemCmp] Split ExpandMemCmp from CodeGen into its own pass.Clement Courbet2017-11-021-710/+0
| | | | | | | | | | | | | | | | | Summary: This is mostly a noop (most of the test diffs are renamed blocks). There are a few temporary register renames (eax<->ecx) and a few blocks are shuffled around. See the discussion in PR33325 for more details. Reviewers: spatel Subscribers: mgorny Differential Revision: https://reviews.llvm.org/D39456 llvm-svn: 317211
* [CGP] Fix the detection of trivial case for addressing modeSerguei Katkov2017-10-311-10/+9
| | | | | | | The address can be presented as a bitcast of baseReg. In this case it is still trivial but OriginalValue != baseReg. llvm-svn: 316980
* [CGP] Fix crash on i96 bit multiplyPhilip Reames2017-10-301-1/+1
| | | | | | | | Issue found by llvm-isel-fuzzer on OSS fuzz, https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=3725 If anyone actually cares about > 64 bit arithmetic, there's a lot more to do in this area. There's a bunch of obviously wrong code in the same function. I don't have the time to fix all of them and am just using this to understand what the workflow for fixing fuzzer cases might look like. llvm-svn: 316967
* [CodeGen][ExpandMemcmp] Allow memcmp to expand to vector loads (2).Clement Courbet2017-10-301-25/+26
| | | | | | | | | | | | - Targets that want to support memcmp expansions now return the list of supported load sizes. - Expansion codegen does not assume that all power-of-two load sizes smaller than the max load size are valid. For examples, this is not the case for x86(32bit)+sse2. Fixes PR34887. llvm-svn: 316905
* [CodeGen] Fix -Wunused-private-field warning on lld-x86_64-darwin13.Clement Courbet2017-10-271-2/+0
| | | | llvm-svn: 316765
* [CodeGen][ExpandMemCmp][NFC] Simplify load sequence generation.Clement Courbet2017-10-271-40/+33
| | | | llvm-svn: 316763
* Revert "[CGP] Merge empty case blocks if no extra moves are added."Balaram Makam2017-10-271-36/+11
| | | | | | This reverts commit r316711. The domtree isn't getting updated correctly. llvm-svn: 316721
* [CGP] Merge empty case blocks if no extra moves are added.Balaram Makam2017-10-261-11/+36
| | | | | | | | | | | | | | | | Summary: Currently we skip merging when extra moves may be added in the header of switch instead of the case block, if the case block is used as an incoming block of a PHI. If all the incoming values of the PHIs are non-constants and the destination block is dominated by the switch block then extra moves are likely not added by ISel, so there is no need to skip merging in this case. Reviewers: efriedma, junbuml, davidxl, hfinkel, qcolombet Reviewed By: efriedma Subscribers: dberlin, kuhar, mcrosier, llvm-commits Differential Revision: https://reviews.llvm.org/D37343 llvm-svn: 316711
* Re-land "[CodeGen][ExpandMemcmp][NFC] Allow memcmp to expand to vector loads ↵Clement Courbet2017-10-251-195/+227
| | | | | | | | | | | | | (1)" Compute the actual decomposition only after deciding whether to expand of not. Else, it's easy to make the compiler OOM with: `memcpy(dst, src, 0xffffffffffffffff);`, which typically happens if someone mistakenly passes a negative value. Add a test. This reverts commit f8fc02fbd4ab33383c010d33675acf9763d0bd44. llvm-svn: 316567
* Revert "[CodeGen][ExpandMemcmp][NFC] Allow memcmp to expand to vector loads (1)"Martin Bohme2017-10-241-201/+196
| | | | | | | | This reverts commit r316417, which causes internal compiles to OOM. I don't unfortunately have a self-contained test case but will follow up with courbet. llvm-svn: 316497
* [CodeGen][ExpandMemcmp][NFC] Allow memcmp to expand to vector loads (1)Clement Courbet2017-10-241-196/+201
| | | | | | | | | | | | | | | | | | | | | | Refactor ExpandMemcmp: - Stop duplicating the logic for computation of the sequence of loads to generate (thsi was done in three different places), this is now done only once in MemCmpExpansion::MemCmpExpansion(). - Add a FIXME to expose a bug with the computation of the number of loads when not all sizes are loadable. For example, on X86-32 + SSE, possible loads are {16,4,2,1} bytes. The current code considers that all loads starting at MaxLoadSize are possible. This is not an issue right now as vector loads are not enabled, so I'm not fixing the issue here to keep the change as small as possible. I'm going to address this in a subsequent revision, where I enable vector loads. See https://bugs.llvm.org/show_bug.cgi?id=34887 Differential Revision: https://reviews.llvm.org/D38498 llvm-svn: 316417
* Reverting r315590; it did not include changes for llvm-tblgen, which is ↵Aaron Ballman2017-10-151-1/+1
| | | | | | | | causing link errors for several people. Error LNK2019 unresolved external symbol "public: void __cdecl `anonymous namespace'::MatchableInfo::dump(void)const " (?dump@MatchableInfo@?A0xf4f1c304@@QEBAXXZ) referenced in function "public: void __cdecl `anonymous namespace'::AsmMatcherEmitter::run(class llvm::raw_ostream &)" (?run@AsmMatcherEmitter@?A0xf4f1c304@@QEAAXAEAVraw_ostream@llvm@@@Z) llvm-tblgen D:\llvm\2017\utils\TableGen\AsmMatcherEmitter.obj 1 llvm-svn: 315854
* [dump] Remove NDEBUG from test to enable dump methods [NFC]Don Hinton2017-10-121-1/+1
| | | | | | | | | | | | | | | Summary: Add LLVM_FORCE_ENABLE_DUMP cmake option, and use it along with LLVM_ENABLE_ASSERTIONS to set LLVM_ENABLE_DUMP. Remove NDEBUG and only use LLVM_ENABLE_DUMP to enable dump methods. Move definition of LLVM_ENABLE_DUMP from config.h to llvm-config.h so it'll be picked up by public headers. Differential Revision: https://reviews.llvm.org/D38406 llvm-svn: 315590
* [CGP] Make optimizeMemoryInst capable of handling multiple AddrModesJohn Brawn2017-10-031-20/+154
| | | | | | | | | | | | | Currently optimizeMemoryInst requires that all of the AddrModes it sees are identical. This patch makes it capable of tracking multiple AddrModes, so long as they differ in at most one field. This patch does nothing by itself, but later patches will make use of it to insert or reuse phi or select instructions for the differing fields. Differential Revision: https://reviews.llvm.org/D38278 llvm-svn: 314795
* [CGP] In optimizeMemoryInst handle select similarly to phiJohn Brawn2017-10-031-7/+16
| | | | | | | | | | This lets us optimize away selects that perform the same address computation in two different ways and is also the first step towards being able to handle selects between two different, but compatible, address computations. Differential Revision: https://reviews.llvm.org/D38242 llvm-svn: 314794
* [CodeGenPrepare][NFC] Rename TargetTransformInfo::expandMemCmp -> ↵Clement Courbet2017-09-251-1/+1
| | | | | | | | | | | | | | | | TargetTransformInfo::enableMemCmpExpansion. Summary: Right now there are two functions with the same name, one does the work and the other one returns true if expansion is needed. Rename TargetTransformInfo::expandMemCmp to make it more consistent with other members of TargetTransformInfo. Remove the unused Instruction* parameter. Differential Revision: https://reviews.llvm.org/D38165 llvm-svn: 314096
OpenPOWER on IntegriCloud