summaryrefslogtreecommitdiffstats
path: root/llvm/lib/Target/SystemZ
Commit message (Collapse)AuthorAgeFilesLines
...
* [SystemZ, MachineScheduler] Improve post-RA scheduling.Jonas Paulsson2017-08-174-41/+264
| | | | | | | | | | | | | | | | | | | | | | The idea of this patch is to continue the scheduler state over an MBB boundary in the case where the successor block has only one predecessor. This means that the scheduler will continue in the successor block (after emitting any branch instructions) with e.g. maintained processor resource counters. Benchmarks have been confirmed to benefit from this. The algorithm in MachineScheduler.cpp that extracts scheduling regions of an MBB has been extended so that the strategy may optionally reverse the order of processing the regions themselves. This is controlled by a new method doMBBSchedRegionsTopDown(), which defaults to false. Handling the top-most region of an MBB first also means that a top-down scheduler can continue the scheduler state across any scheduling boundary between to regions inside MBB. Review: Ulrich Weigand, Matthias Braun, Andy Trick. https://reviews.llvm.org/D35053 llvm-svn: 311072
* Add "Restored" flag to CalleeSavedInfoKrzysztof Parzyszek2017-08-102-2/+2
| | | | | | | | | | | The liveness-tracking code assumes that the registers that were saved in the function's prolog are live outside of the function. Specifically, that registers that were saved are also live-on-exit from the function. This isn't always the case as illustrated by the LR register on ARM. Differential Revision: https://reviews.llvm.org/D36160 llvm-svn: 310619
* [LSR / TTI / SystemZ] Eliminate TargetTransformInfo::isFoldableMemAccess()Jonas Paulsson2017-08-092-16/+9
| | | | | | | | | | | | | | | | | isLegalAddressingMode() has recently gained the extra optional Instruction* parameter, and therefore it can now do the job that previously only isFoldableMemAccess() could do. The SystemZ implementation of isLegalAddressingMode() has gained the functionality of checking for offsets, which used to be done with isFoldableMemAccess(). The isFoldableMemAccess() hook has been removed everywhere. Review: Quentin Colombet, Ulrich Weigand https://reviews.llvm.org/D35933 llvm-svn: 310463
* [SystemZ] Add support for 128-bit atomic load/store/cmpxchgUlrich Weigand2017-08-044-3/+165
| | | | | | | | | | | | This adds support for the main 128-bit atomic operations, using the SystemZ instructions LPQ, STPQ, and CDSG. Generating these instructions is a bit more complex than usual since the i128 type is not legal for the back-end. Therefore, we have to hook the LowerOperationWrapper and ReplaceNodeResults TargetLowering callbacks. llvm-svn: 310094
* [SystemZ] Eliminate unnecessary serialization operationsUlrich Weigand2017-08-041-9/+9
| | | | | | | | | | | We currently emit a serialization operation (bcr 14, 0) before every atomic load and after every atomic store. This is overly conservative. The SystemZ architecture actually does not require any serialization for atomic loads, and a serialization after an atomic store only if we need to enforce sequential consistency. This is what other compilers for the platform implement as well. llvm-svn: 310093
* Delete Default and JITDefault code modelsRafael Espindola2017-08-033-47/+46
| | | | | | | | | | | | | | | IMHO it is an antipattern to have a enum value that is Default. At any given piece of code it is not clear if we have to handle Default or if has already been mapped to a concrete value. In this case in particular, only the target can do the mapping and it is nice to make sure it is always done. This deletes the two default enum values of CodeModel and uses an explicit Optional<CodeModel> when it is possible that it is unspecified. llvm-svn: 309911
* [SystemZ, LoopStrengthReduce]Jonas Paulsson2017-07-214-39/+128
| | | | | | | | | | | | | | | | | | | | | | | | | | This patch makes LSR generate better code for SystemZ in the cases of memory intrinsics, Load->Store pairs or comparison of immediate with memory. In order to achieve this, the following common code changes were made: * New TTI hook: LSRWithInstrQueries(), which defaults to false. Controls if LSR should do instruction-based addressing evaluations by calling isLegalAddressingMode() with the Instruction pointers. * In LoopStrengthReduce: handle address operands of memset, memmove and memcpy as address uses, and call isFoldableMemAccessOffset() for any LSRUse::Address, not just loads or stores. SystemZ changes: * isLSRCostLess() implemented with Insns first, and without ImmCost. * New function supportedAddressingMode() that is a helper for TTI methods looking at Instructions passed via pointers. Review: Ulrich Weigand, Quentin Colombet https://reviews.llvm.org/D35262 https://reviews.llvm.org/D35049 llvm-svn: 308729
* [SystemZ] Minor fixing in SystemZScheduleZ14.tdJonas Paulsson2017-07-191-11/+11
| | | | | | | Some minor corrections for recently added instructions. Review: Ulrich Weigand llvm-svn: 308429
* [SystemZ, AsmParser] Enable the mnemonic spell corrector.Jonas Paulsson2017-07-181-2/+13
| | | | | | | | This enables the suggestions of other mnemonics when invalid ones are specified. Review: Ulrich Weigand llvm-svn: 308280
* [SystemZ] Add support for IBM z14 processor (3/3)Ulrich Weigand2017-07-179-43/+211
| | | | | | | | | | | | | | | | | | | | | | | | | This adds support for the new 128-bit vector float instructions of z14. Note that these instructions actually only operate on the f128 type, since only each 128-bit vector register can hold only one 128-bit float value. However, this is still preferable to the legacy 128-bit float instructions, since those operate on pairs of floating-point registers (so we can hold at most 8 values in registers), while the new instructions use single vector registers (so we hold up to 32 value in registers). Adding support includes: - Enabling the instructions for the assembler/disassembler. - CodeGen for the instructions. This includes allocating the f128 type now to the VR128BitRegClass instead of FP128BitRegClass. - Scheduler description support for the instructions. Note that for a small number of operations, we have no new vector instructions (like integer <-> 128-bit float conversions), and so we use the legacy instruction and then reformat the operand (i.e. copy between a pair of floating-point registers and a vector register). llvm-svn: 308196
* [SystemZ] Add support for IBM z14 processor (2/3)Ulrich Weigand2017-07-177-19/+247
| | | | | | | | | | | | | | This adds support for the new 32-bit vector float instructions of z14. This includes: - Enabling the instructions for the assembler/disassembler. - CodeGen for the instructions, including new LLVM intrinsics. - Scheduler description support for the instructions. - Update to the vector cost function calculations. In general, CodeGen support for the new v4f32 instructions closely matches support for the existing v2f64 instructions. llvm-svn: 308195
* [SystemZ] Add support for IBM z14 processor (1/3)Ulrich Weigand2017-07-1715-15/+2263
| | | | | | | | | | | | | | This patch series adds support for the IBM z14 processor. This part includes: - Basic support for the new processor and its features. - Support for new instructions (except vector 32-bit float and 128-bit float). - CodeGen for new instructions, including new LLVM intrinsics. - Scheduler description for the new processor. - Detection of z14 as host processor. Support for the new 32-bit vector float and 128-bit vector float instructions is provided by separate patches. llvm-svn: 308194
* SystemZCodeGen: Update libdeps. r308024 introduced LoopDataPrefetchPass.NAKAMURA Takumi2017-07-151-1/+1
| | | | llvm-svn: 308086
* [SystemZ] Minor fixing in SystemZScheduleZ196.tdJonas Paulsson2017-07-141-71/+92
| | | | | | | Some minor corrections for the recently added instructions. Review: Ulrich Weigand llvm-svn: 308028
* [SystemZ] Enable LoopDataPrefetch pass.Jonas Paulsson2017-07-142-1/+7
| | | | | | | | Loop data prefetching has shown some improvements on benchmarks, and is enabled at -O1 and above. Review: Ulrich Weigand llvm-svn: 308024
* [SystemZ] Minor fixing in SystemZScheduleZEC12.tdJonas Paulsson2017-07-141-70/+91
| | | | | | | Some minor corrections for the recently added instructions. Review: Ulrich Weigand llvm-svn: 308007
* Fully fix the movw/movt addend.Rafael Espindola2017-07-111-2/+2
| | | | | | | | | | The issue is not if the value is pcrel. It is whether we have a relocation or not. If we have a relocation, the static linker will select the upper bits. If we don't have a relocation, we have to do it. llvm-svn: 307730
* Enhance synchscope representationKonstantin Zhuravlyov2017-07-111-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | OpenCL 2.0 introduces the notion of memory scopes in atomic operations to global and local memory. These scopes restrict how synchronization is achieved, which can result in improved performance. This change extends existing notion of synchronization scopes in LLVM to support arbitrary scopes expressed as target-specific strings, in addition to the already defined scopes (single thread, system). The LLVM IR and MIR syntax for expressing synchronization scopes has changed to use *syncscope("<scope>")*, where <scope> can be "singlethread" (this replaces *singlethread* keyword), or a target-specific name. As before, if the scope is not specified, it defaults to CrossThread/System scope. Implementation details: - Mapping from synchronization scope name/string to synchronization scope id is stored in LLVM context; - CrossThread/System and SingleThread scopes are pre-defined to efficiently check for known scopes without comparing strings; - Synchronization scope names are stored in SYNC_SCOPE_NAMES_BLOCK in the bitcode. Differential Revision: https://reviews.llvm.org/D21723 llvm-svn: 307722
* [CodeGen] Rename DEBUG_TYPE to match passnamesEvandro Menezes2017-07-112-2/+2
| | | | | | | | | Rename missing DEBUG_TYPE "machine-scheduler" from backend files, which were absent from https://reviews.llvm.org/rL303921. Differential revision: https://reviews.llvm.org/D35231 llvm-svn: 307719
* [SystemZ] Minor fixing in SystemZScheduleZ13.tdJonas Paulsson2017-07-111-69/+84
| | | | | | | Some minor corrections for the recently added instructions. Review: Ulrich Weigand llvm-svn: 307658
* [IR] Add Type::isIntOrIntVectorTy(unsigned) similar to the existing ↵Craig Topper2017-07-091-2/+1
| | | | | | isIntegerTy(unsigned), but also works for vectors. llvm-svn: 307492
* [IR] Make use of ↵Craig Topper2017-07-091-1/+1
| | | | | | Type::isPtrOrPtrVectorTy/isIntOrIntVectorTy/isFPOrFPVectorTy to shorten code. NFC llvm-svn: 307491
* [SystemZ] Fix -Wimplicit-fallthrough warnings. NFCI.Simon Pilgrim2017-07-071-0/+2
| | | | llvm-svn: 307376
* [SystemZ] Simplify handling of 128-bit multiply/divide instructionUlrich Weigand2017-07-057-106/+106
| | | | | | | | | | | Several integer multiply/divide instructions require use of a register pair as input and output. This patch moves setting up the input register pair from C++ code to TableGen, simplifying the whole process and making it more easily extensible. No functional change. llvm-svn: 307155
* [SystemZ] Small cleanups to SystemZScheduleZ13.tdUlrich Weigand2017-07-051-25/+36
| | | | | | | | | | Fixes a couple of whitespace errors, re-sorts the vector floating-point instructions to make them more easily extensible, and adds a missing pseudo instruction. No functional change. llvm-svn: 307154
* fix trivial typos in comments; NFCHiroshi Inoue2017-07-031-1/+1
| | | | llvm-svn: 307004
* [SystemZ] Add all remaining instructionsUlrich Weigand2017-06-3015-132/+1169
| | | | | | | | | | | This adds all remaining instructions that were still missing, mostly privileged and semi-privileged system-level instructions. These are provided for use with the assembler and disassembler only. This brings the LLVM assembler / disassembler to parity with the GNU binutils tools. llvm-svn: 306876
* [SystemZ] Add missing high-word facility instructionsUlrich Weigand2017-06-305-0/+70
| | | | | | | | | | | There are a few instructions provided by the high-word facility (z196) that we cannot easily exploit for code generation. This patch at least adds those missing instructions for the assembler and disassembler. This means that now all nonprivileged instructions up to z13 are supported by the LLVM assembler / disassembler. llvm-svn: 306821
* [LoopUnroll] Pass SCEV to getUnrollingPreferences hook. NFCI.Geoff Berry2017-06-282-2/+3
| | | | | | | | | | Reviewers: sanjoy, anna, reames, apilipenko, igor-laevsky, mkuper Subscribers: jholewinski, arsenm, mzolotukhin, nemanjai, nhaehnle, javed.absar, mcrosier, llvm-commits Differential Revision: https://reviews.llvm.org/D34531 llvm-svn: 306554
* [SystemZ] Fix missing emergency spill slot corner caseUlrich Weigand2017-06-261-2/+15
| | | | | | | | | | | | | | | | We sometimes need emergency spill slots for the register scavenger. This may be the case when code needs to access a stack slot that has an offset of 4096 or more relative to the stack pointer. To make that determination, processFunctionBeforeFrameFinalized currently simply checks the total stack frame size of the current function. But this is not enough, since code may need to access stack slots in the caller's stack frame as well, in particular incoming arguments stored on the stack. This commit fixes the problem by taking argument slots into account. llvm-svn: 306305
* [SystemZ] Add a check against zero before calling getTestUnderMaskCond()Jonas Paulsson2017-06-261-0/+2
| | | | | | | | | | | | | | Csmith discovered that this function can be called with a zero argument, in which case an assert for this triggered. This patch also adds a guard before the other call to this function since it was missing, although the test only covers the case where it was discovered. Reduced test case attached as CodeGen/SystemZ/int-cmp-54.ll. Review: Ulrich Weigand llvm-svn: 306287
* Remove redundant argument.Rafael Espindola2017-06-241-2/+2
| | | | llvm-svn: 306189
* ARM: move some logic from processFixupValue to applyFixup.Rafael Espindola2017-06-231-2/+5
| | | | | | | | | | | | processFixupValue is called on every relaxation iteration. applyFixup is only called once at the very end. applyFixup is then the correct place to do last minute changes and value checks. While here, do proper range checks again for fixup_arm_thumb_bl. We used to do it, but dropped because of thumb2. We now do it again, but use the thumb2 range. llvm-svn: 306177
* [SystemZ] Remove unnecessary serialization before volatile loadsUlrich Weigand2017-06-234-17/+6
| | | | | | | | | | | | | | | | | This reverts the use of TargetLowering::prepareVolatileOrAtomicLoad introduced by r196905. Nothing in the semantics of the "volatile" keyword or the definition of the z/Architecture actually requires that volatile loads are preceded by a serialization operation, and no other compiler on the platform actually implements this. Since we've now seen a use case where this additional serialization causes noticable performance degradation, this patch removes it. The patch still leaves in the serialization before atomic loads, which is now implemented directly in lowerATOMIC_LOAD. (This also seems overkill, but that can be addressed separately.) llvm-svn: 306117
* [SystemZ] Fix trap issue and enable expensive checks.Jonas Paulsson2017-06-232-8/+3
| | | | | | | | | | | | | | | | | | | | | | | The isBarrier/isTerminator flags have been removed from the SystemZ trap instructions, so that tests do not fail with EXPENSIVE_CHECKS. This was just an issue at -O0 and did not affect code output on benchmarks. (Like Eli pointed out: "targets are split over whether they consider their "trap" a terminator; x86, AArch64, and NVPTX don't, but ARM, MIPS, PPC, and SystemZ do. We should probably try to be consistent here.". This is still the case, although SystemZ has switched sides). SystemZ now returns true in isMachineVerifierClean() :-) These Generic tests have been modified so that they can be run with or without EXPENSIVE_CHECKS: CodeGen/Generic/llc-start-stop.ll and CodeGen/Generic/print-machineinstrs.ll Review: Ulrich Weigand, Simon Pilgrim, Eli Friedman https://bugs.llvm.org/show_bug.cgi?id=33047 https://reviews.llvm.org/D34143 llvm-svn: 306106
* Use a MutableArrayRef. NFC.Rafael Espindola2017-06-211-4/+4
| | | | llvm-svn: 305968
* Fix build.Rafael Espindola2017-06-211-1/+1
| | | | llvm-svn: 305967
* Mark dump() methods as const. NFCSam Clegg2017-06-211-1/+1
| | | | | | | | | Add const qualifier to any dump() method where adding one was trivial. Differential Revision: https://reviews.llvm.org/D34481 llvm-svn: 305963
* Const correctness for TTI::getRegisterBitWidthDaniel Neilson2017-06-122-2/+2
| | | | | | | | | | | | | | Summary: The method TargetTransformInfo::getRegisterBitWidth() is declared const, but the type erasing implementation classes (TargetTransformInfo::Concept & TargetTransformInfo::Model) that were introduced by Chandler in https://reviews.llvm.org/D7293 do not have the method declared const. This is an NFC to tidy up the const consistency between TTI and its implementation. Reviewers: chandlerc, rnk, reames Reviewed By: reames Subscribers: reames, jfb, arsenm, dschuff, nemanjai, nhaehnle, javed.absar, sbc100, jgravelle-google, llvm-commits Differential Revision: https://reviews.llvm.org/D33903 llvm-svn: 305189
* [SystemZ] Propagate MachineMemOperandsJonas Paulsson2017-06-071-6/+19
| | | | | | | In emitCondStore() and emitMemMemWrapper(). Review: Ulrich Weigand llvm-svn: 304913
* Move Object format code to lib/BinaryFormat.Zachary Turner2017-06-071-1/+1
| | | | | | | | | | | | This creates a new library called BinaryFormat that has all of the headers from llvm/Support containing structure and layout definitions for various types of binary formats like dwarf, coff, elf, etc as well as the code for identifying a file from its magic. Differential Revision: https://reviews.llvm.org/D33843 llvm-svn: 304864
* Sort the remaining #include lines in include/... and lib/....Chandler Carruth2017-06-0610-12/+12
| | | | | | | | | | | | | | | | | | | | | | | | | I did this a long time ago with a janky python script, but now clang-format has built-in support for this. I fed clang-format every line with a #include and let it re-sort things according to the precise LLVM rules for include ordering baked into clang-format these days. I've reverted a number of files where the results of sorting includes isn't healthy. Either places where we have legacy code relying on particular include ordering (where possible, I'll fix these separately) or where we have particular formatting around #include lines that I didn't want to disturb in this patch. This patch is *entirely* mechanical. If you get merge conflicts or anything, just ignore the changes in this patch and run clang-format over your #include lines in the files. Sorry for any noise here, but it is important to keep these things stable. I was seeing an increasing number of patches with irrelevant re-ordering of #include lines because clang-format was used. This patch at least isolates that churn, makes it easy to skip when resolving conflicts, and gets us to a clean baseline (again). llvm-svn: 304787
* TargetMachine: Indicate whether machine verifier passes.Matthias Braun2017-05-311-0/+2
| | | | | | | | | | | | | This adds a callback to the LLVMTargetMachine that lets target indicate that they do not pass the machine verifier checks in all cases yet. This is intended to be a temporary measure while the targets are fixed allowing us to enable the machine verifier by default with EXPENSIVE_CHECKS enabled! Differential Revision: https://reviews.llvm.org/D33696 llvm-svn: 304320
* TargetPassConfig: Keep a reference to an LLVMTargetMachine; NFCMatthias Braun2017-05-301-2/+2
| | | | | | | | | | | TargetPassConfig is not useful for targets that do not use the CodeGen library, so we may just as well store a pointer to an LLVMTargetMachine instead of just to a TargetMachine. While at it, also change the constructor to take a reference instead of a pointer as the TM must not be nullptr. llvm-svn: 304247
* [SystemZ] Add decimal floating-point instructionsUlrich Weigand2017-05-309-1/+666
| | | | | | | | | This adds assembler / disassembler support for the decimal floating-point instructions. Since LLVM does not yet have support for decimal float types, these cannot be used for codegen at this point. llvm-svn: 304203
* [SystemZ] Add hexadecimal floating-point instructionsUlrich Weigand2017-05-307-18/+604
| | | | | | | | This adds assembler / disassembler support for the hexadecimal floating-point instructions. Since the Linux ABI does not use any hex float data types, these are not useful for codegen. llvm-svn: 304202
* [SystemZ] Improve buildVector() in SystemZISelLowering.cpp.Jonas Paulsson2017-05-291-19/+41
| | | | | | | | Use VLREP when inserting one or more loads into a vector. This is more efficient than to first load and then use a VLVGP. Review: Ulrich Weigand llvm-svn: 304152
* LivePhysRegs: Rework constructor + documentation; NFCMatthias Braun2017-05-261-1/+1
| | | | | | | - Take reference instead of pointer to a TRI that cannot be nullptr. - Improve documentation comments. llvm-svn: 304038
* [LoopVectorizer] Let target prefer scalar addressing computations.Jonas Paulsson2017-05-241-0/+1
| | | | | | | | | | | | | | | | | | | | | | The loop vectorizer usually vectorizes any instruction it can and then extracts the elements for a scalarized use. On SystemZ, all elements containing addresses must be extracted into address registers (GRs). Since this extraction is not free, it is better to have the address in a suitable register to begin with. By forcing address arithmetic instructions and loads of addresses to be scalar after vectorization, two benefits result: * No need to extract the register * LSR optimizations trigger (LSR isn't handling vector addresses currently) Benchmarking show improvements on SystemZ with this new behaviour. Any other target could try this by returning false in the new hook prefersVectorizedAddressing(). Review: Renato Golin, Elena Demikhovsky, Ulrich Weigand https://reviews.llvm.org/D32422 llvm-svn: 303744
* [SystemZ] Fix register modelling in expandLoadStackGuard()Jonas Paulsson2017-05-241-16/+14
| | | | | | | | EXPENSIVE_CHECKS found this bug (https://bugs.llvm.org/show_bug.cgi?id=33047), which this patch fixes. The EAR instruction defines a GR32, not a GR64. Review: Ulrich Weigand llvm-svn: 303743
OpenPOWER on IntegriCloud