summaryrefslogtreecommitdiffstats
path: root/llvm/lib/Target/AMDGPU/GCNHazardRecognizer.cpp
Commit message (Collapse)AuthorAgeFilesLines
* [AMDGPU] Add GCNHazardRecognizer::checkInlineAsmHazards() and ↵Mark Searles2017-12-071-17/+60
| | | | | | | | GCNHazardRecognizer::checkVALUHazardsHelper(). checkInlineAsmHazards() checks INLINEASM for hazards that we particularly care about (so not exhaustive); this patch adds a check for INLINEASM that defs vregs that hold data-to-be stored by immediately preceding store of more than 8 bytes. If the instr were not within an INLINEASM, this scenario would be handled by checkVALUHazard(). Add checkVALUHazardsHelper(), which will be called by both checkVALUHazards() and checkInlineAsmHazards(). Differential Revision: https://reviews.llvm.org/D40098 llvm-svn: 320083
* AMDGPU: Move hazard avoidance out of waitcnt pass.Matt Arsenault2017-11-171-37/+54
| | | | | | | | | | This is mostly moving VMEM clause breaking into the hazard recognizer. Also move another hazard currently handled in the waitcnt pass. Also stops breaking clauses unless xnack is enabled. llvm-svn: 318557
* AMDGPU: Replace list of SMEM buffer opcodesMatt Arsenault2017-11-171-10/+1
| | | | llvm-svn: 318506
* AMDGPU: Fix breaking SMEM clausesMatt Arsenault2017-11-171-25/+29
| | | | | | | | This was completely ignoring subregisters, so was not very useful. Also only break them if xnack is actually enabled. llvm-svn: 318505
* AMDGPU: Handle s_buffer_load_dword hazard on SIMarek Olsak2017-10-261-0/+27
| | | | | | | | | | Reviewers: arsenm, nhaehnle Subscribers: kzhuravl, wdng, yaxunl, dstuttard, tpr, llvm-commits, t-tye Differential Revision: https://reviews.llvm.org/D39171 llvm-svn: 316666
* AMDGPU: Make worst-case assumption about the wait states in inline assemblyNicolai Haehnle2017-09-061-1/+2
| | | | | | | | | | | | | | | | Summary: Mesa still uses a hack where empty inline assembly is used as a kind of optimization barrier. This exposed a problem where not enough wait states were inserted, because the hazard recognizer implicitly assumed that each inline assembly "instruction" has at least one wait state. Reviewers: arsenm Subscribers: kzhuravl, wdng, yaxunl, dstuttard, tpr, llvm-commits, t-tye Differential Revision: https://reviews.llvm.org/D37205 llvm-svn: 312635
* AMDGPU: IMPLICIT_DEFs and DBG_VALUEs do not contribute to wait statesNicolai Haehnle2017-09-011-4/+9
| | | | | | | | | | | | | | | Summary: This fixes a bug that was exposed on gfx9 in various GL45-CTS.shaders.loops.*_iterations.select_iteration_count_fragment tests, e.g. GL45-CTS.shaders.loops.do_while_uniform_iterations.select_iteration_count_fragment Reviewers: arsenm Subscribers: kzhuravl, wdng, yaxunl, dstuttard, tpr, t-tye, llvm-commits Differential Revision: https://reviews.llvm.org/D36193 llvm-svn: 312337
* [AMDGPU] Add missing hazard for DPP-after-EXEC-writeConnor Abbott2017-08-041-1/+8
| | | | | | | | | | | | | | Summary: Following the docs, we need at least 5 wait states between an EXEC write and an instruction that uses DPP. Reviewers: tstellar, arsenm Subscribers: kzhuravl, wdng, nhaehnle, yaxunl, dstuttard, tpr, t-tye, llvm-commits Differential Revision: https://reviews.llvm.org/D34849 llvm-svn: 310013
* Sort the remaining #include lines in include/... and lib/....Chandler Carruth2017-06-061-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | I did this a long time ago with a janky python script, but now clang-format has built-in support for this. I fed clang-format every line with a #include and let it re-sort things according to the precise LLVM rules for include ordering baked into clang-format these days. I've reverted a number of files where the results of sorting includes isn't healthy. Either places where we have legacy code relying on particular include ordering (where possible, I'll fix these separately) or where we have particular formatting around #include lines that I didn't want to disturb in this patch. This patch is *entirely* mechanical. If you get merge conflicts or anything, just ignore the changes in this patch and run clang-format over your #include lines in the files. Sorry for any noise here, but it is important to keep these things stable. I was seeing an increasing number of patches with irrelevant re-ordering of #include lines because clang-format was used. This patch at least isolates that churn, makes it easy to skip when resolving conflicts, and gets us to a clean baseline (again). llvm-svn: 304787
* AMDGPU: Fix broken condition in hazard recognizerMatt Arsenault2017-03-171-17/+16
| | | | | | Fixes bug 32248. llvm-svn: 298125
* Reverted unintended commitSimon Pilgrim2017-03-151-2/+2
| | | | llvm-svn: 297841
* Fix Wint-in-bool-context warning (PR32248)Simon Pilgrim2017-03-151-2/+2
| | | | llvm-svn: 297840
* AMDGPU: Merge initial gfx9 supportMatt Arsenault2017-02-181-6/+72
| | | | llvm-svn: 295554
* [AMDGPU] Fix some Clang-tidy modernize and Include What You Use warnings; ↵Eugene Zelenko2017-01-201-7/+15
| | | | | | other minor fixes (NFC). llvm-svn: 292623
* AMDGPU: Rename flat operands to match mubufMatt Arsenault2016-11-291-1/+1
| | | | | | | | | | Use vaddr/vdst for the same purposes. This also fixes a beg in SIInsertWaits for the operand check. The stored value operand is currently called data0 in the single offset case, not data. llvm-svn: 288188
* AMDGPU/GCN: Exit early in hazard recognizer if there is no vreg argumentJan Vesely2016-11-151-0/+4
| | | | | | | | | | wbinvl.* are vector instruction that do not sue vector registers. v2: check only M?BUF instructions Differential Revision: https://reviews.llvm.org/D26633 llvm-svn: 287056
* AMDGPU/SI: Handle hazard with s_rfe_b64Tom Stellard2016-10-271-1/+27
| | | | | | | | | | Reviewers: arsenm Subscribers: kzhuravl, wdng, nhaehnle, yaxunl, llvm-commits, tony-tye Differential Revision: https://reviews.llvm.org/D25638 llvm-svn: 285368
* AMDGPU/SI: Handle hazard with sgpr lane selects for v_{read,write}laneTom Stellard2016-10-271-1/+33
| | | | | | | | | | Reviewers: arsenm Subscribers: kzhuravl, wdng, nhaehnle, yaxunl, tony-tye, llvm-commits Differential Revision: https://reviews.llvm.org/D25637 llvm-svn: 285367
* AMDGPU/SI: Fix unused variable warning on non-debug buildsTom Stellard2016-10-271-0/+1
| | | | llvm-svn: 285363
* AMDGPU/SI: Handle hazard with > 8 byte VMEM storesTom Stellard2016-10-271-20/+107
| | | | | | | | | | Reviewers: arsenm Subscribers: kzhuravl, wdng, nhaehnle, yaxunl, tony-tye, llvm-commits Differential Revision: https://reviews.llvm.org/D25577 llvm-svn: 285359
* AMDGPU/SI: Handle s_setreg hazard in GCNHazardRecognizerTom Stellard2016-10-271-0/+19
| | | | | | | | | | Reviewers: arsenm Subscribers: kzhuravl, wdng, nhaehnle, yaxunl, llvm-commits, tony-tye Differential Revision: https://reviews.llvm.org/D25528 llvm-svn: 285338
* AMDGPU/SI: Handle s_getreg hazard in GCNHazardRecognizerTom Stellard2016-10-151-0/+47
| | | | | | | | | | Reviewers: arsenm Subscribers: kzhuravl, wdng, nhaehnle, yaxunl, llvm-commits, tony-tye Differential Revision: https://reviews.llvm.org/D25526 llvm-svn: 284298
* AMDGPU/SI: Handle div_fmas hazard in GCNHazardRecognizerTom Stellard2016-10-071-0/+22
| | | | | | | | | | Reviewers: arsenm Subscribers: kzhuravl, wdng, nhaehnle, yaxunl, tony-tye, llvm-commits Differential Revision: https://reviews.llvm.org/D25250 llvm-svn: 283622
* AMDGPU: Cleanup subtarget handling.Matt Arsenault2016-06-241-17/+11
| | | | | | | | | Split AMDGPUSubtarget into amdgcn/r600 specific subclasses. This removes most of the static_casting of the basic codegen classes everywhere, and tries to restrict the features visible on the wrong target. llvm-svn: 273652
* Move instances of std::function.Benjamin Kramer2016-06-121-2/+2
| | | | | | Or replace with llvm::function_ref if it's never stored. NFC intended. llvm-svn: 272513
* Silence unused variable warning; NFC.Aaron Ballman2016-05-031-2/+1
| | | | llvm-svn: 268392
* AMDGPU/SI: Use the hazard recognizer to break SMEM soft clausesTom Stellard2016-05-021-2/+70
| | | | | | | | | | | | | | | Summary: Add support for detecting hazards in SMEM soft clauses, so that we only break the clauses when necessary, either by adding s_nop or re-ordering other alu instructions. Reviewers: arsenm Subscribers: arsenm, llvm-commits Differential Revision: http://reviews.llvm.org/D18870 llvm-svn: 268260
* AMDGPU/SI: Use hazard recognizer to detect DPP hazardsTom Stellard2016-05-021-0/+26
| | | | | | | | | | Reviewers: arsenm Subscribers: arsenm, llvm-commits Differential Revision: http://reviews.llvm.org/D18603 llvm-svn: 268247
* Silence unused variable warnings; NFC.Aaron Ballman2016-05-021-9/+4
| | | | llvm-svn: 268234
* AMDGPU/SI: Enable the post-ra schedulerTom Stellard2016-04-301-0/+182
Summary: This includes a hazard recognizer implementation to replace some of the hazard handling we had during frame index elimination. Reviewers: arsenm Subscribers: qcolombet, arsenm, llvm-commits Differential Revision: http://reviews.llvm.org/D18602 llvm-svn: 268143
OpenPOWER on IntegriCloud