summaryrefslogtreecommitdiffstats
path: root/llvm/lib/Target/AMDGPU
Commit message (Collapse)AuthorAgeFilesLines
...
* AMDGPU: Remove deprecated llvm.AMDGPU.kilp intrinsicTom Stellard2018-04-243-11/+0
| | | | | | | | | | | | | | Summary: This is no longer used by mesa since its 18.0.0 release. Reviewers: nhaehnle Reviewed By: nhaehnle Subscribers: arsenm, kzhuravl, wdng, yaxunl, dstuttard, tpr, llvm-commits, t-tye Differential Revision: https://reviews.llvm.org/D45988 llvm-svn: 330775
* AMDGPU/GlobalISel: Fall-back to SelectionDAG for non-void functionsTom Stellard2018-04-241-0/+4
| | | | | | | | | | | | Reviewers: arsenm, nhaehnle Reviewed By: nhaehnle Subscribers: kzhuravl, wdng, nhaehnle, yaxunl, rovka, kristof.beyls, dstuttard, tpr, t-tye, llvm-commits Differential Revision: https://reviews.llvm.org/D45843 llvm-svn: 330774
* AMDGPU/GlobalISel: Add support for amdgpu_ps calling conventionTom Stellard2018-04-241-14/+49
| | | | | | | | | | | | Reviewers: arsenm Reviewed By: arsenm Subscribers: kzhuravl, wdng, nhaehnle, yaxunl, rovka, kristof.beyls, dstuttard, tpr, t-tye, llvm-commits Differential Revision: https://reviews.llvm.org/D45837 llvm-svn: 330767
* [AMDGPU] Truncate packed inline constantStanislav Mekhanoshin2018-04-242-1/+8
| | | | | | | | | | | | | | | | If a packed inline constant is sign extended it must be truncated after the shift. I.e. a constant (0xH0000, 0xHBC00), will be represented as 0xFFFFFFFFBC000000 in the IR because the immediate is sign extended to 64 bit. After the value shifted right by 16 to use it in a low part with op_sel_hi it becomes 0xFFFFFFFFBC00 and does not qualify as inline constant any longer. Fixed the error and added verification code. Without the fix and with the verification bug is causing pk_max_f16_literal.ll to fail. Differential Revision: https://reviews.llvm.org/D45987 llvm-svn: 330752
* [AMDGPU][Waitcnt] NFC. Cleanup some code/naming consistency:Mark Searles2018-04-241-38/+38
| | | | | | - s/SWaitcnt/Waitcnt s/WaitCnt/Waitcnt llvm-svn: 330730
* AMDGPU: Move a flawed assert when spilling SGPRsMatt Arsenault2018-04-232-4/+4
| | | | | | | | It's possible to validly spill the frame offset register in a call sequence to a VGPR. There are definitely issues with SGPR spilling to memory, so move the assert later. llvm-svn: 330612
* AMDGPU: Assign enum name to stack IDMatt Arsenault2018-04-233-2/+10
| | | | | | | | | Also assert that it is correct for SGPRs. There is currently a bug where stack slot coloring replaces SGPR spill FIs with one with the default ID, which results in a more confusing assert later about a dead object. llvm-svn: 330607
* AMDGPU: Fix SDWA peephole for V_AND_B32Nicolai Haehnle2018-04-231-1/+1
| | | | | | | | | | | | | | | | | | | Summary: Found by inspection. We care about the operand that *doesn't* contain the immediate. I believe this is currently not hit because we fold 0xff / 0xffff immediates only later. Change-Id: Ic3cf8538bc7da5eff3200d96eccf9d339e6345a7 Reviewers: arsenm, rampitec Subscribers: kzhuravl, wdng, yaxunl, dstuttard, tpr, t-tye, llvm-commits Differential Revision: https://reviews.llvm.org/D45886 llvm-svn: 330586
* AMDGPU: Fix a corner case crash in SIOptimizeExecMaskingNicolai Haehnle2018-04-231-1/+1
| | | | | | | | | | | | | | | | Summary: See the new test case; this is really unlikely to happen with real code, but I ran into this while attempting to bugpoint-reduce a different issue. Change-Id: I9ade1dc1aa8fd9c4d9fc83661d7b80e310b5c4a6 Reviewers: arsenm, rampitec Subscribers: kzhuravl, wdng, yaxunl, dstuttard, tpr, t-tye, llvm-commits Differential Revision: https://reviews.llvm.org/D45885 llvm-svn: 330585
* Consistently sort add_subdirectory calls in lib/Target/*/CMakeLists.txtNico Weber2018-04-231-2/+2
| | | | llvm-svn: 330584
* AMDGPU: Legalize the operand of SI_INIT_M0Nicolai Haehnle2018-04-201-0/+15
| | | | | | | | | | | | | | | | | | | | Summary: This fixes a case where the argument to a sendmsg intrinsic ends up in a VGPR, for whatever reason. The underlying performance issue is that a multiplication that can be an s_mul_i32 is instead needlessly generated as v_mul_u32_u24, but this is not addressed by this patch. Change-Id: I61fd4034314d5acdf6074632c30b65364dfa7328 Reviewers: arsenm, rampitec Subscribers: kzhuravl, wdng, yaxunl, dstuttard, tpr, t-tye, llvm-commits Differential Revision: https://reviews.llvm.org/D45826 llvm-svn: 330393
* [AMDGPU] Use packed literals with zero either lower or hi partStanislav Mekhanoshin2018-04-192-2/+21
| | | | | | Differential Revision: https://reviews.llvm.org/D45790 llvm-svn: 330365
* [AMDGPU] Do not only rely on BB number when finding bottom loopMark Searles2018-04-191-20/+45
| | | | | | | | We should also check that the "bottom" basic block of a loopis a successor of the "header" basic block, otherwise we don't propagate the information correctly when the CFG is complex. This fixes an important rendering problem with Wolfsentein 2, because of one vector-memory wait was missing. Differential Revision: https://reviews.llvm.org/D43831 llvm-svn: 330337
* [AMDGPU] Fix issues for backend divergence trackingDavid Stuttard2018-04-181-4/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: A change to use divergence analysis in the AMDGPU backend was getting formal arguments incorrect (not tagged as divergent) unless they were VGPR0, VGPR1 or VGPR2 For graphics shaders it is possible to have more than these passed in as VGPR Modified the checking code to check for any VGPR registers passed in as formal arguments. Also, some intrinsics that are sources of divergence may have been lowered during instruction selection and are missed on subsequent calls to isSDNodeSourceOfDivergence - added the relevant AMDGPUISD checks as well. Finally, the FunctionLoweringInfo tracks virtual registers that are live across basic block boundaries. This is used to check for divergence of CopyFromRegister registers using the DivergenceAnalysis analysis. For multiple blocks the lazily evaluated inverted map VirtReg2Value was not cleared when the ValueMap map was. Subscribers: arsenm, kzhuravl, wdng, nhaehnle, yaxunl, tpr, t-tye, llvm-commits Differential Revision: https://reviews.llvm.org/D45372 Change-Id: I112f3bd6dfe0f62e63ce9b43b893982778e4bee3 llvm-svn: 330257
* [AMDGPU] Enabled v2.16 literals for VOP3PStanislav Mekhanoshin2018-04-172-8/+19
| | | | | | | | Literal encoding needs op_sel_hi to select low 16 bit in this case. Differential Revision: https://reviews.llvm.org/D45745 llvm-svn: 330230
* [AMDGPU][MC][VI][GFX9] Added support of SDWA/DPP for v_cndmask_b32Dmitry Preobrazhensky2018-04-162-1/+27
| | | | | | | | | See bug 36356: https://bugs.llvm.org/show_bug.cgi?id=36356 Differential Revision: https://reviews.llvm.org/D45446 Reviewers: artem.tamazov, arsenm, timcorringham llvm-svn: 330123
* [NFC] fix trivial typos in document and commentsHiroshi Inoue2018-04-141-1/+1
| | | | | | "not not" -> "not" etc llvm-svn: 330083
* [NFC] fix trivial typos in commentsHiroshi Inoue2018-04-132-2/+2
| | | | | | "the the" -> "the", "we we" -> "we", etc llvm-svn: 330006
* [MachineScheduler] NFC refactoringJonas Paulsson2018-04-121-26/+32
| | | | | | | | | | | | | | | | This patch makes tryCandidate() virtual and some utility functions like tryLess(), tryGreater(), ... externally available (used to be static). This makes it possible for a target to derive a new MachineSchedStrategy from GenericScheduler and reuse most parts. It was necessary to wrap functions with the same names in AMDGPU/SIMachineScheduler in a local namespace. Review: Andy Trick, Florian Hahn https://reviews.llvm.org/D43329 llvm-svn: 329884
* [AMDGPU] Ensure there are enough registers for wave dispatchTim Renouf2018-04-111-0/+13
| | | | | | | | | | | | | | | | | Summary: This fixes the number of SGPRs and VGPRs in the *_RSRC1 register to allow for registers set up in wave dispatch, even if those registers are not used in the shader. Re-landed after noticing that the buildbot failure from 329808 seemed to be unrelated. Subscribers: arsenm, kzhuravl, wdng, nhaehnle, yaxunl, dstuttard, t-tye, llvm-commits Differential Revision: https://reviews.llvm.org/D45503 Change-Id: I6575f0e0d2a528d1319d0b289f0ebe4510fa5771 llvm-svn: 329826
* [AMDGPU] Fix lowering enqueue_kernelYaxun Liu2018-04-111-20/+30
| | | | | | | | | | | | | | | | | | Two issues were fixed: runtime has difficulty to allocate memory for an external symbol of a kernel and set the address of the external symbol, therefore make the runtime handle of an enqueued kernel an ordinary global variable. Runtime only needs to store the address of the loaded kernel to the handle and has verified that this approach works. handle the situation where __enqueue_kernel* gets inlined therefore the enqueued kernel may be used through a constant expr instead of an instruction. Differential Revision: https://reviews.llvm.org/D45187 llvm-svn: 329815
* Revert "[AMDGPU] Ensure there are enough registers for wave dispatch"Tim Renouf2018-04-111-13/+0
| | | | | | | | | This reverts 329808. That change caused a report of a failure in test/CodeGen/MIR/AMDGPU/mir-canon-multi.mir that I didn't see. I suspect it is an expensive-check-only error. Change-Id: I8133f26f15e7d5ec2b09c687c12cd70e918461b0 llvm-svn: 329811
* [AMDGPU] Ensure there are enough registers for wave dispatchTim Renouf2018-04-111-0/+13
| | | | | | | | | | | | | | Summary: This fixes the number of SGPRs and VGPRs in the *_RSRC1 register to allow for registers set up in wave dispatch, even if those registers are not used in the shader. Subscribers: arsenm, kzhuravl, wdng, nhaehnle, yaxunl, dstuttard, t-tye, llvm-commits Differential Revision: https://reviews.llvm.org/D45503 Change-Id: I6575f0e0d2a528d1319d0b289f0ebe4510fa5771 llvm-svn: 329808
* [AMDGPU][MC][GFX9] Added v_screen_partition_4se_b32Dmitry Preobrazhensky2018-04-111-4/+30
| | | | | | | | | See bug 36845: https://bugs.llvm.org/show_bug.cgi?id=36845 Differential Revision: https://reviews.llvm.org/D45443 Reviewers: artem.tamazov, arsenm, timcorringham llvm-svn: 329801
* AMDGPU: enable 128-bit for local addr space under an optionMarek Olsak2018-04-105-12/+17
| | | | | | | | | | | | | | | | | | | Author: Samuel Pitoiset ds_read_b128 and ds_write_b128 have been recently enabled under the amdgpu-ds128 option because the performance benefit is unclear. Though, using 128-bit loads/stores for the local address space appears to introduce regressions in tessellation shaders. Not sure what is broken, but as ds_read_b128/ds_write_b128 are not enabled by default, just introduce a global option and enable 128-bit only if requested (until it's fixed/used correctly). v2: - fix regressions in merge-stores.ll and multiple_tails.ll Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=105464 llvm-svn: 329764
* AMDGPU/MC: Allow disassembling without symbol infoNicolai Haehnle2018-04-101-0/+3
| | | | | | | | | | | | | | | | | | | | | Summary: We would like the UMR debugging tool[0] to be able to provide disassembly for currently live waves based on plain memory dumps, and we want to leverage the LLVM disassembler for this. This mostly works, except that UMR clearly can't provide real symbol info, so it wants to set DisInfo == nullptr. [0] https://cgit.freedesktop.org/amd/umr/ Reviewers: arsenm, rampitec, artem.tamazov, dp Subscribers: kzhuravl, wdng, yaxunl, dstuttard, tpr, t-tye, llvm-commits Differential Revision: https://reviews.llvm.org/D45477 Change-Id: Ibb2c5af2e66f2e100b4702fd81308e1932bc4ee6 llvm-svn: 329715
* [AMDGPU] For OS type AMDPAL, fixed scratch on compute shaderTim Renouf2018-04-101-2/+6
| | | | | | | | | | | | | | | | | | | | | Summary: For OS type AMDPAL, the scratch descriptor is loaded from offset 0 of the GIT, whose 32 bit pointer is in s0 (s8 for gfx9 merged shaders). This commit fixes that to use offset 0x10 instead of offset 0 for a compute shader, per the PAL ABI spec. V2: Ensure s0 (s8 for gfx9 merged shader) is marked live-in when loading scratch descriptor from GIT. Reviewers: kzhuravl, nhaehnle, timcorringham Subscribers: kzhuravl, wdng, yaxunl, t-tye, llvm-commits, dstuttard, nhaehnle, arsenm Differential Revision: https://reviews.llvm.org/D44468 Change-Id: I93dffa647758e37f613bb5e0dfca840d82e6d26f llvm-svn: 329690
* AMDGPU: Remove max_scratch_backing_memory_byte_size from kernel headerKonstantin Zhuravlyov2018-04-094-10/+10
| | | | | | | | | | | 1. Remove max_scratch_backing_memory_byte_size from kernel header 2. Make it a reserved field 3. Ignore it while parsing assembly for backwards compatibility 4. Bump up minor version of kernel header Differential Revision: https://reviews.llvm.org/D45452 llvm-svn: 329620
* Revert "AMDGPU: enable 128-bit for local addr space under an option"Alex Shlyapnikov2018-04-095-17/+12
| | | | | | | | | | | | | | This reverts commit r329591. It breaks various bots: http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-fast/builds/16516 http://lab.llvm.org:8011/builders/clang-ppc64be-linux/builds/17374 http://lab.llvm.org:8011/builders/clang-ppc64le-linux/builds/15992 http://lab.llvm.org:8011/builders/clang-ppc64be-linux-lnt http://lab.llvm.org:8011/builders/clang-ppc64le-linux-lnt/builds/11251 ... llvm-svn: 329610
* AMDGPU: enable 128-bit for local addr space under an optionMarek Olsak2018-04-095-12/+17
| | | | | | | | | | | | | | | | | Author: Samuel Pitoiset ds_read_b128 and ds_write_b128 have been recently enabled under the amdgpu-ds128 option because the performance benefit is unclear. Though, using 128-bit loads/stores for the local address space appears to introduce regressions in tessellation shaders. Not sure what is broken, but as ds_read_b128/ds_write_b128 are not enabled by default, just introduce a global option and enable 128-bit only if requested (until it's fixed/used correctly). Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=105464 llvm-svn: 329591
* AMDGPU: Initialize GlobalISel passesTom Stellard2018-04-091-0/+1
| | | | | | | | | | | | | | Summary: This fixes AMDGPU GlobalISel test failures when enabling the AMDGPU target without any other targets that use GlobalISel. Reviewers: arsenm Subscribers: kzhuravl, wdng, nhaehnle, yaxunl, rovka, kristof.beyls, dstuttard, tpr, llvm-commits, t-tye Differential Revision: https://reviews.llvm.org/D45353 llvm-svn: 329588
* [AMDGPU][MC][GFX9] Added instructions s_mul_hi_*32, s_lshl*_add_u32Dmitry Preobrazhensky2018-04-091-0/+21
| | | | | | | | | | | See bugs 36841: https://bugs.llvm.org/show_bug.cgi?id=36841 36842: https://bugs.llvm.org/show_bug.cgi?id=36842 Differential Revision: https://reviews.llvm.org/D45251 Reviewers: artem.tamazov, arsenm, timcorringham llvm-svn: 329562
* [AMDGPU][MC][GFX9] Added s_call_b64Dmitry Preobrazhensky2018-04-061-0/+12
| | | | | | | | | See bug 36843: https://bugs.llvm.org/show_bug.cgi?id=36843 Differential Revision: https://reviews.llvm.org/D45268 Reviewers: artem.tamazov, arsenm, timcorringham llvm-svn: 329440
* [AMDGPU][MC][GFX9] Added instruction s_endpgm_ordered_ps_doneDmitry Preobrazhensky2018-04-061-0/+7
| | | | | | | | | See bug 36844: https://bugs.llvm.org/show_bug.cgi?id=36844 Differential Revision: https://reviews.llvm.org/D45313 Reviewers: artem.tamazov, arsenm, timcorringham llvm-svn: 329430
* [AMDGPU][MC][GFX9] Added instructions *saveexec*, *wrexec* and *bitreplicate*Dmitry Preobrazhensky2018-04-061-0/+21
| | | | | | | | | See bug 36840: https://bugs.llvm.org/show_bug.cgi?id=36840 Differential Revision: https://reviews.llvm.org/D45250 Reviewers: artem.tamazov, arsenm, timcorringham llvm-svn: 329419
* [AMDGPU][MC][VI][GFX9] Added s_atc_probe* instructionsDmitry Preobrazhensky2018-04-061-0/+28
| | | | | | | | | See bug 36839: https://bugs.llvm.org/show_bug.cgi?id=36839 Differential Revision: https://reviews.llvm.org/D45249 Reviewers: artem.tamazov, arsenm, timcorringham llvm-svn: 329408
* [AMDGPU][MC][GFX9] Added s_dcache_discard* instructionsDmitry Preobrazhensky2018-04-061-0/+30
| | | | | | | | | See bug 36838: https://bugs.llvm.org/show_bug.cgi?id=36838 Differential Revision: https://reviews.llvm.org/D45247 Reviewers: artem.tamazov, arsenm, timcorringham llvm-svn: 329397
* AMDGPU/Metadata: Always report a fixed number of hidden argumentsKonstantin Zhuravlyov2018-04-051-8/+12
| | | | | | | | | | Currently it is 6. If the "feature" was not used, report dummy hidden argument. Otherwise it does not match the kernarg size reported in the kernel header. Differential Revision: https://reviews.llvm.org/D45129 llvm-svn: 329341
* [SchedModel] Complete models shouldn't match against itineraries when they ↵Simon Pilgrim2018-04-051-1/+1
| | | | | | | | | | | | don't use them (PR35639) For schedule models that don't use itineraries, checkCompleteness still checks that an instruction has a matching itinerary instead of skipping and going straight to matching the InstRWs. That doesn't seem to match what happens in TargetSchedule.cpp This patch causes problems for a number of models that had been incorrectly flagged as complete. Differential Revision: https://reviews.llvm.org/D43235 llvm-svn: 329280
* [AMDGPU][MC] Enabled instruction TBUFFER_LOAD_FORMAT_XYZ for SI/CIDmitry Preobrazhensky2018-04-041-1/+1
| | | | | | | | | See bug 36958: https://bugs.llvm.org/show_bug.cgi?id=36958 Differential Revision: https://reviews.llvm.org/D45099 Reviewers: artem.tamazov, arsenm, timcorringham llvm-svn: 329197
* [AMDGPU][MC] Added support of 3-element addresses for MIMG instructionsDmitry Preobrazhensky2018-04-041-1/+9
| | | | | | | | | See bug 35999: https://bugs.llvm.org/show_bug.cgi?id=35999 Differential Revision: https://reviews.llvm.org/D45084 Reviewers: artem.tamazov, arsenm, timcorringham llvm-svn: 329187
* Sort targetgen calls in lib/Target/*/CMakeLists.Nico Weber2018-04-041-8/+9
| | | | | | | | | | | Makes it easier to see mistakes such as the one fixed in r329178 and makes the different target CMakeLists more consistent. Also remove some stale-looking comments from the Nios2 target cmakefile. No intended behavior change. llvm-svn: 329181
* AMDGPU: Dimension-aware image intrinsicsNicolai Haehnle2018-04-046-3/+271
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: These new image intrinsics contain the texture type as part of their name and have each component of the address/coordinate as individual parameters. This is a preparatory step for implementing the A16 feature, where coordinates are passed as half-floats or -ints, but the Z compare value and texel offsets are still full dwords, making it difficult or impossible to distinguish between A16 on or off in the old-style intrinsics. Additionally, these intrinsics pass the 'texfailpolicy' and 'cachectrl' as i32 bit fields to reduce operand clutter and allow for future extensibility. v2: - gather4 supports 2darray images - fix a bug with 1D images on SI Change-Id: I099f309e0a394082a5901ea196c3967afb867f04 Reviewers: arsenm, rampitec, b-sumner Subscribers: kzhuravl, wdng, yaxunl, dstuttard, tpr, llvm-commits, t-tye Differential Revision: https://reviews.llvm.org/D44939 llvm-svn: 329166
* AMDGPU: Fix copying i1 value out of loop with non-uniform exitNicolai Haehnle2018-04-044-1/+103
| | | | | | | | | | | | | | | | | | | | | | | | Summary: When an i1-value is defined inside of a loop and used outside of it, we cannot simply use the SGPR bitmask from the loop's last iteration. There are also useful and correct cases of an i1-value being copied between basic blocks, e.g. when a condition is computed outside of a loop and used inside it. The concept of dominators is not sufficient to capture what is going on, so I propose the notion of "lane-dominators". Fixes a bug encountered in Nier: Automata. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=103743 Change-Id: If37b969ddc71d823ab3004aeafb9ea050e45bd9a Reviewers: arsenm, rampitec Subscribers: kzhuravl, wdng, mgorny, yaxunl, dstuttard, tpr, llvm-commits, t-tye Differential Revision: https://reviews.llvm.org/D40547 llvm-svn: 329164
* [AMDGPU] performMinMaxCombine should not optimize patterns of vectors to ↵Farhana Aleen2018-04-031-1/+1
| | | | | | | | | | | | | | | | min3/max3. Summary: There are no packed instructions for min3 or max3. So, performMinMaxCombine should not optimize vectors of f16 to min3/max3. Author: FarhanaAleen Reviewed By: arsenm Subscribers: llvm-commits, AMDGPU Differential Revision: https://reviews.llvm.org/D45219 llvm-svn: 329131
* Revert "MSG"Farhana Aleen2018-04-031-1/+1
| | | | | | | | This reverts commit 9a0ce889d1c39c74d69ecad5ce9c875155ae55de. This was committed by mistake. llvm-svn: 329119
* MSGFarhana Aleen2018-04-031-1/+1
| | | | llvm-svn: 329114
* [AMDGPU][MC][GFX9] Added instructions v_cvt_norm_*16_f16, v_sat_pk_u8_i16Dmitry Preobrazhensky2018-04-021-0/+8
| | | | | | | | | See bug 36847: https://bugs.llvm.org/show_bug.cgi?id=36847 Differential Revision: https://reviews.llvm.org/D45097 Reviewers: artem.tamazov, arsenm, timcorringham llvm-svn: 328988
* [AMDGPU][MC][GFX9] Added s_atomic_* and s_buffer_atomic_* instructionsDmitry Preobrazhensky2018-04-024-1/+204
| | | | | | | | | | | Fixed a bug which caused Tablegen crash. See bug 36837: https://bugs.llvm.org/show_bug.cgi?id=36837 Differential Revision: https://reviews.llvm.org/D45085 Reviewers: artem.tamazov, arsenm, timcorringham llvm-svn: 328983
* Revert r328975, it makes TableGen assert on the bots.Nico Weber2018-04-024-204/+1
| | | | llvm-svn: 328978
OpenPOWER on IntegriCloud