summaryrefslogtreecommitdiffstats
path: root/llvm/test/CodeGen/BPF
Commit message (Collapse)AuthorAgeFilesLines
...
* [BPF] Do not generate BTF sections unnecessarilyYonghong Song2019-03-0531-331/+22
| | | | | | | | | | | | If There is no types/non-empty strings, do not generate .BTF section. If there is no func_info/line_info, do not generate .BTF.ext section. Signed-off-by: Yonghong Song <yhs@fb.com> Differential Revision: https://reviews.llvm.org/D58936 llvm-svn: 355360
* bpf: enable sub-register code-gen for XADDJiong Wang2019-02-282-0/+28
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Support sub-register code-gen for XADD is like supporting any other Load and Store patterns. No new instruction is introduced. lock *(u32 *)(r1 + 0) += w2 has exactly the same underlying insn as: lock *(u32 *)(r1 + 0) += r2 BPF_W width modifier has guaranteed they behave the same at runtime. This patch merely teaches BPF back-end that BPF_W width modifier could work GPR32 register class and that's all needed for sub-register code-gen support for XADD. test/CodeGen/BPF/xadd.ll updated to include sub-register code-gen tests. A new testcase test/CodeGen/BPF/xadd_legal.ll is added to make sure the legal case could pass on all code-gen modes. It could also test dead Def check on GPR32. If there is no proper handling like what has been done inside BPFMIChecking.cpp:hasLivingDefs, then this testcase will fail. Acked-by: Yonghong Song <yhs@fb.com> Signed-off-by: Jiong Wang <jiong.wang@netronome.com> llvm-svn: 355126
* [BPF] Don't fail for static variablesYonghong Song2019-02-271-0/+38
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently, the LLVM will print an error like Unsupported relocation: try to compile with -O2 or above, or check your static variable usage if user defines more than one static variables in a single ELF section (e.g., .bss or .data). There is ongoing effort to support static and global variables in libbpf and kernel. This patch removed the assertion so user programs with static variables won't fail compilation. The static variable in-section offset is written to the "imm" field of the corresponding to-be-relocated bpf instruction. Below is an example to show how the application (e.g., libbpf) can relate variable to relocations. -bash-4.4$ cat g1.c static volatile long a = 2; static volatile int b = 3; int test() { return a + b; } -bash-4.4$ clang -target bpf -O2 -c g1.c -bash-4.4$ llvm-readelf -r g1.o Relocation section '.rel.text' at offset 0x158 contains 2 entries: Offset Info Type Symbol's Value Symbol's Name 0000000000000000 0000000400000001 R_BPF_64_64 0000000000000000 .data 0000000000000018 0000000400000001 R_BPF_64_64 0000000000000000 .data -bash-4.4$ llvm-readelf -s g1.o Symbol table '.symtab' contains 6 entries: Num: Value Size Type Bind Vis Ndx Name 0: 0000000000000000 0 NOTYPE LOCAL DEFAULT UND 1: 0000000000000000 0 FILE LOCAL DEFAULT ABS g1.c 2: 0000000000000000 8 OBJECT LOCAL DEFAULT 4 a 3: 0000000000000008 4 OBJECT LOCAL DEFAULT 4 b 4: 0000000000000000 0 SECTION LOCAL DEFAULT 4 5: 0000000000000000 64 FUNC GLOBAL DEFAULT 2 test -bash-4.4$ llvm-objdump -d g1.o g1.o: file format ELF64-BPF Disassembly of section .text: 0000000000000000 test: 0: 18 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 r1 = 0 ll 2: 79 11 00 00 00 00 00 00 r1 = *(u64 *)(r1 + 0) 3: 18 02 00 00 08 00 00 00 00 00 00 00 00 00 00 00 r2 = 8 ll 5: 61 20 00 00 00 00 00 00 r0 = *(u32 *)(r2 + 0) 6: 0f 10 00 00 00 00 00 00 r0 += r1 7: 95 00 00 00 00 00 00 00 exit -bash-4.4$ . from symbol table, static variable "a" is in section #4, offset 0. . from symbol table, static variable "b" is in section #4, offset 8. . the first relocation is against symbol #4: 4: 0000000000000000 0 SECTION LOCAL DEFAULT 4 and in-section offset 0 (see llvm-objdump result) . the second relocation is against symbol #4: 4: 0000000000000000 0 SECTION LOCAL DEFAULT 4 and in-section offset 8 (see llvm-objdump result) . therefore, the first relocation is for variable "a", and the second relocation is for variable "b". Acked-by: Alexei Starovoitov <ast@kernel.org> Signed-off-by: Yonghong Song <yhs@fb.com> llvm-svn: 354954
* [BPF] make test case reloc-btf.ll tolerable for old compilersYonghong Song2019-02-201-2/+2
| | | | | | | | | | | | | | | The test case reloc-btf.ll is generated with an IR containing spFlags introduced by https://reviews.llvm.org/rL347806. In the case of BTF backporting, the old compiler may not have this patch, so this test will fail during validation. This patch removed spFlags from IR in the test case and used the old way for various flags. Acked-by: Alexei Starovoitov <ast@kernel.org> Signed-off-by: Yonghong Song <yhs@fb.com> llvm-svn: 354409
* Revert r354244 "[DAGCombiner] Eliminate dead stores to stack."Clement Courbet2019-02-181-6/+6
| | | | | | Breaks some bots. llvm-svn: 354245
* [DAGCombiner] Eliminate dead stores to stack.Clement Courbet2019-02-181-6/+6
| | | | | | | | | | | | | | | Summary: A store to an object whose lifetime is about to end can be removed. See PR40550 for motivation. Reviewers: niravd Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D57541 llvm-svn: 354244
* [BPF] [BTF] Process FileName with absolute path correctlyYonghong Song2019-02-021-0/+81
| | | | | | | | | | | | | | | | | In IR, sometimes the following attributes for DIFile may be generated: filename: /home/yhs/test.c directory: /tmp The /tmp may represent the working directory of the compilation process. In such cases, since filename is with absolute path, the directory should be ignored by BTF. The filename alone is enough to get the source. Acked-by: Alexei Starovoitov <ast@kernel.org> Signed-off-by: Yonghong Song <yhs@fb.com> llvm-svn: 352952
* Revert "[BPF] [BTF] Process FileName with absolute path correctly"Yonghong Song2019-02-011-83/+0
| | | | | | | | This reverts commit r352939. Some tests failed. Revert to unblock others. llvm-svn: 352941
* [BPF] [BTF] Process FileName with absolute path correctlyYonghong Song2019-02-011-0/+83
| | | | | | | | | | | | | | | | | In IR, sometimes the following attributes for DIFile may be generated: filename: /home/yhs/test.c directory: /tmp The /tmp may represent the working directory of the compilation process. In such cases, since filename is with absolute path, the directory should be ignored by BTF. The filename alone is enough to get the source. Acked-by: Alexei Starovoitov <ast@kernel.org> Signed-off-by: Yonghong Song <yhs@fb.com> llvm-svn: 352939
* [BPF] Fix .BTF.ext reloc type assigment issueYonghong Song2019-01-081-0/+33
| | | | | | | | | | | | | | | | | | | | Commit f1db33c5c1a9 ("[BPF] Disable relocation for .BTF.ext section") assigned relocation type R_BPF_NONE if the fixup type is FK_Data_4 and the symbol is temporary. The reason is we use FK_Data_4 as a fixup type for insn offsets in .BTF.ext section. Just checking whether the symbol is temporary is not enough. For example, .debug_info may reference some strings whose fixup is FK_Data_4 with a temporary symbol as well. To truely reflect the case for .BTF.ext section, this patch further checks that the section associateed with the symbol must be SHF_ALLOC and SHF_EXECINSTR, i.e., in the text section. This fixed the above-mentioned problem. Signed-off-by: Yonghong Song <yhs@fb.com> llvm-svn: 350637
* [BPF] Generate BTF DebugInfo under BPF targetYonghong Song2018-12-1937-0/+2615
| | | | | | | | | | | | | | | | | | | | | | | | | | | This patch implements BTF (BPF Type Format). The BTF is the debug info format for BPF, introduced in the below linux patch: https://github.com/torvalds/linux/commit/69b693f0aefa0ed521e8bd02260523b5ae446ad7#diff-06fb1c8825f653d7e539058b72c83332 and further extended several times, e.g., https://www.spinics.net/lists/netdev/msg534640.html https://www.spinics.net/lists/netdev/msg538464.html https://www.spinics.net/lists/netdev/msg540246.html The main advantage of implementing in LLVM is: . better integration/deployment as no extra tools are needed. . bpf JIT based compilation (like bcc, bpftrace, etc.) can get BTF without much extra effort. . BTF line_info needs selective source codes, which can be easily retrieved when inside the compiler. This patch implemented BTF generation by registering a BPF specific DebugHandler in BPFAsmPrinter. Signed-off-by: Yonghong Song <yhs@fb.com> Differential Revision: https://reviews.llvm.org/D55752 llvm-svn: 349640
* [bpf] Test case for symbol information in object fileYonghong Song2018-09-221-0/+8
| | | | | | | This patch tests the change introduced in r342556. Signed-off-by: Paul Chaignon <paul.chaignon@orange.com> llvm-svn: 342807
* bpf: check illegal usage of XADD insn return valueYonghong Song2018-09-201-0/+59
| | | | | | | | | | | | | | | | | | | | | | | | | Currently, BPF has XADD (locked add) insn support and the asm looks like: lock *(u32 *)(r1 + 0) += r2 lock *(u64 *)(r1 + 0) += r2 The instruction itself does not have a return value. At the source code level, users often use __sync_fetch_and_add() which eventually translates to XADD. The return value of __sync_fetch_and_add() is supposed to be the old value in the xadd memory location. Since BPF::XADD insn does not support such a return value, this patch added a PreEmit phase to check such a usage. If such an illegal usage pattern is detected, a fatal error will be reported like line 4: Invalid usage of the XADD return value if compiled with -g, or Invalid usage of the XADD return value if compiled without -g. Signed-off-by: Yonghong Song <yhs@fb.com> Acked-by: Alexei Starovoitov <ast@kernel.org> llvm-svn: 342692
* bpf: add missing RegState to notify MachineInstr verifier necessary register ↵Yonghong Song2018-07-271-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | usage Errors like the following are reported by: https://urldefense.proofpoint.com/v2/url?u=http-3A__lab.llvm.org-3A8011_builders_llvm-2Dclang-2Dx86-5F64-2Dexpensive-2Dchecks-2Dwin_builds_11261&d=DwIBAg&c=5VD0RTtNlTh3ycd41b3MUw&r=DA8e1B5r073vIqRrFz7MRA&m=929oWPCf7Bf2qQnir4GBtowB8ZAlIRWsAdTfRkDaK-g&s=9k-wbEUVpUm474hhzsmAO29VXVvbxJPWD9RTgCD71fQ&e= *** Bad machine code: Explicit definition marked as use *** - function: cal_align1 - basic block: %bb.0 entry (0x47edd98) - instruction: LDB $r3, $r2, 0 - operand 0: $r3 This is because RegState info was missing for ScratchReg inside expandMEMCPY. This caused incomplete register usage information to MachineInstr verifier which then would complain as there could be potential code-gen issue if the complained MachineInstr is used in place where register usage information matters even though the memcpy expanding is not in such case as it happens at the last stage of IR optimization pipeline. We should always specify those register usage information which compiler couldn't deduct automatically whenever we add a hardware register manually. Reported-by: Builder llvm-clang-x86_64-expensive-checks-win Build #11261 Signed-off-by: Jiong Wang <jiong.wang@netronome.com> Reviewed-by: Yonghong Song <yhs@fb.com> llvm-svn: 338134
* bpf: new option -bpf-expand-memcpy-in-order to expand memcpy in orderYonghong Song2018-07-251-0/+116
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Some BPF JIT backends would want to optimize memcpy in their own architecture specific way. However, at the moment, there is no way for JIT backends to see memcpy semantics in a reliable way. This is due to LLVM BPF backend is expanding memcpy into load/store sequences and could possibly schedule them apart from each other further. So, BPF JIT backends inside kernel can't reliably recognize memcpy semantics by peephole BPF sequence. This patch introduce new intrinsic expand infrastructure to memcpy. To get stable in-order load/store sequence from memcpy, we first lower memcpy into BPF::MEMCPY node which then expanded into in-order load/store sequences in expandPostRAPseudo pass which will happen after instruction scheduling. By this way, kernel JIT backends could reliably recognize memcpy through scanning BPF sequence. This new memcpy expand infrastructure is gated by a new option: -bpf-expand-memcpy-in-order Acked-by: Jakub Kicinski <jakub.kicinski@netronome.com> Signed-off-by: Jiong Wang <jiong.wang@netronome.com> Signed-off-by: Yonghong Song <yhs@fb.com> llvm-svn: 337977
* [DebugInfo] Add DILabel metadata and intrinsic llvm.dbg.label.Shiva Chen2018-05-093-5/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In order to set breakpoints on labels and list source code around labels, we need collect debug information for labels, i.e., label name, the function label belong, line number in the file, and the address label located. In order to keep these information in LLVM IR and to allow backend to generate debug information correctly. We create a new kind of metadata for labels, DILabel. The format of DILabel is !DILabel(scope: !1, name: "foo", file: !2, line: 3) We hope to keep debug information as much as possible even the code is optimized. So, we create a new kind of intrinsic for label metadata to avoid the metadata is eliminated with basic block. The intrinsic will keep existing if we keep it from optimized out. The format of the intrinsic is llvm.dbg.label(metadata !1) It has only one argument, that is the DILabel metadata. The intrinsic will follow the label immediately. Backend could get the label metadata through the intrinsic's parameter. We also create DIBuilder API for labels to be used by Frontend. Frontend could use createLabel() to allocate DILabel objects, and use insertLabel() to insert llvm.dbg.label intrinsic in LLVM IR. Differential Revision: https://reviews.llvm.org/D45024 Patch by Hsiangkai Wang. llvm-svn: 331841
* bpf: fix incorrect SELECT_CC loweringYonghong Song2018-04-033-3/+3
| | | | | | | | | | | | | | | | | | | | | | | Commit 37962a331c77 ("bpf: Improve expanding logic in LowerSELECT_CC") intended to improve code quality for certain jmp conditions. The commit, however, has a couple of issues: (1). In code, just swap is not enough, ConditionalCode CC should also be swapped, otherwise incorrect code will be generated. (2). The ConditionalCode swap should be subject to getHasJmpExt(). If getHasJmpExt() is False, certain conditional codes will not be supported and swap may generate incorrect code. The original goal for this patch is to optimize jmp operations which does not have JmpExt turned on. If JmpExt is on, better code could be generated. For example, the test select_ri.ll is introduced to demonstrate the optimization. The same result can be achieved with -mcpu=v2 flag. Signed-off-by: Yonghong Song <yhs@fb.com> Acked-by: Alexei Starovoitov <ast@kernel.org> llvm-svn: 329043
* bpf: Extends zero extension elimination beyond comparison instructionsYonghong Song2018-03-131-0/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | The current zero extension elimination was restricted to operands of comparison. It actually could be extended to more cases. For example: int *inc_p (int *p, unsigned a) { return p + a; } 'a' will be promoted to i64 during addition, and the zero extension could be eliminated as well. For the elimination optimization, it should be much better to start recognizing the candidate sequence from the SRL instruction instead of J* instructions. This patch makes it an generic zero extension elimination pass instead of one restricted with comparison. Signed-off-by: Jiong Wang <jiong.wang@netronome.com> Signed-off-by: Yonghong Song <yhs@fb.com> llvm-svn: 327367
* bpf: J*_RR should check both operandsYonghong Song2018-03-131-0/+20
| | | | | | | | | | | | | There is a mistake in current code that we "break" out the optimization when the first operand of J*_RR doesn't qualify the elimination. This caused some elimination opportunities missed, for example the one in the testcase. The code should just fall through to handle the second operand. Signed-off-by: Jiong Wang <jiong.wang@netronome.com> Signed-off-by: Yonghong Song <yhs@fb.com> llvm-svn: 327366
* bpf: Tighten subregister definition checkYonghong Song2018-03-131-1/+32
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The current subregister definition check stops after the MOV_32_64 instruction. This means we are thinking all the following instruction sequences are safe to be eliminated: MOV_32_64 rB, wA SLL_ri rB, rB, 32 SRL_ri rB, rB, 32 However, this is *not* true. The source subregister wA of MOV_32_64 could come from a implicit truncation of 64-bit register in which case the high bits of the 64-bit register is not zeroed, therefore we can't eliminate above sequence. For example, for i32_val, we shouldn't do the elimination: long long bar (); int foo (int b, int c) { unsigned int i32_val = (unsigned int) bar(); if (i32_val < 10) return b; else return c; } Signed-off-by: Jiong Wang <jiong.wang@netronome.com> Signed-off-by: Yonghong Song <yhs@fb.com> llvm-svn: 327365
* bpf: Add more check directives in peephole testcaseYonghong Song2018-03-131-0/+4
| | | | | | | | | | | Improve the test accuracy by adding more check directives. Shifts are expected to be eliminated for zero extension but not for signed extension. Signed-off-by: Jiong Wang <jiong.wang@netronome.com> Signed-off-by: Yonghong Song <yhs@fb.com> llvm-svn: 327364
* bpf: New codegen testcases for 32-bit subregister supportYonghong Song2018-02-234-0/+541
| | | | | | | | | | This patch adds some unit tests for 32-bit subregister support. We want to make sure ALU32, subregister load/store and new peephole optimization are truely enabled once -mattr=+alu32 specified. Signed-off-by: Jiong Wang <jiong.wang@netronome.com> Reviewed-by: Yonghong Song <yhs@fb.com> llvm-svn: 325992
* [DebugInfo] Support DWARF v5 source code embedding extensionScott Linder2018-02-231-1/+3
| | | | | | | | | | | | | | | | | | | In DWARF v5 the Line Number Program Header is extensible, allowing values with new content types. In this extension a content type is added, DW_LNCT_LLVM_source, which contains the embedded source code of the file. Add new optional attribute for !DIFile IR metadata called source which contains source text. Use this to output the source to the DWARF line table of code objects. Analogously extend METADATA_FILE in Bitcode and .file directive in ASM to support optional source. Teach llvm-dwarfdump and llvm-objdump about the new values. Update the output format of llvm-dwarfdump to make room for the new attribute on file_names entries, and support embedded sources for the -source option in llvm-objdump. Differential Revision: https://reviews.llvm.org/D42765 llvm-svn: 325970
* [BPF] Return true in enableMultipleCopyHints().Jonas Paulsson2018-02-189-50/+50
| | | | | | | | | | Enable multiple COPY hints to eliminate more COPYs during register allocation. Note that this is something all targets should do, see https://reviews.llvm.org/D38128. Review: Yonghong Song llvm-svn: 325457
* bpf: fix a bug in dag2dag optimization for loads from readonly sectionYonghong Song2018-02-151-0/+53
| | | | | | | | | | | | | | | | | | | | | The reference '&' is missing in the function parameter. If there are back-to-back optimizations in terms of dag node list like below: t29: i64,ch = load<LD4[bitcast (%struct.test_t* @test.t to i8*)+12](dereferenceable), zext from i32> t3, t43, undef:i64 t34: i64,ch = load<LD4[bitcast (%struct.test_t* @test.t to i8*)](dereferenceable), zext from i32> t3, t41, undef:i64 The bug will trigger a segfault for the added test case remove_truncate_5.ll: LLVMSymbolizer: error reading file: No such file or directory #0 0x000000000241c4d9 (llc+0x241c4d9) #1 0x000000000241c56a (llc+0x241c56a) #2 0x000000000241aa50 (llc+0x241aa50) ... #22 0x0000000000fd5edf (llc+0xfd5edf) #23 0x00007f0fe03bec05 __libc_start_main (/lib64/libc.so.6+0x21c05) #24 0x0000000000fd3e69 (llc+0xfd3e69) ... Segmentation fault Signed-off-by: Yonghong Song <yhs@fb.com> llvm-svn: 325267
* bpf: Improve expanding logic in LowerSELECT_CCYonghong Song2018-02-082-1/+26
| | | | | | | | | | | | | | | LowerSELECT_CC is not generating optimal Select_Ri pattern at the moment. It is not guaranteed to place ConstantNode at RHS which would miss matching Select_Ri. A new testcase added into the existing select_ri.ll, also there is an existing case in cmp.ll which would be improved to use Select_Ri after this patch, it is adjusted accordingly. Reported-by: Alexei Starovoitov <alexei.starovoitov@gmail.com> Reviewed-by: Yonghong Song <yhs@fb.com> Signed-off-by: Jiong Wang <jiong.wang@netronome.com> llvm-svn: 324560
* [DWARF] Regularize dumping strings from line tables.Paul Robinson2018-02-051-1/+1
| | | | | | | | | | | | | | | | | The major visible difference here is that in line-table dumps, directory and file names are wrapped in double-quotes; previously, directory names got single quotes and file names were not quoted at all. The improvement in this patch is that when a DWARF v5 line table header has indirect strings, in a verbose dump these will all have their section[offset] printed as well as the name itself. This matches the format used for dumping strings in the .debug_info section. Differential Revision: https://reviews.llvm.org/D42802 llvm-svn: 324270
* Followup on Proposal to move MIR physical register namespace to '$' sigil.Puyan Lotfi2018-01-311-1/+1
| | | | | | | | | | | | Discussed here: http://lists.llvm.org/pipermail/llvm-dev/2018-January/120320.html In preparation for adding support for named vregs we are changing the sigil for physical registers in MIR to '$' from '%'. This will prevent name clashes of named physical register with named vregs. llvm-svn: 323922
* Remove alignment argument from memcpy/memmove/memset in favour of alignment ↵Daniel Neilson2018-01-1911-25/+25
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | attributes (Step 1) Summary: This is a resurrection of work first proposed and discussed in Aug 2015: http://lists.llvm.org/pipermail/llvm-dev/2015-August/089384.html and initially landed (but then backed out) in Nov 2015: http://lists.llvm.org/pipermail/llvm-commits/Week-of-Mon-20151109/312083.html The @llvm.memcpy/memmove/memset intrinsics currently have an explicit argument which is required to be a constant integer. It represents the alignment of the dest (and source), and so must be the minimum of the actual alignment of the two. This change is the first in a series that allows source and dest to each have their own alignments by using the alignment attribute on their arguments. In this change we: 1) Remove the alignment argument. 2) Add alignment attributes to the source & dest arguments. We, temporarily, require that the alignments for source & dest be equal. For example, code which used to read: call void @llvm.memcpy.p0i8.p0i8.i32(i8* %dest, i8* %src, i32 100, i32 4, i1 false) will now read call void @llvm.memcpy.p0i8.p0i8.i32(i8* align 4 %dest, i8* align 4 %src, i32 100, i1 false) Downstream users may have to update their lit tests that check for @llvm.memcpy/memmove/memset call/declaration patterns. The following extended sed script may help with updating the majority of your tests, but it does not catch all possible patterns so some manual checking and updating will be required. s~declare void @llvm\.mem(set|cpy|move)\.p([^(]*)\((.*), i32, i1\)~declare void @llvm.mem\1.p\2(\3, i1)~g s~call void @llvm\.memset\.p([^(]*)i8\(i8([^*]*)\* (.*), i8 (.*), i8 (.*), i32 [01], i1 ([^)]*)\)~call void @llvm.memset.p\1i8(i8\2* \3, i8 \4, i8 \5, i1 \6)~g s~call void @llvm\.memset\.p([^(]*)i16\(i8([^*]*)\* (.*), i8 (.*), i16 (.*), i32 [01], i1 ([^)]*)\)~call void @llvm.memset.p\1i16(i8\2* \3, i8 \4, i16 \5, i1 \6)~g s~call void @llvm\.memset\.p([^(]*)i32\(i8([^*]*)\* (.*), i8 (.*), i32 (.*), i32 [01], i1 ([^)]*)\)~call void @llvm.memset.p\1i32(i8\2* \3, i8 \4, i32 \5, i1 \6)~g s~call void @llvm\.memset\.p([^(]*)i64\(i8([^*]*)\* (.*), i8 (.*), i64 (.*), i32 [01], i1 ([^)]*)\)~call void @llvm.memset.p\1i64(i8\2* \3, i8 \4, i64 \5, i1 \6)~g s~call void @llvm\.memset\.p([^(]*)i128\(i8([^*]*)\* (.*), i8 (.*), i128 (.*), i32 [01], i1 ([^)]*)\)~call void @llvm.memset.p\1i128(i8\2* \3, i8 \4, i128 \5, i1 \6)~g s~call void @llvm\.memset\.p([^(]*)i8\(i8([^*]*)\* (.*), i8 (.*), i8 (.*), i32 ([0-9]*), i1 ([^)]*)\)~call void @llvm.memset.p\1i8(i8\2* align \6 \3, i8 \4, i8 \5, i1 \7)~g s~call void @llvm\.memset\.p([^(]*)i16\(i8([^*]*)\* (.*), i8 (.*), i16 (.*), i32 ([0-9]*), i1 ([^)]*)\)~call void @llvm.memset.p\1i16(i8\2* align \6 \3, i8 \4, i16 \5, i1 \7)~g s~call void @llvm\.memset\.p([^(]*)i32\(i8([^*]*)\* (.*), i8 (.*), i32 (.*), i32 ([0-9]*), i1 ([^)]*)\)~call void @llvm.memset.p\1i32(i8\2* align \6 \3, i8 \4, i32 \5, i1 \7)~g s~call void @llvm\.memset\.p([^(]*)i64\(i8([^*]*)\* (.*), i8 (.*), i64 (.*), i32 ([0-9]*), i1 ([^)]*)\)~call void @llvm.memset.p\1i64(i8\2* align \6 \3, i8 \4, i64 \5, i1 \7)~g s~call void @llvm\.memset\.p([^(]*)i128\(i8([^*]*)\* (.*), i8 (.*), i128 (.*), i32 ([0-9]*), i1 ([^)]*)\)~call void @llvm.memset.p\1i128(i8\2* align \6 \3, i8 \4, i128 \5, i1 \7)~g s~call void @llvm\.mem(cpy|move)\.p([^(]*)i8\(i8([^*]*)\* (.*), i8([^*]*)\* (.*), i8 (.*), i32 [01], i1 ([^)]*)\)~call void @llvm.mem\1.p\2i8(i8\3* \4, i8\5* \6, i8 \7, i1 \8)~g s~call void @llvm\.mem(cpy|move)\.p([^(]*)i16\(i8([^*]*)\* (.*), i8([^*]*)\* (.*), i16 (.*), i32 [01], i1 ([^)]*)\)~call void @llvm.mem\1.p\2i16(i8\3* \4, i8\5* \6, i16 \7, i1 \8)~g s~call void @llvm\.mem(cpy|move)\.p([^(]*)i32\(i8([^*]*)\* (.*), i8([^*]*)\* (.*), i32 (.*), i32 [01], i1 ([^)]*)\)~call void @llvm.mem\1.p\2i32(i8\3* \4, i8\5* \6, i32 \7, i1 \8)~g s~call void @llvm\.mem(cpy|move)\.p([^(]*)i64\(i8([^*]*)\* (.*), i8([^*]*)\* (.*), i64 (.*), i32 [01], i1 ([^)]*)\)~call void @llvm.mem\1.p\2i64(i8\3* \4, i8\5* \6, i64 \7, i1 \8)~g s~call void @llvm\.mem(cpy|move)\.p([^(]*)i128\(i8([^*]*)\* (.*), i8([^*]*)\* (.*), i128 (.*), i32 [01], i1 ([^)]*)\)~call void @llvm.mem\1.p\2i128(i8\3* \4, i8\5* \6, i128 \7, i1 \8)~g s~call void @llvm\.mem(cpy|move)\.p([^(]*)i8\(i8([^*]*)\* (.*), i8([^*]*)\* (.*), i8 (.*), i32 ([0-9]*), i1 ([^)]*)\)~call void @llvm.mem\1.p\2i8(i8\3* align \8 \4, i8\5* align \8 \6, i8 \7, i1 \9)~g s~call void @llvm\.mem(cpy|move)\.p([^(]*)i16\(i8([^*]*)\* (.*), i8([^*]*)\* (.*), i16 (.*), i32 ([0-9]*), i1 ([^)]*)\)~call void @llvm.mem\1.p\2i16(i8\3* align \8 \4, i8\5* align \8 \6, i16 \7, i1 \9)~g s~call void @llvm\.mem(cpy|move)\.p([^(]*)i32\(i8([^*]*)\* (.*), i8([^*]*)\* (.*), i32 (.*), i32 ([0-9]*), i1 ([^)]*)\)~call void @llvm.mem\1.p\2i32(i8\3* align \8 \4, i8\5* align \8 \6, i32 \7, i1 \9)~g s~call void @llvm\.mem(cpy|move)\.p([^(]*)i64\(i8([^*]*)\* (.*), i8([^*]*)\* (.*), i64 (.*), i32 ([0-9]*), i1 ([^)]*)\)~call void @llvm.mem\1.p\2i64(i8\3* align \8 \4, i8\5* align \8 \6, i64 \7, i1 \9)~g s~call void @llvm\.mem(cpy|move)\.p([^(]*)i128\(i8([^*]*)\* (.*), i8([^*]*)\* (.*), i128 (.*), i32 ([0-9]*), i1 ([^)]*)\)~call void @llvm.mem\1.p\2i128(i8\3* align \8 \4, i8\5* align \8 \6, i128 \7, i1 \9)~g The remaining changes in the series will: Step 2) Expand the IRBuilder API to allow creation of memcpy/memmove with differing source and dest alignments. Step 3) Update Clang to use the new IRBuilder API. Step 4) Update Polly to use the new IRBuilder API. Step 5) Update LLVM passes that create memcpy/memmove calls to use the new IRBuilder API, and those that use use MemIntrinsicInst::[get|set]Alignment() to use getDestAlignment() and getSourceAlignment() instead. Step 6) Remove the single-alignment IRBuilder API for memcpy/memmove, and the MemIntrinsicInst::[get|set]Alignment() methods. Reviewers: pete, hfinkel, lhames, reames, bollu Reviewed By: reames Subscribers: niosHD, reames, jholewinski, qcolombet, jfb, sanjoy, arsenm, dschuff, dylanmckay, mehdi_amini, sdardis, nemanjai, david2050, nhaehnle, javed.absar, sbc100, jgravelle-google, eraman, aheejin, kbarton, JDevlieghere, asb, rbar, johnrusso, simoncook, jordy.potman.lists, apazos, sabuasal, llvm-commits Differential Revision: https://reviews.llvm.org/D41675 llvm-svn: 322965
* [BPF] Teach DAG2DAG AND elimination about load intrinsicsYonghong Song2018-01-161-0/+58
| | | | | | | | | | | | | | | | | | As commented on the existing code: // The Reg operand should be a virtual register, which is defined // outside the current basic block. DAG combiner has done a pretty // good job in removing truncating inside a single basic block. However, when the Reg operand comes from bpf_load_[byte | half | word] intrinsics, the generic optimizer doesn't understand their results are zero extended, so these single basic block elimination opportunities were missed. Acked-by: Jakub Kicinski <jakub.kicinski@netronome.com> Acked-by: Yonghong Song <yhs@fb.com> Signed-off-by: Jiong Wang <jiong.wang@netronome.com> llvm-svn: 322534
* bpf: add support for objdump -print-imm-hexYonghong Song2017-12-201-0/+65
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add support for 'objdump -print-imm-hex' for imm64, operand imm and branch target. If user programs encode immediate values as hex numbers, such an option will make it easy to correlate asm insns with source code. This option also makes it easy to correlate imm values with insn encoding. There is one changed behavior in this patch. In old way, we print the 64bit imm as u64: O << (uint64_t)Op.getImm(); and the new way is: O << formatImm(Op.getImm()); The formatImm is defined in llvm/MC/MCInstPrinter.h as format_object<int64_t> formatImm(int64_t Value) So the new way to print 64bit imm is i64 type. If a 64bit value has the highest bit set, the old way will print the value as a positive value and the new way will print as a negative value. The new way is consistent with x86_64. For the code (see the test program): ... if (a == 0xABCDABCDabcdabcdULL) ... x86_64 objdump, with and without -print-imm-hex, looks like: 48 b8 cd ab cd ab cd ab cd ab movabsq $-6067004223159161907, %rax 48 b8 cd ab cd ab cd ab cd ab movabsq $-0x5432543254325433, %rax Signed-off-by: Yonghong Song <yhs@fb.com> llvm-svn: 321215
* [CodeGen] Print register names in lowercase in both MIR and debug outputFrancis Visoiu Mistrih2017-11-281-1/+1
| | | | | | | | | | | As part of the unification of the debug format and the MIR format, always print registers as lowercase. * Only debug printing is affected. It now follows MIR. Differential Revision: https://reviews.llvm.org/D40417 llvm-svn: 319187
* [DAG] Do MergeConsecutiveStores again before Instruction SelectionNirav Dave2017-11-271-7/+4
| | | | | | | | | | | | | | | | Summary: Now that store-merge is only generates type-safe stores, do a second pass just before instruction selection to allow lowered intrinsics to be merged as well. Reviewers: jyknight, hfinkel, RKSimon, efriedma, rnk, jmolloy Subscribers: javed.absar, llvm-commits Differential Revision: https://reviews.llvm.org/D33675 llvm-svn: 319036
* bpf: add a test case for trunc-op optimizationYonghong Song2017-11-201-0/+106
| | | | | | | | | | | | | | | | Commit b5cbc7760ab8 ("[bpf] allow direct and indirect calls") allowed more than one function in the bpf program, and commit 114353884415 ("bpf: fix a bug in trunc-op optimization") fixed a bug in trunc-op optimization which only showed up with more than one function in the bpf program. This patch added a test case for trunc-op optimization for bpf programs with two functions. Reverting commit "bpf: fix a bug in trunc-op optimization" will cause failure for this test case. Signed-off-by: Yonghong Song <yhs@fb.com> llvm-svn: 318695
* [bpf] allow direct and indirect callsAlexei Starovoitov2017-11-197-10/+7
| | | | | | | | | | | | | | | kernel verifier is becoming smarter and soon will support direct and indirect function calls. Remove obsolete error from BPF backend. Make call to use PCRel_4 fixup. 'bpf to bpf' calls are distinguished from 'bpf to kernel' calls by insn->src_reg == BPF_PSEUDO_CALL == 1 which is used as relocation indicator similar to ld_imm64->src_reg == BPF_PSEUDO_MAP_FD == 1 The actual 'call' instruction remains the same for both 'bpf to kernel' and 'bpf to bpf' calls. Signed-off-by: Alexei Starovoitov <ast@kernel.org> llvm-svn: 318614
* bpf: print backward branch target properlyYonghong Song2017-11-162-3/+42
| | | | | | | | | | | | | | | | | | | | | | | | Currently, it prints the backward branch offset as unsigned value like below: 7: 7d 34 0b 00 00 00 00 00 if r4 s>= r3 goto 11 <LBB0_3> 8: b7 00 00 00 00 00 00 00 r0 = 0 LBB0_2: 9: 07 00 00 00 01 00 00 00 r0 += 1 ...... 17: bf 31 00 00 00 00 00 00 r1 = r3 18: 6d 32 f6 ff 00 00 00 00 if r2 s> r3 goto 65526 <LBB0_3+0x7FFB0> The correct print insn 18 should be: 18: 6d 32 f6 ff 00 00 00 00 if r2 s> r3 goto -10 <LBB0_2> To provide better clarity and be consistent with kernel verifier output, the insn 7 output is changed to the following with "+" added to non-negative branch offset: 7: 7d 34 0b 00 00 00 00 00 if r4 s>= r3 goto +11 <LBB0_3> Signed-off-by: Yonghong Song <yhs@fb.com> Acked-by: Alexei Starovoitov <ast@kernel.org> llvm-svn: 318442
* bpf: enable llvm-objdump to print out symbolized jmp targetYonghong Song2017-11-161-0/+69
| | | | | | | | | | | | | | | | | | Add hook in BPF backend so that llvm-objdump can print out the jmp target with label names, e.g., ... if r1 != 2 goto 6 <LBB0_2> ... goto 7 <LBB0_4> ... LBB0_2: ... LBB0_4: ... Signed-off-by: Yonghong Song <yhs@fb.com> Acked-by: Alexei Starovoitov <ast@kernel.org> llvm-svn: 318358
* bpf: fix bug on silently truncating 64-bit immediateYonghong Song2017-10-161-0/+35
| | | | | | | | | | | | | | | | We came across an llvm bug when compiling some testcases that 64-bit immediates are silently truncated into 32-bit and then packed into BPF_JMP | BPF_K encoding. This caused comparison with wrong value. This bug looks to be introduced by r308080. The Select_Ri pattern is supposed to be lowered into J*_Ri while the latter only support 32-bit immediate encoding, therefore Select_Ri should have similar immediate predicate check as what J*_Ri are doing. Reported-by: Jakub Kicinski <jakub.kicinski@netronome.com> Signed-off-by: Jiong Wang <jiong.wang@netronome.com> Reviewed-by: Yonghong Song <yhs@fb.com> llvm-svn: 315889
* bpf: add new insns for bswap_to_le and negationYonghong Song2017-09-282-26/+52
| | | | | | | | | | | | | | | | This patch adds new insn, "reg = be16/be32/be64 reg", for bswap to little endian for big-endian target (bpfeb). It also adds new insn for negation "reg = -reg". Currently, for source code, e.g., b = -a LLVM still prefers to generate: b = 0 - a But "reg = -reg" format can be used in assembly code. Signed-off-by: Yonghong Song <yhs@fb.com> Acked-by: Alexei Starovoitov <ast@kernel.org> llvm-svn: 314376
* bpf: add inline-asm supportYonghong Song2017-09-181-0/+54
| | | | | | Signed-off-by: Yonghong Song <yhs@fb.com> Acked-by: Alexei Starovoitov <ast@kernel.org> llvm-svn: 313593
* bpf: add " ll" in the LD_IMM64 asmstringYonghong Song2017-09-117-9/+9
| | | | | | | | | | | | | | | | | | | | | | | | This partially revert previous fix in commit f5858045aa0b ("bpf: proper print imm64 expression in inst printer"). In that commit, the original suffix "ll" is removed from LD_IMM64 asmstring. In the customer print method, the "ll" suffix is printed if the rhs is an immediate. For example, "r2 = 5ll" => "r2 = 5ll", and "r3 = varll" => "r3 = var". This has an issue though for assembler. Since assembler relies on asmstring to do pattern matching, it will not be able to distiguish between "mov r2, 5" and "ld_imm64 r2, 5" since both asmstring is "r2 = 5". In such cases, the assembler uses 64bit load for all "r = <val>" asm insts. This patch adds back " ll" suffix for ld_imm64 with one additional space for "#reg = #global_var" case. Signed-off-by: Yonghong Song <yhs@fb.com> Acked-by: Alexei Starovoitov <ast@kernel.org> llvm-svn: 312978
* llvm-dwarfdump: Replace -debug-dump=sect option with individual options.Adrian Prantl2017-09-111-2/+2
| | | | | | | | | | | | | | | | | | | | | | As discussed on llvm-dev in http://lists.llvm.org/pipermail/llvm-dev/2017-September/117301.html this changes the command line interface of llvm-dwarfdump to match the one used by the dwarfdump utility shipping on macOS. In addition to being shorter to type this format also has the advantage of allowing more than one section to be specified at the same time. In a nutshell, with this change $ llvm-dwarfdump --debug-dump=info $ llvm-dwarfdump --debug-dump=apple-objc becomes $ dwarfdump --debug-info --apple-objc Differential Revision: https://reviews.llvm.org/D37714 llvm-svn: 312970
* bpf: fix test failures due to previous bpf change of assembly code syntaxYonghong Song2017-09-095-6/+6
| | | | | Signed-off-by: Yonghong Song <yhs@fb.com> llvm-svn: 312840
* Canonicalize the representation of empty an expression in ↵Adrian Prantl2017-08-301-1/+1
| | | | | | | | | | | | | | | | DIGlobalVariableExpression This change simplifies code that has to deal with DIGlobalVariableExpression and mirrors how we treat DIExpressions in debug info intrinsics. Before this change there were two ways of representing empty expressions on globals, a nullptr and an empty !DIExpression(). If someone needs to upgrade out-of-tree testcases: perl -pi -e 's/(!DIGlobalVariableExpression\(var: ![0-9]*)\)/\1, expr: !DIExpression())/g' <MYTEST.ll> will catch 95%. llvm-svn: 312144
* bpf: add variants of -mcpu=# and support for additional jmp insnsYonghong Song2017-08-231-13/+26
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | -mcpu=# will support: . generic: the default insn set . v1: insn set version 1, the same as generic . v2: insn set version 2, version 1 + additional jmp insns . probe: the compiler will probe the underlying kernel to decide proper version of insn set. We did not not use -mcpu=native since llc/llvm will interpret -mcpu=native as the underlying hardware architecture regardless of -march value. Currently, only x86_64 supports -mcpu=probe. Other architecture will silently revert to "generic". Also added -mcpu=help to print available cpu parameters. llvm will print out the information only if there are at least one cpu and at least one feature. Add an unused dummy feature to enable the printout. Examples for usage: $ llc -march=bpf -mcpu=v1 -filetype=asm t.ll $ llc -march=bpf -mcpu=v2 -filetype=asm t.ll $ llc -march=bpf -mcpu=generic -filetype=asm t.ll $ llc -march=bpf -mcpu=probe -filetype=asm t.ll $ llc -march=bpf -mcpu=v3 -filetype=asm t.ll 'v3' is not a recognized processor for this target (ignoring processor) ... $ llc -march=bpf -mcpu=help -filetype=asm t.ll Available CPUs for this target: generic - Select the generic processor. probe - Select the probe processor. v1 - Select the v1 processor. v2 - Select the v2 processor. Available features for this target: dummy - unused feature. Use +feature to enable a feature, or -feature to disable it. For example, llc -mcpu=mycpu -mattr=+feature1,-feature2 ... Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Signed-off-by: Yonghong Song <yhs@fb.com> Acked-by: Alexei Starovoitov <ast@kernel.org> llvm-svn: 311522
* DAG: Undo and->or combine with FrameIndexesMatt Arsenault2017-08-021-18/+18
| | | | | | | | | | | | | | This pattern shows up when lowering byval copies on AMDGPU. The byval object access is split into 4-byte chunks, adding a constant offset to the FixedStack base. When some of the offsets turn into ors, this prevents combining the constant offsets. This makes it not apparent that the object is there when matching addressing modes, so it ends up using a scratch wave offset relative access and the lengthy frame index expansion for that. llvm-svn: 309775
* [DAG] Fix typo preventing some stores merges to truncated stores.Nirav Dave2017-07-231-4/+3
| | | | | | | | | | | | | | | Check the actual memory type stored and not the extended value size when considering if truncated store merge is worthwhile. Reviewers: efriedma, RKSimon, spatel, jyknight Reviewed By: efriedma Subscribers: llvm-commits, nhaehnle Differential Revision: https://reviews.llvm.org/D35623 llvm-svn: 308833
* bpf: generate better lowering code for certain select/setcc instructionsYonghong Song2017-07-152-2/+29
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently, for code like below, === inner_map = bpf_map_lookup_elem(outer_map, &port_key); if (!inner_map) { inner_map = &fallback_map; } === the compiler generates (pseudo) code like the below: === I1: r1 = bpf_map_lookup_elem(outer_map, &port_key); I2: r2 = 0 I3: if (r1 == r2) I4: r6 = &fallback_map I5: ... === During kernel verification process, After I1, r1 holds a state map_ptr_or_null. If I3 condition is not taken (path [I1, I2, I3, I5]), supposedly r1 should become map_ptr. Unfortunately, kernel does not recognize this pattern and r1 remains map_ptr_or_null at insn I5. This will cause verificaiton failure later on. Kernel, however, is able to recognize pattern "if (r1 == 0)" properly and give a map_ptr state to r1 in the above case. LLVM here generates suboptimal code which causes kernel verification failure. This patch fixes the issue by changing BPF insn pattern matching and lowering to generate proper codes if the righthand parameter of the above condition is a constant. A test case is also added. Signed-off-by: Yonghong Song <yhs@fb.com> llvm-svn: 308080
* Rewrite areNonVolatileConsecutiveLoads to use BaseIndexOffsetNirav Dave2017-07-051-31/+27
| | | | | | | | | | | | | | | | | | | | | | | | | | | Relanding after rewriting undef.ll test to avoid host-dependant endianness. As discussed in D34087, rewrite areNonVolatileConsecutiveLoads using generic checks. Also, propagate missing local handling from there to BaseIndexOffset checks. Tests of note: * test/CodeGen/X86/build-vector* - Improved. * test/CodeGen/BPF/undef.ll - Improved store alignment allows an additional store merge * test/CodeGen/X86/clear_upper_vector_element_bits.ll - This is a case we already do not handle well. Here, the DAG is improved, but scheduling causes a code size degradation. Reviewers: RKSimon, craig.topper, spatel, andreadb, filcab Subscribers: nemanjai, llvm-commits Differential Revision: https://reviews.llvm.org/D34472 llvm-svn: 307114
* Revert "[DAG] Rewrite areNonVolatileConsecutiveLoads to use BaseIndexOffset"Nirav Dave2017-06-301-24/+30
| | | | | | | This reverts commit r306819 which appears be exposing underlying issues in a stage1 ppc64be build llvm-svn: 306820
OpenPOWER on IntegriCloud