summaryrefslogtreecommitdiffstats
path: root/lld/ELF/Writer.cpp
Commit message (Collapse)AuthorAgeFilesLines
* [ELF] Allow SHF_LINK_ORDER and non-SHF_LINK_ORDER to be mixedFangrui Song2020-04-141-7/+20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently, `error: incompatible section flags for .rodata` is reported when we mix SHF_LINK_ORDER and non-SHF_LINK_ORDER sections in an output section. This is overconstrained. This patch allows mixed flags with the requirement that SHF_LINK_ORDER sections must be contiguous. Mixing flags is used by Linux aarch64 (https://github.com/ClangBuiltLinux/linux/issues/953) .init.data : { ... KEEP(*(__patchable_function_entries)) ... } When the integrated assembler is enabled, clang's -fpatchable-function-entry=N[,M] implementation sets the SHF_LINK_ORDER flag (D72215) to fix a number of garbage collection issues. Strictly speaking, the ELF specification does not require contiguous SHF_LINK_ORDER sections but for many current uses of SHF_LINK_ORDER like .ARM.exidx/__patchable_function_entries there has been a requirement for the sections to be contiguous on top of the requirements of the ELF specification. This patch also imposes one restriction: SHF_LINK_ORDER sections cannot be separated by a symbol assignment or a BYTE command. Not allowing BYTE is a natural extension that a non-SHF_LINK_ORDER cannot be a separator. Symbol assignments can delimiter the contents of SHF_LINK_ORDER sections. Allowing SHF_LINK_ORDER sections across symbol assignments (especially __start_/__stop_) can make things hard to explain. The restriction should not be a problem for practical use cases. Reviewed By: psmith Differential Revision: https://reviews.llvm.org/D77007 (cherry picked from commit 673e81eee4fa3ffa38736f1063e6c4fa2d9278b0)
* [ELF] -r: don't create .interpFangrui Song2020-01-161-2/+2
| | | | | | | | | | | | | `{clang,gcc} -nostdlib -r a.c` passes --dynamic-linker to the linker, and the expected behavior is to ignore it. If .interp is kept in the relocatable object file, a final link will get PT_INTERP even if --dynamic-linker is not specified. glibc ld.so expects to see PT_DYNAMIC and the executable will likely fail to run. Ignore --dynamic-linker in -r mode as well as -shared. (cherry picked from commit 2d7a8cf90478cd845ffb39763b0e95b7715322d2)
* [ELF] Add -z force-ibt and -z shstk for Intel Control-flow Enforcement ↵Fangrui Song2020-01-131-0/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | Technology This patch is a joint work by Rui Ueyama and me based on D58102 by Xiang Zhang. It adds Intel CET (Control-flow Enforcement Technology) support to lld. The implementation follows the draft version of psABI which you can download from https://github.com/hjl-tools/x86-psABI/wiki/X86-psABI. CET introduces a new restriction on indirect jump instructions so that you can limit the places to which you can jump to using indirect jumps. In order to use the feature, you need to compile source files with -fcf-protection=full. * IBT is enabled if all input files are compiled with the flag. To force enabling ibt, pass -z force-ibt. * SHSTK is enabled if all input files are compiled with the flag, or if -z shstk is specified. IBT-enabled executables/shared objects have two PLT sections, ".plt" and ".plt.sec". For the details as to why we have two sections, please read the comments. Reviewed By: xiangzhangllvm Differential Revision: https://reviews.llvm.org/D59780
* [ELF] Improve the condition to create .interpFangrui Song2019-12-271-1/+1
| | | | | | | | | | | | | | | This restores commit 1417558e4a61794347c6bfbafaff7cd96985b2c3 and its follow-up, reverted by commit c3dbd782f1e0578c7ebc342f2e92f54d9644cff7. After this commit: clang -fuse-ld=bfd -no-pie -nostdlib a.c => .interp not created clang -fuse-ld=bfd -pie -fPIE -nostdlib a.c => .interp created clang -fuse-ld=gold -no-pie -nostdlib a.c => .interp not created clang -fuse-ld=gold -pie -fPIE -nostdlib a.c => .interp created clang -fuse-ld=lld -no-pie -nostdlib a.c => .interp created clang -fuse-ld=lld -pie -fPIE -nostdlib a.c => .interp created
* Revert "[ELF] Improve the condition to create .interp"Reid Kleckner2019-12-271-1/+1
| | | | | | | | This reverts commit 1417558e4a61794347c6bfbafaff7cd96985b2c3. Also reverts commit 019a92bb2832447092bb5c1faf9d03bb03b8c9c8. This causes check-sanitizer to fail. The "-Nolib" variant of the test crashes on startup in the loader.
* [ELF] Improve the condition to create .interpFangrui Song2019-12-261-1/+1
| | | | | | | | | | | | | | | | | Similar to rL362355, but with the `!config->shared` guard. (1) {gcc,clang} -fuse-ld=bfd -pie -fPIE -nostdlib a.c => .interp created (2) {gcc,clang} -fuse-ld=lld -pie -fPIE -nostdlib a.c => .interp not created (3) {gcc,clang} -fuse-ld=lld -pie -fPIE -nostdlib a.c a.so => .interp created The inconsistency of (2) is due to the condition `!Config->SharedFiles.empty()`. To make lld behave more like ld.bfd, we could change the condition to: config->hasDynSymTab && !config->dynamicLinker.empty() && script->needsInterpSection(); However, that would bring another inconsistency as can be observed with: (4) {gcc,clang} -fuse-ld=bfd -no-pie -nostdlib a.c => .interp not created
* [ELF] Add IpltSectionFangrui Song2019-12-171-2/+2
| | | | | | | | | | | | | | | | | | | PltSection is used by both PLT and IPLT. The PLT section may have a header while the IPLT section does not. Split off IpltSection from PltSection to be clearer. Unlike other targets, PPC64 cannot use the same code sequence for PLT and IPLT. This helps make a future PPC64 patch (D71509) more isolated. On EM_386 and EM_X86_64, when PLT is empty while IPLT is not, currently we are inconsistent whether the PLT header is conceptually attached to in.plt or in.iplt . Consistently attach the header to in.plt can make the -z retpolineplt logic simpler. It also makes `jmp` point to an aesthetically better place for non-retpolineplt cases. Reviewed By: grimar, ruiu Differential Revision: https://reviews.llvm.org/D71519
* Revert an accidental commit af5ca40b47b3e85c3add81ccdc0b787c4bc355aeRui Ueyama2019-12-131-5/+1
|
* temporaryRui Ueyama2019-12-131-1/+5
|
* [ELF] --icf: do not fold preemptible symbolsFangrui Song2019-12-101-31/+0
| | | | | | | | | | | | | | | Fixes PR44124. A preemptible symbol may refer to a different definition at runtime. When comparing a pair of relocations, if they refer to different symbols, and either symbol is preemptible, the two containing sections should be considered different. gold has a similar rule https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;a=commit;h=ce97fa81e0c46d216b80b143ad8c02fff6906fef Reviewed By: grimar Differential Revision: https://reviews.llvm.org/D71163
* [LLD][ELF] Add support for PT_GNU_PROPERTYPeter Smith2019-12-051-0/+3
| | | | | | | | | | The PT_GNU_PROPERTY program header describes the location of the .note.gnu.property SHT_NOTES section. The linux kernel uses this program header to find the .note.gnu.property section rather than parsing. Executables that have properties that the kernel needs to act on that don't have the PT_GNU_PROPERTY program header will not boot. Differential Revision: https://reviews.llvm.org/D70961
* [ELF] Replace SymbolTable::forEachSymbol with iterator_range symbols()Fangrui Song2019-11-261-9/+7
| | | | | | | | | | | | | | D62381 introduced forEachSymbol(). It seems that many call sites cannot be parallelized because the body shared some states. Replace forEachSymbol with iterator_range<filter_iterator<...>> symbols() to simplify code and improve debuggability (std::function calls take some frames). It also allows us to use early return to simplify code added in D69650. Reviewed By: grimar Differential Revision: https://reviews.llvm.org/D70505
* [LLD][ELF] Support --[no-]mmap-output-file with F_no_mmapNick Terrell2019-10-291-1/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: Add a flag `F_no_mmap` to `FileOutputBuffer` to support `--[no-]mmap-output-file` in ELF LLD. LLD currently explicitly ignores this flag for compatibility with GNU ld and gold. We need this flag to speed up link time for large binaries in certain scenarios. When we link some of our larger binaries we find that LLD takes 50+ GB of memory, which causes memory pressure. The memory pressure causes the VM to flush dirty pages of the output file to disk. This is normally okay, since we should be flushing cold pages. However, when using BtrFS with compression we need to write 128KB at a time when we flush a page. If any page in that 128KB block is written again, then it must be flushed a second time, and so on. Since LLD doesn't write sequentially this causes write amplification. The same 128KB block will end up being flushed multiple times, causing the linker to many times more IO than necessary. We've observed 3-5x faster builds with -no-mmap-output-file when we hit this scenario. The bad scenario only applies to compressed filesystems, which group together multiple pages into a single compressed block. I've tested BtrFS, but the problem will be present for any compressed filesystem on Linux, since it is caused by the VM. Silently ignoring --no-mmap-output-file caused a silent regression when we switched from gold to lld. We pass --no-mmap-output-file to fix this edge case, but since lld silently ignored the flag we didn't realize it wasn't being respected. Benchmark building a 9 GB binary that exposes this edge case. I linked 3 times with --mmap-output-file and 3 times with --no-mmap-output-file and took the average. The machine has 24 cores @ 2.4 GHz, 112 GB of RAM, BtrFS mounted with -compress-force=zstd, and an 80% full disk. | Mode | Time | |---------|-------| | mmap | 894 s | | no mmap | 126 s | When compression is disabled, BtrFS performs just as well with and without mmap on this benchmark. I was unable to reproduce the regression with any binaries in lld-speed-test. Reviewed By: ruiu, MaskRay Differential Revision: https://reviews.llvm.org/D69294
* [lld] [ELF] Add '-z nognustack' opt to suppress emitting PT_GNU_STACKMichał Górny2019-10-291-8/+10
| | | | | | | | | Add a new '-z nognustack' option that suppresses emitting PT_GNU_STACK segment. This segment is not supported at all on NetBSD (stack is always non-executable), and the option is meant to be used to disable emitting it. Differential Revision: https://reviews.llvm.org/D56554
* Fix a few typos in lld/ELF to cycle botsNico Weber2019-10-281-5/+5
|
* [ELF] Wrap things in `namespace lld { namespace elf {`, NFCFangrui Song2019-10-071-17/+19
| | | | | | | | | | | This makes it clear `ELF/**/*.cpp` files define things in the `lld::elf` namespace and simplifies `elf::foo` to `foo`. Reviewed By: atanasyan, grimar, ruiu Differential Revision: https://reviews.llvm.org/D68323 llvm-svn: 373885
* ELF: Add .interp synthetic sections first in createSyntheticSections().Peter Collingbourne2019-10-011-5/+13
| | | | | | | | | | | | | | | | | | | | | | Our .interp section is not a SyntheticSection. As a result, it terminates the loop in removeUnusedSyntheticSections(). This has at least two consequences: - The synthetic .bss and .bss.rel.ro sections are always present in dynamically linked executables, even when they are not needed. - The synthetic .ARM.exidx (and possibly other) sections are always present in partitions other than the last one, even when not needed. .ARM.exidx in particular is problematic because it assumes that its list of code sections is non-empty in getLinkOrderDep(), which can lead to a crash if the partition does not have any code sections. Fix these problems by moving the creation of the .interp sections to the top of createSyntheticSections(). While here, make the code a little less error-prone by changing the add() lambdas to take a SyntheticSection instead of an InputSectionBase. Differential Revision: https://reviews.llvm.org/D68256 llvm-svn: 373347
* [ELF] Add -z separate-loadable-segments to complement separate-code and ↵Fangrui Song2019-09-251-6/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | noseparate-code D64906 allows PT_LOAD to have overlapping p_offset ranges. In the default R RX RW RW layout + -z noseparate-code case, we do not tail pad segments when transiting to another segment. This can save at most 3*maxPageSize bytes. a) Before D64906, we tail pad R, RX and the first RW. b) With -z separate-code, we tail pad R and RX, but not the first RW (RELRO). In some cases, b) saves one file page. In some cases, b) wastes one virtual memory page. The waste is a concern on Fuchsia. Because it uses compressed binaries, it doesn't benefit from the saved file page. This patch adds -z separate-loadable-segments to restore the behavior before D64906. It can affect section addresses and can thus be used as a debugging mechanism (see PR43214 and ld.so partition bug in crbug.com/998712). Reviewed By: jakehehrlich, ruiu Differential Revision: https://reviews.llvm.org/D67481 llvm-svn: 372807
* [ELF] Make MergeInputSection merging aware of output sectionsFangrui Song2019-09-241-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Fixes PR38748 mergeSections() calls getOutputSectionName() to get output section names. Two MergeInputSections may be merged even if they are made different by SECTIONS commands. This patch moves mergeSections() after processSectionCommands() and addOrphanSections() to fix the issue. The new pass is renamed to OutputSection::finalizeInputSections(). processSectionCommands() and addorphanSections() are changed to add sections to InputSectionDescription::sectionBases. finalizeInputSections() merges MergeInputSections and migrates `sectionBases` to `sections`. For the -r case, we drop an optimization that tries keeping sh_entsize non-zero. This is for the simplicity of addOrphanSections(). The updated merge-entsize2.s reflects the change. Reviewed By: grimar Differential Revision: https://reviews.llvm.org/D67504 llvm-svn: 372734
* [ELF] Error if the linked-to section of a SHF_LINK_ORDER section is discardedFangrui Song2019-09-201-4/+14
| | | | | | | | | | | | | | | | | | | | | | | | | Summary: If st_link(A)=B, and A has the SHF_LINK_ORDER flag, we may dereference a null pointer if B is garbage collected (PR43147): 1. In Wrter.cpp:compareByFilePosition, `aOut->sectionIndex` or `bOut->sectionIndex` 2. In OutputSections::finalize, `d->getParent()->sectionIndex` Simply error and bail out to avoid null pointer dereferences. ld.bfd has a similar error: sh_link of section `.bar' points to discarded section `.foo0' of `a.o' ld.bfd is more permissive in that it just checks whether the linked-to section of the first input section is discarded. This is likely because it sets sh_link of the output section according to the first input section. Reviewed By: grimar Differential Revision: https://reviews.llvm.org/D67761 llvm-svn: 372400
* [ELF][Hexagon] Allow PT_LOAD to have overlapping p_offset ranges on EM_HEXAGONFangrui Song2019-09-171-6/+1
| | | | | | | | Port the D64906 technique to EM_HEXAGON. This concludes the patch series. Differential Revision: https://reviews.llvm.org/D67605 llvm-svn: 372059
* [ELF][ARM] Implement --fix-cortex-a8 to fix erratum 657417Peter Smith2019-09-161-0/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | The --fix-cortex-a8 option implements a linker workaround for the coretex-a8 erratum 657417. A summary of the erratum conditions is: - A 32-bit Thumb-2 branch instruction B.w, Bcc.w, BL, BLX spans two 4KiB regions. - The destination of the branch is to the first 4KiB region. - The instruction before the branch is a 32-bit Thumb-2 non-branch instruction. The linker fix is to redirect the branch to a patch not in the first 4KiB region. The patch forwards the branch on to its target. The cortex-a8, is an old CPU, with the first implementation of this workaround in ld.bfd appearing in 2009. The cortex-a8 has been used in early Android Phones and there are some critical applications that still need to run on a cortex-a8 that have the erratum. The patch is applied roughly 10 times on LLD and 20 on Clang when they are built with --fix-cortex-a8 on an Arm system. The formal erratum description is avaliable in the ARM Core Cortex-A8 (AT400/AT401) Errata Notice document. This is available from Arm on request but it seems to be findable via a web search. Differential Revision: https://reviews.llvm.org/D67284 llvm-svn: 371965
* [ELF][X86] Allow PT_LOAD to have overlapping p_offset ranges on EM_X86_64Fangrui Song2019-09-161-2/+1
| | | | | | | | Port the D64906 technique to EM_X86_64. Differential Revision: https://reviews.llvm.org/D67482 llvm-svn: 371958
* [ELF] Map the ELF header at imageBaseFangrui Song2019-09-161-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If there is no readonly section, we map: * The ELF header at imageBase+maxPageSize * Program headers at imageBase+maxPageSize+sizeof(Ehdr) * The first section .text at imageBase+maxPageSize+sizeof(Ehdr)+sizeof(program headers) Due to the interaction between Writer<ELFT>::fixSectionAlignments and LinkerScript::allocateHeaders, `alignDown(p_vaddr(R PT_LOAD)) = alignDown(p_vaddr(RX PT_LOAD))`. The RX PT_LOAD will override the R PT_LOAD at runtime, which is not ideal: ``` // PHDR at 0x401034, should be 0x400034 PHDR 0x000034 0x00401034 0x00401034 0x000a0 0x000a0 R 0x4 // R PT_LOAD contains just Ehdr and program headers. // At 0x401000, should be 0x400000 LOAD 0x000000 0x00401000 0x00401000 0x000d4 0x000d4 R 0x1000 LOAD 0x0000d4 0x004010d4 0x004010d4 0x00001 0x00001 R E 0x1000 ``` * createPhdrs allocates the headers to the R PT_LOAD. * fixSectionAlignments assigns `imageBase+maxPageSize+sizeof(Ehdr)+sizeof(program headers)` (formula: `alignTo(dot, maxPageSize) + dot % config->maxPageSize`) to addrExpr of .text * allocateHeaders computes the minimum address among SHF_ALLOC sections, i.e. addr(.text) * allocateHeaders sets address of ELF header to `addr(.text)-sizeof(Ehdr)-sizeof(program headers) = imageBase+maxPageSize` The main observation is that when the SECTIONS command is not used, we don't have to call allocateHeaders. This requires an assumption that the presence of PT_PHDR and addresses of headers can be decided regardless of address information. This may seem natural because dot is not manipulated by a linker script. The other thing is that we have to drop the special rule for -T<section> in `getInitialDot`. If -Ttext is smaller than the image base, the headers will not be allocated with the old behavior (allocateHeaders is called) but always allocated with the new behavior. The behavior change is not a problem. Whether and where headers are allocated can vary among linkers, or ld.bfd across different versions (--enable-separate-code or not). It is thus advised to use a linker script with the PHDRS command to have a consistent behavior across linkers. If PT_PHDR is needed, an explicit --image-base can be a simpler alternative. Differential Revision: https://reviews.llvm.org/D67325 llvm-svn: 371957
* [mips] Allow PT_LOAD to have overlapping p_offset ranges on EM_MIPSSimon Atanasyan2019-09-101-1/+0
| | | | | | | | Port the D64906 <https://reviews.llvm.org/D64906> technique to MIPS. Fix PR33131 llvm-svn: 371554
* [ELF] nmagic or omagic: don't allocate PT_PHDR or PF_R PT_LOAD for the ↵Fangrui Song2019-09-091-17/+22
| | | | | | | | | | | | | | | | | | | !hasPhdrsCommands case ``` part.phdrs = script->hasPhdrsCommands() ? script->createPhdrs() : createPhdrs(part); ``` createPhdrs() allocates a PT_PHDR and a PF_R PT_LOAD, which will be deleted later in LinkerScript::allocateHeaders, but leave a gap between the program headers and the first section. Don't allocate the segments to avoid the gap. PT_INTERP is likely not needed as well. Reviewed By: ruiu Differential Revision: https://reviews.llvm.org/D67324 llvm-svn: 371398
* Reland D66717 [ELF] Do not ICF two sections with different output sections ↵Fangrui Song2019-09-061-5/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | (by SECTIONS commands) Recommit r370635 (reverted by r371202), with one change: move addOrphanSections() before ICF. Before, orphan sections in two different partitions may be folded and moved to the main partition. Now, InputSection->OutputSection assignment for orphans happens before ICF. ICF does not fold input sections with different output sections. With the PR43241 reproduce, `llvm-objcopy --extract-partition libvr.so libchrome__combined.so libvr.so` => no error Updated description: Fixes PR39418. Complements D47241 (the non-linker-script case). processSectionCommands() assigns input sections to output sections. ICF is called before it, so .text.foo and .text.bar may be folded even if their output sections are made different by SECTIONS commands. ``` markLive<ELFT>() doIcf<ELFT>() // During ICF, we don't know the output sections writeResult() combineEhSections<ELFT>() script->processSectionCommands() // InputSection -> OutputSection assignment ``` This patch splits processSectionCommands() into processSectionCommands() and processSymbolAssignments(), and moves processSectionCommands()/addOrphanSections() before ICF: ``` markLive<ELFT>() combineEhSections<ELFT>() script->processSectionCommands() script->addOrphanSections(); doIcf<ELFT>() // should remove folded input sections writeResult() script->processSymbolAssignments() ``` An alternative approach is to unfold a section `sec` in processSectionCommands() when we find `sec` and `sec->repl` belong to different output sections. I feel this patch is superior because this can fold more sections and the decouple of SectionCommand/SymbolAssignment gives flexibility: * An ExprValue can't be evaluated before its section is assigned to an output section -> we can delete getOutputSectionVA and simplify another place where we had to check if the output section is null. Moreover, a case in linkerscript/early-assign-symbol.s can be handled now. * processSectionCommands/processSymbolAssignments can be freely moved around. llvm-svn: 371216
* Revert "Revert r370635, it caused PR43241."Fangrui Song2019-09-061-22/+14
| | | | | | This reverts commit 50d2dca22b3b05d0ee4883b0cbf93d7d15f241fc. llvm-svn: 371215
* Revert r370635, it caused PR43241.Nico Weber2019-09-061-14/+22
| | | | llvm-svn: 371202
* [ELF] Initialize PhdrEntry::p_align to maxPageSize for PT_LOADFangrui Song2019-09-051-9/+4
| | | | | | | | | | | | | | | | | | | | | ``` Writer<ELFT>::run assignFileOffsets setFileOffset computeFileOffset os->ptLoad->p_align may be smaller than config->maxPageSize setPhdrs p_align = max(p_align, config->maxPageSize) ``` If we move the config->maxPageSize logic to the constructor of PhdrEntry, computeFileOffset can be simplified. Reviewed By: ruiu Differential Revision: https://reviews.llvm.org/D67211 llvm-svn: 371085
* Align output segments correctlyRui Ueyama2019-09-051-1/+2
| | | | | | | | | | | | | | | | Previously, segments were aligned according to their first section's alignment requirements. That was not correct, but segments are also aligned to a page boundary, and a page boundary is usually much larger than a section alignment requirement, so no one noticed this bug before. Now, lld has --nmagic option which sets maxPageSize to 1 to effectively disable page alignment, which reveals the issue. Fixes https://bugs.llvm.org/show_bug.cgi?id=43212 Differential Revision: https://reviews.llvm.org/D67152 llvm-svn: 371013
* [ELF] Do not ICF two sections with different output sections (by SECTIONS ↵Fangrui Song2019-09-021-22/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | commands) Fixes PR39418. Complements D47241 (the non-linker-script case). processSectionCommands() assigns input sections to output sections. ICF is called before it, so .text.foo and .text.bar may be folded even if their output sections are made different by SECTIONS commands. ``` markLive<ELFT>() doIcf<ELFT>() // During ICF, we don't know the output sections writeResult() combineEhSections<ELFT>() script->processSectionCommands() // InputSection -> OutputSection assignment ``` This patch splits processSectionCommands() into processSectionCommands() and processSymbolAssignments(), and moves processSectionCommands() before ICF: ``` markLive<ELFT>() combineEhSections<ELFT>() script->processSectionCommands() doIcf<ELFT>() // should remove folded input sections writeResult() script->processSymbolAssignments() ``` An alternative approach is to unfold a section `sec` in processSectionCommands() when we find `sec` and `sec->repl` belong to different output sections. I feel this patch is superior because this can fold more sections and the decouple of SectionCommand/SymbolAssignment gives flexibility: * An ExprValue can't be evaluated before its section is assigned to an output section -> we can delete getOutputSectionVA and simplify another place where we had to check if the output section is null. Moreover, a case in linkerscript/early-assign-symbol.s can be handled now. * processSectionCommands/processSymbolAssignments can be freely moved around. Reviewed By: ruiu Differential Revision: https://reviews.llvm.org/D66717 llvm-svn: 370635
* [ELF] Align SHT_LLVM_PART_EHDR to a maximum page size boundaryFangrui Song2019-09-021-2/+10
| | | | | | | | | | | | | | | | | | | | | Fixes https://bugs.chromium.org/p/chromium/issues/detail?id=998712 SHT_LLVM_PART_EHDR marks the start of a partition. The partition sections will be extracted to a separate file. Align to the next maximum page size boundary so that we can find the ELF header at the start. We cannot benefit from overlapping p_offset ranges with the previous segment anyway. It seems we lack some llvm-objcopy --extract-main-partition and --extract-partition sanity checks. It may place EHDR at the start even if p_offset if non zero. Anyway, the lld change is justified for the reasons above. Reviewed By: ruiu Differential Revision: https://reviews.llvm.org/D67032 llvm-svn: 370629
* [ELF][RISCV] Allow PT_LOAD to have overlapping p_offset ranges on EM_RISCVFangrui Song2019-08-281-3/+3
| | | | | | | | Port the D64906 technique to RISC-V. It deletes 3 alignments at PT_LOAD boundaries for the default case: the size of a RISC-V binary decreases by at most 12kb. llvm-svn: 370192
* [ELF][AMDGPU][SPARC] Allow PT_LOAD to have overlapping p_offset ranges on ↵Fangrui Song2019-08-281-4/+3
| | | | | | EM_AMDGPU and EM_SPARCV9 llvm-svn: 370180
* [ELF][RISCV] Assign st_shndx of __global_pointer$ to 1 if .sdata does not existFangrui Song2019-08-281-11/+9
| | | | | | | | | | | | | | | | | | | | | | | | This essentially reverts the code change of D63132 and switches to a simpler approach. In an executable/shared object, st_shndx of a symbol can be: 1) SHN_UNDEF: undefined symbol (or canonical PLT) 2) SHN_ABS: absolute symbol 3) any other value (usually a regular section index) represents a relative symbol. The actual value does not matter. Many ld.so (musl, all archs except MIPS of FreeBSD rtld-elf) even treat 2) and 3) the same. If .sdata does not exist, it does not matter what value/section __global_pointer$ has, as long as it is relative (otherwise there will be a pedantic lld error. See D63132). Just set the st_shndx arbitrarily to 1. Dummy st_shndx=1 may be used by __rela_iplt_start, linker-script-defined symbols outside a section, __dso_handle, etc. Reviewed By: ruiu Differential Revision: https://reviews.llvm.org/D66798 llvm-svn: 370172
* [ELF][ARM] Allow PT_LOAD to have overlapping p_offset ranges on EM_ARMFangrui Song2019-08-271-1/+2
| | | | | | | | | | | | Port the D64906 technique to ARM. It deletes 3 alignments at PT_LOAD boundaries for the default case: the size of an arm binary decreases by at most 12kb. Reviewed By: grimar Differential Revision: https://reviews.llvm.org/D66749 llvm-svn: 370049
* [ELF] EhFrameSection: postpone FDE liveness check to finalizeSectionsFangrui Song2019-08-261-3/+3
| | | | | | | | | | | | | | | EhFrameSection::addSection checks liveness of FDE early. This makes it infeasible to move combineEhSections() before ICF. Postpone the check to EhFrameSection::finalizeContents(). This is what ARMExidxSyntheticSection does and it will make a subsequent patch D66717 simpler. Reviewed By: ruiu Differential Revision: https://reviews.llvm.org/D66727 llvm-svn: 369890
* [ELF] Make LinkerScript::assignAddresses iterativeFangrui Song2019-08-261-10/+22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | PR42990. For `SECTIONS { b = a; . = 0xff00 + (a >> 8); a = .; }`, we currently set st_value(a)=0xff00 while st_value(b)=0xffff. The following call tree demonstrates the problem: ``` link<ELF64LE>(Args); Script->declareSymbols(); // insert a and b as absolute Defined Writer<ELFT>().run(); Script->processSectionCommands(); addSymbol(cmd); // a and b are re-inserted. LinkerScript::getSymbolValue // is lazily called by subsequent evaluation finalizeSections(); forEachRelSec(scanRelocations<ELFT>); processRelocAux // another problem PR42506, not affected by this patch finalizeAddressDependentContent(); // loop executed once script->assignAddresses(); // a = 0, b = 0xff00 script->assignAddresses(); // a = 0xff00, _end = 0xffff ``` We need another assignAddresses() to finalize the value of `a`. This patch 1) modifies assignAddress() to track the original section/value of each symbol and return a symbol whose section/value has changed. 2) moves the post-finalizeSections assignAddress() inside the loop of finalizeAddressDependentContent() and makes it iterative. Symbol assignment may not converge so we make a few attempts before bailing out. Note, assignAddresses() must be called at least twice. The penultimate call finalized section addresses while the last finalized symbol values. It is somewhat obscure and there was no comment. linkerscript/addr-zero.test tests this. Reviewed By: ruiu Differential Revision: https://reviews.llvm.org/D66279 llvm-svn: 369889
* [ELF] Simplify with less_second. NFCFangrui Song2019-08-241-4/+1
| | | | llvm-svn: 369844
* [ELF] Make member function Writer<ELFT>::removeEmptyPTLoad non-member. NFCFangrui Song2019-08-241-3/+1
| | | | llvm-svn: 369838
* [ELF] Align the first section of a PT_LOAD even if its type is SHT_NOBITSFangrui Song2019-08-241-11/+12
| | | | | | | | | | | | | | | | | | | | | Reported at https://reviews.llvm.org/D64930#1642223 If the only section of a PT_LOAD is a SHT_NOBITS section (e.g. .bss), we may not align its sh_offset. p_offset of the PT_LOAD will be set to sh_offset, and we will get p_offset!=p_vaddr (mod p_align). If such executable is mapped by the Linux kernel, it will segfault. After D64906, this may happen the non-linker script case. The linker script case has had this issue for a long time. This was fixed by rL321657 (but the test linkerscript/nobits-offset.s failed to test a SHT_NOBITS section), but broken by rL345154. Reviewed By: peter.smith Differential Revision: https://reviews.llvm.org/D66658 llvm-svn: 369828
* [ELF][PPC] Allow PT_LOAD to have overlapping p_offset ranges on EM_PPCFangrui Song2019-08-201-1/+1
| | | | | | | | Ported the D64906 technique to EM_PPC. Delete ppc-rela.s that is covered by ppc32-abs-pic.s llvm-svn: 369351
* [ELF][X86] Allow PT_LOAD to have overlapping p_offset ranges on EM_386Fangrui Song2019-08-201-2/+3
| | | | | | | | | | | | | | | | | | | | Ported the D64906 technique to EM_386. If `sh_addralign(.tdata) < sh_addralign(.tbss)`, we can potentially make `p_vaddr(PT_TLS)%p_align(PT_TLS) != 0`. ld.so that are known to have problems if p_vaddr%p_align!=0: * FreeBSD 13.0-CURRENT rtld-elf * glibc https://sourceware.org/bugzilla/show_bug.cgi?id=24606 New test i386-tls-vaddr-align.s checks our workaround makes p_vaddr%p_align = 0. Reviewed By: ruiu Differential Revision: https://reviews.llvm.org/D65865 llvm-svn: 369347
* [ELF][AArch64] Allow PT_LOAD to have overlapping p_offset rangesFangrui Song2019-08-201-1/+2
| | | | | | | | | | | | | | | | | | | | | | Ported the D64906 technique to AArch64. It deletes 3 alignments at PT_LOAD boundaries for the default case: the size of an aarch64 binary decreases by at most 192kb. If `sh_addralign(.tdata) < sh_addralign(.tbss)`, we can potentially make `p_vaddr(PT_TLS)%p_align(PT_TLS) != 0`. ld.so that are known to have problems if p_vaddr%p_align!=0: * musl<=1.1.22 * FreeBSD 13.0-CURRENT (and before) rtld-elf arm64 New test aarch64-tls-vaddr-align.s checks that our workaround makes p_vaddr%p_align = 0. Reviewed By: ruiu Differential Revision: https://reviews.llvm.org/D64930 llvm-svn: 369344
* [ELF][PPC] Allow PT_LOAD to have overlapping p_offset rangesFangrui Song2019-08-201-12/+56
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This change affects the non-linker script case (precisely, when the `SECTIONS` command is not used). It deletes 3 alignments at PT_LOAD boundaries for the default case: the size of a powerpc64 binary can be decreased by at most 192kb. The technique can be ported to other targets. Let me demonstrate the idea with a maxPageSize=65536 example: When assigning the address to the first output section of a new PT_LOAD, if the end p_vaddr of the previous PT_LOAD is 0x10020, we advance to the next multiple of maxPageSize: 0x20000. The new PT_LOAD will thus have p_vaddr=0x20000. Because p_offset and p_vaddr are congruent modulo maxPageSize, p_offset will be 0x20000, leaving a p_offset gap [0x10020, 0x20000) in the output. Alternatively, if we advance to 0x20020, the new PT_LOAD will have p_vaddr=0x20020. We can pick either 0x10020 or 0x20020 for p_offset! Obviously 0x10020 is the choice because it leaves no gap. At runtime, p_vaddr will be rounded down by pagesize (65536 if pagesize=maxPageSize). This PT_LOAD will load additional initial contents from p_offset ranges [0x10000,0x10020), which will also be loaded by the previous PT_LOAD. This is fine if -z noseparate-code is in effect or if we are not transiting between executable and non-executable segments. ld.bfd -z noseparate-code leverages this technique to keep output small. This patch implements the technique in lld, which is mostly effective on targets with large defaultMaxPageSize (AArch64/MIPS/PPC: 65536). The 3 removed alignments can save almost 3*65536 bytes. Two places that rely on p_vaddr%pagesize = 0 have to be updated. 1) We used to round p_memsz(PT_GNU_RELRO) up to commonPageSize (defaults to 4096 on all targets). Now p_vaddr%commonPageSize may be non-zero. The updated formula takes account of that factor. 2) Our TP offsets formulae are only correct if p_vaddr%p_align = 0. Fix them. See the updated comments in InputSection.cpp for details. On targets that we enable the technique (only PPC64 now), we can potentially make `p_vaddr(PT_TLS)%p_align(PT_TLS) != 0` if `sh_addralign(.tdata) < sh_addralign(.tbss)` This exposes many problems in ld.so implementations, especially the offsets of dynamic TLS blocks. Known issues: FreeBSD 13.0-CURRENT rtld-elf (i386/amd64/powerpc/arm64) glibc (HEAD) i386 and x86_64 https://sourceware.org/bugzilla/show_bug.cgi?id=24606 musl<=1.1.22 on TLS Variant I architectures (aarch64/powerpc64/...) So, force p_vaddr%p_align = 0 by rounding dot up to p_align(PT_TLS). The technique will be enabled (with updated tests) for other targets in subsequent patches. Reviewed By: ruiu Differential Revision: https://reviews.llvm.org/D64906 llvm-svn: 369343
* [ELF] Don't special case symbolic relocations with 0 addend to ifunc in ↵Fangrui Song2019-08-131-2/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | writable locations Currently the following 3 relocation types do not trigger the creation of a canonical PLT (which changes STT_GNU_IFUNC to STT_FUNC and redirects all references): 1) GOT-generating (`needsGot`) 2) PLT-generating (`needsPlt`) 3) R_ABS with 0 addend in a writable location. This is used for for ifunc function pointers in writable sections such as .data and .toc. This patch deletes case 3) to simplify the R_*_IRELATIVE generating logic added in D57371. Other advantages: * It is guaranteed no more than 1 R_*_IRELATIVE is created for an ifunc. * PPC64: no need to special case ifunc in toc-indirect to toc-relative relaxation. See D65755 The deleted elf::addIRelativeRelocs demonstrates that one-pass scan through relocations makes several optimizations difficult. This is something we can think about in the future. Reviewed By: peter.smith Differential Revision: https://reviews.llvm.org/D65995 llvm-svn: 368661
* [ELF] Simplify handling of exportDynamic and isPreemptibleFangrui Song2019-08-131-8/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In Writer::includeInDynSym(), exportDynamic is used by a Defined with protected or default visibility, to record whether it is required to be exported into .dynsym. It is set when any of the following conditions hold: 1) There is an interposable symbol from a DSO (Undefined or SharedSymbol with default visibility) 2) If -shared or --export-dynamic is specified, any symbol in an object file/bitcode sets this property, unless suppressed by canBeOmittedFromSymbolTable(). 3) --dynamic-list when producing an executable 4) protected symbol from a DSO preempted by copy relocation/canonical PLT when --ignore-{data,function}-address-equality is specified 5) ifunc is exported when -z ifunc-noplt is specified Bullet points 4) and 5) are irrelevant in this patch. Bullet 3) does not play well with 1) and 2). When -shared is specified, exportDynamic of most symbols is true. This makes it incapable to record --dynamic-list marked symbols. We thus have obscure: if (!config->shared) b->exportDynamic = true; else if (b->includeInDynsym()) b->isPreemptible = true; This patch adds another bit `Symbol::inDynamicList` to record 3). We can thus simplify handleDynamicList() by unifying the DSO and executable cases. It also allows us to simplify isPreemptible - now the field is only used in finalizeSections() and later stages. Reviewed By: peter.smith Differential Revision: https://reviews.llvm.org/D66091 llvm-svn: 368659
* [ELF] Consistently prioritize non-* wildcards overs "*" in version scriptsFangrui Song2019-08-051-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We prioritize non-* wildcards overs VER_NDX_LOCAL/VER_NDX_GLOBAL "*". This patch generalizes the rule to "*" of other versions and thus fixes PR40176. I don't feel strongly about this GNU linkers' behavior but the generalization simplifies code. Delete `config->defaultSymbolVersion` which was used to special case VER_NDX_LOCAL/VER_NDX_GLOBAL "*". In `SymbolTable::scanVersionScript`, custom versions are handled the same way as VER_NDX_LOCAL/VER_NDX_GLOBAL. So merge `config->versionScript{Locals,Globals}` into `config->versionDefinitions`. Overall this seems to simplify the code. In `SymbolTable::assign{Exact,Wildcard}Versions`, `sym->verdefIndex == config->defaultSymbolVersion` is changed to `verdefIndex == UINT32_C(-1)`. This allows us to give duplicate assignment diagnostics for `{ global: foo; };` `V1 { global: foo; };` In test/linkerscript/version-script.s: vs_index of an undefined symbol changes from 0 to 1. This doesn't matter (arguably 1 is better because the binding is STB_GLOBAL) because vs_index of an undefined symbol is ignored. Reviewed By: ruiu Differential Revision: https://reviews.llvm.org/D65716 llvm-svn: 367869
* [ELF] Move R_*_IRELATIVE from .rel[a].plt to .rel[a].dyn unless ↵Fangrui Song2019-08-031-17/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | --pack-dyn-relocs=android[+relr] An R_*_IRELATIVE represents the address of a STT_GNU_IFUNC symbol (redirected at runtime) which is non-preemptable and is not associated with a canonical PLT (associated with a symbol with a section index of SHN_UNDEF but a non-zero st_value). .rel[a].plt [DT_JMPREL, DT_JMPREL+DT_JMPRELSZ) contains relocations that can be lazily resolved. R_*_IRELATIVE are always eagerly resolved, so conceptually they do not belong to .rela.plt. "iplt" is mostly a misnomer. glibc powerpc and powerpc64 do not resolve R_*_IRELATIVE if they are in .rela.plt. // a.o - synthesized PLT call stub has an R_*_IRELATIVE void ifunc(); int main() { ifunc(); } // b.o static void real() {} asm (".type ifunc, %gnu_indirect_function"); void *ifunc() { return &real; } The lld-linked executable crashes. ld.bfd places R_*_IRELATIVE in .rela.dyn and the executable works. glibc i386, x86_64, and aarch64 have logic (glibc/sysdeps/*/dl-machine.h:elf_machine_lazy_rel) to eagerly resolve R_*_IRELATIVE in .rel[a].plt so the lld-linked executable works. Move R_*_IRELATIVE from .rel[a].plt to .rel[a].dyn to fix the crashes on glibc powerpc/powerpc64. This also helps simplifying ifunc implementation in FreeBSD rtld-elf powerpc64. If --pack-dyn-relocs=android[+relr] is specified, the Android packed dynamic relocation format is used for .rela.dyn. We cannot name in.relaIplt ".rela.dyn" because the output section will have mixed formats. This can be improved in the future. Reviewed By: pcc Differential Revision: https://reviews.llvm.org/D65651 llvm-svn: 367745
OpenPOWER on IntegriCloud