summaryrefslogtreecommitdiffstats
path: root/llvm/lib/LTO
Commit message (Collapse)AuthorAgeFilesLines
...
* Revert "[ThinLTO] Ensure we always select the same function copy to import"Teresa Johnson2018-07-141-2/+2
| | | | | | | This reverts commits r337050 and r337059. Caused failure in reverse-iteration bot that needs more investigation. llvm-svn: 337081
* [ThinLTO] Ensure we always select the same function copy to importTeresa Johnson2018-07-131-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In order to always import the same copy of a linkonce function, even when encountering it with different thresholds (a higher one then a lower one), keep track of the summary we decided to import. This ensures that the backend only gets a single definition to import for each GUID, so that it doesn't need to choose one. Move the largest threshold the GUID was considered for import into the current module out of the ImportMap (which is part of a larger map maintained across the whole index), and into a new map just maintained for the current module we are computing imports for. This saves some memory since we no longer have the thresholds maintained across the whole index (and throughout the in-process backends when doing a normal non-distributed ThinLTO build), at the cost of some additional information being maintained for each invocation of ComputeImportForModule (the selected summary pointer for each import). There is an additional map lookup for each callee being considered for importing, however, this was able to subsume a map lookup in the Worklist iteration that invokes computeImportForFunction. We also are able to avoid calling selectCallee if we already failed to import at the same or higher threshold. I compared the run time and peak memory for the SPEC2006 471.omnetpp benchmark (running in-process ThinLTO backends), as well as for a large internal benchmark with a distributed ThinLTO build (so just looking at the thin link time/memory). Across a number of runs with and without this change there was no significant change in the time and memory. (I tried a few other variations of the change but they also didn't improve time or peak memory). Reviewers: davidxl Subscribers: mehdi_amini, inglorion, llvm-commits Differential Revision: https://reviews.llvm.org/D48670 llvm-svn: 337050
* [ThinLTO] Use std::map to get determistic imports filesTeresa Johnson2018-07-102-2/+9
| | | | | | | | | | | | | | | | | | | | | | Summary: I noticed that the .imports files emitted for distributed ThinLTO backends do not have consistent ordering. This is because StringMap iteration order is not guaranteed to be deterministic. Since we already have a std::map with this information, used when emitting the individual index files (ModuleToSummariesForIndex), use it for the imports files as well. This issue is likely causing some unnecessary rebuilds of the ThinLTO backends in our distributed build system as the imports files are inputs to those backends. Reviewers: pcc, steven_wu, mehdi_amini Subscribers: mehdi_amini, inglorion, eraman, steven_wu, dexonsmith, llvm-commits Differential Revision: https://reviews.llvm.org/D48783 llvm-svn: 336721
* [ThinLTO] Update ThinLTO cache file atimes when on WindowsAndrew Ng2018-07-042-9/+41
| | | | | | | | | | | | | | | | | | ThinLTO cache file access times are used for expiration based pruning and since Vista, file access times are not updated by Windows by default: https://blogs.technet.microsoft.com/filecab/2006/11/07/disabling-last-access-time-in-windows-vista-to-improve-ntfs-performance This means on Windows, cache files are currently being pruned from creation time. This change manually updates cache files that are accessed by ThinLTO, when on Windows. Patch by Owen Reynolds. Differential Revision: https://reviews.llvm.org/D47266 llvm-svn: 336276
* LTO: Keep file handles open for memory mapped files.Peter Collingbourne2018-06-131-1/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | On Windows we've observed that if you open a file, write to it, map it into memory and close the file handle, the contents of the memory mapping can sometimes be incorrect. That was what we did when adding an entry to the ThinLTO cache using the TempFile and MemoryBuffer classes, and it was causing intermittent build failures on Chromium's ThinLTO bots on Windows. More details are in the associated Chromium bug (crbug.com/786127). We can prevent this from happening by keeping a handle to the file open while the mapping is active. So this patch changes the mapped_file_region class to duplicate the file handle when mapping the file and close it upon unmapping it. One gotcha is that the file handle that we keep open must not have been created with FILE_FLAG_DELETE_ON_CLOSE, as otherwise the operating system will prevent other processes from opening the file. We can achieve this by avoiding the use of FILE_FLAG_DELETE_ON_CLOSE altogether. Instead, we use SetFileInformationByHandle with FileDispositionInfo to manage the delete-on-close bit. This lets us remove the hack that we used to use to clear the delete-on-close bit on a file opened with FILE_FLAG_DELETE_ON_CLOSE. A downside of using SetFileInformationByHandle/FileDispositionInfo as opposed to FILE_FLAG_DELETE_ON_CLOSE is that it prevents us from using CreateFile to open the file while the flag is set, even within the same process. This doesn't seem to matter for almost every client of TempFile, except for LockFileManager, which calls sys::fs::create_link to create a hard link from the lock file, and in the process of doing so tries to open the file. To prevent this change from breaking LockFileManager I changed it to stop using TempFile by effectively reverting r318550. Differential Revision: https://reviews.llvm.org/D48051 llvm-svn: 334630
* [ThinLTO] Rename index IsAnalysis flag to HaveGVs (NFC)Teresa Johnson2018-06-062-2/+2
| | | | | | | | | With the upcoming patch to add summary parsing support, IsAnalysis would be true in contexts where we are not performing module summary analysis. Rename to the more specific and approprate HaveGVs, which is essentially what this flag is indicating. llvm-svn: 334140
* IRGen: Write .dwo files when -split-dwarf-file is used together with ↵Peter Collingbourne2018-05-311-1/+6
| | | | | | | | -fthinlto-index. Differential Revision: https://reviews.llvm.org/D47597 llvm-svn: 333677
* Fix a make_unique ambiguity.Peter Collingbourne2018-05-211-1/+1
| | | | llvm-svn: 332889
* LTO: Replace split dwarf implementation that uses objcopy with one that uses ↵Peter Collingbourne2018-05-211-61/+17
| | | | | | | | | | direct emission. Part of PR37466. Differential Revision: https://reviews.llvm.org/D47091 llvm-svn: 332884
* CodeGen: Add a dwo output file argument to addPassesToEmitFile and hook it ↵Peter Collingbourne2018-05-212-3/+4
| | | | | | | | | | up to dwo output. Part of PR37466. Differential Revision: https://reviews.llvm.org/D47089 llvm-svn: 332881
* Rename DEBUG macro to LLVM_DEBUG.Nicola Zaghen2018-05-141-3/+3
| | | | | | | | | | | | | | | | The DEBUG() macro is very generic so it might clash with other projects. The renaming was done as follows: - git grep -l 'DEBUG' | xargs sed -i 's/\bDEBUG\s\?(/LLVM_DEBUG(/g' - git diff -U0 master | ../clang/tools/clang-format/clang-format-diff.py -i -p1 -style LLVM - Manual change to APInt - Manually chage DOCS as regex doesn't match it. In the transition period the DEBUG() macro is still present and aliased to the LLVM_DEBUG() one. Differential Revision: https://reviews.llvm.org/D43624 llvm-svn: 332240
* [LTO] Handle Task=-1 passed to addSaveTempsTeresa Johnson2018-05-051-3/+5
| | | | | | | | | | | | | | | | | | | Summary: This change is necessary for D46464, which will pass -1 as the Task ID for distributed backends, so that the save temps files don't end up with "4294967295" in their path. For distributed back ends, when -1 is passed, don't append any Task ID. An existing test (tools/clang/test/CodeGen/thinlto_backend.ll) will fail without this change after D46464. Reviewers: pcc Subscribers: mehdi_amini, inglorion, llvm-commits Differential Revision: https://reviews.llvm.org/D46488 llvm-svn: 331591
* [LTO] Allow pass remarks with hotness to be set when emitting to stderrTeresa Johnson2018-05-041-2/+2
| | | | | | | | | | | | | | | | | | | | | Summary: Set setDiagnosticsHotnessRequested before the early exit check for a diagnostic output file, so that pass remarks with hotness works when emitting pass remarks to stderr (e.g. via -pass-remarks=.). Also fix the llvm-lto2 diagnistic handler so that it only calls exit(1) when the diagnistic is an error type. Otherwise the new test invocation of llvm-lto2 with -pass-remarks causes it to fail. The new code is consistent with the diagnostic handler elsewhere (e.g. on the LLVMContext). Reviewers: pcc, davide Subscribers: fhahn, mehdi_amini, llvm-commits, inglorion Differential Revision: https://reviews.llvm.org/D46387 llvm-svn: 331569
* [ThinLTO] Add support for optimization remarks to thinBackendTeresa Johnson2018-05-031-15/+20
| | | | | | | | | | | | | | Summary: Support was added to the regular LTO backend, but not thinBackend. This patch adds that support. Reviewers: pcc, davide Subscribers: mehdi_amini, inglorion, llvm-commits Differential Revision: https://reviews.llvm.org/D46376 llvm-svn: 331481
* IWYU for llvm-config.h in llvm, additions.Nico Weber2018-04-302-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | See r331124 for how I made a list of files missing the include. I then ran this Python script: for f in open('filelist.txt'): f = f.strip() fl = open(f).readlines() found = False for i in xrange(len(fl)): p = '#include "llvm/' if not fl[i].startswith(p): continue if fl[i][len(p):] > 'Config': fl.insert(i, '#include "llvm/Config/llvm-config.h"\n') found = True break if not found: print 'not found', f else: open(f, 'w').write(''.join(fl)) and then looked through everything with `svn diff | diffstat -l | xargs -n 1000 gvim -p` and tried to fix include ordering and whatnot. No intended behavior change. llvm-svn: 331184
* [LTO] Add stats-file option to LTO/Config.h.Florian Hahn2018-04-201-3/+21
| | | | | | | | | | | | | This patch adds a StatsFile option to LTO/Config.h and updates both LLVMGold and llvm-lto2 to set it. Reviewers: MatzeB, tejohnson, espindola Reviewed By: tejohnson Differential Revision: https://reviews.llvm.org/D45531 llvm-svn: 330411
* Rename ObjectMemoryBuffer to SmallVectorMemoryBuffer; NFCIWeiming Zhao2018-04-161-3/+3
| | | | | | | | | | | | | | Summary: As discussed in https://reviews.llvm.org/D45606, it makes more sense to name the class as SmallVectorMemoryBuffer Reviewers: bkramer, dblaikie Reviewed By: dblaikie Subscribers: mehdi_amini, eraman, llvm-commits Differential Revision: https://reviews.llvm.org/D45661 llvm-svn: 330107
* NFC: Move ObjectMemoryBuffer to supportWeiming Zhao2018-04-152-2/+1
| | | | | | | | | | | | | | | | Summary: Since the class is used by both MCJIT and LTO, it makes more sense to move it to Support lib. This is a follow up patch to r329929 and https://reviews.llvm.org/D45244 Reviewers: bkramer, dblaikie Reviewed By: bkramer Subscribers: mehdi_amini, eraman, llvm-commits Differential Revision: https://reviews.llvm.org/D45606 llvm-svn: 330093
* [LTO] Change std::sort to llvm::sort in response to r327219Mandeep Singh Grang2018-04-131-6/+6
| | | | | | | | | | | | | | | | | | | | | | Summary: r327219 added wrappers to std::sort which randomly shuffle the container before sorting. This will help in uncovering non-determinism caused due to undefined sorting order of objects having the same key. To make use of that infrastructure we need to invoke llvm::sort instead of std::sort. Note: This patch is one of a series of patches to replace *all* std::sort to llvm::sort. Refer D44363 for a list of all the required patches. Reviewers: pcc, mehdi_amini, ruiu Reviewed By: ruiu Subscribers: ruiu, inglorion, eraman, llvm-commits Differential Revision: https://reviews.llvm.org/D45137 llvm-svn: 330053
* Enable debug fission for thinLTO linked via gold-pluginYunlian Jiang2018-04-132-0/+68
| | | | | | | | | | | | | | Summary: This enables debug fission on implicit ThinLTO when linked with gold. It will put the .dwo files in a directory specified by user. Reviewers: tejohnson, pcc, dblaikie Reviewed By: pcc Subscribers: JDevlieghere, mehdi_amini, inglorion Differential Revision: https://reviews.llvm.org/D44792 llvm-svn: 329988
* Add missing vtable anchorsWeiming Zhao2018-04-111-0/+1
| | | | | | | | | | | | | | Summary: This patch adds anchor() for MemoryBuffer, raw_fd_ostream, RTDyldMemoryManager, SectionMemoryManager, etc. Reviewers: jlebar, eli.friedman, dblaikie Reviewed By: dblaikie Subscribers: mehdi_amini, mgorny, dblaikie, weimingz, llvm-commits Differential Revision: https://reviews.llvm.org/D45244 llvm-svn: 329861
* Prevent data races in concurrent ThinLTO processes.Ekaterina Romanova2018-03-301-17/+14
| | | | | | | | | | | | Make sure ThinLTO with caching doesn't use non-atomic writes to the cache file (to prevent data races and cache files corruption). 1. Place temp file to the same place where the caching directory is (instead of creating it the directory pointed to by TMP/TEMP variable). This will help to prevent using non-atomic rename and falling back to non-atomic "direct" write to the cache file. 2. if rename failed do not write to the cache file directly (direct write to the file is non-atomic and could cause data race conditions). 3. if cache file doesn't exist (e.g., because 'rename' failed or because some other reasons), bypass using the cache altogether. Differential Revision: https://reviews.llvm.org/D45076 llvm-svn: 328904
* Move TargetLoweringObjectFile from CodeGen to Target to fix layeringDavid Blaikie2018-03-231-1/+1
| | | | | | | It's implemented in Target & include from other Target headers, so the header should be in Target. llvm-svn: 328392
* Sink Analysis/ObjectUtil(canBeOmittedFromSymbolTable) into IR so it can be ↵David Blaikie2018-03-211-2/+1
| | | | | | legitimately be used by Object/IRSymtab llvm-svn: 328135
* [LTO] Return proper error object rather than null LTOModuleAdam Nemet2018-03-131-1/+1
| | | | | | | | This caused a crash in LTOModule::createInLocalContext. rdar://37926841 llvm-svn: 327359
* Revert "[LTO] Support filtering by hotness threshold"Bob Haarman2018-03-084-19/+7
| | | | | | | | This reverts commit 1f3bd185c53beb6aa68446974b7e80837abd6ef0 (r326107) because it fails ThinLTO/X86/diagnostic-handler-remarks-with-hotness.ll. llvm-svn: 326975
* [LTO] Support filtering by hotness thresholdAdam Nemet2018-02-264-7/+19
| | | | | | | | | | | This wires up -pass-remarks-hotness-threshold to LTO and ThinLTO. Next is to change the clang driver to pass this with -fdiagnostics-hotness-threshold. Differential Revision: https://reviews.llvm.org/D41465 llvm-svn: 326107
* [ThinLTO] Always create linked objects file for --thinlto-index-only=Vitaly Buka2018-02-221-20/+8
| | | | | | | | | | | | | | | Summary: ThinLTO indexing may decide to skip all objects. If we don't write something to the list build system may consider this as failure or linker can reuse a file from the previews build. Reviewers: pcc, tejohnson Subscribers: mehdi_amini, inglorion, eraman, hiraditya, llvm-commits Differential Revision: https://reviews.llvm.org/D43415 llvm-svn: 325819
* [LTO] Remove unused Path parameter to AddBufferFnTeresa Johnson2018-02-201-2/+2
| | | | | | | | | | | | | | | Summary: With D43396, no clients use the Path parameter anymore. Depends on D43396. Reviewers: pcc Subscribers: mehdi_amini, inglorion, llvm-commits Differential Revision: https://reviews.llvm.org/D43400 llvm-svn: 325619
* [ThinLTO] Add GraphTraits for FunctionSummariesCharles Saternos2018-02-191-0/+7
| | | | | | | | Add GraphTraits definitions to the FunctionSummary and ModuleSummaryIndex classes. These GraphTraits will be used to construct find SCC's in ThinLTO analysis passes. Third attempt - moved function from lambda to static function due to build failures. llvm-svn: 325506
* Revert: [llvm] r325448 - [ThinLTO] Add GraphTraits for FunctionSummaries Simon Pilgrim2018-02-181-7/+0
| | | | | | | | | | Add GraphTraits definitions to the FunctionSummary and ModuleSummaryIndex classes. These GraphTraits will be used to construct find SCC's in ThinLTO analysis passes. Second attempt, since last patch caused stage2 build to fail (now using function_ref rather than std::function). Reverted due to buildbot failures llvm-svn: 325454
* [ThinLTO] Add GraphTraits for FunctionSummariesCharles Saternos2018-02-171-0/+7
| | | | | | | | Add GraphTraits definitions to the FunctionSummary and ModuleSummaryIndex classes. These GraphTraits will be used to construct find SCC's in ThinLTO analysis passes. Second attempt, since last patch caused stage2 build to fail (now using function_ref rather than std::function). llvm-svn: 325448
* Pass a reference to a module to the bitcode writer.Rafael Espindola2018-02-143-5/+5
| | | | | | | This simplifies most callers as they are already using references or std::unique_ptr. llvm-svn: 325155
* Revert "[ThinLTO] Add GraphTraits for FunctionSummaries"Volodymyr Sapsai2018-02-121-7/+0
| | | | | | | | | It caused assertion failure Assertion failed: (!DD.IsLambda && !MergeDD.IsLambda && "faked up lambda definition?"), function MergeDefinitionData, file /Users/buildslave/jenkins/workspace/clang-stage1-configure-RA/llvm/tools/clang/lib/Serialization/ASTReaderDecl.cpp, line 1675. on the second stage build bots. llvm-svn: 324932
* [ThinLTO] Add GraphTraits for FunctionSummariesCharles Saternos2018-02-111-0/+7
| | | | | | Add GraphTraits definitions to the FunctionSummary and ModuleSummaryIndex classes. These GraphTraits will be used to construct find SCC's in ThinLTO analysis passes. llvm-svn: 324854
* LTO: Include live bit in ThinLTO cache key.Peter Collingbourne2018-02-091-0/+1
| | | | | | | | | | As of r323633, this bit started controlling whether symbol definitions appear in object files, and it also became sensitive to the prevailing bit, so it needs to be included in the key. Differential Revision: https://reviews.llvm.org/D43109 llvm-svn: 324711
* [ThinLTO] Remove dead and dropped symbol declarations when possibleTeresa Johnson2018-02-061-6/+15
| | | | | | | | | | | | | | | | | | | | | Summary: Removing the dropped symbols will prevent indirect call promotion in the ThinLTO Backend from adding a new reference to a symbol, which can result in linker unsats. This can happen when we compile with a sample profile collected from one binary by used for another, which may have profiled targets that aren't used in the new binary. Note that until dropDeadSymbols handles variables and aliases (in progress), we may not be able to remove the declaration and can still have an issue. Reviewers: grimar, davidxl Subscribers: mehdi_amini, inglorion, llvm-commits, eraman Differential Revision: https://reviews.llvm.org/D42816 llvm-svn: 324299
* LTO: Also include dso-local bit for calls in ThinLTO cache key.Peter Collingbourne2018-02-051-1/+3
| | | | | | Differential Revision: https://reviews.llvm.org/D42934 llvm-svn: 324291
* LTO: Include dso-local bit in ThinLTO cache key.Peter Collingbourne2018-02-051-1/+3
| | | | | | Differential Revision: https://reviews.llvm.org/D42713 llvm-svn: 324253
* [ThinLTO] Convert dead alias to declarationsTeresa Johnson2018-02-051-10/+6
| | | | | | | | | | | | | | | | Summary: This complements the fixes in r323633 and r324075 which drop the definitions of dead functions and variables, respectively. Fixes PR36208. Reviewers: grimar, rafael Subscribers: mehdi_amini, llvm-commits, inglorion Differential Revision: https://reviews.llvm.org/D42856 llvm-svn: 324242
* [LTO] - Simplify. NFC.George Rimar2018-02-021-3/+2
| | | | llvm-svn: 324076
* [ThinLTO] - Fix for "ThinLTO inlines variables that should be discarded".George Rimar2018-02-021-6/+12
| | | | | | | | | | | This fixes PR36187. Patch teaches ThinLTO to drop non-prevailing variables, just like we recently did for functions (in r323633). Differential revision: https://reviews.llvm.org/D42798 llvm-svn: 324075
* [ThinLTO/gold] Write empty imports even for modules with symbolsVitaly Buka2018-01-301-7/+15
| | | | | | | | | | | | Summary: ThinLTO may skip object for other reasons, e.g. if there is no summary. Reviewers: pcc, eugenis Subscribers: mehdi_amini, inglorion, eraman, llvm-commits Differential Revision: https://reviews.llvm.org/D42514 llvm-svn: 323818
* [ThinLTO] - Stop internalizing and drop non-prevailing symbols.George Rimar2018-01-293-8/+60
| | | | | | | | | | | Implementation marks non-prevailing symbols as not live in the summary. Then them are dropped in backends. Fixes https://bugs.llvm.org/show_bug.cgi?id=35938 Differential revision: https://reviews.llvm.org/D42107 llvm-svn: 323633
* [LTO] - Introduce GlobalResolution::Prevailing flag.George Rimar2018-01-251-15/+9
| | | | | | | | | It is NFC refactoring change that will make D42107 a bit smaller. Differential revision: https://reviews.llvm.org/D42528 llvm-svn: 323444
* Another try to commit 323321 (aggressive instruction combine).Amjad Aboud2018-01-251-0/+1
| | | | llvm-svn: 323416
* Simplify. NFC.Rafael Espindola2018-01-241-1/+1
| | | | | | Thanks to Teresa Johnson for the suggestion. llvm-svn: 323365
* Reverted 323321.Amjad Aboud2018-01-241-1/+0
| | | | llvm-svn: 323326
* [InstCombine] Introducing Aggressive Instruction Combine pass ↵Amjad Aboud2018-01-241-0/+1
| | | | | | | | | | | | | | | | | | (-aggressive-instcombine). Combine expression patterns to form expressions with fewer, simple instructions. This pass does not modify the CFG. For example, this pass reduce width of expressions post-dominated by TruncInst into smaller width when applicable. It differs from instcombine pass in that it contains pattern optimization that requires higher complexity than the O(1), thus, it should run fewer times than instcombine pass. Differential Revision: https://reviews.llvm.org/D38313 llvm-svn: 323321
* Introduce errorToBool() helper and use it.Nico Weber2018-01-231-16/+4
| | | | | | | | | errorToBool() converts an Error to a bool and puts the Error in a checked state. No behavior change. https://reviews.llvm.org/D42422 llvm-svn: 323238
OpenPOWER on IntegriCloud