summaryrefslogtreecommitdiffstats
path: root/llvm/lib/ExecutionEngine/Orc/CompileOnDemandLayer.cpp
Commit message (Collapse)AuthorAgeFilesLines
* Update the file headers across all of the LLVM projects in the monorepoChandler Carruth2019-01-191-4/+3
| | | | | | | | | | | | | | | | | to reflect the new license. We understand that people may be surprised that we're moving the header entirely to discuss the new license. We checked this carefully with the Foundation's lawyer and we believe this is the correct approach. Essentially, all code in the project is now made available by the LLVM project under our new license, so you will see that the license headers include that license only. Some of our contributors have contributed code under our old license, and accordingly, we have retained a copy of our old license notice in the top-level files in each project and repository. llvm-svn: 351636
* [ORC] Re-apply r345077 with fixes to remove ambiguity in lookup calls.Lang Hames2018-10-231-5/+12
| | | | llvm-svn: 345098
* Revert r345077 "[ORC] Change how non-exported symbols are matched during ↵Reid Kleckner2018-10-231-12/+5
| | | | | | | | | | | | | | | lookup." Doesn't build on Windows. The call to 'lookup' is ambiguous. Clang and MSVC agree, anyway. http://lab.llvm.org:8011/builders/clang-x64-windows-msvc/builds/787 C:\b\slave\clang-x64-windows-msvc\build\llvm.src\unittests\ExecutionEngine\Orc\CoreAPIsTest.cpp(315): error C2668: 'llvm::orc::ExecutionSession::lookup': ambiguous call to overloaded function C:\b\slave\clang-x64-windows-msvc\build\llvm.src\include\llvm/ExecutionEngine/Orc/Core.h(823): note: could be 'llvm::Expected<llvm::JITEvaluatedSymbol> llvm::orc::ExecutionSession::lookup(llvm::ArrayRef<llvm::orc::JITDylib *>,llvm::orc::SymbolStringPtr)' C:\b\slave\clang-x64-windows-msvc\build\llvm.src\include\llvm/ExecutionEngine/Orc/Core.h(817): note: or 'llvm::Expected<llvm::JITEvaluatedSymbol> llvm::orc::ExecutionSession::lookup(const llvm::orc::JITDylibSearchList &,llvm::orc::SymbolStringPtr)' C:\b\slave\clang-x64-windows-msvc\build\llvm.src\unittests\ExecutionEngine\Orc\CoreAPIsTest.cpp(315): note: while trying to match the argument list '(initializer list, llvm::orc::SymbolStringPtr)' llvm-svn: 345078
* [ORC] Change how non-exported symbols are matched during lookup.Lang Hames2018-10-231-5/+12
| | | | | | | | | | | | | | | | | In the new scheme the client passes a list of (JITDylib&, bool) pairs, rather than a list of JITDylibs. For each JITDylib the boolean indicates whether or not to match against non-exported symbols (true means that they should be found, false means that they should not). The MatchNonExportedInJD and MatchNonExported parameters on lookup are removed. The new scheme is more flexible, and easier to understand. This patch also updates JITDylib search orders to be lists of (JITDylib&, bool) pairs to match the new lookup scheme. Error handling is also plumbed through the LLJIT class to allow regression tests to fail predictably when a lookup from a lazy call-through fails. llvm-svn: 345077
* [ORC] Make the VModuleKey optional, propagate it via MaterializationUnit andLang Hames2018-10-161-9/+11
| | | | | | | | | | | | | | | | | | | | | | MaterializationResponsibility. VModuleKeys are intended to enable selective removal of modules from a JIT session, however for a wide variety of use cases selective removal is not needed and introduces unnecessary overhead. As of this commit, the default constructed VModuleKey value is reserved as a "do not track" value, and becomes the default when adding a new module to the JIT. This commit also changes the propagation of VModuleKeys. They were passed alongside the MaterializationResponsibity instance in XXLayer::emit methods, but are now propagated as part of the MaterializationResponsibility instance itself (and as part of MaterializationUnit when stored in a JITDylib). Associating VModuleKeys with MaterializationUnits in this way should allow for a thread-safe module removal mechanism in the future, even when a module is in the process of being compiled, by having the MaterializationResponsibility object check in on its VModuleKey's state before commiting its results to the JITDylib. llvm-svn: 344643
* [ORC] Rename ORC layers to make the "new" ORC layers the default.Lang Hames2018-10-151-15/+15
| | | | | | | | | | | | | This commit adds a 'Legacy' prefix to old ORC layers and utilities, and removes the '2' suffix from the new ORC layers. If you wish to continue using the old ORC layers you will need to add a 'Legacy' prefix to your classes. If you were already using the new ORC layers you will need to drop the '2' suffix. The legacy layers will remain in-tree until the new layers reach feature parity with them. This will involve adding support for removing code from the new layers, and ensuring that performance is comperable. llvm-svn: 344572
* [ORC] Promote and rename private symbols inside the CompileOnDemand layer,Lang Hames2018-10-091-5/+20
| | | | | | | | | | | | | | | rather than require them to have been promoted before being passed in. Dropping this precondition is better for layer composition (CompileOnDemandLayer was the only one that placed pre-conditions on the modules that could be added). It also means that the promoted private symbols do not show up in the target JITDylib's symbol table. Instead, they are confined to the hidden implementation dylib that contains the actual definitions. For the 403.gcc testcase this cut down the public symbol table size from ~15,000 symbols to ~4000, substantially reducing symbol dependence tracking costs. llvm-svn: 344078
* [ORC] Pass symbol name to discard by const reference.Lang Hames2018-10-061-1/+1
| | | | | | This saves some unnecessary atomic ref-counting operations. llvm-svn: 343927
* [ORC] Add partitioning support to CompileOnDemandLayer2.Lang Hames2018-09-291-156/+174
| | | | | | | | | | | | | CompileOnDemandLayer2 now supports user-supplied partition functions (the original CompileOnDemandLayer already supported these). Partition functions are called with the list of requested global values (i.e. global values that currently have queries waiting on them) and have an opportunity to select extra global values to materialize at the same time. Also adds testing infrastructure for the new feature to lli. llvm-svn: 343396
* [ORC] Narrow a cast: the block guarded by the condition only handlesLang Hames2018-09-281-1/+1
| | | | | | GlobalVariables, not all GlobalValues. llvm-svn: 343358
* [ORC] Remove some dead code.Lang Hames2018-09-281-20/+0
| | | | llvm-svn: 343327
* [ORC] Update CompileOnDemandLayer2 to use the new lazyReexports mechanismLang Hames2018-09-261-67/+33
| | | | | | | | | | | | for lazy compilation, rather than a callback manager. The new mechanism does not block compile threads, and does not require function bodies to be renamed. Future modifications should allow laziness on a per-module basis to work without any modification of the input module. llvm-svn: 343065
* [ORC] Add ThreadSafeModule and ThreadSafeContext wrappers to support concurrentLang Hames2018-09-261-75/+46
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | compilation of IR in the JIT. ThreadSafeContext is a pair of an LLVMContext and a mutex that can be used to lock that context when it needs to be accessed from multiple threads. ThreadSafeModule is a pair of a unique_ptr<Module> and a shared_ptr<ThreadSafeContext>. This allows the lifetime of a ThreadSafeContext to be managed automatically in terms of the ThreadSafeModules that refer to it: Once all modules using a ThreadSafeContext are destructed, and providing the client has not held on to a copy of shared context pointer, the context will be automatically destructed. This scheme is necessary due to the following constraits: (1) We need multiple contexts for multithreaded compilation (at least one per compile thread plus one to store any IR not currently being compiled, though one context per module is simpler). (2) We need to free contexts that are no longer being used so that the JIT does not leak memory over time. (3) Module lifetimes are not predictable (modules are compiled as needed depending on the flow of JIT'd code) so there is no single point where contexts could be reclaimed. JIT clients not using concurrency can safely use one ThreadSafeContext for all ThreadSafeModules. JIT clients who want to be able to compile concurrently should use a different ThreadSafeContext for each module, or call setCloneToNewContextOnEmit on their top-level IRLayer. The former reduces compile latency (since no clone step is needed) at the cost of additional memory overhead for uncompiled modules (as every uncompiled module will duplicate the LLVM types, constants and metadata that have been shared). llvm-svn: 343055
* [ORC] Rename VSO to JITDylib.Lang Hames2018-08-171-8/+9
| | | | | | | | | | | | | VSO was a little close to VDSO (an acronym on Linux for Virtual Dynamic Shared Object) for comfort. It also risks giving the impression that instances of this class could be shared between ExecutionSessions, which they can not. JITDylib seems moderately less confusing, while still hinting at how this class is intended to be used, i.e. as a JIT-compiled stand-in for a dynamic library (code that would have been a dynamic library if you had wanted to compile it ahead of time). llvm-svn: 340084
* [ORC] Replace SymbolResolvers in the new ORC layers with search orders on VSOs.Lang Hames2018-07-201-47/+16
| | | | | | | | | | | | | | | A search order is a list of VSOs to be searched linearly to find symbols. Each VSO now has a search order that will be used when fixing up definitions in that VSO. Each VSO's search order defaults to just that VSO itself. This is a first step towards removing symbol resolvers from ORC altogether. In practice symbol resolvers tended to be used to implement a search order anyway, sometimes with additional programatic generation of symbols. Now that VSOs support programmatic generation of definitions via fallback generators, search orders provide a cleaner way to achieve the desired effect (while removing a lot of boilerplate). llvm-svn: 337593
* [ORC] Rename MaterializationResponsibility::delegate to replace and add a newLang Hames2018-07-091-2/+2
| | | | | | | | | | | | | delegate method (and unit test). The name 'replace' better captures what the old delegate method did: it returned materialization responsibility for a set of symbols to the VSO. The new delegate method delegates responsibility for a set of symbols to a new MaterializationResponsibility instance. This can be used to split responsibility between multiple threads, or multiple materialization methods. llvm-svn: 336603
* [ORC] In CompileOnDemandLayer2, clone modules on to different contexts byLang Hames2018-07-051-77/+79
| | | | | | | | | | | | | | | writing them to a buffer and re-loading them. Also introduces a multithreaded variant of SimpleCompiler (MultiThreadedSimpleCompiler) for compiling IR concurrently on multiple threads. These changes are required to JIT IR on multiple threads correctly. No test case yet. I will be looking at how to modify LLI / LLJIT to test multithreaded JIT support soon. llvm-svn: 336385
* [ORC] Verify modules when running LLLazyJIT in LLI, and deal with fallout.Lang Hames2018-07-021-3/+7
| | | | | | | | The verifier identified several modules that were broken due to incorrect linkage on declarations. To fix this, CompileOnDemandLayer2::extractFunction has been updated to change decls to external linkage. llvm-svn: 336150
* [ORC] Add LLJIT and LLLazyJIT, and replace OrcLazyJIT in LLI with LLLazyJIT.Lang Hames2018-06-261-8/+48
| | | | | | | | | | | | | | | | | | | LLJIT is a prefabricated ORC based JIT class that is meant to be the go-to replacement for MCJIT. Unlike OrcMCJITReplacement (which will continue to be supported) it is not API or bug-for-bug compatible, but targets the same use cases: Simple, non-lazy compilation and execution of LLVM IR. LLLazyJIT extends LLJIT with support for function-at-a-time lazy compilation, similar to what was provided by LLVM's original (now long deprecated) JIT APIs. This commit also contains some simple utility classes (CtorDtorRunner2, LocalCXXRuntimeOverrides2, JITTargetMachineBuilder) to support LLJIT and LLLazyJIT. Both of these classes are works in progress. Feedback from JIT clients is very welcome! llvm-svn: 335670
* [ORC] Add an initial implementation of a replacement CompileOnDemandLayer.Lang Hames2018-06-181-0/+328
CompileOnDemandLayer2 is a replacement for CompileOnDemandLayer built on the ORC Core APIs. Functions in added modules are extracted and compiled lazily. CompileOnDemandLayer2 supports multithreaded JIT'd code, and compilation on multiple threads. llvm-svn: 334967
OpenPOWER on IntegriCloud