diff options
author | Duncan P. N. Exon Smith <dexonsmith@apple.com> | 2014-12-09 18:38:53 +0000 |
---|---|---|
committer | Duncan P. N. Exon Smith <dexonsmith@apple.com> | 2014-12-09 18:38:53 +0000 |
commit | 5bf8fef58013e2c97180236fa6973faa40435d5f (patch) | |
tree | d43dbdd5a9ee2cf0031cf3cd953d365a5e4e141d /llvm/lib/Transforms | |
parent | b39e22bdc504ba77e2cd385538b6634808cda66e (diff) | |
download | bcm5719-llvm-5bf8fef58013e2c97180236fa6973faa40435d5f.tar.gz bcm5719-llvm-5bf8fef58013e2c97180236fa6973faa40435d5f.zip |
IR: Split Metadata from Value
Split `Metadata` away from the `Value` class hierarchy, as part of
PR21532. Assembly and bitcode changes are in the wings, but this is the
bulk of the change for the IR C++ API.
I have a follow-up patch prepared for `clang`. If this breaks other
sub-projects, I apologize in advance :(. Help me compile it on Darwin
I'll try to fix it. FWIW, the errors should be easy to fix, so it may
be simpler to just fix it yourself.
This breaks the build for all metadata-related code that's out-of-tree.
Rest assured the transition is mechanical and the compiler should catch
almost all of the problems.
Here's a quick guide for updating your code:
- `Metadata` is the root of a class hierarchy with three main classes:
`MDNode`, `MDString`, and `ValueAsMetadata`. It is distinct from
the `Value` class hierarchy. It is typeless -- i.e., instances do
*not* have a `Type`.
- `MDNode`'s operands are all `Metadata *` (instead of `Value *`).
- `TrackingVH<MDNode>` and `WeakVH` referring to metadata can be
replaced with `TrackingMDNodeRef` and `TrackingMDRef`, respectively.
If you're referring solely to resolved `MDNode`s -- post graph
construction -- just use `MDNode*`.
- `MDNode` (and the rest of `Metadata`) have only limited support for
`replaceAllUsesWith()`.
As long as an `MDNode` is pointing at a forward declaration -- the
result of `MDNode::getTemporary()` -- it maintains a side map of its
uses and can RAUW itself. Once the forward declarations are fully
resolved RAUW support is dropped on the ground. This means that
uniquing collisions on changing operands cause nodes to become
"distinct". (This already happened fairly commonly, whenever an
operand went to null.)
If you're constructing complex (non self-reference) `MDNode` cycles,
you need to call `MDNode::resolveCycles()` on each node (or on a
top-level node that somehow references all of the nodes). Also,
don't do that. Metadata cycles (and the RAUW machinery needed to
construct them) are expensive.
- An `MDNode` can only refer to a `Constant` through a bridge called
`ConstantAsMetadata` (one of the subclasses of `ValueAsMetadata`).
As a side effect, accessing an operand of an `MDNode` that is known
to be, e.g., `ConstantInt`, takes three steps: first, cast from
`Metadata` to `ConstantAsMetadata`; second, extract the `Constant`;
third, cast down to `ConstantInt`.
The eventual goal is to introduce `MDInt`/`MDFloat`/etc. and have
metadata schema owners transition away from using `Constant`s when
the type isn't important (and they don't care about referring to
`GlobalValue`s).
In the meantime, I've added transitional API to the `mdconst`
namespace that matches semantics with the old code, in order to
avoid adding the error-prone three-step equivalent to every call
site. If your old code was:
MDNode *N = foo();
bar(isa <ConstantInt>(N->getOperand(0)));
baz(cast <ConstantInt>(N->getOperand(1)));
bak(cast_or_null <ConstantInt>(N->getOperand(2)));
bat(dyn_cast <ConstantInt>(N->getOperand(3)));
bay(dyn_cast_or_null<ConstantInt>(N->getOperand(4)));
you can trivially match its semantics with:
MDNode *N = foo();
bar(mdconst::hasa <ConstantInt>(N->getOperand(0)));
baz(mdconst::extract <ConstantInt>(N->getOperand(1)));
bak(mdconst::extract_or_null <ConstantInt>(N->getOperand(2)));
bat(mdconst::dyn_extract <ConstantInt>(N->getOperand(3)));
bay(mdconst::dyn_extract_or_null<ConstantInt>(N->getOperand(4)));
and when you transition your metadata schema to `MDInt`:
MDNode *N = foo();
bar(isa <MDInt>(N->getOperand(0)));
baz(cast <MDInt>(N->getOperand(1)));
bak(cast_or_null <MDInt>(N->getOperand(2)));
bat(dyn_cast <MDInt>(N->getOperand(3)));
bay(dyn_cast_or_null<MDInt>(N->getOperand(4)));
- A `CallInst` -- specifically, intrinsic instructions -- can refer to
metadata through a bridge called `MetadataAsValue`. This is a
subclass of `Value` where `getType()->isMetadataTy()`.
`MetadataAsValue` is the *only* class that can legally refer to a
`LocalAsMetadata`, which is a bridged form of non-`Constant` values
like `Argument` and `Instruction`. It can also refer to any other
`Metadata` subclass.
(I'll break all your testcases in a follow-up commit, when I propagate
this change to assembly.)
llvm-svn: 223802
Diffstat (limited to 'llvm/lib/Transforms')
-rw-r--r-- | llvm/lib/Transforms/IPO/StripSymbols.cpp | 4 | ||||
-rw-r--r-- | llvm/lib/Transforms/InstCombine/InstCombineCalls.cpp | 17 | ||||
-rw-r--r-- | llvm/lib/Transforms/Instrumentation/AddressSanitizer.cpp | 33 | ||||
-rw-r--r-- | llvm/lib/Transforms/Instrumentation/SanitizerCoverage.cpp | 2 | ||||
-rw-r--r-- | llvm/lib/Transforms/ObjCARC/ObjCARCOpts.cpp | 8 | ||||
-rw-r--r-- | llvm/lib/Transforms/Scalar/LoopUnrollPass.cpp | 16 | ||||
-rw-r--r-- | llvm/lib/Transforms/Scalar/SROA.cpp | 16 | ||||
-rw-r--r-- | llvm/lib/Transforms/Scalar/ScalarReplAggregates.cpp | 16 | ||||
-rw-r--r-- | llvm/lib/Transforms/Utils/AddDiscriminators.cpp | 2 | ||||
-rw-r--r-- | llvm/lib/Transforms/Utils/CloneFunction.cpp | 4 | ||||
-rw-r--r-- | llvm/lib/Transforms/Utils/InlineFunction.cpp | 23 | ||||
-rw-r--r-- | llvm/lib/Transforms/Utils/Local.cpp | 20 | ||||
-rw-r--r-- | llvm/lib/Transforms/Utils/LoopUnrollRuntime.cpp | 13 | ||||
-rw-r--r-- | llvm/lib/Transforms/Utils/PromoteMemoryToRegister.cpp | 9 | ||||
-rw-r--r-- | llvm/lib/Transforms/Utils/SimplifyCFG.cpp | 11 | ||||
-rw-r--r-- | llvm/lib/Transforms/Utils/ValueMapper.cpp | 175 | ||||
-rw-r--r-- | llvm/lib/Transforms/Vectorize/LoopVectorize.cpp | 22 |
17 files changed, 243 insertions, 148 deletions
diff --git a/llvm/lib/Transforms/IPO/StripSymbols.cpp b/llvm/lib/Transforms/IPO/StripSymbols.cpp index 3412b9e915e..816978ea9ce 100644 --- a/llvm/lib/Transforms/IPO/StripSymbols.cpp +++ b/llvm/lib/Transforms/IPO/StripSymbols.cpp @@ -301,8 +301,8 @@ bool StripDeadDebugInfo::runOnModule(Module &M) { // For each compile unit, find the live set of global variables/functions and // replace the current list of potentially dead global variables/functions // with the live list. - SmallVector<Value *, 64> LiveGlobalVariables; - SmallVector<Value *, 64> LiveSubprograms; + SmallVector<Metadata *, 64> LiveGlobalVariables; + SmallVector<Metadata *, 64> LiveSubprograms; DenseSet<const MDNode *> VisitedSet; for (DICompileUnit DIC : F.compile_units()) { diff --git a/llvm/lib/Transforms/InstCombine/InstCombineCalls.cpp b/llvm/lib/Transforms/InstCombine/InstCombineCalls.cpp index 1dd828602a3..f70d8f3530a 100644 --- a/llvm/lib/Transforms/InstCombine/InstCombineCalls.cpp +++ b/llvm/lib/Transforms/InstCombine/InstCombineCalls.cpp @@ -119,15 +119,14 @@ Instruction *InstCombiner::SimplifyMemTransfer(MemIntrinsic *MI) { // If the memcpy has metadata describing the members, see if we can // get the TBAA tag describing our copy. if (MDNode *M = MI->getMetadata(LLVMContext::MD_tbaa_struct)) { - if (M->getNumOperands() == 3 && - M->getOperand(0) && - isa<ConstantInt>(M->getOperand(0)) && - cast<ConstantInt>(M->getOperand(0))->isNullValue() && + if (M->getNumOperands() == 3 && M->getOperand(0) && + mdconst::hasa<ConstantInt>(M->getOperand(0)) && + mdconst::extract<ConstantInt>(M->getOperand(0))->isNullValue() && M->getOperand(1) && - isa<ConstantInt>(M->getOperand(1)) && - cast<ConstantInt>(M->getOperand(1))->getValue() == Size && - M->getOperand(2) && - isa<MDNode>(M->getOperand(2))) + mdconst::hasa<ConstantInt>(M->getOperand(1)) && + mdconst::extract<ConstantInt>(M->getOperand(1))->getValue() == + Size && + M->getOperand(2) && isa<MDNode>(M->getOperand(2))) CopyMD = cast<MDNode>(M->getOperand(2)); } } @@ -1129,7 +1128,7 @@ Instruction *InstCombiner::visitCallInst(CallInst &CI) { cast<Constant>(RHS)->isNullValue()) { LoadInst* LI = cast<LoadInst>(LHS); if (isValidAssumeForContext(II, LI, DL, DT)) { - MDNode* MD = MDNode::get(II->getContext(), ArrayRef<Value*>()); + MDNode *MD = MDNode::get(II->getContext(), None); LI->setMetadata(LLVMContext::MD_nonnull, MD); return EraseInstFromFunction(*II); } diff --git a/llvm/lib/Transforms/Instrumentation/AddressSanitizer.cpp b/llvm/lib/Transforms/Instrumentation/AddressSanitizer.cpp index 5072d09b237..b6fcea8ae13 100644 --- a/llvm/lib/Transforms/Instrumentation/AddressSanitizer.cpp +++ b/llvm/lib/Transforms/Instrumentation/AddressSanitizer.cpp @@ -220,8 +220,10 @@ struct LocationMetadata { assert(MDN->getNumOperands() == 3); MDString *MDFilename = cast<MDString>(MDN->getOperand(0)); Filename = MDFilename->getString(); - LineNo = cast<ConstantInt>(MDN->getOperand(1))->getLimitedValue(); - ColumnNo = cast<ConstantInt>(MDN->getOperand(2))->getLimitedValue(); + LineNo = + mdconst::extract<ConstantInt>(MDN->getOperand(1))->getLimitedValue(); + ColumnNo = + mdconst::extract<ConstantInt>(MDN->getOperand(2))->getLimitedValue(); } }; @@ -249,23 +251,22 @@ class GlobalsMetadata { for (auto MDN : Globals->operands()) { // Metadata node contains the global and the fields of "Entry". assert(MDN->getNumOperands() == 5); - Value *V = MDN->getOperand(0); + auto *GV = mdconst::extract_or_null<GlobalVariable>(MDN->getOperand(0)); // The optimizer may optimize away a global entirely. - if (!V) + if (!GV) continue; - GlobalVariable *GV = cast<GlobalVariable>(V); // We can already have an entry for GV if it was merged with another // global. Entry &E = Entries[GV]; - if (Value *Loc = MDN->getOperand(1)) - E.SourceLoc.parse(cast<MDNode>(Loc)); - if (Value *Name = MDN->getOperand(2)) { - MDString *MDName = cast<MDString>(Name); - E.Name = MDName->getString(); - } - ConstantInt *IsDynInit = cast<ConstantInt>(MDN->getOperand(3)); + if (auto *Loc = cast_or_null<MDNode>(MDN->getOperand(1))) + E.SourceLoc.parse(Loc); + if (auto *Name = cast_or_null<MDString>(MDN->getOperand(2))) + E.Name = Name->getString(); + ConstantInt *IsDynInit = + mdconst::extract<ConstantInt>(MDN->getOperand(3)); E.IsDynInit |= IsDynInit->isOne(); - ConstantInt *IsBlacklisted = cast<ConstantInt>(MDN->getOperand(4)); + ConstantInt *IsBlacklisted = + mdconst::extract<ConstantInt>(MDN->getOperand(4)); E.IsBlacklisted |= IsBlacklisted->isOne(); } } @@ -496,9 +497,9 @@ struct FunctionStackPoisoner : public InstVisitor<FunctionStackPoisoner> { AllocaForValueMapTy AllocaForValue; FunctionStackPoisoner(Function &F, AddressSanitizer &ASan) - : F(F), ASan(ASan), DIB(*F.getParent()), C(ASan.C), - IntptrTy(ASan.IntptrTy), IntptrPtrTy(PointerType::get(IntptrTy, 0)), - Mapping(ASan.Mapping), + : F(F), ASan(ASan), DIB(*F.getParent(), /*AllowUnresolved*/ false), + C(ASan.C), IntptrTy(ASan.IntptrTy), + IntptrPtrTy(PointerType::get(IntptrTy, 0)), Mapping(ASan.Mapping), StackAlignment(1 << Mapping.Scale) {} bool runOnFunction() { diff --git a/llvm/lib/Transforms/Instrumentation/SanitizerCoverage.cpp b/llvm/lib/Transforms/Instrumentation/SanitizerCoverage.cpp index 34f5ae96b13..7d86e1c8af0 100644 --- a/llvm/lib/Transforms/Instrumentation/SanitizerCoverage.cpp +++ b/llvm/lib/Transforms/Instrumentation/SanitizerCoverage.cpp @@ -275,7 +275,7 @@ void SanitizerCoverageModule::InjectCoverageAtBlock(Function &F, Load->setAtomic(Monotonic); Load->setAlignment(1); Load->setMetadata(F.getParent()->getMDKindID("nosanitize"), - MDNode::get(*C, ArrayRef<llvm::Value *>())); + MDNode::get(*C, None)); Value *Cmp = IRB.CreateICmpEQ(Constant::getNullValue(Int8Ty), Load); Instruction *Ins = SplitBlockAndInsertIfThen( Cmp, IP, false, MDBuilder(*C).createBranchWeights(1, 100000)); diff --git a/llvm/lib/Transforms/ObjCARC/ObjCARCOpts.cpp b/llvm/lib/Transforms/ObjCARC/ObjCARCOpts.cpp index 95c66742d95..76932e6b600 100644 --- a/llvm/lib/Transforms/ObjCARC/ObjCARCOpts.cpp +++ b/llvm/lib/Transforms/ObjCARC/ObjCARCOpts.cpp @@ -880,11 +880,9 @@ static void AppendMDNodeToInstForPtr(unsigned NodeId, Sequence OldSeq, Sequence NewSeq) { MDNode *Node = nullptr; - Value *tmp[3] = {PtrSourceMDNodeID, - SequenceToMDString(Inst->getContext(), - OldSeq), - SequenceToMDString(Inst->getContext(), - NewSeq)}; + Metadata *tmp[3] = {PtrSourceMDNodeID, + SequenceToMDString(Inst->getContext(), OldSeq), + SequenceToMDString(Inst->getContext(), NewSeq)}; Node = MDNode::get(Inst->getContext(), tmp); Inst->setMetadata(NodeId, Node); diff --git a/llvm/lib/Transforms/Scalar/LoopUnrollPass.cpp b/llvm/lib/Transforms/Scalar/LoopUnrollPass.cpp index f60d9903a55..7844008eb34 100644 --- a/llvm/lib/Transforms/Scalar/LoopUnrollPass.cpp +++ b/llvm/lib/Transforms/Scalar/LoopUnrollPass.cpp @@ -272,7 +272,8 @@ static unsigned UnrollCountPragmaValue(const Loop *L) { if (MD) { assert(MD->getNumOperands() == 2 && "Unroll count hint metadata should have two operands."); - unsigned Count = cast<ConstantInt>(MD->getOperand(1))->getZExtValue(); + unsigned Count = + mdconst::extract<ConstantInt>(MD->getOperand(1))->getZExtValue(); assert(Count >= 1 && "Unroll count must be positive."); return Count; } @@ -288,9 +289,9 @@ static void SetLoopAlreadyUnrolled(Loop *L) { if (!LoopID) return; // First remove any existing loop unrolling metadata. - SmallVector<Value *, 4> Vals; + SmallVector<Metadata *, 4> MDs; // Reserve first location for self reference to the LoopID metadata node. - Vals.push_back(nullptr); + MDs.push_back(nullptr); for (unsigned i = 1, ie = LoopID->getNumOperands(); i < ie; ++i) { bool IsUnrollMetadata = false; MDNode *MD = dyn_cast<MDNode>(LoopID->getOperand(i)); @@ -298,17 +299,18 @@ static void SetLoopAlreadyUnrolled(Loop *L) { const MDString *S = dyn_cast<MDString>(MD->getOperand(0)); IsUnrollMetadata = S && S->getString().startswith("llvm.loop.unroll."); } - if (!IsUnrollMetadata) Vals.push_back(LoopID->getOperand(i)); + if (!IsUnrollMetadata) + MDs.push_back(LoopID->getOperand(i)); } // Add unroll(disable) metadata to disable future unrolling. LLVMContext &Context = L->getHeader()->getContext(); - SmallVector<Value *, 1> DisableOperands; + SmallVector<Metadata *, 1> DisableOperands; DisableOperands.push_back(MDString::get(Context, "llvm.loop.unroll.disable")); MDNode *DisableNode = MDNode::get(Context, DisableOperands); - Vals.push_back(DisableNode); + MDs.push_back(DisableNode); - MDNode *NewLoopID = MDNode::get(Context, Vals); + MDNode *NewLoopID = MDNode::get(Context, MDs); // Set operand 0 to refer to the loop id itself. NewLoopID->replaceOperandWith(0, NewLoopID); L->setLoopID(NewLoopID); diff --git a/llvm/lib/Transforms/Scalar/SROA.cpp b/llvm/lib/Transforms/Scalar/SROA.cpp index f9ebd75ec71..45b66674bc9 100644 --- a/llvm/lib/Transforms/Scalar/SROA.cpp +++ b/llvm/lib/Transforms/Scalar/SROA.cpp @@ -807,12 +807,14 @@ public: void run(const SmallVectorImpl<Instruction*> &Insts) { // Retain the debug information attached to the alloca for use when // rewriting loads and stores. - if (MDNode *DebugNode = MDNode::getIfExists(AI.getContext(), &AI)) { - for (User *U : DebugNode->users()) - if (DbgDeclareInst *DDI = dyn_cast<DbgDeclareInst>(U)) - DDIs.push_back(DDI); - else if (DbgValueInst *DVI = dyn_cast<DbgValueInst>(U)) - DVIs.push_back(DVI); + if (auto *L = LocalAsMetadata::getIfExists(&AI)) { + if (auto *DebugNode = MetadataAsValue::getIfExists(AI.getContext(), L)) { + for (User *U : DebugNode->users()) + if (DbgDeclareInst *DDI = dyn_cast<DbgDeclareInst>(U)) + DDIs.push_back(DDI); + else if (DbgValueInst *DVI = dyn_cast<DbgValueInst>(U)) + DVIs.push_back(DVI); + } } LoadAndStorePromoter::run(Insts); @@ -3614,7 +3616,7 @@ bool SROA::promoteAllocas(Function &F) { DEBUG(dbgs() << "Promoting allocas with SSAUpdater...\n"); SSAUpdater SSA; - DIBuilder DIB(*F.getParent()); + DIBuilder DIB(*F.getParent(), /*AllowUnresolved*/ false); SmallVector<Instruction *, 64> Insts; // We need a worklist to walk the uses of each alloca. diff --git a/llvm/lib/Transforms/Scalar/ScalarReplAggregates.cpp b/llvm/lib/Transforms/Scalar/ScalarReplAggregates.cpp index f7fa9171862..20644611d2e 100644 --- a/llvm/lib/Transforms/Scalar/ScalarReplAggregates.cpp +++ b/llvm/lib/Transforms/Scalar/ScalarReplAggregates.cpp @@ -1068,12 +1068,14 @@ public: void run(AllocaInst *AI, const SmallVectorImpl<Instruction*> &Insts) { // Remember which alloca we're promoting (for isInstInList). this->AI = AI; - if (MDNode *DebugNode = MDNode::getIfExists(AI->getContext(), AI)) { - for (User *U : DebugNode->users()) - if (DbgDeclareInst *DDI = dyn_cast<DbgDeclareInst>(U)) - DDIs.push_back(DDI); - else if (DbgValueInst *DVI = dyn_cast<DbgValueInst>(U)) - DVIs.push_back(DVI); + if (auto *L = LocalAsMetadata::getIfExists(AI)) { + if (auto *DebugNode = MetadataAsValue::getIfExists(AI->getContext(), L)) { + for (User *U : DebugNode->users()) + if (DbgDeclareInst *DDI = dyn_cast<DbgDeclareInst>(U)) + DDIs.push_back(DDI); + else if (DbgValueInst *DVI = dyn_cast<DbgValueInst>(U)) + DVIs.push_back(DVI); + } } LoadAndStorePromoter::run(Insts); @@ -1420,7 +1422,7 @@ bool SROA::performPromotion(Function &F) { AssumptionTracker *AT = &getAnalysis<AssumptionTracker>(); BasicBlock &BB = F.getEntryBlock(); // Get the entry node for the function - DIBuilder DIB(*F.getParent()); + DIBuilder DIB(*F.getParent(), /*AllowUnresolved*/ false); bool Changed = false; SmallVector<Instruction*, 64> Insts; while (1) { diff --git a/llvm/lib/Transforms/Utils/AddDiscriminators.cpp b/llvm/lib/Transforms/Utils/AddDiscriminators.cpp index f8e5af5c768..820544bcebf 100644 --- a/llvm/lib/Transforms/Utils/AddDiscriminators.cpp +++ b/llvm/lib/Transforms/Utils/AddDiscriminators.cpp @@ -167,7 +167,7 @@ bool AddDiscriminators::runOnFunction(Function &F) { bool Changed = false; Module *M = F.getParent(); LLVMContext &Ctx = M->getContext(); - DIBuilder Builder(*M); + DIBuilder Builder(*M, /*AllowUnresolved*/ false); // Traverse all the blocks looking for instructions in different // blocks that are at the same file:line location. diff --git a/llvm/lib/Transforms/Utils/CloneFunction.cpp b/llvm/lib/Transforms/Utils/CloneFunction.cpp index 6b86378bc2e..affe0194d9a 100644 --- a/llvm/lib/Transforms/Utils/CloneFunction.cpp +++ b/llvm/lib/Transforms/Utils/CloneFunction.cpp @@ -164,8 +164,8 @@ static MDNode* FindSubprogram(const Function *F, DebugInfoFinder &Finder) { // Add an operand to an existing MDNode. The new operand will be added at the // back of the operand list. -static void AddOperand(DICompileUnit CU, DIArray SPs, Value *NewSP) { - SmallVector<Value *, 16> NewSPs; +static void AddOperand(DICompileUnit CU, DIArray SPs, Metadata *NewSP) { + SmallVector<Metadata *, 16> NewSPs; NewSPs.reserve(SPs->getNumOperands() + 1); for (unsigned I = 0, E = SPs->getNumOperands(); I != E; ++I) NewSPs.push_back(SPs->getOperand(I)); diff --git a/llvm/lib/Transforms/Utils/InlineFunction.cpp b/llvm/lib/Transforms/Utils/InlineFunction.cpp index 2d0b7dcc71c..147237edbcb 100644 --- a/llvm/lib/Transforms/Utils/InlineFunction.cpp +++ b/llvm/lib/Transforms/Utils/InlineFunction.cpp @@ -308,7 +308,7 @@ static void CloneAliasScopeMetadata(CallSite CS, ValueToValueMapTy &VMap) { // Walk the existing metadata, adding the complete (perhaps cyclic) chain to // the set. - SmallVector<const Value *, 16> Queue(MD.begin(), MD.end()); + SmallVector<const Metadata *, 16> Queue(MD.begin(), MD.end()); while (!Queue.empty()) { const MDNode *M = cast<MDNode>(Queue.pop_back_val()); for (unsigned i = 0, ie = M->getNumOperands(); i != ie; ++i) @@ -320,12 +320,12 @@ static void CloneAliasScopeMetadata(CallSite CS, ValueToValueMapTy &VMap) { // Now we have a complete set of all metadata in the chains used to specify // the noalias scopes and the lists of those scopes. SmallVector<MDNode *, 16> DummyNodes; - DenseMap<const MDNode *, TrackingVH<MDNode> > MDMap; + DenseMap<const MDNode *, TrackingMDNodeRef> MDMap; for (SetVector<const MDNode *>::iterator I = MD.begin(), IE = MD.end(); I != IE; ++I) { MDNode *Dummy = MDNode::getTemporary(CalledFunc->getContext(), None); DummyNodes.push_back(Dummy); - MDMap[*I] = Dummy; + MDMap[*I].reset(Dummy); } // Create new metadata nodes to replace the dummy nodes, replacing old @@ -333,17 +333,17 @@ static void CloneAliasScopeMetadata(CallSite CS, ValueToValueMapTy &VMap) { // node. for (SetVector<const MDNode *>::iterator I = MD.begin(), IE = MD.end(); I != IE; ++I) { - SmallVector<Value *, 4> NewOps; + SmallVector<Metadata *, 4> NewOps; for (unsigned i = 0, ie = (*I)->getNumOperands(); i != ie; ++i) { - const Value *V = (*I)->getOperand(i); + const Metadata *V = (*I)->getOperand(i); if (const MDNode *M = dyn_cast<MDNode>(V)) NewOps.push_back(MDMap[M]); else - NewOps.push_back(const_cast<Value *>(V)); + NewOps.push_back(const_cast<Metadata *>(V)); } - MDNode *NewM = MDNode::get(CalledFunc->getContext(), NewOps), - *TempM = MDMap[*I]; + MDNode *NewM = MDNode::get(CalledFunc->getContext(), NewOps); + MDNodeFwdDecl *TempM = cast<MDNodeFwdDecl>(MDMap[*I]); TempM->replaceAllUsesWith(NewM); } @@ -516,7 +516,7 @@ static void AddAliasScopeMetadata(CallSite CS, ValueToValueMapTy &VMap, // need to go through several PHIs to see it, and thus could be // repeated in the Objects list. SmallPtrSet<const Value *, 4> ObjSet; - SmallVector<Value *, 4> Scopes, NoAliases; + SmallVector<Metadata *, 4> Scopes, NoAliases; SmallSetVector<const Argument *, 4> NAPtrArgs; for (unsigned i = 0, ie = PtrArgs.size(); i != ie; ++i) { @@ -869,8 +869,9 @@ static void fixupLineNumbers(Function *Fn, Function::iterator FI, if (DbgValueInst *DVI = dyn_cast<DbgValueInst>(BI)) { LLVMContext &Ctx = BI->getContext(); MDNode *InlinedAt = BI->getDebugLoc().getInlinedAt(Ctx); - DVI->setOperand(2, createInlinedVariable(DVI->getVariable(), - InlinedAt, Ctx)); + DVI->setOperand(2, MetadataAsValue::get( + Ctx, createInlinedVariable(DVI->getVariable(), + InlinedAt, Ctx))); } } } diff --git a/llvm/lib/Transforms/Utils/Local.cpp b/llvm/lib/Transforms/Utils/Local.cpp index f8f62a9273d..5decfe50626 100644 --- a/llvm/lib/Transforms/Utils/Local.cpp +++ b/llvm/lib/Transforms/Utils/Local.cpp @@ -137,7 +137,8 @@ bool llvm::ConstantFoldTerminator(BasicBlock *BB, bool DeleteDeadConditions, SmallVector<uint32_t, 8> Weights; for (unsigned MD_i = 1, MD_e = MD->getNumOperands(); MD_i < MD_e; ++MD_i) { - ConstantInt* CI = dyn_cast<ConstantInt>(MD->getOperand(MD_i)); + ConstantInt *CI = + mdconst::dyn_extract<ConstantInt>(MD->getOperand(MD_i)); assert(CI); Weights.push_back(CI->getValue().getZExtValue()); } @@ -208,8 +209,10 @@ bool llvm::ConstantFoldTerminator(BasicBlock *BB, bool DeleteDeadConditions, SI->getDefaultDest()); MDNode *MD = SI->getMetadata(LLVMContext::MD_prof); if (MD && MD->getNumOperands() == 3) { - ConstantInt *SICase = dyn_cast<ConstantInt>(MD->getOperand(2)); - ConstantInt *SIDef = dyn_cast<ConstantInt>(MD->getOperand(1)); + ConstantInt *SICase = + mdconst::dyn_extract<ConstantInt>(MD->getOperand(2)); + ConstantInt *SIDef = + mdconst::dyn_extract<ConstantInt>(MD->getOperand(1)); assert(SICase && SIDef); // The TrueWeight should be the weight for the single case of SI. NewBr->setMetadata(LLVMContext::MD_prof, @@ -1048,7 +1051,7 @@ static bool isArray(AllocaInst *AI) { /// LowerDbgDeclare - Lowers llvm.dbg.declare intrinsics into appropriate set /// of llvm.dbg.value intrinsics. bool llvm::LowerDbgDeclare(Function &F) { - DIBuilder DIB(*F.getParent()); + DIBuilder DIB(*F.getParent(), /*AllowUnresolved*/ false); SmallVector<DbgDeclareInst *, 4> Dbgs; for (auto &FI : F) for (BasicBlock::iterator BI : FI) @@ -1091,10 +1094,11 @@ bool llvm::LowerDbgDeclare(Function &F) { /// FindAllocaDbgDeclare - Finds the llvm.dbg.declare intrinsic describing the /// alloca 'V', if any. DbgDeclareInst *llvm::FindAllocaDbgDeclare(Value *V) { - if (MDNode *DebugNode = MDNode::getIfExists(V->getContext(), V)) - for (User *U : DebugNode->users()) - if (DbgDeclareInst *DDI = dyn_cast<DbgDeclareInst>(U)) - return DDI; + if (auto *L = LocalAsMetadata::getIfExists(V)) + if (auto *MDV = MetadataAsValue::getIfExists(V->getContext(), L)) + for (User *U : MDV->users()) + if (DbgDeclareInst *DDI = dyn_cast<DbgDeclareInst>(U)) + return DDI; return nullptr; } diff --git a/llvm/lib/Transforms/Utils/LoopUnrollRuntime.cpp b/llvm/lib/Transforms/Utils/LoopUnrollRuntime.cpp index 3d9133684db..f12cd61d463 100644 --- a/llvm/lib/Transforms/Utils/LoopUnrollRuntime.cpp +++ b/llvm/lib/Transforms/Utils/LoopUnrollRuntime.cpp @@ -217,9 +217,9 @@ static void CloneLoopBlocks(Loop *L, Value *NewIter, const bool UnrollProlog, } if (NewLoop) { // Add unroll disable metadata to disable future unrolling for this loop. - SmallVector<Value *, 4> Vals; + SmallVector<Metadata *, 4> MDs; // Reserve first location for self reference to the LoopID metadata node. - Vals.push_back(nullptr); + MDs.push_back(nullptr); MDNode *LoopID = NewLoop->getLoopID(); if (LoopID) { // First remove any existing loop unrolling metadata. @@ -230,17 +230,18 @@ static void CloneLoopBlocks(Loop *L, Value *NewIter, const bool UnrollProlog, const MDString *S = dyn_cast<MDString>(MD->getOperand(0)); IsUnrollMetadata = S && S->getString().startswith("llvm.loop.unroll."); } - if (!IsUnrollMetadata) Vals.push_back(LoopID->getOperand(i)); + if (!IsUnrollMetadata) + MDs.push_back(LoopID->getOperand(i)); } } LLVMContext &Context = NewLoop->getHeader()->getContext(); - SmallVector<Value *, 1> DisableOperands; + SmallVector<Metadata *, 1> DisableOperands; DisableOperands.push_back(MDString::get(Context, "llvm.loop.unroll.disable")); MDNode *DisableNode = MDNode::get(Context, DisableOperands); - Vals.push_back(DisableNode); + MDs.push_back(DisableNode); - MDNode *NewLoopID = MDNode::get(Context, Vals); + MDNode *NewLoopID = MDNode::get(Context, MDs); // Set operand 0 to refer to the loop id itself. NewLoopID->replaceOperandWith(0, NewLoopID); NewLoop->setLoopID(NewLoopID); diff --git a/llvm/lib/Transforms/Utils/PromoteMemoryToRegister.cpp b/llvm/lib/Transforms/Utils/PromoteMemoryToRegister.cpp index 1fd70711f53..1b1da6963cf 100644 --- a/llvm/lib/Transforms/Utils/PromoteMemoryToRegister.cpp +++ b/llvm/lib/Transforms/Utils/PromoteMemoryToRegister.cpp @@ -284,7 +284,8 @@ public: PromoteMem2Reg(ArrayRef<AllocaInst *> Allocas, DominatorTree &DT, AliasSetTracker *AST, AssumptionTracker *AT) : Allocas(Allocas.begin(), Allocas.end()), DT(DT), - DIB(*DT.getRoot()->getParent()->getParent()), AST(AST), AT(AT) {} + DIB(*DT.getRoot()->getParent()->getParent(), /*AllowUnresolved*/ false), + AST(AST), AT(AT) {} void run(); @@ -415,7 +416,8 @@ static bool rewriteSingleStoreAlloca(AllocaInst *AI, AllocaInfo &Info, // Record debuginfo for the store and remove the declaration's // debuginfo. if (DbgDeclareInst *DDI = Info.DbgDeclare) { - DIBuilder DIB(*AI->getParent()->getParent()->getParent()); + DIBuilder DIB(*AI->getParent()->getParent()->getParent(), + /*AllowUnresolved*/ false); ConvertDebugDeclareToDebugValue(DDI, Info.OnlyStore, DIB); DDI->eraseFromParent(); LBI.deleteValue(DDI); @@ -498,7 +500,8 @@ static void promoteSingleBlockAlloca(AllocaInst *AI, const AllocaInfo &Info, StoreInst *SI = cast<StoreInst>(AI->user_back()); // Record debuginfo for the store before removing it. if (DbgDeclareInst *DDI = Info.DbgDeclare) { - DIBuilder DIB(*AI->getParent()->getParent()->getParent()); + DIBuilder DIB(*AI->getParent()->getParent()->getParent(), + /*AllowUnresolved*/ false); ConvertDebugDeclareToDebugValue(DDI, SI, DIB); } SI->eraseFromParent(); diff --git a/llvm/lib/Transforms/Utils/SimplifyCFG.cpp b/llvm/lib/Transforms/Utils/SimplifyCFG.cpp index daa576cfbdc..5f55b89b0f9 100644 --- a/llvm/lib/Transforms/Utils/SimplifyCFG.cpp +++ b/llvm/lib/Transforms/Utils/SimplifyCFG.cpp @@ -713,8 +713,7 @@ SimplifyEqualityComparisonWithOnlyPredecessor(TerminatorInst *TI, if (HasWeight) for (unsigned MD_i = 1, MD_e = MD->getNumOperands(); MD_i < MD_e; ++MD_i) { - ConstantInt* CI = dyn_cast<ConstantInt>(MD->getOperand(MD_i)); - assert(CI); + ConstantInt *CI = mdconst::extract<ConstantInt>(MD->getOperand(MD_i)); Weights.push_back(CI->getValue().getZExtValue()); } for (SwitchInst::CaseIt i = SI->case_end(), e = SI->case_begin(); i != e;) { @@ -819,7 +818,7 @@ static void GetBranchWeights(TerminatorInst *TI, MDNode *MD = TI->getMetadata(LLVMContext::MD_prof); assert(MD); for (unsigned i = 1, e = MD->getNumOperands(); i < e; ++i) { - ConstantInt *CI = cast<ConstantInt>(MD->getOperand(i)); + ConstantInt *CI = mdconst::extract<ConstantInt>(MD->getOperand(i)); Weights.push_back(CI->getValue().getZExtValue()); } @@ -2037,8 +2036,10 @@ static bool ExtractBranchMetadata(BranchInst *BI, "Looking for probabilities on unconditional branch?"); MDNode *ProfileData = BI->getMetadata(LLVMContext::MD_prof); if (!ProfileData || ProfileData->getNumOperands() != 3) return false; - ConstantInt *CITrue = dyn_cast<ConstantInt>(ProfileData->getOperand(1)); - ConstantInt *CIFalse = dyn_cast<ConstantInt>(ProfileData->getOperand(2)); + ConstantInt *CITrue = + mdconst::dyn_extract<ConstantInt>(ProfileData->getOperand(1)); + ConstantInt *CIFalse = + mdconst::dyn_extract<ConstantInt>(ProfileData->getOperand(2)); if (!CITrue || !CIFalse) return false; ProbTrue = CITrue->getValue().getZExtValue(); ProbFalse = CIFalse->getValue().getZExtValue(); diff --git a/llvm/lib/Transforms/Utils/ValueMapper.cpp b/llvm/lib/Transforms/Utils/ValueMapper.cpp index a2f69d1f10b..d735e279d08 100644 --- a/llvm/lib/Transforms/Utils/ValueMapper.cpp +++ b/llvm/lib/Transforms/Utils/ValueMapper.cpp @@ -40,7 +40,7 @@ Value *llvm::MapValue(const Value *V, ValueToValueMapTy &VM, RemapFlags Flags, // Global values do not need to be seeded into the VM if they // are using the identity mapping. - if (isa<GlobalValue>(V) || isa<MDString>(V)) + if (isa<GlobalValue>(V)) return VM[V] = const_cast<Value*>(V); if (const InlineAsm *IA = dyn_cast<InlineAsm>(V)) { @@ -56,57 +56,24 @@ Value *llvm::MapValue(const Value *V, ValueToValueMapTy &VM, RemapFlags Flags, return VM[V] = const_cast<Value*>(V); } - - if (const MDNode *MD = dyn_cast<MDNode>(V)) { + if (const auto *MDV = dyn_cast<MetadataAsValue>(V)) { + const Metadata *MD = MDV->getMetadata(); // If this is a module-level metadata and we know that nothing at the module // level is changing, then use an identity mapping. - if (!MD->isFunctionLocal() && (Flags & RF_NoModuleLevelChanges)) - return VM[V] = const_cast<Value*>(V); - - // Create a dummy node in case we have a metadata cycle. - MDNode *Dummy = MDNode::getTemporary(V->getContext(), None); - VM[V] = Dummy; - - // Check all operands to see if any need to be remapped. - for (unsigned i = 0, e = MD->getNumOperands(); i != e; ++i) { - Value *OP = MD->getOperand(i); - if (!OP) continue; - Value *Mapped_OP = MapValue(OP, VM, Flags, TypeMapper, Materializer); - // Use identity map if Mapped_Op is null and we can ignore missing - // entries. - if (Mapped_OP == OP || - (Mapped_OP == nullptr && (Flags & RF_IgnoreMissingEntries))) - continue; - - // Ok, at least one operand needs remapping. - SmallVector<Value*, 4> Elts; - Elts.reserve(MD->getNumOperands()); - for (i = 0; i != e; ++i) { - Value *Op = MD->getOperand(i); - if (!Op) - Elts.push_back(nullptr); - else { - Value *Mapped_Op = MapValue(Op, VM, Flags, TypeMapper, Materializer); - // Use identity map if Mapped_Op is null and we can ignore missing - // entries. - if (Mapped_Op == nullptr && (Flags & RF_IgnoreMissingEntries)) - Mapped_Op = Op; - Elts.push_back(Mapped_Op); - } - } - MDNode *NewMD = MDNode::get(V->getContext(), Elts); - Dummy->replaceAllUsesWith(NewMD); - VM[V] = NewMD; - MDNode::deleteTemporary(Dummy); - return NewMD; - } + if (!isa<LocalAsMetadata>(MD) && (Flags & RF_NoModuleLevelChanges)) + return VM[V] = const_cast<Value *>(V); - VM[V] = const_cast<Value*>(V); - MDNode::deleteTemporary(Dummy); + auto *MappedMD = MapValue(MD, VM, Flags, TypeMapper, Materializer); + if (MD == MappedMD || (!MappedMD && (Flags & RF_IgnoreMissingEntries))) + return VM[V] = const_cast<Value *>(V); - // No operands needed remapping. Use an identity mapping. - return const_cast<Value*>(V); + // FIXME: This assert crashes during bootstrap, but I think it should be + // correct. For now, just match behaviour from before the metadata/value + // split. + // + // assert(MappedMD && "Referenced metadata value not in value map"); + return VM[V] = MetadataAsValue::get(V->getContext(), MappedMD); } // Okay, this either must be a constant (which may or may not be mappable) or @@ -177,6 +144,120 @@ Value *llvm::MapValue(const Value *V, ValueToValueMapTy &VM, RemapFlags Flags, return VM[V] = ConstantPointerNull::get(cast<PointerType>(NewTy)); } +static Metadata *map(ValueToValueMapTy &VM, const Metadata *Key, + Metadata *Val) { + VM.MD()[Key].reset(Val); + return Val; +} + +static Metadata *mapToSelf(ValueToValueMapTy &VM, const Metadata *MD) { + return map(VM, MD, const_cast<Metadata *>(MD)); +} + +static Metadata *MapValueImpl(const Metadata *MD, ValueToValueMapTy &VM, + RemapFlags Flags, + ValueMapTypeRemapper *TypeMapper, + ValueMaterializer *Materializer) { + // If the value already exists in the map, use it. + if (Metadata *NewMD = VM.MD().lookup(MD).get()) + return NewMD; + + if (isa<MDString>(MD)) + return mapToSelf(VM, MD); + + if (isa<ConstantAsMetadata>(MD)) + if ((Flags & RF_NoModuleLevelChanges)) + return mapToSelf(VM, MD); + + if (const auto *VMD = dyn_cast<ValueAsMetadata>(MD)) { + Value *MappedV = + MapValue(VMD->getValue(), VM, Flags, TypeMapper, Materializer); + if (VMD->getValue() == MappedV || + (!MappedV && (Flags & RF_IgnoreMissingEntries))) + return mapToSelf(VM, MD); + + // FIXME: This assert crashes during bootstrap, but I think it should be + // correct. For now, just match behaviour from before the metadata/value + // split. + // + // assert(MappedV && "Referenced metadata not in value map!"); + if (MappedV) + return map(VM, MD, ValueAsMetadata::get(MappedV)); + return nullptr; + } + + const MDNode *Node = cast<MDNode>(MD); + assert(Node->isResolved() && "Unexpected unresolved node"); + + auto getMappedOp = [&](Metadata *Op) -> Metadata *{ + if (!Op) + return nullptr; + if (Metadata *MappedOp = + MapValueImpl(Op, VM, Flags, TypeMapper, Materializer)) + return MappedOp; + // Use identity map if MappedOp is null and we can ignore missing entries. + if (Flags & RF_IgnoreMissingEntries) + return Op; + + // FIXME: This assert crashes during bootstrap, but I think it should be + // correct. For now, just match behaviour from before the metadata/value + // split. + // + // llvm_unreachable("Referenced metadata not in value map!"); + return nullptr; + }; + + // If this is a module-level metadata and we know that nothing at the + // module level is changing, then use an identity mapping. + if (Flags & RF_NoModuleLevelChanges) + return mapToSelf(VM, MD); + + // Create a dummy node in case we have a metadata cycle. + MDNodeFwdDecl *Dummy = MDNode::getTemporary(Node->getContext(), None); + map(VM, Node, Dummy); + + // Check all operands to see if any need to be remapped. + for (unsigned I = 0, E = Node->getNumOperands(); I != E; ++I) { + Metadata *Op = Node->getOperand(I); + Metadata *MappedOp = getMappedOp(Op); + if (Op == MappedOp) + continue; + + // Ok, at least one operand needs remapping. + SmallVector<Metadata *, 4> Elts; + Elts.reserve(Node->getNumOperands()); + for (I = 0; I != E; ++I) + Elts.push_back(getMappedOp(Node->getOperand(I))); + + MDNode *NewMD = MDNode::get(Node->getContext(), Elts); + Dummy->replaceAllUsesWith(NewMD); + MDNode::deleteTemporary(Dummy); + return map(VM, Node, NewMD); + } + + // No operands needed remapping. Use an identity mapping. + mapToSelf(VM, MD); + MDNode::deleteTemporary(Dummy); + return const_cast<Metadata *>(MD); +} + +Metadata *llvm::MapValue(const Metadata *MD, ValueToValueMapTy &VM, + RemapFlags Flags, ValueMapTypeRemapper *TypeMapper, + ValueMaterializer *Materializer) { + Metadata *NewMD = MapValueImpl(MD, VM, Flags, TypeMapper, Materializer); + if (NewMD && NewMD != MD) + if (auto *G = dyn_cast<GenericMDNode>(NewMD)) + G->resolveCycles(); + return NewMD; +} + +MDNode *llvm::MapValue(const MDNode *MD, ValueToValueMapTy &VM, + RemapFlags Flags, ValueMapTypeRemapper *TypeMapper, + ValueMaterializer *Materializer) { + return cast<MDNode>(MapValue(static_cast<const Metadata *>(MD), VM, Flags, + TypeMapper, Materializer)); +} + /// RemapInstruction - Convert the instruction operands from referencing the /// current values into those specified by VMap. /// diff --git a/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp b/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp index 83e19f406de..3c543530a43 100644 --- a/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp +++ b/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp @@ -1097,7 +1097,7 @@ private: for (unsigned i = 1, ie = LoopID->getNumOperands(); i < ie; ++i) { const MDString *S = nullptr; - SmallVector<Value*, 4> Args; + SmallVector<Metadata *, 4> Args; // The expected hint is either a MDString or a MDNode with the first // operand a MDString. @@ -1123,12 +1123,12 @@ private: } /// Checks string hint with one operand and set value if valid. - void setHint(StringRef Name, Value *Arg) { + void setHint(StringRef Name, Metadata *Arg) { if (!Name.startswith(Prefix())) return; Name = Name.substr(Prefix().size(), StringRef::npos); - const ConstantInt *C = dyn_cast<ConstantInt>(Arg); + const ConstantInt *C = mdconst::dyn_extract<ConstantInt>(Arg); if (!C) return; unsigned Val = C->getZExtValue(); @@ -1147,9 +1147,10 @@ private: /// Create a new hint from name / value pair. MDNode *createHintMetadata(StringRef Name, unsigned V) const { LLVMContext &Context = TheLoop->getHeader()->getContext(); - Value *Vals[] = {MDString::get(Context, Name), - ConstantInt::get(Type::getInt32Ty(Context), V)}; - return MDNode::get(Context, Vals); + Metadata *MDs[] = {MDString::get(Context, Name), + ConstantAsMetadata::get( + ConstantInt::get(Type::getInt32Ty(Context), V))}; + return MDNode::get(Context, MDs); } /// Matches metadata with hint name. @@ -1170,7 +1171,7 @@ private: return; // Reserve the first element to LoopID (see below). - SmallVector<Value*, 4> Vals(1); + SmallVector<Metadata *, 4> MDs(1); // If the loop already has metadata, then ignore the existing operands. MDNode *LoopID = TheLoop->getLoopID(); if (LoopID) { @@ -1178,18 +1179,17 @@ private: MDNode *Node = cast<MDNode>(LoopID->getOperand(i)); // If node in update list, ignore old value. if (!matchesHintMetadataName(Node, HintTypes)) - Vals.push_back(Node); + MDs.push_back(Node); } } // Now, add the missing hints. for (auto H : HintTypes) - Vals.push_back( - createHintMetadata(Twine(Prefix(), H.Name).str(), H.Value)); + MDs.push_back(createHintMetadata(Twine(Prefix(), H.Name).str(), H.Value)); // Replace current metadata node with new one. LLVMContext &Context = TheLoop->getHeader()->getContext(); - MDNode *NewLoopID = MDNode::get(Context, Vals); + MDNode *NewLoopID = MDNode::get(Context, MDs); // Set operand 0 to refer to the loop id itself. NewLoopID->replaceOperandWith(0, NewLoopID); |