[CodeView] Don't output S_UDTs for nested typedefs.

S_UDT records are basically the "bridge" between the debugger's
expression evaluator and the type information. If you type
(Foo*)nullptr into the watch window, the debugger looks for an
S_UDT record named Foo. If it can find one, it displays your type.
Otherwise you get an error.

We have always understood this to mean that if you have code like
this:

  struct A {
    int X;
  };

  struct B {
    typedef A AT;
    AT Member;
  };

that you will get 3 S_UDT records. "A", "B", and "B::AT". Because
if you were to type (B::AT*)nullptr into the debugger, it would
need to find an S_UDT record named "B::AT".

But "B::AT" is actually the S_UDT record that would be generated
if B were a namespace, not a struct. So the debugger needs to be
able to distinguish this case. So what it does is:

  1. Look for an S_UDT named "B::AT". If it finds one, it knows
     that AT is in a namespace.
  2. If it doesn't find one, split at the scope resolution operator,
     and look for an S_UDT named B. If it finds one, look up the type
     for B, and then look for AT as one of its members.

With this algorithm, S_UDT records for nested typedefs are not just
unnecessary, but actually wrong!

The results of implementing this in clang are dramatic. It cuts
our /DEBUG:FASTLINK PDB sizes by more than 50%, and we go from
being ~20% larger than MSVC PDBs on average, to ~40% smaller.

It also slightly speeds up link time. We get about 10% faster
links than without this patch.

Differential Revision: https://reviews.llvm.org/D37410

llvm-svn: 312583

1 files changed, 15 insertions, 0 deletions

diff --git a/llvm/lib/CodeGen/AsmPrinter/CodeViewDebug.cpp b/llvm/lib/CodeGen/AsmPrinter/CodeViewDebug.cpp
index e4084856272..42a0d7991d2 100644
--- a/llvm/lib/CodeGen/AsmPrinter/CodeViewDebug.cpp
+++ b/llvm/lib/CodeGen/AsmPrinter/CodeViewDebug.cpp
@@ -1139,6 +1139,21 @@ void CodeViewDebug::beginFunctionImpl(const MachineFunction *MF) {
 }
 
 static bool shouldEmitUdt(const DIType *T) {
+  if (!T)
+    return false;
+
+  // MSVC does not emit UDTs for typedefs that are scoped to classes.
+  if (T->getTag() == dwarf::DW_TAG_typedef) {
+    if (DIScope *Scope = T->getScope().resolve()) {
+      switch (Scope->getTag()) {
+      case dwarf::DW_TAG_structure_type:
+      case dwarf::DW_TAG_class_type:
+      case dwarf::DW_TAG_union_type:
+        return false;
+      }
+    }
+  }
+
   while (true) {
     if (!T || T->isForwardDecl())
       return false;