From 030d7d6daa1e93685660c6c03667126e72518101 Mon Sep 17 00:00:00 2001
From: "Duncan P. N. Exon Smith" <dexonsmith@apple.com>
Date: Mon, 20 Mar 2017 17:58:26 +0000
Subject: Reapply "Modules: Cache PCMs in memory and avoid a use-after-free"

This reverts commit r298185, effectively reapplying r298165, after fixing the
new unit tests (PR32338).  The memory buffer generator doesn't null-terminate
the MemoryBuffer it creates; this version of the commit informs getMemBuffer
about that to avoid the assert.

Original commit message follows:

----

Clang's internal build system for implicit modules uses lock files to
ensure that after a process writes a PCM it will read the same one back
in (without contention from other -cc1 commands).  Since PCMs are read
from disk repeatedly while invalidating, building, and importing, the
lock is not released quickly.  Furthermore, the LockFileManager is not
robust in every environment.  Other -cc1 commands can stall until
timeout (after about eight minutes).

This commit changes the lock file from being necessary for correctness
to a (possibly dubious) performance hack.  The remaining benefit is to
reduce duplicate work in competing -cc1 commands which depend on the
same module.  Follow-up commits will change the internal build system to
continue after a timeout, and reduce the timeout.  Perhaps we should
reconsider blocking at all.

This also fixes a use-after-free, when one part of a compilation
validates a PCM and starts using it, and another tries to swap out the
PCM for something new.

The PCMCache is a new type called MemoryBufferCache, which saves memory
buffers based on their filename.  Its ownership is shared by the
CompilerInstance and ModuleManager.

- The ModuleManager stores PCMs there that it loads from disk, never
touching the disk if the cache is hot.

- When modules fail to validate, they're removed from the cache.

- When a CompilerInstance is spawned to build a new module, each
already-loaded PCM is assumed to be valid, and is frozen to avoid
the use-after-free.

- Any newly-built module is written directly to the cache to avoid the
round-trip to the filesystem, making lock files unnecessary for
correctness.

Original patch by Manman Ren; most testcases by Adrian Prantl!

llvm-svn: 298278
---
 clang/lib/Basic/MemoryBufferCache.cpp | 48 +++++++++++++++++++++++++++++++++++
 1 file changed, 48 insertions(+)
 create mode 100644 clang/lib/Basic/MemoryBufferCache.cpp

(limited to 'clang/lib/Basic/MemoryBufferCache.cpp')

diff --git a/clang/lib/Basic/MemoryBufferCache.cpp b/clang/lib/Basic/MemoryBufferCache.cpp
new file mode 100644
index 00000000000..c1fc571ec9b
--- /dev/null
+++ b/clang/lib/Basic/MemoryBufferCache.cpp
@@ -0,0 +1,48 @@
+//===- MemoryBufferCache.cpp - Cache for loaded memory buffers ------------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#include "clang/Basic/MemoryBufferCache.h"
+#include "llvm/Support/MemoryBuffer.h"
+
+using namespace clang;
+
+llvm::MemoryBuffer &
+MemoryBufferCache::addBuffer(llvm::StringRef Filename,
+                             std::unique_ptr<llvm::MemoryBuffer> Buffer) {
+  auto Insertion =
+      Buffers.insert({Filename, BufferEntry{std::move(Buffer), NextIndex++}});
+  assert(Insertion.second && "Already has a buffer");
+  return *Insertion.first->second.Buffer;
+}
+
+llvm::MemoryBuffer *MemoryBufferCache::lookupBuffer(llvm::StringRef Filename) {
+  auto I = Buffers.find(Filename);
+  if (I == Buffers.end())
+    return nullptr;
+  return I->second.Buffer.get();
+}
+
+bool MemoryBufferCache::isBufferFinal(llvm::StringRef Filename) {
+  auto I = Buffers.find(Filename);
+  if (I == Buffers.end())
+    return false;
+  return I->second.Index < FirstRemovableIndex;
+}
+
+bool MemoryBufferCache::tryToRemoveBuffer(llvm::StringRef Filename) {
+  auto I = Buffers.find(Filename);
+  assert(I != Buffers.end() && "No buffer to remove...");
+  if (I->second.Index < FirstRemovableIndex)
+    return true;
+
+  Buffers.erase(I);
+  return false;
+}
+
+void MemoryBufferCache::finalizeCurrentBuffers() { FirstRemovableIndex = NextIndex; }
-- 
cgit v1.2.3