[libFuzzer] Avoid undefined behavior, properly discard output to stdout/stderr.

Fix libFuzzer when setting -close_fd_mask to a non-zero value.
In previous implementation, libFuzzer closes the file descriptors for
stdout/stderr. This has some disavantages:

For `fuzzer-fdmask.test`, we write directly to stdout and stderr using the
file streams stdout and stderr, after the file descriptors are closed, which is
undefined behavior. In Windows, in particular, this was making the test fail.

Also, if we close stdout and we open a new file in libFuzzer, we get the file
descriptor 1, which could generate problem if some code assumes file descriptors
refers to stdout and works directly writing to the file descriptor 1, but it
will be writing to the opened file (for example using std::cout).

Instead of closing the file descriptors, I redirect the output to /dev/null on
linux and nul on Windows.

Differential Revision: https://reviews.llvm.org/D28718

llvm-svn: 292743

4 files changed, 20 insertions, 2 deletions

diff --git a/llvm/lib/Fuzzer/FuzzerIO.cpp b/llvm/lib/Fuzzer/FuzzerIO.cpp
index eda8e877293..45445fa3ba4 100644
--- a/llvm/lib/Fuzzer/FuzzerIO.cpp
+++ b/llvm/lib/Fuzzer/FuzzerIO.cpp
@@ -97,13 +97,13 @@ void DupAndCloseStderr() {
       OutputFile = NewOutputFile;
       if (EF->__sanitizer_set_report_fd)
         EF->__sanitizer_set_report_fd(reinterpret_cast<void *>(OutputFd));
-      CloseFile(2);
+      DiscardOutput(2);
     }
   }
 }
 
 void CloseStdout() {
-  CloseFile(1);
+  DiscardOutput(1);
 }
 
 void Printf(const char *Fmt, ...) {
diff --git a/llvm/lib/Fuzzer/FuzzerIO.h b/llvm/lib/Fuzzer/FuzzerIO.h
index 17cf2ab6ac1..1f79f3632ba 100644
--- a/llvm/lib/Fuzzer/FuzzerIO.h
+++ b/llvm/lib/Fuzzer/FuzzerIO.h
@@ -64,6 +64,8 @@ int DuplicateFile(int Fd);
 
 void RemoveFile(const std::string &Path);
 
+void DiscardOutput(int Fd);
+
 }  // namespace fuzzer
 
 #endif  // LLVM_FUZZER_IO_H
diff --git a/llvm/lib/Fuzzer/FuzzerIOPosix.cpp b/llvm/lib/Fuzzer/FuzzerIOPosix.cpp
index 494ec9e6c83..2dc2c61b999 100644
--- a/llvm/lib/Fuzzer/FuzzerIOPosix.cpp
+++ b/llvm/lib/Fuzzer/FuzzerIOPosix.cpp
@@ -75,6 +75,14 @@ void RemoveFile(const std::string &Path) {
   unlink(Path.c_str());
 }
 
+void DiscardOutput(int Fd) {
+  FILE* Temp = fopen("/dev/null", "w");
+  if (!Temp)
+    return;
+  dup2(fileno(Temp), Fd);
+  fclose(Temp);
+}
+
 std::string DirName(const std::string &FileName) {
   char *Tmp = new char[FileName.size() + 1];
   memcpy(Tmp, FileName.c_str(), FileName.size() + 1);
diff --git a/llvm/lib/Fuzzer/FuzzerIOWindows.cpp b/llvm/lib/Fuzzer/FuzzerIOWindows.cpp
index 983cb6c2e4a..1d82f416272 100644
--- a/llvm/lib/Fuzzer/FuzzerIOWindows.cpp
+++ b/llvm/lib/Fuzzer/FuzzerIOWindows.cpp
@@ -141,6 +141,14 @@ void RemoveFile(const std::string &Path) {
   _unlink(Path.c_str());
 }
 
+void DiscardOutput(int Fd) {
+  FILE* Temp = fopen("nul", "w");
+  if (!Temp)
+    return;
+  _dup2(_fileno(Temp), Fd);
+  fclose(Temp);
+}
+
 static bool IsSeparator(char C) {
   return C == '\\' || C == '/';
 }