[libc++] Set __file_ to 0 in basic_filebuf::close() even if fclose fails

This issue was detected by ASan in one of our tests. This test manually
invokes basic_filebuf::cloe(). fclose(__h.release() returned a non-zero
exit status, so __file_ wasn't set to 0. Later when basic_filebuf
destructor ran, we would enter the if (__file_) block again leading to
heap-use-after-free error.

The POSIX specification for fclose says that independently of the return
value, fclose closes the underlying file descriptor and any further
access (including another call to fclose()) to the stream results in
undefined behavior. This is exactly what happened in our test case.

To avoid this issue, we have to always set __file_ to 0 independently of
the fclose return value.

Differential Revision: https://reviews.llvm.org/D64979

llvm-svn: 366730

1 files changed, 56 insertions, 0 deletions

diff --git a/libcxx/test/std/input.output/file.streams/fstreams/filebuf.members/close.pass.cpp b/libcxx/test/std/input.output/file.streams/fstreams/filebuf.members/close.pass.cpp
new file mode 100644
index 00000000000..b545041e428
--- /dev/null
+++ b/libcxx/test/std/input.output/file.streams/fstreams/filebuf.members/close.pass.cpp
@@ -0,0 +1,56 @@
+//===----------------------------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+// <fstream>
+
+// basic_filebuf<charT,traits>* close();
+
+#include <fstream>
+#include <cassert>
+#if defined(__unix__)
+#include <fcntl.h>
+#include <unistd.h>
+#endif
+#include "test_macros.h"
+#include "platform_support.h"
+
+int main(int, char**)
+{
+    std::string temp = get_temp_file_name();
+    {
+        std::filebuf f;
+        assert(!f.is_open());
+        assert(f.open(temp.c_str(), std::ios_base::out) != 0);
+        assert(f.is_open());
+        assert(f.close() != nullptr);
+        assert(!f.is_open());
+        assert(f.close() == nullptr);
+        assert(!f.is_open());
+    }
+#if defined(__unix__)
+    {
+        std::filebuf f;
+        assert(!f.is_open());
+        // Use open directly to get the file descriptor.
+        int fd = open(temp.c_str(), O_RDWR);
+        assert(fd >= 0);
+        // Use the internal method to create filebuf from the file descriptor.
+        assert(f.__open(fd, std::ios_base::out) != 0);
+        assert(f.is_open());
+        // Close the file descriptor directly to force filebuf::close to fail.
+        assert(close(fd) == 0);
+        // Ensure that filebuf::close handles the failure.
+        assert(f.close() == nullptr);
+        assert(!f.is_open());
+        assert(f.close() == nullptr);
+    }
+#endif
+    std::remove(temp.c_str());
+
+    return 0;
+}