summaryrefslogtreecommitdiffstats
path: root/llvm/unittests/Support/FileOutputBufferTest.cpp
Commit message (Collapse)AuthorAgeFilesLines
* [LLD][ELF] Support --[no-]mmap-output-file with F_no_mmapNick Terrell2019-10-291-0/+22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: Add a flag `F_no_mmap` to `FileOutputBuffer` to support `--[no-]mmap-output-file` in ELF LLD. LLD currently explicitly ignores this flag for compatibility with GNU ld and gold. We need this flag to speed up link time for large binaries in certain scenarios. When we link some of our larger binaries we find that LLD takes 50+ GB of memory, which causes memory pressure. The memory pressure causes the VM to flush dirty pages of the output file to disk. This is normally okay, since we should be flushing cold pages. However, when using BtrFS with compression we need to write 128KB at a time when we flush a page. If any page in that 128KB block is written again, then it must be flushed a second time, and so on. Since LLD doesn't write sequentially this causes write amplification. The same 128KB block will end up being flushed multiple times, causing the linker to many times more IO than necessary. We've observed 3-5x faster builds with -no-mmap-output-file when we hit this scenario. The bad scenario only applies to compressed filesystems, which group together multiple pages into a single compressed block. I've tested BtrFS, but the problem will be present for any compressed filesystem on Linux, since it is caused by the VM. Silently ignoring --no-mmap-output-file caused a silent regression when we switched from gold to lld. We pass --no-mmap-output-file to fix this edge case, but since lld silently ignored the flag we didn't realize it wasn't being respected. Benchmark building a 9 GB binary that exposes this edge case. I linked 3 times with --mmap-output-file and 3 times with --no-mmap-output-file and took the average. The machine has 24 cores @ 2.4 GHz, 112 GB of RAM, BtrFS mounted with -compress-force=zstd, and an 80% full disk. | Mode | Time | |---------|-------| | mmap | 894 s | | no mmap | 126 s | When compression is disabled, BtrFS performs just as well with and without mmap on this benchmark. I was unable to reproduce the regression with any binaries in lld-speed-test. Reviewed By: ruiu, MaskRay Differential Revision: https://reviews.llvm.org/D69294
* Revert "raw_ostream: add operator<< overload for std::error_code"Pavel Labath2019-08-141-1/+10
| | | | | | | | | | | This reverts commit r368849, because it breaks some bots (e.g. llvm-clang-x86_64-win-fast). It turns out this is not as NFC as we had hoped, because operator== will consider two std::error_codes to be distinct even though they both hold "success" values if they have different categories. llvm-svn: 368854
* raw_ostream: add operator<< overload for std::error_codePavel Labath2019-08-141-10/+1
| | | | | | | | | | | | | | | | | | | | | | | Summary: The main motivation for this is unit tests, which contain a large macro for pretty-printing std::error_code, and this macro is duplicated in every file that needs to do this. However, the functionality may be useful elsewhere too. In this patch I have reimplemented the existing ASSERT_NO_ERROR macros to reuse the new functionality, but I have kept the macro (as a one-liner) as it is slightly more readable than ASSERT_EQ(..., std::error_code()). Reviewers: sammccall, ilya-biryukov Subscribers: zturner, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D65643 llvm-svn: 368849
* Update the file headers across all of the LLVM projects in the monorepoChandler Carruth2019-01-191-4/+3
| | | | | | | | | | | | | | | | | to reflect the new license. We understand that people may be surprised that we're moving the header entirely to discuss the new license. We checked this carefully with the Foundation's lawyer and we believe this is the correct approach. Essentially, all code in the project is now made available by the LLVM project under our new license, so you will see that the license headers include that license only. Some of our contributors have contributed code under our old license, and accordingly, we have retained a copy of our old license notice in the top-level files in each project and repository. llvm-svn: 351636
* Remove F_modify flag from FileOutputBuffer.Rui Ueyama2019-01-191-49/+0
| | | | | | | | This code is dead. There is no use of the feature in the entire LLVM codebase. Differential Revision: https://reviews.llvm.org/D56939 llvm-svn: 351613
* [SupportTests] Silence -Wsign-compare warningsBenjamin Kramer2018-06-281-2/+2
| | | | llvm-svn: 335921
* Add a flag to FileOutputBuffer that allows modification.Zachary Turner2018-06-281-0/+50
| | | | | | | | | | | | | FileOutputBuffer creates a temp file and on commit atomically renames the temp file to the destination file. Sometimes we want to modify an existing file in place, but still have the atomicity guarantee. To do this we can initialize the contents of the temp file from the destination file (if it exists), that way the resulting FileOutputBuffer can have only selective bytes modified. Committing will then atomically replace the destination file as desired. llvm-svn: 335902
* Convert FileOutputBuffer::commit to Error.Rafael Espindola2017-11-081-3/+3
| | | | llvm-svn: 317656
* Update unittest too.Rafael Espindola2017-11-081-8/+8
| | | | llvm-svn: 317651
* Untabify.NAKAMURA Takumi2017-08-281-3/+3
| | | | llvm-svn: 311875
* Re-sort #include lines for unittests. This uses a slightly modifiedChandler Carruth2017-06-061-1/+1
| | | | | | | | | | | | | | | clang-format (https://reviews.llvm.org/D33932) to keep primary headers at the top and handle new utility headers like 'gmock' consistently with other utility headers. No other change was made. I did no manual edits, all of this is clang-format. This should allow other changes to have more clear and focused diffs, and is especially motivated by moving some headers into more focused libraries. llvm-svn: 304786
* Fix a real temp file leak in FileOutputBufferReid Kleckner2016-09-021-5/+8
| | | | | | | | | | | | | If we failed to commit the buffer but did not die to a signal, the temp file would remain on disk on Windows. Having an open file mapping and file handle prevents the file from being deleted. I am choosing not to add an assertion of success on the temp file removal, since virus scanners and other environmental things can often cause removal to fail in real world tools. Also fix more temp file leaks in unit tests. llvm-svn: 280445
* Return ErrorOr from FileOutputBuffer::create. NFC.Rafael Espindola2015-08-131-9/+16
| | | | llvm-svn: 244848
* [cleanup] Re-sort all the #include lines in LLVM usingChandler Carruth2015-01-141-1/+1
| | | | | | | | | | | utils/sort_includes.py. I clearly haven't done this in a while, so more changed than usual. This even uncovered a missing include from the InstrProf library that I've added. No functionality changed here, just mechanical cleanup of the include order. llvm-svn: 225974
* Remove unused feature. NFC.Rafael Espindola2014-12-121-3/+2
| | | | llvm-svn: 224135
* Misc cleanups to the FileSytem api.Rafael Espindola2014-09-111-3/+3
| | | | | | | | | | | | | | | | The main difference is the removal of std::error_code exists(const Twine &path, bool &result); It was an horribly redundant interface since a file not existing is also a valid error_code. Now we have an access function that returns just an error_code. This is the only function that has to be implemented for Unix and Windows. The functions can_write, exists and can_execute an now just wrappers. One still has to be very careful using these function to avoid introducing race conditions (Time of check to time of use). llvm-svn: 217625
* Remove the last uses of 'using std::error_code'Rafael Espindola2014-06-131-7/+7
| | | | | | This finishes the transition to std::error_code. llvm-svn: 210877
* Don't use 'using std::error_code' in include/llvm.Rafael Espindola2014-06-121-0/+1
| | | | | | This should make sure that most new uses use the std prefix. llvm-svn: 210835
* Remove unused has_magic.Rafael Espindola2014-06-111-10/+2
| | | | | | | This will allow inlining get_magic, which should in turn fix one of the mingw build problems after the switch to std::error_code. llvm-svn: 210712
* Use std::unique_ptr instead of OwningPtr in the MemoryBuffer unittests.Craig Topper2014-05-181-5/+4
| | | | llvm-svn: 209102
* Fix known typosAlp Toker2014-01-241-1/+1
| | | | | | | Sweep the codebase for common typos. Includes some changes to visible function names that were misspelt. llvm-svn: 200018
* Remove remove_all. A compiler has no need for recursively deleting a directory.Rafael Espindola2014-01-101-2/+5
| | | | llvm-svn: 198955
* Add a convenience createUniqueDirectory function.Rafael Espindola2013-06-271-5/+1
| | | | | | | | | | | There are a few valid situation where we care about the structure inside a directory, but not about the directory itself. A simple example is for unit testing directory traversal. PathV1 had a function like this, add one to V2 and port existing users of the created temp file and delete it hack to using it. llvm-svn: 185059
* Move PathV2.h to Path.hRafael Espindola2013-06-111-1/+1
| | | | | | | Most clients have already been moved from Path V1 to V2. The ones using V1 now include PathV1.h explicitly. llvm-svn: 183801
* Sort the #include lines for unittest/...Chandler Carruth2012-12-041-2/+0
| | | | llvm-svn: 169250
* [Support] Make FileOutputBuffer work on Windows.Michael J. Spencer2012-12-031-18/+7
| | | | llvm-svn: 169167
* Initial commit of new FileOutputBuffer support class. Nick Kledzik2012-08-011-0/+137
Since the llvm::sys::fs::map_file_pages() support function it relies on is not yet implemented on Windows, the unit tests for FileOutputBuffer are currently conditionalized to run only on unix. llvm-svn: 161099
OpenPOWER on IntegriCloud