Revamp test-suite documentation

- Remove duplication: Both TestingGuide and TestSuiteMakefileGuide
  would give a similar overview over the test-suite.
- Present cmake/lit as the default/normal way of running the test-suite:
- Move information about the cmake/lit testsuite into the new
  TestSuiteGuide.rst file. Mark the remaining information in
  TestSuiteMakefilesGuide.rst as deprecated.
- General simplification and shorting of language.
- Remove paragraphs about tests known to fail as everything should pass
  nowadays.
- Remove paragraph about zlib requirement; it's not required anymore
  since we copied a zlib source snapshot into the test-suite.
- Remove paragraph about comparison with "native compiler". Correctness is
  always checked against reference outputs nowadays.
- Change cmake/lit quickstart section to recommend `pip` for installing
  lit and use `CMAKE_C_COMPILER` and a cache file in the example as that
  is what most people will end up doing anyway. Also a section about
  compare.py to quickstart.
- Document `Bitcode` and `MicroBenchmarks` directories.
- Add section with commonly used cmake configuration options.
- Add section about showing and comparing result files via compare.py.
- Add section about using external benchmark suites.
- Add section about using custom benchmark suites.
- Add section about profile guided optimization.
- Add section about cross-compilation and running on external devices.

Differential Revision: https://reviews.llvm.org/D51465

llvm-svn: 341260

1 files changed, 6 insertions, 70 deletions

diff --git a/llvm/docs/TestingGuide.rst b/llvm/docs/TestingGuide.rst
index 3cc8cf4589b..e3db60dd9e7 100644
--- a/llvm/docs/TestingGuide.rst
+++ b/llvm/docs/TestingGuide.rst
@@ -8,6 +8,7 @@ LLVM Testing Infrastructure Guide
 .. toctree::
    :hidden:
 
+   TestSuiteGuide
    TestSuiteMakefileGuide
 
 Overview
@@ -25,11 +26,7 @@ In order to use the LLVM testing infrastructure, you will need all of the
 software required to build LLVM, as well as `Python <http://python.org>`_ 2.7 or
 later.
 
-If you intend to run the :ref:`test-suite <test-suite-overview>`, you will also
-need a development version of zlib (zlib1g-dev is known to work on several Linux
-distributions).
-
-LLVM testing infrastructure organization
+LLVM Testing Infrastructure Organization
 ========================================
 
 The LLVM testing infrastructure contains two major categories of tests:
@@ -77,6 +74,8 @@ LLVM compiles, optimizes, and generates code.
 
 The test-suite is located in the ``test-suite`` Subversion module.
 
+See the :doc:`TestSuiteGuide` for details.
+
 Debugging Information tests
 ---------------------------
 
@@ -96,9 +95,8 @@ regressions tests are in the main "llvm" module under the directory
 ``llvm/test`` (so you get these tests for free with the main LLVM tree).
 Use ``make check-all`` to run the regression tests after building LLVM.
 
-The more comprehensive test suite that includes whole programs in C and C++
-is in the ``test-suite`` module. See :ref:`test-suite Quickstart
-<test-suite-quickstart>` for more information on running these tests.
+The ``test-suite`` module contains more comprehensive tests including whole C
+and C++ programs. See the :doc:`TestSuiteGuide` for details.
 
 Regression tests
 ----------------
@@ -585,65 +583,3 @@ the last RUN: line. This has two side effects:
 
 (b) it speeds things up for really big test cases by avoiding
     interpretation of the remainder of the file.
-
-.. _test-suite-overview:
-
-``test-suite`` Overview
-=======================
-
-The ``test-suite`` module contains a number of programs that can be
-compiled and executed. The ``test-suite`` includes reference outputs for
-all of the programs, so that the output of the executed program can be
-checked for correctness.
-
-``test-suite`` tests are divided into three types of tests: MultiSource,
-SingleSource, and External.
-
--  ``test-suite/SingleSource``
-
-   The SingleSource directory contains test programs that are only a
-   single source file in size. These are usually small benchmark
-   programs or small programs that calculate a particular value. Several
-   such programs are grouped together in each directory.
-
--  ``test-suite/MultiSource``
-
-   The MultiSource directory contains subdirectories which contain
-   entire programs with multiple source files. Large benchmarks and
-   whole applications go here.
-
--  ``test-suite/External``
-
-   The External directory contains Makefiles for building code that is
-   external to (i.e., not distributed with) LLVM. The most prominent
-   members of this directory are the SPEC 95 and SPEC 2000 benchmark
-   suites. The ``External`` directory does not contain these actual
-   tests, but only the Makefiles that know how to properly compile these
-   programs from somewhere else. When using ``LNT``, use the
-   ``--test-externals`` option to include these tests in the results.
-
-.. _test-suite-quickstart:
-
-``test-suite`` Quickstart
--------------------------
-
-The modern way of running the ``test-suite`` is focused on testing and
-benchmarking complete compilers using the
-`LNT <http://llvm.org/docs/lnt>`_ testing infrastructure.
-
-For more information on using LNT to execute the ``test-suite``, please
-see the `LNT Quickstart <http://llvm.org/docs/lnt/quickstart.html>`_
-documentation.
-
-``test-suite`` Makefiles
-------------------------
-
-Historically, the ``test-suite`` was executed using a complicated setup
-of Makefiles. The LNT based approach above is recommended for most
-users, but there are some testing scenarios which are not supported by
-the LNT approach. In addition, LNT currently uses the Makefile setup
-under the covers and so developers who are interested in how LNT works
-under the hood may want to understand the Makefile based setup.
-
-For more information on the ``test-suite`` Makefile setup, please see
-the :doc:`Test Suite Makefile Guide <TestSuiteMakefileGuide>`.