diff options
Diffstat (limited to 'openmp/testsuite/README')
| -rw-r--r-- | openmp/testsuite/README | 334 |
1 files changed, 334 insertions, 0 deletions
diff --git a/openmp/testsuite/README b/openmp/testsuite/README new file mode 100644 index 00000000000..8ca7c61aa06 --- /dev/null +++ b/openmp/testsuite/README @@ -0,0 +1,334 @@ + ============================================================================ +| OpenMP Validation Suite V 3.0 | +| High Performance Computing Center, Stuttgart | +| High Performance Computing and Tools, University of Houston | +| Jan. 2012 | + ============================================================================ + + +TABLE OF CONTENTS + +I INTRODUCTION +I.1. Aims and general function +I.2. Files and directories + +II USAGE +II.1. First run with make +II.2 Where to search for results +II.3. Using the runtest script + +III Adding and modifying tests +III.1. The template structure + +IV Known Issues and Workaround + +V Contact and Support + +------------------------------------------------------------------------------ + +I. INTRODUCTION +============================================================================== + +I.1. Aims and general function +-------------------------------- + +The OpenMP validation suite is designed to verify the correct implementation +of OpenMP in compilers. It is capable of checking Fortran as well as c +compiler. + +Testing the implementation is based on statistics. Each directive is tested +by computing and verifying the result with the already known value. In most +cases a wrong implementation can result in the right values. So the tests are +run several times. + +Additionally, the validation suite creates so called crosstests for each +directive. These are tests in which either the directive is missing or used +with different arguments. If this so called crosstest fails, this indicates +strongly that the previous test is capable of testing the directive. + +Lastly, an orphaned test is also run to determine if the directive being +tested is able to correctly run when 'orphaned' from the main function. +Essentially, the directive's code is placed into its own function which is +called during execution of the main function and often inside a parallel +region. + + +I.2. Files and directories +---------------------------- + + +d c directory containing the templates for the c tests +d fortran directory containing the templates for the Fortran + tests + Makefile Makefile containing options for compilation + common_utility.f + omp_my_sleep.h thread save sleep function + omp_testsuite.f Fortran header file + omp_testsuite.h autogenerated c-header file + ompts-c.conf configuration file for the c tests about how often + the tests shall be executed or how large the loop size + is + ompts_makeHeader.pl perl module for automatically generation of an up + to date header file + ompts_parserFunctions.pm perl module containing general functions for + the parser.pl script + ompts_parser.pl script for generating the source code out of the templates + ompts_standaloneProc.c framework for the c tests + ompts_standaloneProc.f framework for the Fortran tests + README the README file you've already found ;-) + LICENSE contains license information + runtest.pl the frame program of the test suite + testlist-f.txt test list containing the available tests for Fortran + testlist-c.txt test list containing the available tests for c + + +------------------------------------------------------------------------------ + +II. USAGE +============================================================================== + + +II.1. First run with make +-------------------------- + + +You can do a first simple run of the testsuite only after one step of +configuration: + +1) Modify the ompts.conf and ompts-c.conf file, change the number of threads +and number of repetitions of each test. + +2) Modify the Makefile, uncommenting the CC/FC and CFLAGS/FFLAGS variables for +the compiler of your choice. + +And now you can run the testsuite either for a C compiler with: + +> make ctest + +or for a Fortran compiler with: + +> make ftest + + +II.2. Running custom tests +-------------------------- + + +In order to run single tests or custom groups of tests, two make commmands +are defined: make stest and make fstest. These two command reference the file +customtest.txt when looking for a testlist to use. Simply edit customtest.txt +to include the desired test or tests. If customtest.txt contains c tests, + +> make stest + +or for fortran tests + +> make fstest + +In order to change the number of threads used in the tests, simply edit the +Makefile variables MINTHREADS and MAXTHREADS. By default, they are configured +to use 2 threads. To change the number of times each test is run, for c tests +edit the REPETITIONS variable in the file ompts-c.conf. The LOOPCOUNT and +SLEEPTIME variables can also be changed here. For fortran tests, edit the file +omp_testsuite.f to change both the LOOPCOUNT and the number of times each test +is run. + + + +II.3. Understanding the results +--------------------------------- + + +When running the testsuite the results will be shown on the screen. + +If you need the results for further purpose you can use the results.txt, which +is a simple list containing the results for each directive +in a single line. Each line starts with the name of the directive. Then follows +the result of the test given in the percentage of the passed tests. If 100% of +the tests passed successfully, the second number gives the result of the +corresponding crosstest. Crosstests will only be run if the normal test passes +with 100% accuracy. If a crosstest was not run or a test does not exist, it is +denotated by a "-". +After the results of the normal tests, there follow a series of tests in the +orphaned mode. If there were no orphaned tests available this is shown by a "-". + +If you run the testsuite with different numbers of threads (e.g. using the +runtest.pl script) the results are shown in blocks of 4 columns for each number +of threads. + +If a test fails you can find more detailed information in the ompts.log, +bin/c/*.out and *.log files. While the ompts.log file contains all compiler +error messages for all tests, the *.out and *.log files contain detailed inforamtion +on the execution process of the single tests. +In the *.out files there are listed all the results of the single executions of +the tests. In the *.log files there are error messages of the tests itself. + + +II.4. Cleaning Up +----------------- + + +Because many files are generated for each tested directive, it is often necessary +to clean the main directory after a battery of tests. To clean all generated files +in the main directory including the results and log files, + +> make clean + + +To clean only the logs and out files, + +> make cleanlogs + +To clean only the results, + +> make cleanresults + + + +II.4. Using the runtest script +------------------------------- + + +So for special purpose you can use the runtest.pl script, which allows a lot +more options for the execution process than the execution with make. + +Using the runtest.pl script is rather easy. You can use the the test suite only +after two steps of modifications: + +1.) Modify the Makefile to your wishes choosing your compiler and the necessary + compiler flags. +2.) If necessary edit one of the test lists (testlist-c.txt) and comment out the + tests you do not want to run using # at the beginning of a line. Testlists for Fortran end + with -f.txt while test lists for c with -c.txt. + +And now you can run the test suite either for Fortran using + # > ./runtest.pl -lang=fortran -d=fortran testlist-f.txt +or for c + # > ./runtest.pl -lang=c -d=c testlist-c.txt + +With the --help option you can show the complete list of options and their +explanations. + +The test results are summarized in cresults or fresults.txt while *.log keep +details for individual tests. There is also a file (ompts.log) keeping +compilation information. (see section II.2 ) + +If you don't want to test the directives in orphaned mode you can use the +-norphan option. You also can use the runtest.pl script either to compile all +tests or run compiled tests e.g. for cross compilation on other platforms. For +this there are the options -norun and -nocompile. + +Happy testing! + + +------------------------------------------------------------------------------ + +III. How to add new tests / The structure of test templates +============================================================================== + +III.1 The template structure +------------------------------ + +The test suite is based on templates so that you only have one file for test, +crosstest and the orphaned versions of them. + + A) Description of the template structure + +The syntax of the templates is much like xml. So each test begins with +'<ompts:test>' and ends with '</ompts:test>'. + +In between there are several other blocks holding information: + +- <ompts:testdescription> </ompts:testdescription> In between this tag you can + give a description on what the test checks and how it works. + +- <ompts:ompversion> </ompts:version> This tag is used to specify the + OpenMP-version which includes the tested directive. + +- <ompts:directive> </ompts:directive> Used to specify the directive how it is + called in the programming language. + +- <ompts:dependences> </ompts:dependences> With this tag you can specify other + omp directives which are necessary for the correct execution of your test. +The directives have to be listed by their directive names as it is called in +the programming language. Multiple directives are separated by ','. + +- <ompts:testcode> </ompts:testcode> In this tag stands the whole source code +for the test / crosstest. Each test has to be written as a function. The +syntax of the functions differs between C and Fortran: In C it has to take a +file pointer 'FILE * logFile' and return an int. If the test has been passed +successful it has to return a value unequal to 0. The file pointer can be used +to write information into a log file. In Fortran the function takes no +argument and the function name must not exceed XX characters. The return value +has to be specified using the '<testfunctionname>' tags. It has to be 1 if the +test succeeded and 0 if the test failed. For details see the example. + +To tell the test suite the name of your test function you have to enclose it +into the '<ompts:testcode:functionname> </ompts:testcode:functionname>' tag. + +If there are differences between test and crosstest you can use the +<ompts:check> </ompts:check> and <ompts:crosscheck> </ompts:crosscheck> tag. +When generating the test the parser will use the code enclosed in +<ompts:check> tags and cut out the code written in <ompts:crosscheck> tags. So +you have two possibilities to write your template for test and crosstest: The +first way you can write the complete code is to write the test in one +<ompts:check> tag and later the crosstest in one <ompts:crosscheck> tag. The +second way is to write both tests only by enclosing differing parts in +corresponding tags. + +The first method should be preferred if test and crosstest differ much from +each other. The second e.g. if you only want to change a few options like +replacing an omp singleprivate clause by an omp private clause or to cut out +a directive like omp flush. When you use the first way you have to take care +of the function name! You have to declare it twice with +<ompts:testcode:functionname>! + +- <ompts:orphan> </ompts:orphan> This tag can be used if you want to enable + your test to check the directive in orphan regions, too. The code enclosed +in this part will be written to a separate function which will be called +instead. If you have variables which are used outside this region you have to +declare them as global variables enclosed in an <ompts:orphan:vars> tag. For +further information see the description of the <ompts:orphan:vars> tag. + +- <ompts:orphan:vars> </ompts:orphan:vars> This tag is used to specify global + variables for an orphan region which allow the exchange of values between +the main program and the orphaned functions. The usage differs between C and +Fortran. In C you have to use a single declaration for each variable. You can +either declare all variables in one single or in several different regions. You +must not initialize the variables inside! In Fortran you have to put all +declarations in one single tag. Because there exist no global variables as in +C you have to use common blocks. For further information see the examples. + +III.2. Adding tests to the test lists +-------------------------------------- + +After you have created a new test you have to add them to a testlist. Simply +add the function name in a new Line into a file. + + + +------------------------------------------------------------------------------ + + +IV. Known Issues and Workaround +============================================================================== + +The Sun OS has a problem with the -maxdepth option on the 'make cleanall' +command. This prevents the tests from being removed from the working directory +and can cause problems with future tests. To remedy, edit the makefile line +under the clean command: + + -rm [cf]test*.[cf] [cf]crosstest*.[cf] [cf]ctest*.[cf] [cf]orphan*.[cf] + +Change to: + + -rm [cf]test* [cf]crosstest* [cf]ctest* [cf]orphan* + + + +------------------------------------------------------------------------------ + +V. Contact and Support +============================================================================== + +Contact: http://www.cs.uh.edu/~hpctools |
