| Commit message (Collapse) | Author | Age | Files | Lines |
... | |
|
|
|
|
|
|
|
|
|
| |
If the system is under heavy load 1 second might not be long enough
for it to produce output which could lead to spurious test failures.
What matters is that the right test cases reach a timeout.
rdar://problem/40221572
llvm-svn: 332985
|
|
|
|
|
|
| |
failing on Windows.
llvm-svn: 332563
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Summary:
This sequence ends the CDATA block so any characters after that are no
longer escaped. This can be fixed by replacing "]]>" with "]]]]><![CDATA[>".
Reviewers: cmatthews
Reviewed By: cmatthews
Differential Revision: https://reviews.llvm.org/D46886
llvm-svn: 332440
|
|
|
|
| |
llvm-svn: 332424
|
|
|
|
|
|
|
| |
As per review feedback, make sure we rm temp files, and make the return
code checking for lit more specific.
llvm-svn: 332423
|
|
|
|
|
|
|
| |
JUnit xml allows for a message attribute to be displayed on skips. Lets
populate that with an analysis of why we skipped the test.
llvm-svn: 332156
|
|
|
|
|
|
|
| |
CDATA blocks don't need to have XML stuff escaped. Makes sense to wrap
output in them instead of escaping.
llvm-svn: 332116
|
|
|
|
|
|
|
|
| |
We were reporting "Unsupported" tests in xunit as passes, however since
they are not run, it make more sense to mark them as skipped. The Junit
xml standard has support for that, so lets use it.
llvm-svn: 332065
|
|
|
|
|
|
| |
Failures will increase coverage.
llvm-svn: 332056
|
|
|
|
|
|
| |
I forgot to commit this file.
llvm-svn: 331946
|
|
|
|
|
|
| |
Looks like results can come in either way in this file. Loosen the ordering constraints.
llvm-svn: 331945
|
|
|
|
|
|
|
|
|
|
| |
Lit creates malformed xml when the test case has an & in the name.
Escape those correctly.
This also adds a test case which I will add other nasty encoding issues to in some followup commits.
llvm-svn: 331942
|
|
|
|
|
|
|
| |
It is causing many tests to fail on Windows buildbots:
http://lab.llvm.org:8011/builders/clang-x64-ninja-win7/builds/10211
llvm-svn: 330848
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When debugging test failures with -vv (or -v in the case of the
internal shell), this makes it easier to locate the RUN line that
failed. For example, clang's test/Driver/linux-ld.c has 892 total RUN
lines, and clang's test/Driver/arm-cortex-cpus.c has 424 RUN lines
after concatenation for line continuations.
When reading the generated shell script, this also makes it easier to
locate the RUN line that produced each command.
To support reporting RUN line numbers in the case of the internal
shell, this patch extends the internal shell to support the null
command, ":", except pipelines are not supported.
Reviewed By: asmith, delcypher
Differential Revision: https://reviews.llvm.org/D44598
llvm-svn: 330755
|
|
|
|
|
|
|
|
|
|
| |
`shtest-xunit-output.py` test.
Although there is no `-` file Jeremy Morse has reported to me that it
causes problems in their setup because lit tries to find it and ends up
loading an out of tree lit configuration file.
llvm-svn: 330728
|
|
|
|
|
|
|
|
|
|
|
| |
XML printer.
A test has been added that tries to comprehensively test emitting
XUnit XML output for shell tests.
Differential Revision: https://reviews.llvm.org/D45567
llvm-svn: 330409
|
|
|
|
|
|
|
|
|
|
| |
This reverts commit 771829b640a5494ab65c810dd6b4330522bf3a33 (rr328598)
Hopefully the test will now pass on the bots.
rdar://problem/38774530
llvm-svn: 328703
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The `shtest-timeout.py` test was failing intermittently. It looks like
the issue is that on a resource constrained system lit is unable to run
`quick_then_slow.py` twice and print out the messages the tests expects
within the one second timeout.
The underlying issue is that the test is dependent on the performance of
the host machine is a rather fragile way. This is due to hardcoding
timeout values and having assumptions that the host machine is able to
perform a certain amount of work within the hardcoded timeout values.
We could increase the timeout values but that doesn't really fix the
underlying issue. Instead this patch removes one of fragile assumptions
in the hope that this will be enough to fix the bots.
There are other fragile assumptions in this test (e.g. `quick.py` can be
executed in less than 1 second). If the bots continue to fail we'll have
to revisit this.
rdar://problem/38774530
llvm-svn: 328702
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Summary:
This reverts commit r328596.
Checking if the arguments are strings before testing if they contain "/dev/null".
Reviewers: rnk
Reviewed By: rnk
Subscribers: delcypher, llvm-commits
Differential Revision: https://reviews.llvm.org/D44914
llvm-svn: 328603
|
|
|
|
|
|
|
|
| |
Disabled until fixed in order to avoid random failures on green dragon.
rdar://problem/38774530
llvm-svn: 328598
|
|
|
|
|
|
| |
This reverts commit ca7fdbb974384ce5a05528b22a41d46b1cc13e92.
llvm-svn: 328596
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Generalized /dev/null remapping on Windows, and added test.
Reviewers: rnk
Reviewed By: rnk
Subscribers: amccarth, zturner, delcypher, llvm-commits
Differential Revision: https://reviews.llvm.org/D44771
llvm-svn: 328589
|
|
|
|
|
|
|
|
|
|
| |
Fixes PR36449
Patch by Chamal de Silva
Differential Revision: https://reviews.llvm.org/D43501
llvm-svn: 328563
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
support for microbenchmarks.
Summary:
These changes are to allow to a Result object to have nested Result objects in
order to support microbenchmarks. Currently lit is restricted to reporting one
result object for one test, this change provides support tests that want to
report individual timings for individual kernels.
This revision is the result of the discussions in
https://reviews.llvm.org/D32272#794759,
https://reviews.llvm.org/D37421#f8003b27 and https://reviews.llvm.org/D38496.
It is a separation of the changes purposed in https://reviews.llvm.org/D40077.
This change will enable adding LCALS (Livermore Compiler Analysis Loop Suite)
collection of loop kernels to the llvm test suite using the google benchmark
library (https://reviews.llvm.org/D43319) with tracking of individual kernel
timings.
Previously microbenchmarks had been handled by using macros to section groups
of microbenchmarks together and build many executables while still getting a
grouped timing (MultiSource/TSVC). Recently the google benchmark library was
added to the test suite and utilized with a litsupport plugin. However the
limitation of 1 test 1 result limited its use to passing a runtime option to
run only 1 microbenchmark with several hand written tests
(MicroBenchmarks/XRay). This runs the same executable many times with different
hand-written tests. I will update the litsupport plugin to utilize the new
functionality (https://reviews.llvm.org/D43316).
These changes allow lit to report micro test results if desired in order to get
many precise timing results from 1 run of 1 test executable.
Reviewers: MatzeB, hfinkel, rengolin, delcypher
Differential Revision: https://reviews.llvm.org/D43314
llvm-svn: 327422
|
|
|
|
| |
llvm-svn: 326359
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Summary:
That would allow to recursively compare directories in tests using
"diff -r" on Windows in a similar way as it can be done on Linux or Mac.
Reviewers: zturner, morehouse, vsk
Reviewed By: zturner
Subscribers: kcc, llvm-commits
Differential Revision: https://reviews.llvm.org/D41776
llvm-svn: 322102
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
internally
Summary:
The internal shell already supports 'cd', ‘export’ and ‘echo’ commands.
This patch adds implementation of non-pipelined ‘mkdir’, ‘diff’ and ‘rm’
commands as the internal shell builtins.
Reviewed by: Zachary Turner, Reid Kleckner
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D39567
llvm-svn: 319528
|
|
|
|
|
|
| |
Differential Revision: https://reviews.llvm.org/D40520
llvm-svn: 319329
|
|
|
|
| |
llvm-svn: 318914
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
internally
Summary:
The internal shell already supports 'cd', ‘export’ and ‘echo’ commands.
This patch adds implementation of non-pipelined ‘mkdir’, ‘diff’ and ‘rm’
commands as the internal shell builtins.
Reviewers: Zachary Turner, Reid Kleckner
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D39567
llvm-svn: 318911
|
|
|
|
|
|
|
| |
This reverts r315697 and my ill-fated attempts to fix it on Windows.
I'll try again when I get access to a Windows machine.
llvm-svn: 315793
|
|
|
|
|
|
|
|
|
|
| |
"No such file or directory: C:\\...\\tests\\Output\\shared-output.py.tmp/Output/Shared/SHARED.tmp"
And yet other forward-slashes don't seem to be causing the same
problem. I'll see if I can get ahold of a Windows machine to poke at
this directly later.
llvm-svn: 315792
|
|
|
|
|
|
|
| |
I don't have access to a Windows machine at the moment, so if this
doesn't fix it I'll just revert for now.
llvm-svn: 315782
|
|
|
|
|
|
|
| |
I didn't think about '%{inputs}' having the same problem. This one
should be a fully Windows path name.
llvm-svn: 315779
|
|
|
|
| |
llvm-svn: 315773
|
|
|
|
|
|
|
|
|
|
|
|
| |
This refers to a temporary path that can be shared across all tests,
identified by a particular label. This can be used for things like
caches.
At the moment, the character set for the LABEL is limited to C
identifier characters, plus '-', '+', '=', and '.'. This is the same
set of characters currently allowed in REQUIRES clause identifiers.
llvm-svn: 315697
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Fix llvm_tools_dir attribute access not to fail when the variable is not
present. This directory is not really necessary to run lit tests,
and the code already accounts for it being None.
The reference was added in r313407, and it breaks the stand-alone lit
package in Gentoo.
Differential Revision: https://reviews.llvm.org/D38442
llvm-svn: 314620
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This has gone back and forth, but it seems this is necessary
after all. realpath is not sufficient because if you have a
file named 'C:\foo.txt', then both realpath('c:\foo.txt') and
realpath(C:\foo.txt') return the string that was passed to them
exactly as is, meaning the case of the drive-letter won't match.
The problem before was not that we were normalizing the case of
items going into the config map, but rather that we were
normalizing the case of something we needed to print. The value
that is used to key on the config map should never be printed.
llvm-svn: 313918
|
|
|
|
|
|
|
|
| |
This makes all paths lowercase on Windows, which seemed like a
good idea at the time, but it means that tests can't properly
use FileCheck to match expected path names.
llvm-svn: 313889
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Config map is not exposed through the command line, so testing this
is somewhat tricky. But basically we need a test that if a custom
driver builds a config map and passes it to main, it gets respected.
A config map allows config files in the source tree to be mapped
to alternate config files in the build tree. This particular test
works by having two config files in separate directories, and
setting up a config map to have that redirects A/lit.site.cfg
to B/altconfig. Then, we print a message in A/lit.site.cfg
and B/altconfig and check that we do see the output from B
but don't see the output from A. Additionally we test that
the test suite specified by A's config map is properly discovered.
Differential Revision: https://reviews.llvm.org/D38105
llvm-svn: 313887
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Many editors and Python-related diagnostics tools such as
debuggers break or fail in mysterious ways when python files
don't end in .py. This is especially true on Windows, but
still exists on other platforms. I don't want to be too heavy
handed in changing everything across the board, but I do want
to at least *allow* lit configs to have .py extensions. This
patch makes the discovery process first look for a config file
with a .py extension, and if one is not found, then looks for
a config file using the old method. So for existing users, there
should be no functional change.
Differential Revision: https://reviews.llvm.org/D37838
llvm-svn: 313849
|
|
|
|
|
|
|
|
| |
A few tests were manually constructing a LitConfig object, since
I added a new argument to it this was triggering some failures
I didn't detect. `ninja check-lit` passes now.
llvm-svn: 313461
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This is a resubmission of r313270. It broke standalone builds of
compiler-rt because we were not correctly generating the llvm-lit
script in the standalone build directory.
The fixes incorporated here attempt to find llvm/utils/llvm-lit
from the source tree returned by llvm-config. If present, it
will generate llvm-lit into the output directory. Regardless,
the user can specify -DLLVM_EXTERNAL_LIT to point to a specific
lit.py on their file system. This supports the use case of
someone installing lit via a package manager. If it cannot find
a source tree, and -DLLVM_EXTERNAL_LIT is either unspecified or
invalid, then we print a warning that tests will not be able
to run.
Differential Revision: https://reviews.llvm.org/D37756
llvm-svn: 313407
|
|
|
|
|
|
|
|
| |
This patch is still breaking several multi-stage compiler-rt bots.
I already know what the fix is, but I want to get the bots green
for now and then try re-applying in the morning.
llvm-svn: 313335
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This patch simplifies LLVM's lit infrastructure by enforcing an ordering
that a site config is always run before a source-tree config.
A significant amount of the complexity from lit config files arises from
the fact that inside of a source-tree config file, we don't yet know if
the site config has been run. However it is *always* required to run
a site config first, because it passes various variables down through
CMake that the main config depends on. As a result, every config
file has to do a bunch of magic to try to reverse-engineer the location
of the site config file if they detect (heuristically) that the site
config file has not yet been run.
This patch solves the problem by emitting a mapping from source tree
config file to binary tree site config file in llvm-lit.py. Then, during
discovery when we find a config file, we check to see if we have a
target mapping for it, and if so we use that instead.
This mechanism is generic enough that it does not affect external users
of lit. They will just not have a config mapping defined, and everything
will work as normal.
On the other hand, for us it allows us to make many simplifications:
* We are guaranteed that a site config will be executed first
* Inside of a main config, we no longer have to assume that attributes
might not be present and use getattr everywhere.
* We no longer have to pass parameters such as --param llvm_site_config=<path>
on the command line.
* It is future-proof, meaning you don't have to edit llvm-lit.in to add
support for new projects.
* All of the duplicated logic of trying various fallback mechanisms of
finding a site config from the main config are now gone.
One potentially noteworthy thing that was required to implement this
change is that whereas the ninja check targets previously used the first
method to spawn lit, they now use the second. In particular, you can no
longer run lit.py against the source tree while specifying the various
`foo_site_config=<path>` parameters. Instead, you need to run
llvm-lit.py.
Differential Revision: https://reviews.llvm.org/D37756
llvm-svn: 313270
|
|
|
|
| |
llvm-svn: 312288
|
|
|
|
|
|
|
|
|
|
|
|
| |
It was marked as unsupported on Windows in r311230 because on some Win10
machines it failed or caused hang. The problem was that on these machines
system bash (C:\Windows\System32\bash.exe) was used which requires paths to be
passed like '/mnt/c/path/to/my/script' instead of 'C:\path\to\my\script'.
TODO: we should make lit detect if system bash is used instead of msys and set
appropriate path format.
llvm-svn: 311558
|
|
|
|
|
|
| |
When run manually it fails, but when run under buildbot it causes hang.
llvm-svn: 311230
|
|
|
|
|
|
|
|
|
|
|
|
| |
This is an updated version of https://reviews.llvm.org/D22144 by @jlpeyton.
The patch was accepted but not landed.
This is useful functionality and I would like to use this to enable lit tests for environment variable behaviour.
Differential Revision: https://reviews.llvm.org/D36403
llvm-svn: 311180
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Multi-configuration CMake generators such as those for Visual Studio or Xcode do not
specify a build config at configure time, but let the user choose at build
time. In these cases binaries go into build/${Configuration}/bin rather than
build/bin. Prior to this commit, check-lit would fail when using multi-configuration
generators as it did not know how to resolve ${Configuration} in order
to find tools such as FileCheck. This commit teaches it to resolve
llvm_tools_dir within lit using the value specified with --param
build_mode.
Differential Revision: https://reviews.llvm.org/D36263
llvm-svn: 309967
|