talos-obmc-linux/tools/perf, branch dev-5.0

perf script python: Add trace_context extension module to sys.modules

2019-04-05T20:34:48+00:00

[ Upstream commit cc437642255224e4140fed1f3e3156fc8ad91903 ] In Python3, the result of PyModule_Create (called from scripts/python/Perf-Trace-Util/Context.c) is not automatically added to sys.modules. See: https://bugs.python.org/issue4592 Below is the observed behavior without the fix: # ldd /usr/bin/perf | grep -i python libpython3.6m.so.1.0 => /usr/lib64/libpython3.6m.so.1.0 (0x00007f8e1dfb2000) # perf record /bin/false [ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0.015 MB perf.data (17 samples) ] # perf script -g python | cat generated Python script: perf-script.py # perf script -s ./perf-script.py Traceback (most recent call last): File "./perf-script.py", line 18, in from perf_trace_context import * ModuleNotFoundError: No module named 'perf_trace_context' Error running python script ./perf-script.py # Committer notes: To build with python3 use: $ make -C tools/perf PYTHON=python3 Use a non-const variable to pass the 'name' arg to PyImport_AppendInittab(), as python2.6 has that as 'char *', which ends up trowing this in some environments: CC /tmp/build/perf/util/parse-branch-options.o util/scripting-engines/trace-event-python.c: In function 'python_start_script': util/scripting-engines/trace-event-python.c:1520:2: error: passing argument 1 of 'PyImport_AppendInittab' discards 'const' qualifier from pointer target type [-Werror] PyImport_AppendInittab("perf_trace_context", initfunc); ^ In file included from /usr/include/python2.6/Python.h:130:0, from util/scripting-engines/trace-event-python.c:22: /usr/include/python2.6/import.h:54:17: note: expected 'char *' but argument is of type 'const char *' PyAPI_FUNC(int) PyImport_AppendInittab(char *name, void (*initfunc)(void)); ^ cc1: all warnings being treated as errors Signed-off-by: Tony Jones Acked-by: Jiri Olsa Tested-by: Arnaldo Carvalho de Melo Cc: Jaroslav Škarvada Cc: Jonathan Corbet Cc: Ravi Bangoria Cc: Seeteena Thoufeek Fixes: 66dfdff03d19 ("perf tools: Add Python 3 support") Link: http://lkml.kernel.org/r/20190124005229.16146-2-tonyj@suse.de Signed-off-by: Arnaldo Carvalho de Melo Signed-off-by: Sasha Levin

perf script python: Use PyBytes for attr in trace-event-python

2019-04-05T20:34:47+00:00

[ Upstream commit 72e0b15cb24a497d7d0d4707cf51ff40c185ae8c ] With Python3. PyUnicode_FromStringAndSize is unsafe to call on attr and will return NULL. Use _PyBytes_FromStringAndSize (as with raw_buf). Below is the observed behavior without the fix. Note it is first necessary to apply the prior fix (Add trace_context extension module to sys,modules): # ldd /usr/bin/perf | grep -i python libpython3.6m.so.1.0 => /usr/lib64/libpython3.6m.so.1.0 (0x00007f8e1dfb2000) # perf record -e raw_syscalls:sys_enter /bin/false [ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0.018 MB perf.data (21 samples) ] # perf script -g python | cat generated Python script: perf-script.py # perf script -s ./perf-script.py in trace_begin Segmentation fault (core dumped) Signed-off-by: Tony Jones Acked-by: Jiri Olsa Tested-by: Arnaldo Carvalho de Melo Cc: Jaroslav Škarvada Cc: Jonathan Corbet Cc: Ravi Bangoria Cc: Seeteena Thoufeek Fixes: 66dfdff03d19 ("perf tools: Add Python 3 support") Link: http://lkml.kernel.org/r/20190124005229.16146-3-tonyj@suse.de Signed-off-by: Arnaldo Carvalho de Melo Signed-off-by: Sasha Levin

perf trace: Fixup etcsnoop example

2019-04-05T20:34:47+00:00

[ Upstream commit 1d59cb1bbd4cbe5a8f8032242cdacea5658129cf ] Where we don't have "raw_syscalls:sys_enter", so we need to look for a "*syscalls:sys_enter*" to initialize the offsets for the __augmented_syscalls__ evsel, which is the case with etcsnoop, that was segfaulting, fixed: # trace -e /home/acme/git/perf/tools/perf/examples/bpf/etcsnoop.c 0.000 ( ): gnome-shell/2105 openat(dfd: CWD, filename: "/etc/localtime") ... 631.834 ( ): cat/6521 openat(dfd: CWD, filename: "/etc/ld.so.cache", flags: RDONLY|CLOEXEC) ... 632.637 ( ): bash/6521 openat(dfd: CWD, filename: "/etc/passwd") ... ^C# Cc: Adrian Hunter Cc: Jiri Olsa Cc: Luis Cláudio Gonçalves Cc: Namhyung Kim Cc: Wang Nan Fixes: b9b6a2ea2baf ("perf trace: Do not hardcode the size of the tracepoint common_ fields") Link: https://lkml.kernel.org/n/tip-0tjwcit8qitsmh4nyvf2b0jo@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo Signed-off-by: Sasha Levin

perf coresight: Do not test for libopencsd by default

2019-04-05T20:34:37+00:00

[ Upstream commit 1c3b28fd7ae80c8f6bf1a09e1848e20a953c9ce4 ] Since it is not yet that generally available, avoid testing for the presence of libcoresight in the fast path test-all.bin feature test. # dnf search opencsd No matches found. # dnf search OpenCSD No matches found. # cat /etc/fedora-release Fedora release 29 (Twenty Nine) # I.e. right now, in my system test-all.bin is failing all the time since Fedora29 doesn't have libopencsd available: $ cat /tmp/build/perf/feature/test-all.make.output In file included from test-all.c:174: test-libopencsd.c:2:10: fatal error: opencsd/c_api/opencsd_c_api.h: No such file or directory #include ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ compilation terminated. See: 6ab2b762befd ("perf build: Disable libbabeltrace check by default") For the rationale, as soon as libopencsd becomes more generally packaged and available, we do the same thing we did with babeltrace, enabling it by default, as done in: 24787afbcd01 ("perf tools: Enable LIBBABELTRACE by default") For now, to explicitely ask for opencsd, make sure you have it installed and use: make -C tools/perf CORESIGHT=1 The feature test output will be there as an empty file: $ ls -la /tmp/build/perf/feature/test-libopencsd.make.output Because the binary used for the feature check was successfully built: $ ls -la /tmp/build/perf/feature/test-libopencsd.bin -rwxrwxr-x. 1 acme acme 18336 Feb 12 14:49 /tmp/build/perf/feature/test-libopencsd.bin $ ldd /tmp/build/perf/feature/test-libopencsd.bin linux-vdso.so.1 (0x00007fffe18cc000) libopencsd_c_api.so.0 => /lib64/libopencsd_c_api.so.0 (0x00007fb8e67f6000) libopencsd.so.0 => /lib64/libopencsd.so.0 (0x00007fb8e676f000) libc.so.6 => /lib64/libc.so.6 (0x00007fb8e65a9000) libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007fb8e6411000) libm.so.6 => /lib64/libm.so.6 (0x00007fb8e628d000) libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007fb8e6272000) /lib64/ld-linux-x86-64.so.2 (0x00007fb8e6828000) $ And the resulting perf binary will be linked with it: -rw-rw-r--. 1 acme acme 0 Feb 12 14:49 /tmp/build/perf/feature/test-libopencsd.make.output $ ldd ~/bin/perf | grep opencsd libopencsd_c_api.so.0 => /lib64/libopencsd_c_api.so.0 (0x00007fd43097f000) libopencsd.so.0 => /lib64/libopencsd.so.0 (0x00007fd4308f8000) $ To make sure this gets built before pushing things upstream I have a ubuntu:19.04-x-arm64 container that has: [root@quaco x-arm64]# grep CORESIGHT Dockerfile ENV EXTRA_MAKE_ARGS=CORESIGHT=1 [root@quaco x-arm64]# So that I always build with libopencsd before pushing things upstream. Cc: Adrian Hunter Cc: Alexander Shishkin Cc: Jiri Olsa Cc: Kim Phillips Cc: linux-arm-kernel@lists.infradead.org Cc: Mathieu Poirier Cc: Mike Leach Cc: Namhyung Kim Cc: Peter Zijlstra Cc: Suzuki Poulouse Link: https://lkml.kernel.org/n/tip-20vyy39jw9jgrijesi30fgox@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo Signed-off-by: Sasha Levin

perf report: Add s390 diagnosic sampling descriptor size

2019-04-05T20:34:37+00:00

[ Upstream commit 2187d87eacd46f6214ce3dc9cfd7a558375a4153 ] On IBM z13 machine types 2964 and 2965 the descriptor sizes for sampling and diagnostic sampling entries might be missing in the trailer entry and are set to zero. This leads to a perf report failure when processing diagnostic sampling entries. This patch adds missing descriptor sizes when the trailer entry contains zero for these fields. Output before: [root@s38lp82 perf]# ./perf report --stdio | fgrep Samples 0xabbf0 [0x8]: failed to process type: 68 Error: failed to process sample [root@s38lp82 perf]# Output after: [root@s38lp82 perf]# ./perf report --stdio | fgrep Samples # Total Lost Samples: 0 # Samples: 3K of event 'SF_CYCLES_BASIC_DIAG' # Samples: 162 of event 'CF_DIAG' [root@s38lp82 perf]# Fixes: 2b1444f2e28b ("perf report: Add raw report support for s390 auxiliary trace") Signed-off-by: Thomas Richter Reviewed-by: Hendrik Brueckner Cc: Heiko Carstens Cc: Martin Schwidefsky Link: http://lkml.kernel.org/r/20190211100627.85714-1-tmricht@linux.ibm.com Signed-off-by: Arnaldo Carvalho de Melo Signed-off-by: Sasha Levin

perf trace: Check if the 'fd' is negative when mapping it to pathname

2019-04-05T20:34:37+00:00

[ Upstream commit 051074867434cc520c08f188479d4757dcfdaef8 ] We were crashing when processing a negative fd: Program received signal SIGSEGV, Segmentation fault. 0x0000000000609bbf in syscall_arg__scnprintf_ioctl_cmd (bf=0x1172eca "", size=2038, arg=0x7fffffff8360) at trace/beauty/ioctl.c:182 182 if (file->dev_maj == USB_DEVICE_MAJOR) Missing separate debuginfos, use: dnf debuginfo-install bzip2-libs-1.0.6-28.fc29.x86_64 elfutils-libelf-0.174-5.fc29.x86_64 elfutils-libs-0.174-5.fc29.x86_64 glib2-2.58.3-1.fc29.x86_64 libbabeltrace-1.5.6-1.fc29.x86_64 libunwind-1.2.1-6.fc29.x86_64 libuuid-2.32.1-1.fc29.x86_64 libxcrypt-4.4.3-2.fc29.x86_64 numactl-libs-2.0.12-1.fc29.x86_64 openssl-libs-1.1.1a-1.fc29.x86_64 pcre-8.42-6.fc29.x86_64 perl-libs-5.28.1-427.fc29.x86_64 popt-1.16-15.fc29.x86_64 python2-libs-2.7.15-11.fc29.x86_64 slang-2.3.2-4.fc29.x86_64 xz-libs-5.2.4-3.fc29.x86_64 (gdb) bt #0 0x0000000000609bbf in syscall_arg__scnprintf_ioctl_cmd (bf=0x1172eca "", size=2038, arg=0x7fffffff8360) at trace/beauty/ioctl.c:182 #1 0x000000000048e295 in syscall__scnprintf_val (sc=0x123b500, bf=0x1172eca "", size=2038, arg=0x7fffffff8360, val=21519) at builtin-trace.c:1594 #2 0x000000000048e60d in syscall__scnprintf_args (sc=0x123b500, bf=0x1172ec6 "-1, ", size=2042, args=0x7ffff6a7c034 "\377\377\377\377", augmented_args=0x7ffff6a7c064, augmented_args_size=4, trace=0x7fffffffa8d0, thread=0x1175cd0) at builtin-trace.c:1661 #3 0x000000000048f04e in trace__sys_enter (trace=0x7fffffffa8d0, evsel=0xb260b0, event=0x7ffff6a7bfe8, sample=0x7fffffff84f0) at builtin-trace.c:1880 #4 0x00000000004915a4 in trace__handle_event (trace=0x7fffffffa8d0, event=0x7ffff6a7bfe8, sample=0x7fffffff84f0) at builtin-trace.c:2590 #5 0x0000000000491eed in __trace__deliver_event (trace=0x7fffffffa8d0, event=0x7ffff6a7bfe8) at builtin-trace.c:2818 #6 0x0000000000492030 in trace__deliver_event (trace=0x7fffffffa8d0, event=0x7ffff6a7bfe8) at builtin-trace.c:2845 #7 0x0000000000492896 in trace__run (trace=0x7fffffffa8d0, argc=0, argv=0x7fffffffdb58) at builtin-trace.c:3040 #8 0x000000000049603a in cmd_trace (argc=0, argv=0x7fffffffdb58) at builtin-trace.c:3952 #9 0x00000000004d5103 in main (argc=1, argv=0x7fffffffdb58) at perf.c:474 (gdb) p fd $1 = -1 (gdb) p file $7 = (struct file *) 0xfffffffffffffff0 (gdb) p ((struct thread_trace *)arg->thread)->files.table + fd $8 = (struct file *) 0xfffffffffffffff0 (gdb) Check for that and return NULL instead. This problem was introduced recently, the other codepaths leading to thread_trace__files_entry() check for negative fds, like thread__fd_path(), but we need to do it at thread_trace__files_entry() as more users are now calling it directly. Cc: Adrian Hunter Cc: Jiri Olsa Cc: Luis Cláudio Gonçalves Cc: Namhyung Kim Cc: Wang Nan Fixes: 2d473389f87a ("perf trace beauty: Export function to get the files for a thread") Link: https://lkml.kernel.org/n/tip-oq7bvaaf07gsd4yqty3107u2@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo Signed-off-by: Sasha Levin

perf beauty waitid options: Fix up prefix showing logic

2019-04-05T20:34:37+00:00

[ Upstream commit 1da7e0022784b0e05b49bf73521fa2cc4633af85 ] When introducing the possibility for selecting if the common prefix to options such as the waitid ones, i.e. all 'waitid' options start with 'W', so, to make it make it more compact if configured to suppress it, 'perf trace' will do so, other examples include mmap's PROT_ prefix for its 'prot' argument, etc, which, when showing the syscall argument name ends up producing duplicated info that clutters the screen, i.e.: # perf trace -e mmap --max-events 2 sleep 1 0.000 ( 0.014 ms): sleep/20886 mmap(len: 112595, prot: PROT_READ, flags: MAP_PRIVATE, fd: 3) = 0x7f3e986d2000 0.041 ( 0.005 ms): sleep/20886 mmap(len: 8192, prot: PROT_READ|PROT_WRITE, flags: MAP_PRIVATE|MAP_ANONYMOUS) = 0x7f3e986d0000 # So it is possible to suppress that and make it more compact by having this in your ~/.perfconfig: # cat ~/.perfconfig [trace] show_prefix = no # # perf trace -e mmap --max-events 2 sleep 1 0.000 ( 0.014 ms): sleep/8009 mmap(len: 112595, prot: READ, flags: PRIVATE, fd: 3) = 0x7ff2373de000 0.040 ( 0.005 ms): sleep/8009 mmap(len: 8192, prot: READ|WRITE, flags: PRIVATE|ANONYMOUS) = 0x7ff2373dc000 # To have it look more like strace's output, we instead want to suppress the arg name and show the prefix, so use: # cat ~/.perfconfig [trace] show_prefix = yes show_arg_names = no # # perf trace -e mmap --max-events 2 sleep 1 0.000 ( 0.006 ms): sleep/15513 mmap(NULL, 112595, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f7a9b6d3000 0.020 ( 0.002 ms): sleep/15513 mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS) = 0x7f7a9b6d1000 # When this logic was introduced a bug came with it when processing the waitid 'option' arg that ended up expecting 3 strings when just two were being provided, fix it. Cc: Adrian Hunter Cc: Jiri Olsa Cc: Luis Cláudio Gonçalves Cc: Namhyung Kim Cc: Wang Nan Fixes: c65c83ffe904 ("perf trace: Allow asking for not suppressing common string prefixes") Signed-off-by: Arnaldo Carvalho de Melo Signed-off-by: Sasha Levin

tools build: Add -lrt to FEATURE_CHECK_LDFLAGS-libaio

2019-04-05T20:34:37+00:00

[ Upstream commit aa8f9c517ebce7a0959da064ef2660ea03f133f8 ] Since we need it to resolve the AIO symbols, otherwise we fail with: $ cat /tmp/build/perf/feature/test-all.make.output /usr/bin/ld: /tmp/ccEqrj36.o: undefined reference to symbol 'aio_return64@@GLIBC_2.2.5' /usr/bin/ld: //usr/lib64/librt.so.1: error adding symbols: DSO missing from command line collect2: error: ld returned 1 exit status $ When we added the aio support in 'perf record' only the test-libaio.bin target got the -lrt, i.e. the feature detection slow path. Fix it. Cc: Adrian Hunter Cc: Alexander Shishkin Cc: Alexey Budankov Cc: Andi Kleen Cc: Jiri Olsa Cc: Namhyung Kim Cc: Peter Zijlstra Fixes: 2a07d814747b ("tools build feature: Check if libaio is available") Signed-off-by: Arnaldo Carvalho de Melo Signed-off-by: Sasha Levin

perf report: Don't shadow inlined symbol with different addr range

2019-04-05T20:34:34+00:00

[ Upstream commit 7346195e8643482968f547483e0d823ec1982fab ] We can't assume inlined symbols with the same name are equal, because their address range may be different. This will cause the symbols with different addresses be shadowed when adding to the hist entry, and lead to ERANGE error when checking the symbol address during sample parse, the addr should be within the range of [sym.start, sym.end]. The error message is like: "0x36aea60 [0x8]: failed to process type: 68". The second parameter of symbol__new() is the length of the fake symbol for the inline frame, which is the subtraction of the end and start address of base_sym. Signed-off-by: He Kuang Acked-by: Jiri Olsa Cc: Alexander Shishkin Cc: Milian Wolff Cc: Namhyung Kim Cc: Peter Zijlstra Fixes: aa441895f7b4 ("perf report: Compare symbol name for inlined frames when sorting") Link: http://lkml.kernel.org/r/20190219130531.15692-1-hekuang@huawei.com Signed-off-by: Arnaldo Carvalho de Melo Signed-off-by: Sasha Levin

perf test: Fix failure of 'evsel-tp-sched' test on s390

2019-04-05T20:34:33+00:00

[ Upstream commit 03d309711d687460d1345de8a0363f45b1c8cd11 ] Commit 489338a717a0 ("perf tests evsel-tp-sched: Fix bitwise operator") causes test case 14 "Parse sched tracepoints fields" to fail on s390. This test succeeds on x86. In fact this test now fails on all architectures with type char treated as type unsigned char. The root cause is the signed-ness of character arrays in the tracepoints sched_switch for structure members prev_comm and next_comm. On s390 the output of: [root@m35lp76 perf]# cat /sys/kernel/debug/tracing/events/sched/sched_switch/format name: sched_switch ID: 287 format: field:unsigned short common_type; offset:0; size:2; signed:0; ... field:char prev_comm[16]; offset:8; size:16; signed:0; ... field:char next_comm[16]; offset:40; size:16; signed:0; reveals the character arrays prev_comm and next_comm are per default unsigned char and have values in the range of 0..255. On x86 both fields are signed as this output shows: [root@f29]# cat /sys/kernel/debug/tracing/events/sched/sched_switch/format name: sched_switch ID: 287 format: field:unsigned short common_type; offset:0; size:2; signed:0; ... field:char prev_comm[16]; offset:8; size:16; signed:1; ... field:char next_comm[16]; offset:40; size:16; signed:1; and the character arrays prev_comm and next_comm are per default signed char and have values in the range of -1..127. The implementation of type char is architecture specific. Since the character arrays in both tracepoints sched_switch and sched_wakeup should contain ascii characters, simply omit the check for signedness in the test case. Output before: [root@m35lp76 perf]# ./perf test -F 14 14: Parse sched tracepoints fields : --- start --- sched:sched_switch: "prev_comm" signedness(0) is wrong, should be 1 sched:sched_switch: "next_comm" signedness(0) is wrong, should be 1 sched:sched_wakeup: "comm" signedness(0) is wrong, should be 1 ---- end ---- 14: Parse sched tracepoints fields : FAILED! [root@m35lp76 perf]# Output after: [root@m35lp76 perf]# ./perf test -Fv 14 14: Parse sched tracepoints fields : --- start --- ---- end ---- Parse sched tracepoints fields: Ok [root@m35lp76 perf]# Fixes: 489338a717a0 ("perf tests evsel-tp-sched: Fix bitwise operator") Signed-off-by: Thomas Richter Cc: Heiko Carstens Cc: Hendrik Brueckner Cc: Martin Schwidefsky Link: http://lkml.kernel.org/r/20190219153639.31267-1-tmricht@linux.ibm.com Signed-off-by: Arnaldo Carvalho de Melo Signed-off-by: Sasha Levin