[llvm-profdata] Add support for weighted merge of profile data

This change adds support for an optional weight when merging profile data with the llvm-profdata tool.
Weights are specified by adding an option ':<weight>' suffix to the input file names.

Adding support for arbitrary weighting of input profile data allows for relative importance to be placed on the
input data from multiple training runs.

Both sampled and instrumented profiles are supported.

Reviewers: dnovillo, bogner, davidxl

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D14547

llvm-svn: 254669

6 files changed, 114 insertions, 0 deletions

diff --git a/llvm/test/tools/llvm-profdata/Inputs/weight-instr-bar.profdata b/llvm/test/tools/llvm-profdata/Inputs/weight-instr-bar.profdata
new file mode 100644
index 00000000000..4ed07660f65
--- /dev/null
+++ b/llvm/test/tools/llvm-profdata/Inputs/weight-instr-bar.profdata
diff --git a/llvm/test/tools/llvm-profdata/Inputs/weight-instr-foo.profdata b/llvm/test/tools/llvm-profdata/Inputs/weight-instr-foo.profdata
new file mode 100644
index 00000000000..581ef39a55b
--- /dev/null
+++ b/llvm/test/tools/llvm-profdata/Inputs/weight-instr-foo.profdata
diff --git a/llvm/test/tools/llvm-profdata/Inputs/weight-sample-bar.proftext b/llvm/test/tools/llvm-profdata/Inputs/weight-sample-bar.proftext
new file mode 100644
index 00000000000..a910f745e6c
--- /dev/null
+++ b/llvm/test/tools/llvm-profdata/Inputs/weight-sample-bar.proftext
@@ -0,0 +1,8 @@
+bar:1772037:35370
+ 17: 35370
+ 18: 35370
+ 19: 7005
+ 20: 29407
+ 21: 12170
+ 23: 18150 bar:19829
+ 25: 36666
diff --git a/llvm/test/tools/llvm-profdata/Inputs/weight-sample-foo.proftext b/llvm/test/tools/llvm-profdata/Inputs/weight-sample-foo.proftext
new file mode 100644
index 00000000000..155ec5d0031
--- /dev/null
+++ b/llvm/test/tools/llvm-profdata/Inputs/weight-sample-foo.proftext
@@ -0,0 +1,8 @@
+foo:1763288:35327
+ 7: 35327
+ 8: 35327
+ 9: 6930
+ 10: 29341
+ 11: 11906
+ 13: 18185 foo:19531
+ 15: 36458
diff --git a/llvm/test/tools/llvm-profdata/weight-instr.test b/llvm/test/tools/llvm-profdata/weight-instr.test
new file mode 100644
index 00000000000..bc0b5061647
--- /dev/null
+++ b/llvm/test/tools/llvm-profdata/weight-instr.test
@@ -0,0 +1,55 @@
+Tests for weighted merge of instrumented profiles.
+
+1- Merge the foo and bar profiles with unity weight and verify the combined output
+RUN: llvm-profdata merge --instr %p/Inputs/weight-instr-bar.profdata:1 %p/Inputs/weight-instr-foo.profdata:1 -o %t
+RUN: llvm-profdata show --instr -all-functions %t | FileCheck %s --check-prefix=WEIGHT1
+WEIGHT1: Counters:
+WEIGHT1:   usage:
+WEIGHT1:     Hash: 0x0000000000000000
+WEIGHT1:     Counters: 1
+WEIGHT1:     Function count: 0
+WEIGHT1:   foo:
+WEIGHT1:     Hash: 0x000000000000028a
+WEIGHT1:     Counters: 3
+WEIGHT1:     Function count: 866988873
+WEIGHT1:   bar:
+WEIGHT1:     Hash: 0x000000000000028a
+WEIGHT1:     Counters: 3
+WEIGHT1:     Function count: 866988873
+WEIGHT1:   main:
+WEIGHT1:     Hash: 0x7d31c47ea98f8248
+WEIGHT1:     Counters: 60
+WEIGHT1:     Function count: 2
+WEIGHT1: Functions shown: 4
+WEIGHT1: Total functions: 4
+WEIGHT1: Maximum function count: 866988873
+WEIGHT1: Maximum internal block count: 267914296
+
+2- Merge the foo and bar profiles with weight 3x and 5x respectively and verify the combined output
+RUN: llvm-profdata merge --instr %p/Inputs/weight-instr-bar.profdata:3 %p/Inputs/weight-instr-foo.profdata:5 -o %t
+RUN: llvm-profdata show --instr -all-functions %t | FileCheck %s --check-prefix=WEIGHT2
+WEIGHT2: Counters:
+WEIGHT2:   usage:
+WEIGHT2:     Hash: 0x0000000000000000
+WEIGHT2:     Counters: 1
+WEIGHT2:     Function count: 0
+WEIGHT2:   foo:
+WEIGHT2:     Hash: 0x000000000000028a
+WEIGHT2:     Counters: 3
+WEIGHT2:     Function count: 4334944365
+WEIGHT2:   bar:
+WEIGHT2:     Hash: 0x000000000000028a
+WEIGHT2:     Counters: 3
+WEIGHT2:     Function count: 2600966619
+WEIGHT2:   main:
+WEIGHT2:     Hash: 0x7d31c47ea98f8248
+WEIGHT2:     Counters: 60
+WEIGHT2:     Function count: 8
+WEIGHT2: Functions shown: 4
+WEIGHT2: Total functions: 4
+WEIGHT2: Maximum function count: 4334944365
+WEIGHT2: Maximum internal block count: 1339571480
+
+3- Bad merge: foo and bar profiles with invalid weights
+RUN: not llvm-profdata merge --instr %p/Inputs/weight-instr-bar.profdata:3 %p/Inputs/weight-instr-foo.profdata:-5 -o %t.out 2>&1 | FileCheck %s --check-prefix=ERROR3
+ERROR3: error: Input weight must be a positive integer.
diff --git a/llvm/test/tools/llvm-profdata/weight-sample.test b/llvm/test/tools/llvm-profdata/weight-sample.test
new file mode 100644
index 00000000000..a1fe1df1b6d
--- /dev/null
+++ b/llvm/test/tools/llvm-profdata/weight-sample.test
@@ -0,0 +1,43 @@
+Tests for weighted merge of sample profiles.
+
+1- Merge the foo and bar profiles with unity weight and verify the combined output
+RUN: llvm-profdata merge --sample --text %p/Inputs/weight-sample-bar.proftext:1 %p/Inputs/weight-sample-foo.proftext:1 -o - | FileCheck %s --check-prefix=WEIGHT1
+WEIGHT1: foo:1763288:35327
+WEIGHT1:  7: 35327
+WEIGHT1:  8: 35327
+WEIGHT1:  9: 6930
+WEIGHT1:  10: 29341
+WEIGHT1:  11: 11906
+WEIGHT1:  13: 18185 foo:19531
+WEIGHT1:  15: 36458
+WEIGHT1: bar:1772037:35370
+WEIGHT1:  17: 35370
+WEIGHT1:  18: 35370
+WEIGHT1:  19: 7005
+WEIGHT1:  20: 29407
+WEIGHT1:  21: 12170
+WEIGHT1:  23: 18150 bar:19829
+WEIGHT1:  25: 36666
+
+2- Merge the foo and bar profiles with weight 3x and 5x respectively and verify the combined output
+RUN: llvm-profdata merge --sample --text %p/Inputs/weight-sample-bar.proftext:3 %p/Inputs/weight-sample-foo.proftext:5 -o - | FileCheck %s --check-prefix=WEIGHT2
+WEIGHT2: foo:8816440:176635
+WEIGHT2:  7: 176635
+WEIGHT2:  8: 176635
+WEIGHT2:  9: 34650
+WEIGHT2:  10: 146705
+WEIGHT2:  11: 59530
+WEIGHT2:  13: 90925 foo:97655
+WEIGHT2:  15: 182290
+WEIGHT2: bar:5316111:106110
+WEIGHT2:  17: 106110
+WEIGHT2:  18: 106110
+WEIGHT2:  19: 21015
+WEIGHT2:  20: 88221
+WEIGHT2:  21: 36510
+WEIGHT2:  23: 54450 bar:59487
+WEIGHT2:  25: 109998
+
+3- Bad merge: foo and bar profiles with invalid weights
+RUN: not llvm-profdata merge --sample --text %p/Inputs/weight-sample-bar.proftext:3 %p/Inputs/weight-sample-foo.proftext:-5 -o %t.out 2>&1 | FileCheck %s --check-prefix=ERROR3
+ERROR3: error: Input weight must be a positive integer.