[libFuzzer] Handle unstable edges by disregarding unstable edges

Summary:
Added a new mode within flag -handle_unstable for new unstable handling algorithm that does the following:
    When an edge is shown as unstable, copy to UnstableCounters the value 0.
    During ApplyUnstableCounters we copy back the value 0 to ModuleInline8bitCounters if the edge was unstable.

This way we would be ignoring completely features that were collected through non-determinism.
Unstable hits would be counted as if it never hit.

Reviewers: metzman, Dor1s, kcc, morehouse

Reviewed By: metzman, morehouse

Subscribers: delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D49684

llvm-svn: 337853

1 files changed, 2 insertions, 1 deletions

diff --git a/compiler-rt/lib/fuzzer/FuzzerLoop.cpp b/compiler-rt/lib/fuzzer/FuzzerLoop.cpp
index e63ee7361c5..4bc88365a0b 100644
--- a/compiler-rt/lib/fuzzer/FuzzerLoop.cpp
+++ b/compiler-rt/lib/fuzzer/FuzzerLoop.cpp
@@ -472,7 +472,8 @@ void Fuzzer::CheckForUnstableCounters(const uint8_t *Data, size_t Size) {
   TPC.UpdateUnstableCounters(Options.HandleUnstable);
 
   // Move minimum hit counts back to ModuleInline8bitCounters
-  if (Options.HandleUnstable)
+  if (Options.HandleUnstable == TracePC::MinUnstable ||
+      Options.HandleUnstable == TracePC::ZeroUnstable)
     TPC.ApplyUnstableCounters();
 }