[asan] Avoid instrumenting duplicated memory access in the same basic block

Like what Address Sanitizer does in LLVM, this patch avoids instrumented
duplicated memory accesses in the same basic blocks.

The approach taken is very conservative, to keep the pass simple, for
a start.

A memory access is considered to be a pair made of an expression tree
representing the beginning of the memory region that is accessed and
a the size of the access, in byte.  For now that size is either 1, 2,
4, 8 or 16 bytes.

The patch builds a hash table of the memory accesses that have been
instrumented in the current basic block.  Then it walks the gimple
statements of the current basic block.  For each statement, it tests
if the memory regions it references have already been instrumented.
If not, the statement is instrumented and each memory references that
are actually instrumented are added to the hash table.  When a memory
region is accessed (usually through builtin functions like memset),
then what gets added to the hash table is actually two memory
accesses: one for the beginning of the region, and the other for the
its end.

When the patch crosses a function call that is not a built-in function
that we ought to instrument, the hash table is cleared, because that
function call can possibly e.g free some memory that was instrumented.

Likewise, when a new basic block is visited, the hash table is
cleared.  I guess we could be smarter than just unconditionally
clearing the hash table in this later case, but this is what asan@llvm
does, and for now, I thought starting in a conservative manner might
have some value.

The hash table is destroyed at the end of the pass.

Bootstrapped and tested against trunk on x86-64-unknown-linux-gnu.

gcc/
	* Makefile.in (asan.o): Add new dependency on hash-table.h
	* asan.c (struct asan_mem_ref, struct mem_ref_hasher): New types.
	(asan_mem_ref_init, asan_mem_ref_get_end, get_mem_ref_hash_table)
	(has_stmt_been_instrumented_p, empty_mem_ref_hash_table)
	(free_mem_ref_resources, has_mem_ref_been_instrumented)
	(has_stmt_been_instrumented_p, update_mem_ref_hash_table)
	(get_mem_ref_of_assignment): New functions.
	(get_mem_refs_of_builtin_call): Extract from
	instrument_builtin_call and tweak a little bit to make it fit with
	the new signature.
	(instrument_builtin_call): Use the new
	get_mem_refs_of_builtin_call.  Use gimple_call_builtin_p instead
	of is_gimple_builtin_call.
	(instrument_derefs, instrument_mem_region_access): Insert the
	instrumented memory reference into the hash table.
	(maybe_instrument_assignment): Renamed instrument_assignment into
	this, and change it to advance the iterator when instrumentation
	actually happened and return true in that case.  This makes it
	homogeneous with maybe_instrument_assignment, and thus give a
	chance to callers to be more 'regular'.
	(transform_statements): Clear the memory reference hash table
	whenever we enter a new BB, when we cross a function call, or when
	we are done transforming statements.  Use
	maybe_instrument_assignment instead of instrumentation.  No more
	need to special case maybe_instrument_assignment and advance the
	iterator after calling it; it's now handled just like
	maybe_instrument_call.  Update comment.

gcc/testsuite/

	* c-c++-common/asan/no-redundant-instrumentation-1.c: New test.
	* testsuite/c-c++-common/asan/no-redundant-instrumentation-2.c: Likewise.
	* testsuite/c-c++-common/asan/no-redundant-instrumentation-3.c: Likewise.
	* testsuite/c-c++-common/asan/inc.c: Likewise.

git-svn-id: svn+ssh://gcc.gnu.org/svn/gcc/trunk@196008 138bc75d-0d04-0410-961f-82ee72b054a4

0 files changed, 0 insertions, 0 deletions