Linemap infrastructure for virtual locations

This is the first instalment of a set which goal is to track locations
of tokens across macro expansions.  Tom Tromey did the original work
and attached the patch to PR preprocessor/7263.  This opus is a
derivative of that original work.

This patch modifies the linemap module of libcpp to add virtual
locations support.

A virtual location is a mapped location that can resolve to several
different physical locations.  It can always resolve to the spelling
location of a token.  For tokens resulting from macro expansion it can
resolve to:
  - either the location of the expansion point of the macro.
  - or the location of the token in the definition of the
  macro
  - or, if the token is an argument of a function-like macro,
  the location of the use of the matching macro parameter in
  the definition of the macro

The patch creates a new type of line map called a macro map.  For every
single macro expansion, there is a macro map that generates a virtual
location for every single resulting token of the expansion.

The good old type of line map we all know is now called an ordinary
map.  That one still encodes spelling locations as it has always had.

As a result linemap_lookup as been extended to return a macro map when
given a virtual location resulting from a macro expansion.  The layout
of structs line_map has changed to support this new type of map.  So
did the layout of struct line_maps.  Accessor macros have been
introduced to avoid messing with the implementation details of these
datastructures directly.  This helped already as we have been testing
different ways of arranging these datastructure.  Having to constantly
adjust client code that is too tied with the internals of line_map and
line_maps would have been even more painful.

Of course, many new public functions have been added to the linemap
module to handle the resolution of virtual locations.

This patch introduces the infrastructure but no part of the compiler
uses virtual locations yet.

However the client code of the linemap data structures has been
adjusted as per the changes.  E.g, it's not anymore reliable for a
client code to manipulate struct line_map directly if it just wants to
deal with spelling locations, because struct line_map can now
represent a macro map as well.  In that case, it's better to use the
convenient API to resolve the initial (possibly virtual) location to a
spelling location (or to an ordinary map) and use that.

This is the reason why the patch adjusts the Java, Ada and Fortran
front ends.

Also, note that virtual locations are not supposed to be ordered for
relations '<' and '>' anymore.  To test if a virtual location appears
"before" another one, one has to use a new operator exposed by the
line map interface.  The patch updates the only spot (in the
diagnostics module) I have found that was making the assumption that
locations were ordered for these relations.  This is the only change
that introduces a use of the new line map API in this patch, so I am
adding a regression test for it only.

git-svn-id: svn+ssh://gcc.gnu.org/svn/gcc/trunk@180081 138bc75d-0d04-0410-961f-82ee72b054a4

1 files changed, 118 insertions, 0 deletions

diff --git a/libcpp/ChangeLog b/libcpp/ChangeLog
index 4ecda03cfec..b961ef5a810 100644
--- a/libcpp/ChangeLog
+++ b/libcpp/ChangeLog
@@ -1,3 +1,121 @@
+2011-10-15  Tom Tromey  <tromey@redhat>
+	    Dodji Seketeli  <dodji@redhat.com>
+
+	* include/line-map.h (enum lc_reason)<LC_ENTER_MACRO>: New enum
+	member.
+	(MAX_SOURCE_LOCATION): New constant.
+	(struct line_map_ordinary, struct line_map_macro): New structs.
+    	(struct line_map): Turn this into a union of the two above.  Add
+	comments.
+	(struct maps_info): New struct.
+	(struct line_maps)<info_ordinary, info_macro>: Two new fields.
+	These now carry the map information that was previously scattered
+	in struct line_maps.
+	(struct map_info::allocated): Fix comment.
+	(MAP_START_LOCATION, ORDINARY_MAP_FILE_NAME)
+	(ORDINARY_MAP_STARTING_LINE_NUMBER)
+	(ORDINARY_MAP_INCLUDER_FILE_INDEX)
+	(ORDINARY_MAP_IN_SYSTEM_HEADER_P)
+	(ORDINARY_MAP_NUMBER_OF_COLUMN_BITS, MACRO_MAP_MACRO)
+	(MACRO_MAP_NUM_MACRO_TOKENS MACRO_MAP_LOCATIONS)
+	(MACRO_MAP_EXPANSION_POINT_LOCATION)
+	(LOCATION_POSSIBLY_IN_MACRO_MAP_P, LINEMAPS_MAP_INFO)
+	(LINEMAPS_MAPS, LINEMAPS_ALLOCATE, LINEMAPS_USED, LINEMAPS_CACHE)
+	(LINEMAPS_LAST_MAP, LINEMAPS_LAST_ALLOCATED_MAP)
+	(LINEMAPS_ORDINARY_MAPS, LINEMAPS_ORDINARY_ALLOCATED)
+	(LINEMAPS_ORDINARY_USED, LINEMAPS_ORDINARY_CACHE)
+	(LINEMAPS_LAST_ORDINARY_MAP, LINEMAPS_LAST_ALLOCATED_ORDINARY_MAP)
+	(LINEMAPS_MACRO_MAPS, LINEMAPS_MACRO_ALLOCATED)
+	(LINEMAPS_MACRO_USED, LINEMAPS_MACRO_CACHE)
+	(LINEMAPS_LAST_MACRO_MAP, LINEMAPS_LAST_ALLOCATED_MACRO_MAP)
+	(LINEMAPS_MAP_AT, LINEMAPS_ORDINARY_MAP_AT)
+	(LINEMAPS_MACRO_MAP_AT): New accessors for ordinary and macro map
+	information.
+	(linemap_check_ordinary, linemap_assert)
+	(linemap_location_before_p): New macros.
+	(linemap_position_for_line_and_column)
+	(linemap_tracks_macro_expansion_locs_p, linemap_add_macro_token)
+	(linemap_macro_expansion_map_p)
+	(linemap_macro_map_loc_to_def_point)
+	(linemap_macro_map_loc_unwind_once)
+	(linemap_macro_map_loc_to_exp_point, linemap_step_out_once)
+	(linemap_get_source_line linemap_get_source_column)
+	(linemap_map_get_macro_name, linemap_get_file_path)
+	(linemap_location_in_system_header_p)
+	(linemap_location_from_macro_expansion_p): Declare new functions.
+	(SOURCE_LINE, SOURCE_COLUMN, LAST_SOURCE_LINE_LOCATION)
+	(LINEMAP_FILE, LINEMAP_LINE, LINEMAP_SYSP): Assert that this
+	accessors act on ordinary maps only.
+	(INCLUDED_FROM): Return NULL for main files; use the new
+	accessors.
+	(LINEMAP_POSITION_FOR_COLUMN): Use the new accessors.
+	(struct expanded_location): Move here from gcc/input.h
+	(linemap_resolve_location, linemap_expand_location)
+	(linemap_expand_location_full): Declare new functions.
+	* line-map.c: Include cpplib.h, internal.h
+	(linemap_enter_macro, linemap_add_macro_token)
+	(linemap_get_expansion_line, linemap_get_expansion_filename): New
+	functions that are private to libcpp.
+	(linemap_assert): New macro.
+	(linemap_macro_loc_to_exp_point, linemap_macro_loc_to_exp_point)
+	(linemap_macro_loc_unwind, linemap_macro_map_loc_to_def_point)
+	(linemap_macro_map_loc_unwind_toward_spelling)
+	(linemap_macro_map_loc_to_exp_point)
+	(first_map_in_common_1, first_map_in_common): New static
+	functions.
+	(new_linemap): Define new static functions.  Extracted and
+	enhanced from ...
+	(linemap_add): ... here.  Use linemap_assert in lieu of abort
+	previously.
+	(linemap_tracks_macro_expansion_locs_p)
+	(linemap_add_macro_token, linemap_macro_expansion_map_p)
+	(linemap_check_ordinary, linemap_macro_map_loc_to_exp_point)
+	(linemap_macro_map_loc_to_def_point)
+	(linemap_macro_map_loc_unwind_once)
+	(linemap_step_out_once, linemap_map_get_index)
+	(linemap_get_source_line,linemap_get_source_column)
+	(linemap_get_file_path, linemap_map_get_macro_name)
+	(linemap_location_in_system_header_p)
+	(linemap_location_originated_from_system_header_p)
+	(linemap_location_from_macro_expansion_p)
+	(linemap_tracks_macro_expansion_locs_p)
+	(linemap_resolve_location, linemap_expand_location)
+	(linemap_expand_location_full)
+	(linemap_tracks_macro_expansion_locs_p)
+	(linemap_position_for_line_and_column, linemap_compare_locations):
+	Define new public functions.
+	(linemap_init): Initialize ordinary and macro maps information in
+	the map set.
+	(linemap_check_files_exited): Use the new accessors.
+	(linemap_free): Remove this dead code.
+	(linemap_line_start): Assert this uses an ordinary map.  Adjust to
+	use the new ordinary map accessors and data structures.  Don't
+	overflow past the lowest possible macro token's location.
+	(linemap_position_for_column): Assert the ordinary maps of the map
+	set are really ordinary.  Use ordinary map accessors.
+	(linemap_lookup): Keep the same logic but generalize to allow
+	lookup of both ordinary and macro maps.  Do not crash when called
+	with an empty line table.
+	* directives-only.c (_cpp_preprocess_dir_only): Adjust to use the
+	new API of line-map.h.
+	* directives.c (start_directive, do_line, do_linemarker)
+	(do_linemarker): Likewise.
+	* files.c (_cpp_find_file, _cpp_stack_include, open_file_failed)
+	(make_cpp_dir, cpp_make_system_header): Likewise.
+	* init.c (cpp_read_main_file): Likewise.
+	* internal.h (CPP_INCREMENT_LINE): Likewise.
+	(linemap_enter_macro, linemap_add_macro_token)
+	(linemap_get_expansion_line, linemap_get_expansion_filename): New
+	functions private to libcpp.
+	* lex.c (_cpp_process_line_notes, _cpp_skip_block_comment)
+	(skip_line_comment, skip_whitespace, lex_raw_string)
+	(_cpp_lex_direct): Likewise.
+	* macro.c (_cpp_builtin_macro_text): Likewise.
+	(_cpp_aligned_alloc): Initialize the new name member of the macro.
+	* traditional.c (copy_comment, _cpp_scan_out_logical_line):
+	Likewise.
+	* errors.c (cpp_diagnostic): Adjust to new linemap API.
+
 2011-08-28  Dodji Seketeli  <dodji@redhat.com>
 
 	* line-map.c (linemap_add): Assert that reason must not be